[00:07.930 --> 00:14.570]  So today we're going to talk about the duality of Chad GPT and copilot in secure development.
[00:14.790 --> 00:19.090]  And so my name is Kalyani Pawar. Let's talk a little bit about me.
[00:19.970 --> 00:24.050]  So hi, I am Kalyani Pawar. I also go by Kaylee.
[00:24.130 --> 00:30.350]  I currently run the application security and vendor security side of things at a drone company called Zipline.
[00:30.390 --> 00:33.850]  I've been doing security for nearly half a decade now.
[00:35.570 --> 00:38.850]  I enjoy doing what I do in my free time.
[00:39.050 --> 00:44.370]  All the opinions throughout all these slides are mine. I am not representing my employer in any way.
[00:45.190 --> 00:50.590]  This is my Twitter or X handle. I don't post a lot, but I promise everything that I learn,
[00:50.590 --> 00:54.950]  all the prompts that we're talking about today, I'm going to talk about them on the Twitter soon.
[00:56.250 --> 00:58.690]  Things that I do outside of my work.
[00:58.690 --> 01:05.330]  I do review talks that get submitted to Grace Hoppers and AppSec Village every year at DEF CON.
[01:05.330 --> 01:07.590]  So that's about me.
[01:08.010 --> 01:12.950]  Now, what are we going to talk about today?
[01:12.990 --> 01:19.130]  Obviously, Chad GPT and Copilot, but some fun, cool AppSec-y things that you can do with them.
[01:19.330 --> 01:24.230]  And while we are exploring the fun side of things, let us proceed with caution
[01:24.230 --> 01:29.550]  and not fall directly for the responses that it is generating for us.
[01:29.610 --> 01:32.230]  All right, now that we know what we're going to talk about...
[01:32.990 --> 01:34.890]  Okay, quick question.
[01:35.210 --> 01:40.250]  Who in the audience today has worked with Chad GPT or Chad GPT-like models?
[01:40.810 --> 01:43.130]  Fancy. That is what I expect.
[01:43.470 --> 01:48.450]  So here is a quick primer for people who are not familiar with Chad GPT.
[01:48.830 --> 01:52.550]  It's an AI language model developed by OpenAI.
[01:52.550 --> 01:56.170]  It is based on generative pre-trained transformer architecture.
[01:56.170 --> 01:58.930]  It is out of scope. The whole architecture is out of scope for this talk.
[01:58.930 --> 02:00.150]  You don't need to know this information.
[02:00.150 --> 02:03.690]  All you've got to know is how to type in text on a browser.
[02:05.250 --> 02:10.710]  So what I like about Chad GPT is it is trained on a range of diverse data that you find on the Internet,
[02:10.710 --> 02:14.430]  on various books and sources of information that are available online.
[02:14.470 --> 02:20.430]  But what I find beautiful about this tool is that it has the ability to generate human-like text.
[02:20.430 --> 02:23.150]  Once you input whatever question you have for it.
[02:23.170 --> 02:29.630]  The fact that you can speak to a machine which can understand the context and nuances of whatever you talk to it.
[02:30.790 --> 02:34.150]  And it's like talking to a human interlocutor at the end of the day.
[02:34.150 --> 02:35.770]  So that is what I like about it.
[02:35.770 --> 02:39.070]  And you can just imagine the kind of functionality that this tool provides you.
[02:39.070 --> 02:40.990]  You can ask it all sorts of questions.
[02:41.170 --> 02:45.150]  You can ask it to translate into different languages you don't know about.
[02:45.150 --> 02:51.570]  You can use it to learn different things that you probably would be investing an hour reading a book.
[02:51.570 --> 02:53.570]  Rather ask Chad GPT about it.
[02:54.090 --> 02:58.270]  And, yes, when I don't know the answer to a question, that's what I do.
[02:58.270 --> 03:02.570]  Ask Chad GPT, guys. I don't know the answer to this. Use Chad GPT for this.
[03:05.290 --> 03:08.050]  How many of y'all have used Copilot?
[03:09.030 --> 03:10.330]  Y'all like it?
[03:11.290 --> 03:13.990]  Sort of? Yeah, same, same. Sort of.
[03:13.990 --> 03:16.930]  For people who don't know what Copilot is,
[03:16.930 --> 03:21.490]  it is an AI pair programmer developed by GitHub in collaboration with OpenAI.
[03:22.550 --> 03:27.930]  It is trained on a diverse range of public code repositories that are available,
[03:27.930 --> 03:33.750]  so it does understand the coding styles of millions of developers all over the world,
[03:33.750 --> 03:37.050]  and what their documentation styles would be.
[03:37.050 --> 03:42.490]  How many of y'all have struggled with code that has the worst comments in it?
[03:43.130 --> 03:45.330]  Most of us? Most of us? We don't, like...
[03:45.330 --> 03:47.090]  What the f... What is this?
[03:47.610 --> 03:49.750]  What is this? Right?
[03:49.750 --> 03:53.790]  So, what I like about Chad GPT is that, while you are coding,
[03:53.790 --> 03:58.030]  it does also generate comments so that someone else is going to read your code,
[03:58.030 --> 04:00.830]  does understand what have you done, the beauty that you've created.
[04:01.790 --> 04:02.590]  Awesome.
[04:04.370 --> 04:04.770]  So...
[04:04.770 --> 04:06.610]  Oh, yeah, this is something else I like about it,
[04:06.610 --> 04:10.110]  that it can seamlessly integrate within VS Code, which is my choice of IDE.
[04:10.110 --> 04:14.150]  I don't have to, like, open up a different browser or an instance, like, launch this tool.
[04:14.150 --> 04:17.430]  As I'm typing, it's going to generate real-time suggestions about,
[04:17.430 --> 04:19.470]  hey, this is probably what you should be writing,
[04:19.470 --> 04:21.890]  instead of whatever you're thinking of writing.
[04:21.970 --> 04:24.770]  For example, if I just ask Chad GPT Copilot that,
[04:24.770 --> 04:27.150]  can you generate a program that's going to create...
[04:27.150 --> 04:29.190]  that's going to generate a random number for you?
[04:29.190 --> 04:30.370]  It would just...
[04:30.370 --> 04:33.770]  Bam! You just press tap, and it writes the whole program for you.
[04:35.210 --> 04:35.570]  So...
[04:35.570 --> 04:37.530]  Now that we know what Copilot is,
[04:37.530 --> 04:43.390]  I want to talk about the problems that the unsung heroes of our community face.
[04:43.390 --> 04:45.470]  Application security engineers.
[04:45.590 --> 04:46.970]  What is the problem?
[04:47.050 --> 04:50.810]  The problem is that there's a bunch of problems that we're facing.
[04:51.230 --> 04:53.310]  And I'm going to run through them real quick.
[04:54.430 --> 05:00.990]  How many of us have been part of these meetings where they're like,
[05:00.990 --> 05:04.030]  hey, I found something vulnerable in my code.
[05:04.030 --> 05:07.270]  And you're like, okay, how long ago did you push this code into Prod?
[05:07.270 --> 05:09.330]  And they're like, yeah, many sprints ago.
[05:09.330 --> 05:11.890]  And you're like, are you telling me this right now?
[05:11.890 --> 05:13.310]  Like, this is what you did?
[05:13.310 --> 05:17.290]  So I understand with advent of DevOps and Agile,
[05:17.290 --> 05:23.330]  we are shipping and deploying code at 100 miles an hour, which is amazing,
[05:23.330 --> 05:26.610]  but also it's leaving potential security gaps
[05:26.610 --> 05:29.630]  because we are falling short on providing coverage
[05:29.630 --> 05:31.410]  and all the code that has been deployed.
[05:32.530 --> 05:36.130]  Second, lack of security education.
[05:36.130 --> 05:38.990]  According to dark reading, about half of the developers,
[05:38.990 --> 05:42.230]  48% of the developers lack security education.
[05:42.230 --> 05:45.050]  They simply develop code keeping functionality in mind
[05:45.050 --> 05:47.950]  and completely ignore the security aspect of it.
[05:47.950 --> 05:51.630]  So you're pushing insecure code full of vulnerabilities from the get-go.
[05:51.630 --> 05:55.310]  That is another pressing issue that we're facing.
[05:56.820 --> 05:59.070]  The resource constraints.
[05:59.070 --> 06:02.670]  Were you at Tanya Janka's talk earlier this week?
[06:02.670 --> 06:04.250]  Any hands? Okay.
[06:04.250 --> 06:06.670]  So she was talking about... I'm sure...
[06:06.670 --> 06:07.970]  Okay, raise your hands, okay?
[06:07.970 --> 06:10.370]  How many times have you been in an organization
[06:10.370 --> 06:12.550]  where there's like 200 developers
[06:12.550 --> 06:15.410]  and only one AppSec engineer supporting them?
[06:16.790 --> 06:18.690]  Unsung heroes, I tell you.
[06:23.370 --> 06:25.830]  Security teams are usually smaller in size
[06:25.830 --> 06:29.110]  and it's getting difficult to literally review every line of code
[06:29.110 --> 06:32.190]  that is generated by your organization.
[06:32.190 --> 06:34.110]  So what do you do to combat this?
[06:34.670 --> 06:37.250]  You onboard different tools for it.
[06:37.290 --> 06:40.730]  Now, I am guilty of onboarding two, three different tools
[06:40.730 --> 06:44.070]  and just integrating it within my CICD and all of that.
[06:44.070 --> 06:46.230]  But the problem with this is that
[06:46.230 --> 06:48.950]  it is generating a bunch of false positives.
[06:48.950 --> 06:51.170]  And now we're all sitting and investing our time
[06:51.170 --> 06:53.010]  and sifting through what the true positives are,
[06:53.010 --> 06:55.870]  what the false positives are, and it just does not end there.
[06:55.870 --> 06:58.110]  It's about, okay, now we got to prioritize
[06:58.110 --> 06:59.990]  what risk this vulnerability holds
[06:59.990 --> 07:02.410]  and how are we going to clear this backlog?
[07:02.410 --> 07:04.470]  And there's just a lot of time that we spend
[07:04.470 --> 07:07.050]  in strategizing how to deal with this monster.
[07:08.670 --> 07:11.150]  And this is another problem.
[07:11.350 --> 07:14.970]  Ever-emerging threats, ever-emerging attack vectors.
[07:14.970 --> 07:17.120]  Every day there's something new that I read online
[07:17.710 --> 07:19.230]  about some new attack vector.
[07:19.230 --> 07:22.290]  And it's just difficult for this one brain of mine
[07:22.290 --> 07:26.110]  to constantly catch up with everything that's going on outside
[07:26.110 --> 07:27.810]  on the threat landscape.
[07:27.810 --> 07:30.010]  Last but not the least,
[07:30.010 --> 07:34.250]  the hammer that is the legal regulation stick.
[07:34.850 --> 07:37.390]  Depending on the kind of product you're making
[07:37.390 --> 07:39.270]  or the kind of jurisdiction you're part of
[07:39.270 --> 07:40.710]  or the business that you are in,
[07:40.710 --> 07:43.090]  you've got to make sure that your product is compliant
[07:43.090 --> 07:45.550]  with certain industry regulated standards
[07:45.550 --> 07:48.070]  like SOC 2 or GDPR.
[07:48.070 --> 07:50.050]  How many of us take that seriously?
[07:51.150 --> 07:53.090]  We are trying, right?
[07:53.090 --> 07:53.810]  We're trying.
[07:53.810 --> 07:56.090]  It's just like an additional audit task somewhere
[07:56.090 --> 07:57.570]  in the sprint.
[07:58.830 --> 07:59.330]  Awesome.
[07:59.810 --> 08:02.150]  So, now that we know
[08:02.150 --> 08:05.150]  the bunch of problems that we are facing,
[08:05.290 --> 08:07.510]  a possible solution, a possible solution
[08:07.510 --> 08:10.290]  would be to leverage AI for AppSec.
[08:10.290 --> 08:13.410]  When I say AI, I'm trying to loosely refer to
[08:13.410 --> 08:16.950]  Chad, GPT, and Copilot for the rest of this whole conversation.
[08:18.510 --> 08:21.170]  How can we leverage AI for AppSec?
[08:21.410 --> 08:24.370]  Anyone in the crowd?
[08:26.530 --> 08:27.410]  Yes.
[08:30.030 --> 08:31.050]  Training, yes.
[08:31.830 --> 08:33.090]  Anyone else?
[08:35.210 --> 08:36.730]  Yes. Perfect. Bang on.
[08:36.730 --> 08:37.710]  Anyone else?
[08:41.160 --> 08:42.360]  Exactly. Yes.
[08:42.360 --> 08:44.160]  So, a couple applications here.
[08:44.160 --> 08:46.380]  You can ask Chad, GPT, like,
[08:46.380 --> 08:48.760]  hey, this is my code. Tell me what's vulnerable in it
[08:48.760 --> 08:50.620]  and how can I fix this.
[08:50.700 --> 08:53.020]  You can also go like, hey, Copilot,
[08:53.020 --> 08:54.960]  I'm generating this code right now.
[08:55.260 --> 08:57.480]  Can you give me secure coding suggestions
[08:57.480 --> 08:59.780]  about how this whole code is supposed to go?
[09:00.920 --> 09:03.000]  You can also be like, oh, also for, like,
[09:03.100 --> 09:04.860]  a developer especially, they can go like,
[09:04.860 --> 09:07.140]  hey, what is Heartbleed vulnerability?
[09:07.140 --> 09:08.620]  Explain to me in simple terms.
[09:08.620 --> 09:10.160]  They ask this question to Chad, GPT,
[09:10.160 --> 09:11.960]  and they don't have to go through, like,
[09:12.080 --> 09:13.740]  a bunch of web pages to understand
[09:13.740 --> 09:15.880]  what Heartbleed is, right?
[09:15.880 --> 09:18.820]  So, there's certain applications to it.
[09:18.820 --> 09:20.740]  Now, what I see on the bright side for this
[09:20.740 --> 09:23.940]  is that how many times do you see yourself
[09:23.940 --> 09:24.920]  context switching?
[09:24.920 --> 09:27.080]  Like, one moment you're doing API security,
[09:27.080 --> 09:28.700]  one moment you're threat modeling,
[09:28.700 --> 09:31.700]  next moment you're source code reviewing.
[09:31.700 --> 09:33.520]  How many times does that happen to you in a day?
[09:34.320 --> 09:36.240]  All the time. All the time, right?
[09:36.240 --> 09:38.980]  This context switching is leading to burnout
[09:38.980 --> 09:39.800]  on the team.
[09:39.800 --> 09:41.540]  I feel like a hamster on a wheel all the time
[09:41.540 --> 09:43.000]  because I'm constantly switching
[09:43.000 --> 09:45.000]  into, like, different projects all the time
[09:45.000 --> 09:47.960]  because it comes down to that 200 developers
[09:47.960 --> 09:50.160]  being supported by this one AppSec engineer.
[09:50.240 --> 09:52.160]  So, why not use these tools a little bit?
[09:52.160 --> 09:53.660]  Why not hack the game a little?
[09:55.120 --> 09:58.860]  So, now that we spoke about what about this approach,
[09:58.860 --> 10:00.100]  why this approach?
[10:00.880 --> 10:02.840]  Because here are certain reasons I think
[10:02.840 --> 10:06.480]  we should think this approach should be fit.
[10:06.700 --> 10:08.020]  One is efficiency.
[10:08.020 --> 10:09.600]  Since these tools use machine learning,
[10:09.600 --> 10:13.440]  they have the ability to generate speedy outcomes.
[10:14.380 --> 10:16.000]  It is instructional.
[10:16.380 --> 10:17.880]  Since it has the ability,
[10:17.880 --> 10:19.660]  ChatGPT has the ability to have, like,
[10:19.740 --> 10:21.320]  a conversation with you,
[10:21.320 --> 10:22.720]  you can ask it questions
[10:22.720 --> 10:24.940]  and you can learn new things from it.
[10:26.780 --> 10:27.660]  Scalability.
[10:27.760 --> 10:29.340]  They can handle complex tasks
[10:29.340 --> 10:30.540]  and large volumes of data
[10:30.540 --> 10:32.520]  with little human attention.
[10:33.960 --> 10:34.520]  Automation.
[10:34.520 --> 10:36.460]  The fact that I can seamlessly integrate Copilot
[10:36.460 --> 10:38.380]  within my IDE.
[10:38.980 --> 10:42.180]  I feel that that brings some amount of convenience.
[10:42.620 --> 10:43.540]  And ease of use.
[10:43.540 --> 10:44.880]  The fact that I can just open up a browser
[10:44.880 --> 10:46.500]  and go, like, chat, open AI,
[10:46.500 --> 10:49.760]  and then just ask a bunch of questions to the tool.
[10:49.760 --> 10:51.780]  The fact that it is easy to use
[10:51.780 --> 10:52.780]  and user-friendly,
[10:52.780 --> 10:54.300]  that is what appeals to me.
[10:54.300 --> 10:55.660]  So, these are some reasons
[10:55.660 --> 10:58.020]  why this approach might be deemed fit.
[10:58.820 --> 11:02.280]  So, now that we've done the what and the why,
[11:02.280 --> 11:04.260]  we should do the how, right?
[11:05.540 --> 11:07.880]  Let's dive into some fun AppSec things
[11:07.880 --> 11:08.980]  that we got to do.
[11:08.980 --> 11:11.100]  This is what y'all are here for, I guess, right?
[11:12.480 --> 11:15.300]  So, for the sake of time and convenience,
[11:15.300 --> 11:18.100]  I'm going to try to keep it to simple examples.
[11:18.820 --> 11:21.140]  But if you're open to, like,
[11:21.140 --> 11:22.940]  if you want to know more about other research
[11:22.940 --> 11:23.880]  that I've done on this,
[11:23.880 --> 11:25.520]  feel free to reach out to me.
[11:28.080 --> 11:28.980]  Okay.
[11:28.980 --> 11:31.480]  First application is vulnerability identification
[11:31.480 --> 11:33.740]  and remediation with ChatGPD.
[11:33.980 --> 11:36.620]  I have broken this down into a number of steps
[11:36.620 --> 11:38.700]  and how you can do it.
[11:39.920 --> 11:42.300]  One, you input your code, obvious.
[11:42.300 --> 11:44.220]  Whatever script that you want to check
[11:44.220 --> 11:45.780]  if it is vulnerable or not,
[11:45.780 --> 11:47.340]  you input your code.
[11:47.340 --> 11:49.240]  Then you engineer a prompt.
[11:49.240 --> 11:50.840]  When you're engineering a prompt,
[11:50.840 --> 11:51.560]  you ask for...
[11:52.040 --> 11:52.800]  You're straightforward.
[11:52.800 --> 11:53.860]  It's a straightforward process.
[11:53.860 --> 11:55.640]  You ask for vulnerabilities and remediations
[11:55.640 --> 11:59.660]  for whatever this vulnerability might be.
[11:59.680 --> 12:02.020]  You assess the response.
[12:02.020 --> 12:04.340]  Do not just copy-paste anything
[12:04.340 --> 12:07.060]  that ChatGPD gives you, please.
[12:07.060 --> 12:09.380]  For the love of God, do not.
[12:09.580 --> 12:11.060]  Assess response.
[12:11.300 --> 12:13.580]  Validate the recommended fixes
[12:13.580 --> 12:15.680]  that they have given you.
[12:16.400 --> 12:18.380]  Test and review.
[12:18.380 --> 12:19.440]  Do not.
[12:19.440 --> 12:21.540]  Please do not just copy-paste
[12:21.540 --> 12:23.940]  and push it to prod, please.
[12:25.260 --> 12:26.380]  Okay.
[12:26.720 --> 12:30.200]  So, here is a prompt for you.
[12:30.200 --> 12:33.160]  And whoever gets the vulnerability right,
[12:33.160 --> 12:34.780]  I have some stickers to share.
[12:35.700 --> 12:36.820]  Okay.
[12:37.560 --> 12:38.120]  So...
[12:38.120 --> 12:39.940]  And please raise your hands.
[12:40.340 --> 12:42.120]  So what is the vulnerable code here?
[12:42.120 --> 12:44.400]  And the prompt that I've given ChatGPD is that,
[12:44.400 --> 12:46.580]  okay, is this code vulnerable to what?
[12:46.580 --> 12:49.080]  And please suggest what the remediation for this is.
[12:49.080 --> 12:50.200]  Guesses, guesses.
[12:50.860 --> 12:51.900]  Buffer overflow.
[12:51.900 --> 12:53.400]  Thank you, thank you.
[12:53.860 --> 12:55.260]  Please meet me after,
[12:55.260 --> 12:57.060]  so I can give you the sticker.
[12:58.020 --> 12:58.640]  Okay.
[12:59.100 --> 13:02.280]  So, ChatGPD identifies two issues with it.
[13:02.280 --> 13:05.680]  One, that there is an incorrect assignment here.
[13:05.760 --> 13:06.980]  That it should have been...
[13:06.980 --> 13:08.660]  That string two should have been assigned
[13:08.660 --> 13:10.440]  as an array or a pointer.
[13:10.660 --> 13:12.680]  Second, it does understand
[13:12.680 --> 13:14.740]  that there's a vulnerable function in there,
[13:14.740 --> 13:16.700]  that is string copy, right?
[13:16.760 --> 13:19.820]  The secure form of doing the same thing
[13:19.820 --> 13:22.440]  is to use something like string copy,
[13:22.440 --> 13:25.480]  which is the remediation it suggests next.
[13:25.480 --> 13:27.760]  So it does fix the code in a certain way.
[13:27.760 --> 13:28.700]  First, it says,
[13:28.700 --> 13:31.740]  it adjusts the size of your source buffer
[13:31.740 --> 13:34.160]  to destination buffer and says 15,
[13:34.160 --> 13:37.340]  because you see string two has ABCD or N, right?
[13:37.340 --> 13:38.620]  Those are 15 characters.
[13:38.620 --> 13:41.340]  So it adjusts the size of string one.
[13:41.340 --> 13:43.740]  Then it does the correct character assignment,
[13:43.740 --> 13:44.900]  variable assignment.
[13:44.900 --> 13:47.280]  Then it uses the correct, more secure form
[13:47.280 --> 13:48.880]  of the string copy.
[13:48.880 --> 13:50.580]  It uses string copy,
[13:50.580 --> 13:52.360]  which checks on the size of the buffer.
[13:52.360 --> 13:54.920]  And then it also ensures null termination.
[13:55.400 --> 13:58.000]  So now, clear how this thing works, right?
[13:58.000 --> 13:59.980]  This straightforward process.
[14:00.520 --> 14:02.660]  Now let's see what goes wrong.
[14:04.780 --> 14:06.580]  Who can guess the code?
[14:07.020 --> 14:08.080]  What's vulnerable?
[14:08.260 --> 14:09.020]  Yes.
[14:11.580 --> 14:13.720]  SQL injection, thank you.
[14:14.060 --> 14:15.920]  So this is SQL injection.
[14:16.040 --> 14:18.260]  Now I'm giving it a certain background, okay?
[14:18.260 --> 14:20.560]  I'm like, hey, we have three metrics,
[14:20.560 --> 14:21.620]  which we are going to use
[14:21.620 --> 14:23.860]  to calculate the severity risk score.
[14:23.860 --> 14:26.560]  One, how easy it is to exploit this vulnerability.
[14:26.820 --> 14:28.720]  Two, where does this sit?
[14:28.720 --> 14:30.700]  How hostile is my threat environment?
[14:30.900 --> 14:33.740]  Three, what is the impact that is going to happen
[14:33.740 --> 14:35.480]  if I exploit this vulnerability?
[14:35.560 --> 14:38.000]  And we're going to give it score between zero to 10.
[14:38.100 --> 14:39.880]  And we calculate the severity risk
[14:39.880 --> 14:42.780]  by adding all those three and divide them by three.
[14:42.780 --> 14:45.340]  Then I give it additional context.
[14:45.340 --> 14:48.660]  That, hey, this is used to fetch username information
[14:48.660 --> 14:50.960]  on a public-facing website.
[14:51.120 --> 14:53.320]  And is this program vulnerable?
[14:53.320 --> 14:54.140]  To what?
[14:54.140 --> 14:56.620]  Help me calculate the severity risk for this.
[14:57.620 --> 14:58.560]  Beautiful.
[14:58.560 --> 15:00.980]  It identifies it's SQL injection.
[15:01.000 --> 15:02.440]  So kudos on that.
[15:02.620 --> 15:04.840]  But this is where my problem lies.
[15:04.960 --> 15:08.820]  It says ease of exploit development is 10
[15:08.820 --> 15:13.140]  because, hey, there's so many resources online to find this,
[15:13.140 --> 15:14.280]  how to exploit this.
[15:14.280 --> 15:16.080]  Hostility, it's a public-facing website,
[15:16.080 --> 15:17.280]  let's give it a score of 10.
[15:17.280 --> 15:19.560]  Impact, you can modify the database,
[15:19.560 --> 15:22.340]  you can get whatever sorts of secrets,
[15:22.340 --> 15:25.280]  I mean, user data and all of that, so 10.
[15:25.420 --> 15:31.340]  But what I see is it depends on the context of your application.
[15:31.340 --> 15:34.140]  What if there are certain guardrails on your public-facing website?
[15:34.400 --> 15:37.640]  What if the database isn't actually bringing out
[15:37.640 --> 15:40.060]  that kind of sensitive username information?
[15:40.060 --> 15:44.080]  These scores vary depending on the context of your application.
[15:44.080 --> 15:45.680]  I would have, depending on the guardrails
[15:45.680 --> 15:47.540]  I've given my public-facing website,
[15:47.540 --> 15:50.660]  I would have given hostility score 8 or 9.
[15:51.480 --> 15:55.700]  So that it does, one thing I want to point out here
[15:55.700 --> 15:59.340]  is that the more clear you are with your questions,
[15:59.340 --> 16:02.740]  the more context you give chat GPT,
[16:02.740 --> 16:05.660]  the better is the quality of the output
[16:05.660 --> 16:07.400]  or the response it generates.
[16:08.080 --> 16:11.600]  But again, do not blindly trust it.
[16:11.600 --> 16:12.700]  Trust but verify.
[16:13.880 --> 16:15.860]  The second application I want to talk about
[16:15.860 --> 16:18.300]  is secure coding suggestions with Copilot.
[16:19.360 --> 16:22.520]  Why I like this is because I'm writing a function
[16:23.000 --> 16:25.640]  and I can be like, help me give it a suggestion,
[16:25.640 --> 16:27.940]  like, help me write this in a secure fashion.
[16:27.940 --> 16:29.440]  And then it will give me all these different
[16:29.440 --> 16:31.500]  secure coding suggestions to do this.
[16:31.760 --> 16:33.860]  Again, breaking it down into steps.
[16:33.860 --> 16:36.560]  First, obviously, write your code.
[16:37.080 --> 16:39.280]  Then, while you're writing your code,
[16:39.280 --> 16:41.400]  Copilot is going to give you real-time coding suggestions
[16:41.400 --> 16:43.840]  about what you should be typing next.
[16:44.420 --> 16:48.600]  You assess the code that is generated by Copilot.
[16:48.700 --> 16:50.880]  Again, do not blindly believe.
[16:51.860 --> 16:53.840]  If you want to change your prompt,
[16:53.840 --> 16:55.380]  you can refine your prompt.
[16:55.960 --> 16:58.940]  And then, lastly, review and test.
[16:58.980 --> 17:01.020]  Why I say refine the prompt is because
[17:01.020 --> 17:03.200]  sometimes, again, looping back,
[17:03.200 --> 17:05.500]  the more context you give, the better output
[17:05.500 --> 17:06.740]  it's going to generate.
[17:07.040 --> 17:08.600]  Hence, refine your prompts.
[17:09.360 --> 17:12.740]  So, I have a vulnerable snippet.
[17:13.120 --> 17:15.060]  We just saw this. Any guesses?
[17:17.780 --> 17:19.940]  Exactly. Yes, it is deceiving.
[17:19.940 --> 17:21.880]  I know I tried to deceive, but you all are smart.
[17:23.200 --> 17:26.880]  Okay, so the suggestion that I'm giving Copilot is,
[17:26.880 --> 17:29.300]  hey, identify the vulnerability in this code snippet
[17:29.300 --> 17:30.920]  and fix it for me.
[17:31.640 --> 17:34.180]  Now, it does a nice job.
[17:34.180 --> 17:36.920]  This is the whole, the first box is the whole,
[17:37.720 --> 17:39.500]  it's all suggested by Copilot.
[17:39.500 --> 17:41.360]  This is not something I've typed by myself.
[17:41.360 --> 17:44.360]  This whole slide is everything that Copilot has suggested.
[17:44.480 --> 17:46.720]  It identifies buffer overflow.
[17:46.760 --> 17:49.720]  It identifies the vulnerable function that is string copy.
[17:49.720 --> 17:52.400]  And it tells you replace it by the safer version of it.
[17:52.400 --> 17:55.540]  And it tells you why you should use string copy.
[17:55.540 --> 17:56.660]  Good, good enough context.
[17:56.660 --> 17:59.020]  There's some nuggets that I can take from this.
[17:59.020 --> 18:02.240]  Now, it does correct the code as well,
[18:02.240 --> 18:06.220]  but it would have been more secure
[18:06.220 --> 18:08.660]  if it would have adjusted the size of string one
[18:08.660 --> 18:11.120]  to accommodate string two.
[18:11.380 --> 18:11.900]  Right?
[18:11.900 --> 18:14.000]  So, yeah.
[18:14.000 --> 18:16.340]  So now we understand how this whole thing works,
[18:16.340 --> 18:17.600]  the whole gamut.
[18:17.920 --> 18:19.700]  Next prompt for you.
[18:20.180 --> 18:21.540]  What is this?
[18:21.540 --> 18:22.520]  Guesses.
[18:24.480 --> 18:25.560]  Guesses.
[18:25.560 --> 18:29.100]  You can go.
[18:29.100 --> 18:30.780]  You've gone once already.
[18:31.440 --> 18:32.100]  Yes?
[18:33.340 --> 18:35.280]  Part traversal, directory traversal, yes.
[18:35.280 --> 18:36.260]  Okay, perfect.
[18:36.500 --> 18:39.160]  So, again, you give it the suggestion.
[18:39.500 --> 18:41.480]  Identify this vulnerability for me.
[18:41.480 --> 18:43.340]  Rewrite the code to eliminate it.
[18:44.620 --> 18:45.980]  Now, beautiful.
[18:45.980 --> 18:48.140]  It identifies part traversal.
[18:48.140 --> 18:50.420]  But this is where the problem lies.
[18:50.420 --> 18:53.040]  It's giving you like eight different suggestions.
[18:53.180 --> 18:55.060]  It's like, use OS part join.
[18:55.060 --> 18:58.280]  Use OS part apps part, common prefix, exists,
[18:58.280 --> 19:00.500]  is file, rel part, norm part.
[19:00.500 --> 19:03.540]  And then it goes on to generate this 25-line code
[19:03.540 --> 19:06.120]  about how this can be fixed.
[19:06.440 --> 19:10.960]  But if you add the human touch to it,
[19:10.960 --> 19:12.920]  the domain-specific knowledge to it,
[19:13.680 --> 19:15.720]  you would rather realize that you should use
[19:15.720 --> 19:17.300]  something like OS part base name
[19:17.300 --> 19:20.760]  because it would directly break your whole part
[19:20.760 --> 19:22.720]  into two different parts,
[19:22.720 --> 19:23.660]  the head and the tail,
[19:23.660 --> 19:25.540]  separating the whole directory component.
[19:25.540 --> 19:29.000]  And then you completely keep the last file name
[19:29.000 --> 19:29.960]  and focus on it.
[19:29.960 --> 19:32.720]  Albeit, this is a more secure fix,
[19:32.720 --> 19:36.480]  but you can still make it vulnerable.
[19:36.480 --> 19:38.500]  You can still give it some maliciously crafted regex
[19:38.500 --> 19:39.860]  and try to break this.
[19:42.640 --> 19:45.840]  The last one, the last application I want to talk about
[19:45.840 --> 19:50.520]  is security test case generation using Copilot.
[19:51.800 --> 19:53.860]  Again, we're going to break it down into...
[19:53.860 --> 19:54.980]  Wait, I'm curious.
[19:54.980 --> 19:57.320]  How many people here do security test cases?
[19:59.660 --> 20:02.900]  Three, four, five, six?
[20:02.980 --> 20:03.660]  Okay.
[20:04.100 --> 20:04.670]  Okay, that's...
[20:05.060 --> 20:06.080]  I hope this...
[20:06.080 --> 20:07.060]  We're going to work on this.
[20:07.060 --> 20:07.940]  We're going to increase this.
[20:07.940 --> 20:09.280]  Thank you to Copilot.
[20:09.280 --> 20:12.420]  Maybe we try to generate more test cases going forward.
[20:13.820 --> 20:15.020]  The breakdown.
[20:15.020 --> 20:17.200]  You identify the vulnerability.
[20:17.840 --> 20:20.560]  You obviously launch Copilot in your IDE.
[20:21.760 --> 20:22.640]  Oops, okay.
[20:22.640 --> 20:25.700]  Then you write a prompt to generate security test cases
[20:25.700 --> 20:26.620]  like we saw earlier.
[20:26.620 --> 20:27.620]  Hey, help me.
[20:27.620 --> 20:29.180]  Like we saw, identify the vulnerability.
[20:29.180 --> 20:30.180]  This time you're going to be like,
[20:30.180 --> 20:32.320]  hey, generate security test cases for me.
[20:32.460 --> 20:35.660]  And then Copilot will generate all those test cases for you.
[20:35.660 --> 20:37.600]  You execute those test cases.
[20:37.600 --> 20:40.880]  Again, review the output that it generates.
[20:42.020 --> 20:42.760]  Oh.
[20:43.660 --> 20:44.560]  Did you get that?
[20:47.310 --> 20:48.190]  Okay.
[20:48.810 --> 20:50.710]  Snippet one.
[20:50.890 --> 20:52.110]  What is this?
[20:52.110 --> 20:53.830]  What is it vulnerable to?
[20:54.350 --> 20:56.250]  Guesses, guesses.
[20:57.370 --> 20:59.110]  Not buffer overflow.
[21:01.290 --> 21:02.610]  Integer overflow.
[21:02.610 --> 21:03.250]  Perfect.
[21:03.490 --> 21:06.170]  So this is a Python program.
[21:06.170 --> 21:10.150]  Python is used to handling really large numbers,
[21:10.150 --> 21:13.690]  but you can make it vulnerable to integer overflow
[21:13.690 --> 21:16.290]  by using C-compatible data types.
[21:16.510 --> 21:18.990]  So what is basically happening in this program
[21:18.990 --> 21:21.330]  is that if you give it big numbers,
[21:21.330 --> 21:24.370]  for example, A is using the maximum size
[21:24.370 --> 21:26.530]  of the 32-bit sign integer,
[21:26.530 --> 21:28.770]  so 2x minus 31 minus 1,
[21:28.770 --> 21:30.330]  and you give it B, 1,
[21:30.330 --> 21:34.910]  you would expect it to be some positive big number, right?
[21:35.050 --> 21:38.710]  But instead, since this program is vulnerable to integer overflow,
[21:38.710 --> 21:40.210]  it's going to give you a negative value.
[21:40.210 --> 21:41.990]  It's going to do a whole wraparound.
[21:41.990 --> 21:44.430]  So that is how this program is vulnerable.
[21:45.030 --> 21:48.210]  Now, let's go ahead to application.
[21:49.670 --> 21:51.190]  So this is what I'm saying.
[21:51.190 --> 21:54.390]  Hey, write unit tests for vulnerable addition function
[21:54.390 --> 21:55.750]  and vulnerable pi.
[21:57.130 --> 22:01.590]  Then it does a good job of suggesting two cases for me.
[22:01.590 --> 22:02.030]  Or three.
[22:02.030 --> 22:03.890]  It's breaking it down into two lines here.
[22:03.890 --> 22:06.230]  Test one, normal addition.
[22:06.370 --> 22:09.810]  Test two, it should raise an exception
[22:09.810 --> 22:14.670]  when there's an overflow and underflow.
[22:14.670 --> 22:17.810]  Then it proceeds to write,
[22:18.330 --> 22:21.250]  one, normal addition, you're adding 1 and 2,
[22:21.250 --> 22:22.930]  should return 3, straightforward.
[22:24.090 --> 22:26.890]  Two, it should raise an overflow error
[22:26.890 --> 22:29.550]  when you're adding a really big number,
[22:29.550 --> 22:31.230]  2x minus 31 with 1,
[22:31.230 --> 22:33.010]  should raise an overflow error.
[22:33.010 --> 22:34.450]  Three, same thing.
[22:34.450 --> 22:38.350]  You're adding the smallest value of the 32-bit sign to minus 1,
[22:38.350 --> 22:40.310]  it should raise an overflow error.
[22:40.310 --> 22:41.150]  Straightforward?
[22:41.150 --> 22:43.430]  Are we following any questions so far?
[22:43.590 --> 22:44.890]  Okay, perfect.
[22:47.070 --> 22:52.710]  And it does, thankfully, generate two failures.
[22:52.710 --> 22:54.370]  It says that vulnerable addition
[22:54.370 --> 22:58.030]  is not generating overflow error for both the cases,
[22:58.030 --> 23:00.230]  the maximum values and the minimum values.
[23:00.230 --> 23:02.130]  So it works fine, right?
[23:02.130 --> 23:04.410]  This is what we were expecting it to do.
[23:04.410 --> 23:09.910]  So, this is the bad example.
[23:09.930 --> 23:12.290]  Did we follow what I just spoke about?
[23:12.330 --> 23:13.310]  Okay, great.
[23:13.310 --> 23:15.870]  So now that we understand how it's generating unit test cases,
[23:15.870 --> 23:17.250]  this is the second example.
[23:17.470 --> 23:19.250]  What is the vulnerability?
[23:20.850 --> 23:22.270]  We saw this.
[23:24.940 --> 23:26.980]  SQL injection, right.
[23:27.620 --> 23:29.100]  SQL injection, right.
[23:29.100 --> 23:31.580]  So now, I go ahead and tell it,
[23:31.580 --> 23:34.440]  generate security-focused test cases for me,
[23:34.440 --> 23:36.280]  for the previous function,
[23:36.280 --> 23:38.580]  and then it does this thing.
[23:38.580 --> 23:40.480]  It does this really weird thing.
[23:41.120 --> 23:44.440]  One, it looks, hey, is this generating,
[23:45.520 --> 23:47.840]  is when I enter the correct username,
[23:47.840 --> 23:51.680]  is it rendering, when I enter the correct combination
[23:51.680 --> 23:54.460]  for the username and password, is it a success?
[23:54.480 --> 23:57.340]  And if I go for a non-existent user,
[23:57.340 --> 24:00.360]  is it going to raise an is-none, right?
[24:00.360 --> 24:05.060]  But it is completely ignoring the SQL injection.
[24:05.120 --> 24:10.180]  A good addition would have been this, right?
[24:10.180 --> 24:12.680]  Give it a maliciously crafted input, right?
[24:12.680 --> 24:15.240]  It completely ignores the fact
[24:15.240 --> 24:20.500]  that there is SQL injection in here, right?
[24:20.500 --> 24:23.320]  So these are the only two test cases it generates.
[24:23.320 --> 24:27.760]  Okay, so now we've seen security-focused test cases.
[24:28.260 --> 24:29.820]  Everybody follow so far?
[24:29.820 --> 24:31.060]  Okay, great.
[24:33.280 --> 24:35.160]  Another weird application.
[24:35.160 --> 24:37.060]  How many of you all use CodeQL?
[24:38.740 --> 24:40.500]  You find it annoying?
[24:41.420 --> 24:43.220]  Sort of, but does the job? Yeah?
[24:43.480 --> 24:44.700]  Yeah, okay.
[24:44.700 --> 24:47.140]  So for people who don't know what CodeQL is,
[24:47.860 --> 24:50.620]  it's an amazing tool that is generated by GitHub
[24:50.620 --> 24:53.940]  to find out if there are any vulnerabilities
[24:53.940 --> 24:56.180]  in the code that you've pushed, right?
[24:56.240 --> 24:58.440]  So think of it as a SAST.
[24:58.980 --> 25:00.760]  So now I ask...
[25:00.760 --> 25:05.340]  this is the correct query that I have generated.
[25:05.340 --> 25:07.440]  Say I want to find in my whole...
[25:08.120 --> 25:10.580]  in the entire repository,
[25:10.580 --> 25:12.840]  if I want to find all of the to-do comments,
[25:12.840 --> 25:14.700]  this is the query you give CodeQL.
[25:15.300 --> 25:16.700]  Okay? Looks beautiful.
[25:16.780 --> 25:19.180]  Straightforward, it's like, hey, from a comment,
[25:19.180 --> 25:21.300]  you get text, find this regex,
[25:21.300 --> 25:23.500]  it should have to-do in it, right?
[25:23.500 --> 25:24.260]  Okay.
[25:24.800 --> 25:27.360]  When I ask the same question to ChatGPD,
[25:27.360 --> 25:29.420]  it gives me this...
[25:30.000 --> 25:30.960]  I don't know what this is.
[25:30.960 --> 25:32.260]  It gives me this whole thing.
[25:32.580 --> 25:34.540]  When I try to run it,
[25:34.540 --> 25:35.980]  the results are so bad,
[25:35.980 --> 25:37.220]  there's so many errors in it,
[25:37.220 --> 25:39.400]  it doesn't even fit in a screenshot.
[25:39.920 --> 25:41.180]  And this is what I felt.
[25:41.260 --> 25:42.100]  This is what I felt.
[25:42.100 --> 25:43.820]  I would have rather invested all my time
[25:43.820 --> 25:46.720]  understanding how CodeQL works and making the query,
[25:46.720 --> 25:49.520]  but instead, ChatGPD generated this query for me
[25:49.520 --> 25:51.240]  for five hours, and then I was trying to debug it
[25:51.240 --> 25:52.360]  and debug it and debug it,
[25:52.360 --> 25:53.720]  and I invested my whole day,
[25:53.720 --> 25:54.700]  and then I was irritated.
[25:54.700 --> 26:00.420]  So, it's not a good CodeQL query developer.
[26:01.380 --> 26:02.620]  Okay.
[26:02.920 --> 26:03.880]  Praise Jesus!
[26:03.880 --> 26:05.620]  Our jobs are safe, guys!
[26:08.480 --> 26:10.640]  Our jobs are completely safe.
[26:10.640 --> 26:13.040]  It hasn't taken over our job yet.
[26:13.740 --> 26:15.720]  But, why?
[26:15.720 --> 26:18.020]  So certain limitations that I found
[26:18.020 --> 26:21.280]  throughout this whole research of mine,
[26:21.280 --> 26:22.860]  I'm just gonna list them out.
[26:22.860 --> 26:27.200]  One, ChatGPD, the cutoff data for it is September 2021,
[26:27.200 --> 26:29.420]  so all of the vulnerabilities that came out after,
[26:29.420 --> 26:31.060]  it has no idea about those.
[26:31.060 --> 26:33.580]  If I even give it a requirements.txt
[26:33.580 --> 26:36.280]  with a bunch of vulnerable packages,
[26:36.280 --> 26:39.200]  it's not gonna identify which is vulnerable
[26:39.200 --> 26:42.180]  because it does not have most updated knowledge on it.
[26:42.420 --> 26:44.440]  Same thing with Copilot as well.
[26:44.940 --> 26:48.680]  And dependence on prompts, like we saw.
[26:48.680 --> 26:49.960]  The more context you give,
[26:49.960 --> 26:52.460]  the more breakdown in your prompts that you give,
[26:52.460 --> 26:57.020]  the less vagueness that your prompts have,
[26:57.020 --> 27:02.380]  the better the output of ChatGPD and Copilot will be.
[27:02.940 --> 27:05.620]  Intellectual property concerns.
[27:05.620 --> 27:06.280]  Okay.
[27:06.300 --> 27:10.040]  So, do not enter, for the love of God,
[27:10.040 --> 27:11.600]  do not put business critical information
[27:11.600 --> 27:13.420]  in any of these tools.
[27:13.420 --> 27:15.080]  Please, please do not.
[27:15.220 --> 27:18.980]  Do not even put your secret, your PIIs,
[27:18.980 --> 27:20.840]  like your SSNs or something like that.
[27:20.840 --> 27:22.960]  If you're trying to use,
[27:22.960 --> 27:24.360]  if you're trying to make,
[27:24.360 --> 27:25.520]  if you're trying to generate a response
[27:25.520 --> 27:26.700]  that contains IP addresses,
[27:26.700 --> 27:27.820]  give it some fake IP address
[27:27.820 --> 27:29.780]  or give it some fake website name,
[27:29.780 --> 27:32.860]  like this is www.example.com or something like that.
[27:33.660 --> 27:36.800]  Basically, don't give anything that is intellectual property.
[27:37.700 --> 27:40.700]  Since these tools are in the nascent phases of development,
[27:40.700 --> 27:42.440]  they do have potential bias
[27:42.440 --> 27:45.140]  and they are still prone to false positives and negatives,
[27:45.140 --> 27:46.440]  like we saw throughout the presentation
[27:46.440 --> 27:48.220]  in different examples.
[27:48.220 --> 27:53.120]  And lastly, we saw in all of these examples,
[27:53.120 --> 27:56.500]  it is introducing unintended, insecure consequences.
[27:56.500 --> 27:59.920]  It is not developing more secure form of the code from get-go.
[28:00.660 --> 28:03.800]  So, these are certain limitations that we have.
[28:04.840 --> 28:07.320]  So, if there's one thing you can take home today
[28:07.320 --> 28:08.480]  from your flight back,
[28:08.480 --> 28:09.520]  from this whole talk,
[28:09.520 --> 28:13.080]  would be please do not fall for the false sense of security
[28:13.080 --> 28:15.500]  that these tools are providing, please.
[28:17.300 --> 28:18.940]  And here's a fun meme for you.
[28:18.940 --> 28:20.060]  Thanks, Drake.
[28:20.300 --> 28:22.500]  Please do not blindly trust AI.
[28:22.540 --> 28:26.180]  Assess and review the responses that are generated by AI.
[28:28.620 --> 28:32.220]  And here is a chat GPT prompt engineering hack
[28:32.220 --> 28:34.260]  that I stumbled upon during my research.
[28:34.260 --> 28:38.500]  If you want to create more clearer prompts, use this.
[28:38.780 --> 28:40.740]  It's basically an iterative process
[28:40.740 --> 28:41.980]  where you tell chat GPT,
[28:41.980 --> 28:43.240]  hey, this is my question.
[28:43.240 --> 28:44.780]  It comes up with its own suggestions.
[28:44.780 --> 28:47.160]  It comes up with a revised prompt,
[28:47.160 --> 28:48.840]  more additional questions you should be asking
[28:48.840 --> 28:50.580]  to generate a better prompt.
[28:50.940 --> 28:53.100]  And, yeah, this whole iteration process continues
[28:53.100 --> 28:54.080]  until you let it know that,
[28:54.080 --> 28:55.460]  hey, I'm happy with my prompt.
[28:58.460 --> 28:58.860]  And...
[29:02.600 --> 29:03.600]  I'll wait for you.
[29:13.070 --> 29:16.650]  And here's a slide of helpful links.
[29:16.650 --> 29:18.830]  Thanks to Gavin Clonlight from AI Village.
[29:19.550 --> 29:21.570]  He has done some really amazing research.
[29:21.570 --> 29:23.290]  Feel free to check him out on Twitter.
[29:23.290 --> 29:25.430]  Some really helpful links
[29:25.430 --> 29:27.710]  if you want to learn prompt engineering,
[29:27.710 --> 29:30.030]  different stances that OpenAI has,
[29:30.030 --> 29:31.910]  things like that.
[29:36.800 --> 29:37.560]  And...
[29:38.560 --> 29:39.500]  Okay.
[29:39.500 --> 29:41.320]  And that's it. Thank you.