WEBVTT 00:07.930 --> 00:14.570 So today we're going to talk about the duality of Chad GPT and copilot in secure development. 00:14.790 --> 00:17.230 And so my name is Kalyani Pawar. 00:17.230 --> 00:19.090 Let's talk a little bit about me. 00:19.970 --> 00:22.430 So hi, I am Kalyani Pawar. 00:22.430 --> 00:24.050 I also go by Kaylee. 00:24.130 --> 00:30.350 I currently run the application security and vendor security side of things at a drone company called Zipline. 00:30.390 --> 00:33.850 I've been doing security for nearly half a decade now. 00:35.570 --> 00:38.850 I enjoy doing what I do in my free time. 00:39.050 --> 00:41.850 All the opinions throughout all these slides are mine. 00:41.850 --> 00:44.370 I am not representing my employer in any way. 00:45.190 --> 00:47.510 This is my Twitter or X handle. 00:47.510 --> 00:54.950 I don't post a lot, but I promise everything that I learn, all the prompts that we're talking about today, I'm going to talk about them on the Twitter soon. 00:56.250 --> 00:58.690 Things that I do outside of my work. 00:58.690 --> 01:05.330 I do review talks that get submitted to Grace Hoppers and AppSec Village every year at DEF CON. 01:05.330 --> 01:07.590 So that's about me. 01:08.010 --> 01:12.950 Now, what are we going to talk about today? 01:12.990 --> 01:19.130 Obviously, Chad GPT and Copilot, but some fun, cool AppSec-y things that you can do with them. 01:19.330 --> 01:29.550 And while we are exploring the fun side of things, let us proceed with caution and not fall directly for the responses that it is generating for us. 01:29.610 --> 01:32.230 All right, now that we know what we're going to talk about... 01:32.990 --> 01:34.890 Okay, quick question. 01:35.210 --> 01:40.250 Who in the audience today has worked with Chad GPT or Chad GPT-like models? 01:40.810 --> 01:41.530 Fancy. 01:41.530 --> 01:43.130 That is what I expect. 01:43.470 --> 01:48.450 So here is a quick primer for people who are not familiar with Chad GPT. 01:48.830 --> 01:52.550 It's an AI language model developed by OpenAI. 01:52.550 --> 01:56.170 It is based on generative pre-trained transformer architecture. 01:56.170 --> 01:57.250 It is out of scope. 01:57.250 --> 01:58.930 The whole architecture is out of scope for this talk. 01:58.930 --> 02:00.150 You don't need to know this information. 02:00.150 --> 02:03.690 All you've got to know is how to type in text on a browser. 02:05.250 --> 02:14.430 So what I like about Chad GPT is it is trained on a range of diverse data that you find on the Internet, on various books and sources of information that are available online. 02:14.470 --> 02:20.430 But what I find beautiful about this tool is that it has the ability to generate human-like text. 02:20.430 --> 02:23.150 Once you input whatever question you have for it. 02:23.170 --> 02:29.630 The fact that you can speak to a machine which can understand the context and nuances of whatever you talk to it. 02:30.790 --> 02:34.150 And it's like talking to a human interlocutor at the end of the day. 02:34.150 --> 02:35.770 So that is what I like about it. 02:35.770 --> 02:39.070 And you can just imagine the kind of functionality that this tool provides you. 02:39.070 --> 02:40.990 You can ask it all sorts of questions. 02:41.170 --> 02:45.150 You can ask it to translate into different languages you don't know about. 02:45.150 --> 02:51.570 You can use it to learn different things that you probably would be investing an hour reading a book. 02:51.570 --> 02:53.570 Rather ask Chad GPT about it. 02:54.090 --> 02:58.270 And, yes, when I don't know the answer to a question, that's what I do. 02:58.270 --> 02:59.770 Ask Chad GPT, guys. 02:59.770 --> 03:00.930 I don't know the answer to this. 03:00.930 --> 03:02.570 Use Chad GPT for this. 03:05.290 --> 03:08.050 How many of y'all have used Copilot? 03:09.030 --> 03:10.330 Y'all like it? 03:11.290 --> 03:12.070 Sort of? 03:12.070 --> 03:13.170 Yeah, same, same. 03:13.170 --> 03:13.990 Sort of. 03:13.990 --> 03:21.490 For people who don't know what Copilot is, it is an AI pair programmer developed by GitHub in collaboration with OpenAI. 03:22.550 --> 03:37.050 It is trained on a diverse range of public code repositories that are available, so it does understand the coding styles of millions of developers all over the world, and what their documentation styles would be. 03:37.050 --> 03:42.490 How many of y'all have struggled with code that has the worst comments in it? 03:43.130 --> 03:44.010 Most of us? 03:44.010 --> 03:44.630 Most of us? 03:44.630 --> 03:45.330 We don't, like... 03:45.330 --> 03:47.090 What the f... What is this? 03:47.610 --> 03:48.950 What is this? 03:49.150 --> 03:49.750 Right? 03:49.750 --> 04:00.830 So, what I like about Chad GPT is that, while you are coding, it does also generate comments so that someone else is going to read your code, does understand what have you done, the beauty that you've created. 04:01.790 --> 04:02.590 Awesome. 04:04.370 --> 04:04.770 So... 04:04.770 --> 04:10.110 Oh, yeah, this is something else I like about it, that it can seamlessly integrate within VS Code, which is my choice of IDE. 04:10.110 --> 04:14.150 I don't have to, like, open up a different browser or an instance, like, launch this tool. 04:14.150 --> 04:21.890 As I'm typing, it's going to generate real-time suggestions about, hey, this is probably what you should be writing, instead of whatever you're thinking of writing. 04:21.970 --> 04:27.150 For example, if I just ask Chad GPT Copilot that, can you generate a program that's going to create... 04:27.150 --> 04:29.190 that's going to generate a random number for you? 04:29.190 --> 04:30.370 It would just... 04:30.370 --> 04:30.710 Bam! 04:30.710 --> 04:33.770 You just press tap, and it writes the whole program for you. 04:35.210 --> 04:35.570 So... 04:35.570 --> 04:43.390 Now that we know what Copilot is, I want to talk about the problems that the unsung heroes of our community face. 04:43.390 --> 04:45.470 Application security engineers. 04:45.590 --> 04:46.970 What is the problem? 04:47.050 --> 04:50.810 The problem is that there's a bunch of problems that we're facing. 04:51.230 --> 04:53.310 And I'm going to run through them real quick. 04:54.430 --> 05:04.030 How many of us have been part of these meetings where they're like, hey, I found something vulnerable in my code. 05:04.030 --> 05:07.270 And you're like, okay, how long ago did you push this code into Prod? 05:07.270 --> 05:09.330 And they're like, yeah, many sprints ago. 05:09.330 --> 05:11.890 And you're like, are you telling me this right now? 05:11.890 --> 05:13.310 Like, this is what you did? 05:13.310 --> 05:31.410 So I understand with advent of DevOps and Agile, we are shipping and deploying code at 100 miles an hour, which is amazing, but also it's leaving potential security gaps because we are falling short on providing coverage and all the code that has been deployed. 05:32.530 --> 05:36.130 Second, lack of security education. 05:36.130 --> 05:42.230 According to dark reading, about half of the developers, 48% of the developers lack security education. 05:42.230 --> 05:47.950 They simply develop code keeping functionality in mind and completely ignore the security aspect of it. 05:47.950 --> 05:51.630 So you're pushing insecure code full of vulnerabilities from the get-go. 05:51.630 --> 05:55.310 That is another pressing issue that we're facing. 05:56.820 --> 05:59.070 The resource constraints. 05:59.070 --> 06:02.670 Were you at Tanya Janka's talk earlier this week? 06:02.670 --> 06:03.570 Any hands? 06:03.570 --> 06:04.250 Okay. 06:04.250 --> 06:06.670 So she was talking about... I'm sure... 06:06.670 --> 06:07.970 Okay, raise your hands, okay? 06:07.970 --> 06:15.410 How many times have you been in an organization where there's like 200 developers and only one AppSec engineer supporting them? 06:16.790 --> 06:18.690 Unsung heroes, I tell you. 06:23.370 --> 06:32.190 Security teams are usually smaller in size and it's getting difficult to literally review every line of code that is generated by your organization. 06:32.190 --> 06:34.110 So what do you do to combat this? 06:34.670 --> 06:37.250 You onboard different tools for it. 06:37.290 --> 06:44.070 Now, I am guilty of onboarding two, three different tools and just integrating it within my CICD and all of that. 06:44.070 --> 06:48.950 But the problem with this is that it is generating a bunch of false positives. 06:48.950 --> 06:55.870 And now we're all sitting and investing our time and sifting through what the true positives are, what the false positives are, and it just does not end there. 06:55.870 --> 07:02.410 It's about, okay, now we got to prioritize what risk this vulnerability holds and how are we going to clear this backlog? 07:02.410 --> 07:07.050 And there's just a lot of time that we spend in strategizing how to deal with this monster. 07:08.670 --> 07:11.150 And this is another problem. 07:11.350 --> 07:14.970 Ever-emerging threats, ever-emerging attack vectors. 07:14.970 --> 07:19.230 Every day there's something new that I read online about some new attack vector. 07:19.230 --> 07:27.810 And it's just difficult for this one brain of mine to constantly catch up with everything that's going on outside on the threat landscape. 07:27.810 --> 07:34.250 Last but not the least, the hammer that is the legal regulation stick. 07:34.850 --> 07:48.070 Depending on the kind of product you're making or the kind of jurisdiction you're part of or the business that you are in, you've got to make sure that your product is compliant with certain industry regulated standards like SOC 2 or GDPR. 07:48.070 --> 07:50.050 How many of us take that seriously? 07:51.150 --> 07:53.090 We are trying, right? 07:53.090 --> 07:53.810 We're trying. 07:53.810 --> 07:57.570 It's just like an additional audit task somewhere in the sprint. 07:58.830 --> 07:59.330 Awesome. 07:59.810 --> 08:10.290 So, now that we know the bunch of problems that we are facing, a possible solution, a possible solution would be to leverage AI for AppSec. 08:10.290 --> 08:16.950 When I say AI, I'm trying to loosely refer to Chad, GPT, and Copilot for the rest of this whole conversation. 08:18.510 --> 08:21.170 How can we leverage AI for AppSec? 08:21.410 --> 08:24.370 Anyone in the crowd? 08:26.530 --> 08:27.410 Yes. 08:30.030 --> 08:31.050 Training, yes. 08:31.830 --> 08:33.090 Anyone else? 08:35.210 --> 08:35.810 Yes. 08:35.810 --> 08:36.190 Perfect. 08:36.190 --> 08:36.730 Bang on. 08:36.730 --> 08:37.710 Anyone else? 08:41.160 --> 08:41.900 Exactly. 08:41.900 --> 08:42.360 Yes. 08:42.360 --> 08:44.160 So, a couple applications here. 08:44.160 --> 08:47.500 You can ask Chad, GPT, like, hey, this is my code. 08:47.500 --> 08:50.620 Tell me what's vulnerable in it and how can I fix this. 08:50.700 --> 08:54.960 You can also go like, hey, Copilot, I'm generating this code right now. 08:55.260 --> 08:59.780 Can you give me secure coding suggestions about how this whole code is supposed to go? 09:00.920 --> 09:07.140 You can also be like, oh, also for, like, a developer especially, they can go like, hey, what is Heartbleed vulnerability? 09:07.140 --> 09:08.620 Explain to me in simple terms. 09:08.620 --> 09:15.880 They ask this question to Chad, GPT, and they don't have to go through, like, a bunch of web pages to understand what Heartbleed is, right? 09:15.880 --> 09:18.820 So, there's certain applications to it. 09:18.820 --> 09:24.920 Now, what I see on the bright side for this is that how many times do you see yourself context switching? 09:24.920 --> 09:31.700 Like, one moment you're doing API security, one moment you're threat modeling, next moment you're source code reviewing. 09:31.700 --> 09:33.520 How many times does that happen to you in a day? 09:34.320 --> 09:35.240 All the time. 09:35.240 --> 09:36.240 All the time, right? 09:36.240 --> 09:39.800 This context switching is leading to burnout on the team. 09:39.800 --> 09:50.160 I feel like a hamster on a wheel all the time because I'm constantly switching into, like, different projects all the time because it comes down to that 200 developers being supported by this one AppSec engineer. 09:50.240 --> 09:52.160 So, why not use these tools a little bit? 09:52.160 --> 09:53.660 Why not hack the game a little? 09:55.120 --> 10:00.100 So, now that we spoke about what about this approach, why this approach? 10:00.880 --> 10:06.480 Because here are certain reasons I think we should think this approach should be fit. 10:06.700 --> 10:08.020 One is efficiency. 10:08.020 --> 10:13.440 Since these tools use machine learning, they have the ability to generate speedy outcomes. 10:14.380 --> 10:16.000 It is instructional. 10:16.380 --> 10:24.940 Since it has the ability, ChatGPT has the ability to have, like, a conversation with you, you can ask it questions and you can learn new things from it. 10:26.780 --> 10:27.660 Scalability. 10:27.760 --> 10:32.520 They can handle complex tasks and large volumes of data with little human attention. 10:33.960 --> 10:34.520 Automation. 10:34.520 --> 10:38.380 The fact that I can seamlessly integrate Copilot within my IDE. 10:38.980 --> 10:42.180 I feel that that brings some amount of convenience. 10:42.620 --> 10:43.540 And ease of use. 10:43.540 --> 10:49.760 The fact that I can just open up a browser and go, like, chat, open AI, and then just ask a bunch of questions to the tool. 10:49.760 --> 10:54.300 The fact that it is easy to use and user-friendly, that is what appeals to me. 10:54.300 --> 10:58.020 So, these are some reasons why this approach might be deemed fit. 10:58.820 --> 11:04.260 So, now that we've done the what and the why, we should do the how, right? 11:05.540 --> 11:08.980 Let's dive into some fun AppSec things that we got to do. 11:08.980 --> 11:11.100 This is what y'all are here for, I guess, right? 11:12.480 --> 11:18.100 So, for the sake of time and convenience, I'm going to try to keep it to simple examples. 11:18.820 --> 11:25.520 But if you're open to, like, if you want to know more about other research that I've done on this, feel free to reach out to me. 11:28.080 --> 11:28.980 Okay. 11:28.980 --> 11:33.740 First application is vulnerability identification and remediation with ChatGPD. 11:33.980 --> 11:38.700 I have broken this down into a number of steps and how you can do it. 11:39.920 --> 11:42.300 One, you input your code, obvious. 11:42.300 --> 11:47.340 Whatever script that you want to check if it is vulnerable or not, you input your code. 11:47.340 --> 11:49.240 Then you engineer a prompt. 11:49.240 --> 11:51.560 When you're engineering a prompt, you ask for... 11:52.040 --> 11:52.800 You're straightforward. 11:52.800 --> 11:53.860 It's a straightforward process. 11:53.860 --> 11:59.660 You ask for vulnerabilities and remediations for whatever this vulnerability might be. 11:59.680 --> 12:02.020 You assess the response. 12:02.020 --> 12:07.060 Do not just copy-paste anything that ChatGPD gives you, please. 12:07.060 --> 12:09.380 For the love of God, do not. 12:09.580 --> 12:11.060 Assess response. 12:11.300 --> 12:15.680 Validate the recommended fixes that they have given you. 12:16.400 --> 12:18.380 Test and review. 12:18.380 --> 12:19.440 Do not. 12:19.440 --> 12:23.940 Please do not just copy-paste and push it to prod, please. 12:25.260 --> 12:26.380 Okay. 12:26.720 --> 12:30.200 So, here is a prompt for you. 12:30.200 --> 12:34.780 And whoever gets the vulnerability right, I have some stickers to share. 12:35.700 --> 12:36.820 Okay. 12:37.560 --> 12:38.120 So... 12:38.120 --> 12:39.940 And please raise your hands. 12:40.340 --> 12:42.120 So what is the vulnerable code here? 12:42.120 --> 12:46.580 And the prompt that I've given ChatGPD is that, okay, is this code vulnerable to what? 12:46.580 --> 12:49.080 And please suggest what the remediation for this is. 12:49.080 --> 12:50.200 Guesses, guesses. 12:50.860 --> 12:51.900 Buffer overflow. 12:51.900 --> 12:53.400 Thank you, thank you. 12:53.860 --> 12:57.060 Please meet me after, so I can give you the sticker. 12:58.020 --> 12:58.640 Okay. 12:59.100 --> 13:02.280 So, ChatGPD identifies two issues with it. 13:02.280 --> 13:05.680 One, that there is an incorrect assignment here. 13:05.760 --> 13:06.980 That it should have been... 13:06.980 --> 13:10.440 That string two should have been assigned as an array or a pointer. 13:10.660 --> 13:16.700 Second, it does understand that there's a vulnerable function in there, that is string copy, right? 13:16.760 --> 13:25.480 The secure form of doing the same thing is to use something like string copy, which is the remediation it suggests next. 13:25.480 --> 13:27.760 So it does fix the code in a certain way. 13:27.760 --> 13:37.340 First, it says, it adjusts the size of your source buffer to destination buffer and says 15, because you see string two has ABCD or N, right? 13:37.340 --> 13:38.620 Those are 15 characters. 13:38.620 --> 13:41.340 So it adjusts the size of string one. 13:41.340 --> 13:44.900 Then it does the correct character assignment, variable assignment. 13:44.900 --> 13:48.880 Then it uses the correct, more secure form of the string copy. 13:48.880 --> 13:52.360 It uses string copy, which checks on the size of the buffer. 13:52.360 --> 13:54.920 And then it also ensures null termination. 13:55.400 --> 13:58.000 So now, clear how this thing works, right? 13:58.000 --> 13:59.980 This straightforward process. 14:00.520 --> 14:02.660 Now let's see what goes wrong. 14:04.780 --> 14:06.580 Who can guess the code? 14:07.020 --> 14:08.080 What's vulnerable? 14:08.260 --> 14:09.020 Yes. 14:11.580 --> 14:13.720 SQL injection, thank you. 14:14.060 --> 14:15.920 So this is SQL injection. 14:16.040 --> 14:18.260 Now I'm giving it a certain background, okay? 14:18.260 --> 14:23.860 I'm like, hey, we have three metrics, which we are going to use to calculate the severity risk score. 14:23.860 --> 14:26.560 One, how easy it is to exploit this vulnerability. 14:26.820 --> 14:28.720 Two, where does this sit? 14:28.720 --> 14:30.700 How hostile is my threat environment? 14:30.900 --> 14:35.480 Three, what is the impact that is going to happen if I exploit this vulnerability? 14:35.560 --> 14:38.000 And we're going to give it score between zero to 10. 14:38.100 --> 14:42.780 And we calculate the severity risk by adding all those three and divide them by three. 14:42.780 --> 14:45.340 Then I give it additional context. 14:45.340 --> 14:50.960 That, hey, this is used to fetch username information on a public-facing website. 14:51.120 --> 14:53.320 And is this program vulnerable? 14:53.320 --> 14:54.140 To what? 14:54.140 --> 14:56.620 Help me calculate the severity risk for this. 14:57.620 --> 14:58.560 Beautiful. 14:58.560 --> 15:00.980 It identifies it's SQL injection. 15:01.000 --> 15:02.440 So kudos on that. 15:02.620 --> 15:04.840 But this is where my problem lies. 15:04.960 --> 15:14.280 It says ease of exploit development is 10 because, hey, there's so many resources online to find this, how to exploit this. 15:14.280 --> 15:17.280 Hostility, it's a public-facing website, let's give it a score of 10. 15:17.280 --> 15:25.280 Impact, you can modify the database, you can get whatever sorts of secrets, I mean, user data and all of that, so 10. 15:25.420 --> 15:31.340 But what I see is it depends on the context of your application. 15:31.340 --> 15:34.140 What if there are certain guardrails on your public-facing website? 15:34.400 --> 15:40.060 What if the database isn't actually bringing out that kind of sensitive username information? 15:40.060 --> 15:44.080 These scores vary depending on the context of your application. 15:44.080 --> 15:50.660 I would have, depending on the guardrails I've given my public-facing website, I would have given hostility score 8 or 9. 15:51.480 --> 16:07.400 So that it does, one thing I want to point out here is that the more clear you are with your questions, the more context you give chat GPT, the better is the quality of the output or the response it generates. 16:08.080 --> 16:11.600 But again, do not blindly trust it. 16:11.600 --> 16:12.700 Trust but verify. 16:13.880 --> 16:18.300 The second application I want to talk about is secure coding suggestions with Copilot. 16:19.360 --> 16:27.940 Why I like this is because I'm writing a function and I can be like, help me give it a suggestion, like, help me write this in a secure fashion. 16:27.940 --> 16:31.500 And then it will give me all these different secure coding suggestions to do this. 16:31.760 --> 16:33.860 Again, breaking it down into steps. 16:33.860 --> 16:36.560 First, obviously, write your code. 16:37.080 --> 16:43.840 Then, while you're writing your code, Copilot is going to give you real-time coding suggestions about what you should be typing next. 16:44.420 --> 16:48.600 You assess the code that is generated by Copilot. 16:48.700 --> 16:50.880 Again, do not blindly believe. 16:51.860 --> 16:55.380 If you want to change your prompt, you can refine your prompt. 16:55.960 --> 16:58.940 And then, lastly, review and test. 16:58.980 --> 17:06.740 Why I say refine the prompt is because sometimes, again, looping back, the more context you give, the better output it's going to generate. 17:07.040 --> 17:08.600 Hence, refine your prompts. 17:09.360 --> 17:12.740 So, I have a vulnerable snippet. 17:13.120 --> 17:14.120 We just saw this. 17:14.120 --> 17:15.060 Any guesses? 17:17.780 --> 17:18.620 Exactly. 17:18.620 --> 17:19.940 Yes, it is deceiving. 17:19.940 --> 17:21.880 I know I tried to deceive, but you all are smart. 17:23.200 --> 17:30.920 Okay, so the suggestion that I'm giving Copilot is, hey, identify the vulnerability in this code snippet and fix it for me. 17:31.640 --> 17:34.180 Now, it does a nice job. 17:34.180 --> 17:39.500 This is the whole, the first box is the whole, it's all suggested by Copilot. 17:39.500 --> 17:41.360 This is not something I've typed by myself. 17:41.360 --> 17:44.360 This whole slide is everything that Copilot has suggested. 17:44.480 --> 17:46.720 It identifies buffer overflow. 17:46.760 --> 17:49.720 It identifies the vulnerable function that is string copy. 17:49.720 --> 17:52.400 And it tells you replace it by the safer version of it. 17:52.400 --> 17:55.540 And it tells you why you should use string copy. 17:55.540 --> 17:56.660 Good, good enough context. 17:56.660 --> 17:59.020 There's some nuggets that I can take from this. 17:59.020 --> 18:11.120 Now, it does correct the code as well, but it would have been more secure if it would have adjusted the size of string one to accommodate string two. 18:11.380 --> 18:11.900 Right ? 18:11.900 --> 18:14.000 So, yeah. 18:14.000 --> 18:17.600 So now we understand how this whole thing works, the whole gamut. 18:17.920 --> 18:19.700 Next prompt for you. 18:20.180 --> 18:21.540 What is this? 18:21.540 --> 18:22.520 Guesses. 18:24.480 --> 18:25.560 Guesses. 18:25.560 --> 18:29.100 You can go. 18:29.100 --> 18:30.780 You've gone once already. 18:31.440 --> 18:32.100 Yes? 18:33.340 --> 18:35.280 Part traversal, directory traversal, yes. 18:35.280 --> 18:36.260 Okay, perfect. 18:36.500 --> 18:39.160 So, again, you give it the suggestion. 18:39.500 --> 18:41.480 Identify this vulnerability for me. 18:41.480 --> 18:43.340 Rewrite the code to eliminate it. 18:44.620 --> 18:45.980 Now, beautiful. 18:45.980 --> 18:48.140 It identifies part traversal. 18:48.140 --> 18:50.420 But this is where the problem lies. 18:50.420 --> 18:53.040 It's giving you like eight different suggestions. 18:53.180 --> 18:55.060 It's like, use OS part join. 18:55.060 --> 19:00.500 Use OS part apps part, common prefix, exists, is file, rel part, norm part. 19:00.500 --> 19:06.120 And then it goes on to generate this 25-line code about how this can be fixed. 19:06.440 --> 19:25.540 But if you add the human touch to it, the domain-specific knowledge to it, you would rather realize that you should use something like OS part base name because it would directly break your whole part into two different parts, the head and the tail, separating the whole directory component. 19:25.540 --> 19:29.960 And then you completely keep the last file name and focus on it. 19:29.960 --> 19:36.480 Albeit, this is a more secure fix, but you can still make it vulnerable. 19:36.480 --> 19:39.860 You can still give it some maliciously crafted regex and try to break this. 19:42.640 --> 19:50.520 The last one, the last application I want to talk about is security test case generation using Copilot. 19:51.800 --> 19:53.860 Again, we're going to break it down into... 19:53.860 --> 19:54.980 Wait, I'm curious. 19:54.980 --> 19:57.320 How many people here do security test cases? 19:59.660 --> 20:02.900 Three, four, five, six? 20:02.980 --> 20:03.660 Okay. 20:04.100 --> 20:04.670 Okay, that's... 20:05.060 --> 20:06.080 I hope this... 20:06.080 --> 20:07.060 We're going to work on this. 20:07.060 --> 20:07.940 We're going to increase this. 20:07.940 --> 20:09.280 Thank you to Copilot. 20:09.280 --> 20:12.420 Maybe we try to generate more test cases going forward. 20:13.820 --> 20:15.020 The breakdown. 20:15.020 --> 20:17.200 You identify the vulnerability. 20:17.840 --> 20:20.560 You obviously launch Copilot in your IDE. 20:21.760 --> 20:22.640 Oops, okay. 20:22.640 --> 20:26.620 Then you write a prompt to generate security test cases like we saw earlier. 20:26.620 --> 20:27.620 Hey, help me. 20:27.620 --> 20:29.180 Like we saw, identify the vulnerability. 20:29.180 --> 20:32.320 This time you're going to be like, hey, generate security test cases for me. 20:32.460 --> 20:35.660 And then Copilot will generate all those test cases for you. 20:35.660 --> 20:37.600 You execute those test cases. 20:37.600 --> 20:40.880 Again, review the output that it generates. 20:42.020 --> 20:42.760 Oh. 20:43.660 --> 20:44.560 Did you get that? 20:47.310 --> 20:48.190 Okay. 20:48.810 --> 20:50.710 Snippet one. 20:50.890 --> 20:52.110 What is this? 20:52.110 --> 20:53.830 What is it vulnerable to? 20:54.350 --> 20:56.250 Guesses, guesses. 20:57.370 --> 20:59.110 Not buffer overflow. 21:01.290 --> 21:02.610 Integer overflow. 21:02.610 --> 21:03.250 Perfect. 21:03.490 --> 21:06.170 So this is a Python program. 21:06.170 --> 21:16.290 Python is used to handling really large numbers, but you can make it vulnerable to integer overflow by using C-compatible data types. 21:16.510 --> 21:34.910 So what is basically happening in this program is that if you give it big numbers, for example, A is using the maximum size of the 32-bit sign integer, so 2x minus 31 minus 1, and you give it B, 1, you would expect it to be some positive big number, right? 21:35.050 --> 21:40.210 But instead, since this program is vulnerable to integer overflow, it's going to give you a negative value. 21:40.210 --> 21:41.990 It's going to do a whole wraparound. 21:41.990 --> 21:44.430 So that is how this program is vulnerable. 21:45.030 --> 21:48.210 Now, let's go ahead to application. 21:49.670 --> 21:51.190 So this is what I'm saying. 21:51.190 --> 21:55.750 Hey, write unit tests for vulnerable addition function and vulnerable pi. 21:57.130 --> 22:01.590 Then it does a good job of suggesting two cases for me. 22:01.590 --> 22:02.030 Or three. 22:02.030 --> 22:03.890 It's breaking it down into two lines here. 22:03.890 --> 22:06.230 Test one, normal addition. 22:06.370 --> 22:14.670 Test two, it should raise an exception when there's an overflow and underflow. 22:14.670 --> 22:22.930 Then it proceeds to write, one, normal addition, you're adding 1 and 2, should return 3, straightforward. 22:24.090 --> 22:33.010 Two, it should raise an overflow error when you're adding a really big number, 2x minus 31 with 1, should raise an overflow error. 22:33.010 --> 22:34.450 Three, same thing. 22:34.450 --> 22:40.310 You're adding the smallest value of the 32-bit sign to minus 1, it should raise an overflow error. 22:40.310 --> 22:41.150 Straightforward? 22:41.150 --> 22:43.430 Are we following any questions so far? 22:43.590 --> 22:44.890 Okay, perfect. 22:47.070 --> 22:52.710 And it does, thankfully, generate two failures. 22:52.710 --> 23:00.230 It says that vulnerable addition is not generating overflow error for both the cases, the maximum values and the minimum values. 23:00.230 --> 23:02.130 So it works fine, right? 23:02.130 --> 23:04.410 This is what we were expecting it to do. 23:04.410 --> 23:09.910 So, this is the bad example. 23:09.930 --> 23:12.290 Did we follow what I just spoke about? 23:12.330 --> 23:13.310 Okay, great. 23:13.310 --> 23:17.250 So now that we understand how it's generating unit test cases, this is the second example. 23:17.470 --> 23:19.250 What is the vulnerability? 23:20.850 --> 23:22.270 We saw this. 23:24.940 --> 23:26.980 SQL injection, right. 23:27.620 --> 23:29.100 SQL injection, right. 23:29.100 --> 23:38.580 So now, I go ahead and tell it, generate security-focused test cases for me, for the previous function, and then it does this thing. 23:38.580 --> 23:40.480 It does this really weird thing. 23:41.120 --> 23:54.460 One, it looks, hey, is this generating, is when I enter the correct username, is it rendering, when I enter the correct combination for the username and password, is it a success? 23:54.480 --> 24:00.360 And if I go for a non-existent user, is it going to raise an is-none, right? 24:00.360 --> 24:05.060 But it is completely ignoring the SQL injection. 24:05.120 --> 24:10.180 A good addition would have been this, right? 24:10.180 --> 24:12.680 Give it a maliciously crafted input, right? 24:12.680 --> 24:20.500 It completely ignores the fact that there is SQL injection in here, right? 24:20.500 --> 24:23.320 So these are the only two test cases it generates. 24:23.320 --> 24:27.760 Okay, so now we've seen security-focused test cases. 24:28.260 --> 24:29.820 Everybody follow so far? 24:29.820 --> 24:31.060 Okay, great. 24:33.280 --> 24:35.160 Another weird application. 24:35.160 --> 24:37.060 How many of you all use CodeQL? 24:38.740 --> 24:40.500 You find it annoying? 24:41.420 --> 24:43.020 Sort of, but does the job? 24:43.020 --> 24:43.220 Yeah? 24:43.480 --> 24:44.700 Yeah, okay. 24:44.700 --> 24:56.180 So for people who don't know what CodeQL is, it's an amazing tool that is generated by GitHub to find out if there are any vulnerabilities in the code that you've pushed, right? 24:56.240 --> 24:58.440 So think of it as a SAST. 24:58.980 --> 25:00.760 So now I ask... 25:00.760 --> 25:05.340 this is the correct query that I have generated. 25:05.340 --> 25:07.440 Say I want to find in my whole... 25:08.120 --> 25:14.700 in the entire repository, if I want to find all of the to-do comments, this is the query you give CodeQL. 25:15.300 --> 25:15.960 Okay? 25:15.960 --> 25:16.700 Looks beautiful. 25:16.780 --> 25:23.500 Straightforward, it's like, hey, from a comment, you get text, find this regex, it should have to-do in it, right? 25:23.500 --> 25:24.260 Okay. 25:24.800 --> 25:29.420 When I ask the same question to ChatGPD, it gives me this... 25:30.000 --> 25:30.960 I don't know what this is. 25:30.960 --> 25:32.260 It gives me this whole thing. 25:32.580 --> 25:39.400 When I try to run it, the results are so bad, there's so many errors in it, it doesn't even fit in a screenshot. 25:39.920 --> 25:41.180 And this is what I felt. 25:41.260 --> 25:42.100 This is what I felt. 25:42.100 --> 25:54.700 I would have rather invested all my time understanding how CodeQL works and making the query, but instead, ChatGPD generated this query for me for five hours, and then I was trying to debug it and debug it and debug it, and I invested my whole day, and then I was irritated. 25:54.700 --> 26:00.420 So, it's not a good CodeQL query developer. 26:01.380 --> 26:02.620 Okay. 26:02.920 --> 26:03.880 Praise Jesus! 26:03.880 --> 26:05.620 Our jobs are safe, guys! 26:08.480 --> 26:10.640 Our jobs are completely safe. 26:10.640 --> 26:13.040 It hasn't taken over our job yet. 26:13.740 --> 26:15.720 But, why? 26:15.720 --> 26:22.860 So certain limitations that I found throughout this whole research of mine, I'm just gonna list them out. 26:22.860 --> 26:31.060 One, ChatGPD, the cutoff data for it is September 2021, so all of the vulnerabilities that came out after, it has no idea about those. 26:31.060 --> 26:42.180 If I even give it a requirements.txt with a bunch of vulnerable packages, it's not gonna identify which is vulnerable because it does not have most updated knowledge on it. 26:42.420 --> 26:44.440 Same thing with Copilot as well. 26:44.940 --> 26:48.680 And dependence on prompts, like we saw. 26:48.680 --> 27:02.380 The more context you give, the more breakdown in your prompts that you give, the less vagueness that your prompts have, the better the output of ChatGPD and Copilot will be. 27:02.940 --> 27:05.620 Intellectual property concerns. 27:05.620 --> 27:06.280 Okay. 27:06.300 --> 27:13.420 So, do not enter, for the love of God, do not put business critical information in any of these tools. 27:13.420 --> 27:15.080 Please, please do not. 27:15.220 --> 27:20.840 Do not even put your secret, your PIIs, like your SSNs or something like that. 27:20.840 --> 27:32.860 If you're trying to use, if you're trying to make, if you're trying to generate a response that contains IP addresses, give it some fake IP address or give it some fake website name, like this is www.example.com or something like that. 27:33.660 --> 27:36.800 Basically, don't give anything that is intellectual property. 27:37.700 --> 27:48.220 Since these tools are in the nascent phases of development, they do have potential bias and they are still prone to false positives and negatives, like we saw throughout the presentation in different examples. 27:48.220 --> 27:56.500 And lastly, we saw in all of these examples, it is introducing unintended, insecure consequences. 27:56.500 --> 27:59.920 It is not developing more secure form of the code from get-go. 28:00.660 --> 28:03.800 So, these are certain limitations that we have. 28:04.840 --> 28:15.500 So, if there's one thing you can take home today from your flight back, from this whole talk, would be please do not fall for the false sense of security that these tools are providing, please. 28:17.300 --> 28:18.940 And here's a fun meme for you. 28:18.940 --> 28:20.060 Thanks, Drake. 28:20.300 --> 28:22.500 Please do not blindly trust AI. 28:22.540 --> 28:26.180 Assess and review the responses that are generated by AI. 28:28.620 --> 28:34.260 And here is a chat GPT prompt engineering hack that I stumbled upon during my research. 28:34.260 --> 28:38.500 If you want to create more clearer prompts, use this. 28:38.780 --> 28:43.240 It's basically an iterative process where you tell chat GPT, hey, this is my question. 28:43.240 --> 28:44.780 It comes up with its own suggestions. 28:44.780 --> 28:50.580 It comes up with a revised prompt, more additional questions you should be asking to generate a better prompt. 28:50.940 --> 28:55.460 And, yeah, this whole iteration process continues until you let it know that, hey, I'm happy with my prompt. 28:58.460 --> 28:58.860 And... 29:02.600 --> 29:03.600 I'll wait for you. 29:13.070 --> 29:16.650 And here's a slide of helpful links. 29:16.650 --> 29:18.830 Thanks to Gavin Clonlight from AI Village. 29:19.550 --> 29:21.570 He has done some really amazing research. 29:21.570 --> 29:23.290 Feel free to check him out on Twitter. 29:23.290 --> 29:31.910 Some really helpful links if you want to learn prompt engineering, different stances that OpenAI has, things like that. 29:36.800 --> 29:37.560 And... 29:38.560 --> 29:39.500 Okay. 29:39.500 --> 29:40.620 And that's it. 29:40.620 --> 29:41.320 Thank you.