start	end	text
7930	14570	So today we're going to talk about the duality of Chad GPT and copilot in secure development.
14790	19090	And so my name is Kalyani Pawar. Let's talk a little bit about me.
19970	24050	So hi, I am Kalyani Pawar. I also go by Kaylee.
24130	30350	I currently run the application security and vendor security side of things at a drone company called Zipline.
30390	33850	I've been doing security for nearly half a decade now.
35570	38850	I enjoy doing what I do in my free time.
39050	44370	All the opinions throughout all these slides are mine. I am not representing my employer in any way.
45190	50590	This is my Twitter or X handle. I don't post a lot, but I promise everything that I learn,
50590	54950	all the prompts that we're talking about today, I'm going to talk about them on the Twitter soon.
56250	58690	Things that I do outside of my work.
58690	65330	I do review talks that get submitted to Grace Hoppers and AppSec Village every year at DEF CON.
65330	67590	So that's about me.
68010	72950	Now, what are we going to talk about today?
72990	79130	Obviously, Chad GPT and Copilot, but some fun, cool AppSec-y things that you can do with them.
79330	84230	And while we are exploring the fun side of things, let us proceed with caution
84230	89550	and not fall directly for the responses that it is generating for us.
89610	92230	All right, now that we know what we're going to talk about...
92990	94890	Okay, quick question.
95210	100250	Who in the audience today has worked with Chad GPT or Chad GPT-like models?
100810	103130	Fancy. That is what I expect.
103470	108450	So here is a quick primer for people who are not familiar with Chad GPT.
108830	112550	It's an AI language model developed by OpenAI.
112550	116170	It is based on generative pre-trained transformer architecture.
116170	118930	It is out of scope. The whole architecture is out of scope for this talk.
118930	120150	You don't need to know this information.
120150	123690	All you've got to know is how to type in text on a browser.
125250	130710	So what I like about Chad GPT is it is trained on a range of diverse data that you find on the Internet,
130710	134430	on various books and sources of information that are available online.
134470	140430	But what I find beautiful about this tool is that it has the ability to generate human-like text.
140430	143150	Once you input whatever question you have for it.
143170	149630	The fact that you can speak to a machine which can understand the context and nuances of whatever you talk to it.
150790	154150	And it's like talking to a human interlocutor at the end of the day.
154150	155770	So that is what I like about it.
155770	159070	And you can just imagine the kind of functionality that this tool provides you.
159070	160990	You can ask it all sorts of questions.
161170	165150	You can ask it to translate into different languages you don't know about.
165150	171570	You can use it to learn different things that you probably would be investing an hour reading a book.
171570	173570	Rather ask Chad GPT about it.
174090	178270	And, yes, when I don't know the answer to a question, that's what I do.
178270	182570	Ask Chad GPT, guys. I don't know the answer to this. Use Chad GPT for this.
185290	188050	How many of y'all have used Copilot?
189030	190330	Y'all like it?
191290	193990	Sort of? Yeah, same, same. Sort of.
193990	196930	For people who don't know what Copilot is,
196930	201490	it is an AI pair programmer developed by GitHub in collaboration with OpenAI.
202550	207930	It is trained on a diverse range of public code repositories that are available,
207930	213750	so it does understand the coding styles of millions of developers all over the world,
213750	217050	and what their documentation styles would be.
217050	222490	How many of y'all have struggled with code that has the worst comments in it?
223130	225330	Most of us? Most of us? We don't, like...
225330	227090	What the f... What is this?
227610	229750	What is this? Right?
229750	233790	So, what I like about Chad GPT is that, while you are coding,
233790	238030	it does also generate comments so that someone else is going to read your code,
238030	240830	does understand what have you done, the beauty that you've created.
241790	242590	Awesome.
244370	244770	So...
244770	246610	Oh, yeah, this is something else I like about it,
246610	250110	that it can seamlessly integrate within VS Code, which is my choice of IDE.
250110	254150	I don't have to, like, open up a different browser or an instance, like, launch this tool.
254150	257430	As I'm typing, it's going to generate real-time suggestions about,
257430	259470	hey, this is probably what you should be writing,
259470	261890	instead of whatever you're thinking of writing.
261970	264770	For example, if I just ask Chad GPT Copilot that,
264770	267150	can you generate a program that's going to create...
267150	269190	that's going to generate a random number for you?
269190	270370	It would just...
270370	273770	Bam! You just press tap, and it writes the whole program for you.
275210	275570	So...
275570	277530	Now that we know what Copilot is,
277530	283390	I want to talk about the problems that the unsung heroes of our community face.
283390	285470	Application security engineers.
285590	286970	What is the problem?
287050	290810	The problem is that there's a bunch of problems that we're facing.
291230	293310	And I'm going to run through them real quick.
294430	300990	How many of us have been part of these meetings where they're like,
300990	304030	hey, I found something vulnerable in my code.
304030	307270	And you're like, okay, how long ago did you push this code into Prod?
307270	309330	And they're like, yeah, many sprints ago.
309330	311890	And you're like, are you telling me this right now?
311890	313310	Like, this is what you did?
313310	317290	So I understand with advent of DevOps and Agile,
317290	323330	we are shipping and deploying code at 100 miles an hour, which is amazing,
323330	326610	but also it's leaving potential security gaps
326610	329630	because we are falling short on providing coverage
329630	331410	and all the code that has been deployed.
332530	336130	Second, lack of security education.
336130	338990	According to dark reading, about half of the developers,
338990	342230	48% of the developers lack security education.
342230	345050	They simply develop code keeping functionality in mind
345050	347950	and completely ignore the security aspect of it.
347950	351630	So you're pushing insecure code full of vulnerabilities from the get-go.
351630	355310	That is another pressing issue that we're facing.
356820	359070	The resource constraints.
359070	362670	Were you at Tanya Janka's talk earlier this week?
362670	364250	Any hands? Okay.
364250	366670	So she was talking about... I'm sure...
366670	367970	Okay, raise your hands, okay?
367970	370370	How many times have you been in an organization
370370	372550	where there's like 200 developers
372550	375410	and only one AppSec engineer supporting them?
376790	378690	Unsung heroes, I tell you.
383370	385830	Security teams are usually smaller in size
385830	389110	and it's getting difficult to literally review every line of code
389110	392190	that is generated by your organization.
392190	394110	So what do you do to combat this?
394670	397250	You onboard different tools for it.
397290	400730	Now, I am guilty of onboarding two, three different tools
400730	404070	and just integrating it within my CICD and all of that.
404070	406230	But the problem with this is that
406230	408950	it is generating a bunch of false positives.
408950	411170	And now we're all sitting and investing our time
411170	413010	and sifting through what the true positives are,
413010	415870	what the false positives are, and it just does not end there.
415870	418110	It's about, okay, now we got to prioritize
418110	419990	what risk this vulnerability holds
419990	422410	and how are we going to clear this backlog?
422410	424470	And there's just a lot of time that we spend
424470	427050	in strategizing how to deal with this monster.
428670	431150	And this is another problem.
431350	434970	Ever-emerging threats, ever-emerging attack vectors.
434970	437120	Every day there's something new that I read online
437710	439230	about some new attack vector.
439230	442290	And it's just difficult for this one brain of mine
442290	446110	to constantly catch up with everything that's going on outside
446110	447810	on the threat landscape.
447810	450010	Last but not the least,
450010	454250	the hammer that is the legal regulation stick.
454850	457390	Depending on the kind of product you're making
457390	459270	or the kind of jurisdiction you're part of
459270	460710	or the business that you are in,
460710	463090	you've got to make sure that your product is compliant
463090	465550	with certain industry regulated standards
465550	468070	like SOC 2 or GDPR.
468070	470050	How many of us take that seriously?
471150	473090	We are trying, right?
473090	473810	We're trying.
473810	476090	It's just like an additional audit task somewhere
476090	477570	in the sprint.
478830	479330	Awesome.
479810	482150	So, now that we know
482150	485150	the bunch of problems that we are facing,
485290	487510	a possible solution, a possible solution
487510	490290	would be to leverage AI for AppSec.
490290	493410	When I say AI, I'm trying to loosely refer to
493410	496950	Chad, GPT, and Copilot for the rest of this whole conversation.
498510	501170	How can we leverage AI for AppSec?
501410	504370	Anyone in the crowd?
506530	507410	Yes.
510030	511050	Training, yes.
511830	513090	Anyone else?
515210	516730	Yes. Perfect. Bang on.
516730	517710	Anyone else?
521160	522360	Exactly. Yes.
522360	524160	So, a couple applications here.
524160	526380	You can ask Chad, GPT, like,
526380	528760	hey, this is my code. Tell me what's vulnerable in it
528760	530620	and how can I fix this.
530700	533020	You can also go like, hey, Copilot,
533020	534960	I'm generating this code right now.
535260	537480	Can you give me secure coding suggestions
537480	539780	about how this whole code is supposed to go?
540920	543000	You can also be like, oh, also for, like,
543100	544860	a developer especially, they can go like,
544860	547140	hey, what is Heartbleed vulnerability?
547140	548620	Explain to me in simple terms.
548620	550160	They ask this question to Chad, GPT,
550160	551960	and they don't have to go through, like,
552080	553740	a bunch of web pages to understand
553740	555880	what Heartbleed is, right?
555880	558820	So, there's certain applications to it.
558820	560740	Now, what I see on the bright side for this
560740	563940	is that how many times do you see yourself
563940	564920	context switching?
564920	567080	Like, one moment you're doing API security,
567080	568700	one moment you're threat modeling,
568700	571700	next moment you're source code reviewing.
571700	573520	How many times does that happen to you in a day?
574320	576240	All the time. All the time, right?
576240	578980	This context switching is leading to burnout
578980	579800	on the team.
579800	581540	I feel like a hamster on a wheel all the time
581540	583000	because I'm constantly switching
583000	585000	into, like, different projects all the time
585000	587960	because it comes down to that 200 developers
587960	590160	being supported by this one AppSec engineer.
590240	592160	So, why not use these tools a little bit?
592160	593660	Why not hack the game a little?
595120	598860	So, now that we spoke about what about this approach,
598860	600100	why this approach?
600880	602840	Because here are certain reasons I think
602840	606480	we should think this approach should be fit.
606700	608020	One is efficiency.
608020	609600	Since these tools use machine learning,
609600	613440	they have the ability to generate speedy outcomes.
614380	616000	It is instructional.
616380	617880	Since it has the ability,
617880	619660	ChatGPT has the ability to have, like,
619740	621320	a conversation with you,
621320	622720	you can ask it questions
622720	624940	and you can learn new things from it.
626780	627660	Scalability.
627760	629340	They can handle complex tasks
629340	630540	and large volumes of data
630540	632520	with little human attention.
633960	634520	Automation.
634520	636460	The fact that I can seamlessly integrate Copilot
636460	638380	within my IDE.
638980	642180	I feel that that brings some amount of convenience.
642620	643540	And ease of use.
643540	644880	The fact that I can just open up a browser
644880	646500	and go, like, chat, open AI,
646500	649760	and then just ask a bunch of questions to the tool.
649760	651780	The fact that it is easy to use
651780	652780	and user-friendly,
652780	654300	that is what appeals to me.
654300	655660	So, these are some reasons
655660	658020	why this approach might be deemed fit.
658820	662280	So, now that we've done the what and the why,
662280	664260	we should do the how, right?
665540	667880	Let's dive into some fun AppSec things
667880	668980	that we got to do.
668980	671100	This is what y'all are here for, I guess, right?
672480	675300	So, for the sake of time and convenience,
675300	678100	I'm going to try to keep it to simple examples.
678820	681140	But if you're open to, like,
681140	682940	if you want to know more about other research
682940	683880	that I've done on this,
683880	685520	feel free to reach out to me.
688080	688980	Okay.
688980	691480	First application is vulnerability identification
691480	693740	and remediation with ChatGPD.
693980	696620	I have broken this down into a number of steps
696620	698700	and how you can do it.
699920	702300	One, you input your code, obvious.
702300	704220	Whatever script that you want to check
704220	705780	if it is vulnerable or not,
705780	707340	you input your code.
707340	709240	Then you engineer a prompt.
709240	710840	When you're engineering a prompt,
710840	711560	you ask for...
712040	712800	You're straightforward.
712800	713860	It's a straightforward process.
713860	715640	You ask for vulnerabilities and remediations
715640	719660	for whatever this vulnerability might be.
719680	722020	You assess the response.
722020	724340	Do not just copy-paste anything
724340	727060	that ChatGPD gives you, please.
727060	729380	For the love of God, do not.
729580	731060	Assess response.
731300	733580	Validate the recommended fixes
733580	735680	that they have given you.
736400	738380	Test and review.
738380	739440	Do not.
739440	741540	Please do not just copy-paste
741540	743940	and push it to prod, please.
745260	746380	Okay.
746720	750200	So, here is a prompt for you.
750200	753160	And whoever gets the vulnerability right,
753160	754780	I have some stickers to share.
755700	756820	Okay.
757560	758120	So...
758120	759940	And please raise your hands.
760340	762120	So what is the vulnerable code here?
762120	764400	And the prompt that I've given ChatGPD is that,
764400	766580	okay, is this code vulnerable to what?
766580	769080	And please suggest what the remediation for this is.
769080	770200	Guesses, guesses.
770860	771900	Buffer overflow.
771900	773400	Thank you, thank you.
773860	775260	Please meet me after,
775260	777060	so I can give you the sticker.
778020	778640	Okay.
779100	782280	So, ChatGPD identifies two issues with it.
782280	785680	One, that there is an incorrect assignment here.
785760	786980	That it should have been...
786980	788660	That string two should have been assigned
788660	790440	as an array or a pointer.
790660	792680	Second, it does understand
792680	794740	that there's a vulnerable function in there,
794740	796700	that is string copy, right?
796760	799820	The secure form of doing the same thing
799820	802440	is to use something like string copy,
802440	805480	which is the remediation it suggests next.
805480	807760	So it does fix the code in a certain way.
807760	808700	First, it says,
808700	811740	it adjusts the size of your source buffer
811740	814160	to destination buffer and says 15,
814160	817340	because you see string two has ABCD or N, right?
817340	818620	Those are 15 characters.
818620	821340	So it adjusts the size of string one.
821340	823740	Then it does the correct character assignment,
823740	824900	variable assignment.
824900	827280	Then it uses the correct, more secure form
827280	828880	of the string copy.
828880	830580	It uses string copy,
830580	832360	which checks on the size of the buffer.
832360	834920	And then it also ensures null termination.
835400	838000	So now, clear how this thing works, right?
838000	839980	This straightforward process.
840520	842660	Now let's see what goes wrong.
844780	846580	Who can guess the code?
847020	848080	What's vulnerable?
848260	849020	Yes.
851580	853720	SQL injection, thank you.
854060	855920	So this is SQL injection.
856040	858260	Now I'm giving it a certain background, okay?
858260	860560	I'm like, hey, we have three metrics,
860560	861620	which we are going to use
861620	863860	to calculate the severity risk score.
863860	866560	One, how easy it is to exploit this vulnerability.
866820	868720	Two, where does this sit?
868720	870700	How hostile is my threat environment?
870900	873740	Three, what is the impact that is going to happen
873740	875480	if I exploit this vulnerability?
875560	878000	And we're going to give it score between zero to 10.
878100	879880	And we calculate the severity risk
879880	882780	by adding all those three and divide them by three.
882780	885340	Then I give it additional context.
885340	888660	That, hey, this is used to fetch username information
888660	890960	on a public-facing website.
891120	893320	And is this program vulnerable?
893320	894140	To what?
894140	896620	Help me calculate the severity risk for this.
897620	898560	Beautiful.
898560	900980	It identifies it's SQL injection.
901000	902440	So kudos on that.
902620	904840	But this is where my problem lies.
904960	908820	It says ease of exploit development is 10
908820	913140	because, hey, there's so many resources online to find this,
913140	914280	how to exploit this.
914280	916080	Hostility, it's a public-facing website,
916080	917280	let's give it a score of 10.
917280	919560	Impact, you can modify the database,
919560	922340	you can get whatever sorts of secrets,
922340	925280	I mean, user data and all of that, so 10.
925420	931340	But what I see is it depends on the context of your application.
931340	934140	What if there are certain guardrails on your public-facing website?
934400	937640	What if the database isn't actually bringing out
937640	940060	that kind of sensitive username information?
940060	944080	These scores vary depending on the context of your application.
944080	945680	I would have, depending on the guardrails
945680	947540	I've given my public-facing website,
947540	950660	I would have given hostility score 8 or 9.
951480	955700	So that it does, one thing I want to point out here
955700	959340	is that the more clear you are with your questions,
959340	962740	the more context you give chat GPT,
962740	965660	the better is the quality of the output
965660	967400	or the response it generates.
968080	971600	But again, do not blindly trust it.
971600	972700	Trust but verify.
973880	975860	The second application I want to talk about
975860	978300	is secure coding suggestions with Copilot.
979360	982520	Why I like this is because I'm writing a function
983000	985640	and I can be like, help me give it a suggestion,
985640	987940	like, help me write this in a secure fashion.
987940	989440	And then it will give me all these different
989440	991500	secure coding suggestions to do this.
991760	993860	Again, breaking it down into steps.
993860	996560	First, obviously, write your code.
997080	999280	Then, while you're writing your code,
999280	1001400	Copilot is going to give you real-time coding suggestions
1001400	1003840	about what you should be typing next.
1004420	1008600	You assess the code that is generated by Copilot.
1008700	1010880	Again, do not blindly believe.
1011860	1013840	If you want to change your prompt,
1013840	1015380	you can refine your prompt.
1015960	1018940	And then, lastly, review and test.
1018980	1021020	Why I say refine the prompt is because
1021020	1023200	sometimes, again, looping back,
1023200	1025500	the more context you give, the better output
1025500	1026740	it's going to generate.
1027040	1028600	Hence, refine your prompts.
1029360	1032740	So, I have a vulnerable snippet.
1033120	1035060	We just saw this. Any guesses?
1037780	1039940	Exactly. Yes, it is deceiving.
1039940	1041880	I know I tried to deceive, but you all are smart.
1043200	1046880	Okay, so the suggestion that I'm giving Copilot is,
1046880	1049300	hey, identify the vulnerability in this code snippet
1049300	1050920	and fix it for me.
1051640	1054180	Now, it does a nice job.
1054180	1056920	This is the whole, the first box is the whole,
1057720	1059500	it's all suggested by Copilot.
1059500	1061360	This is not something I've typed by myself.
1061360	1064360	This whole slide is everything that Copilot has suggested.
1064480	1066720	It identifies buffer overflow.
1066760	1069720	It identifies the vulnerable function that is string copy.
1069720	1072400	And it tells you replace it by the safer version of it.
1072400	1075540	And it tells you why you should use string copy.
1075540	1076660	Good, good enough context.
1076660	1079020	There's some nuggets that I can take from this.
1079020	1082240	Now, it does correct the code as well,
1082240	1086220	but it would have been more secure
1086220	1088660	if it would have adjusted the size of string one
1088660	1091120	to accommodate string two.
1091380	1091900	Right?
1091900	1094000	So, yeah.
1094000	1096340	So now we understand how this whole thing works,
1096340	1097600	the whole gamut.
1097920	1099700	Next prompt for you.
1100180	1101540	What is this?
1101540	1102520	Guesses.
1104480	1105560	Guesses.
1105560	1109100	You can go.
1109100	1110780	You've gone once already.
1111440	1112100	Yes?
1113340	1115280	Part traversal, directory traversal, yes.
1115280	1116260	Okay, perfect.
1116500	1119160	So, again, you give it the suggestion.
1119500	1121480	Identify this vulnerability for me.
1121480	1123340	Rewrite the code to eliminate it.
1124620	1125980	Now, beautiful.
1125980	1128140	It identifies part traversal.
1128140	1130420	But this is where the problem lies.
1130420	1133040	It's giving you like eight different suggestions.
1133180	1135060	It's like, use OS part join.
1135060	1138280	Use OS part apps part, common prefix, exists,
1138280	1140500	is file, rel part, norm part.
1140500	1143540	And then it goes on to generate this 25-line code
1143540	1146120	about how this can be fixed.
1146440	1150960	But if you add the human touch to it,
1150960	1152920	the domain-specific knowledge to it,
1153680	1155720	you would rather realize that you should use
1155720	1157300	something like OS part base name
1157300	1160760	because it would directly break your whole part
1160760	1162720	into two different parts,
1162720	1163660	the head and the tail,
1163660	1165540	separating the whole directory component.
1165540	1169000	And then you completely keep the last file name
1169000	1169960	and focus on it.
1169960	1172720	Albeit, this is a more secure fix,
1172720	1176480	but you can still make it vulnerable.
1176480	1178500	You can still give it some maliciously crafted regex
1178500	1179860	and try to break this.
1182640	1185840	The last one, the last application I want to talk about
1185840	1190520	is security test case generation using Copilot.
1191800	1193860	Again, we're going to break it down into...
1193860	1194980	Wait, I'm curious.
1194980	1197320	How many people here do security test cases?
1199660	1202900	Three, four, five, six?
1202980	1203660	Okay.
1204100	1204670	Okay, that's...
1205060	1206080	I hope this...
1206080	1207060	We're going to work on this.
1207060	1207940	We're going to increase this.
1207940	1209280	Thank you to Copilot.
1209280	1212420	Maybe we try to generate more test cases going forward.
1213820	1215020	The breakdown.
1215020	1217200	You identify the vulnerability.
1217840	1220560	You obviously launch Copilot in your IDE.
1221760	1222640	Oops, okay.
1222640	1225700	Then you write a prompt to generate security test cases
1225700	1226620	like we saw earlier.
1226620	1227620	Hey, help me.
1227620	1229180	Like we saw, identify the vulnerability.
1229180	1230180	This time you're going to be like,
1230180	1232320	hey, generate security test cases for me.
1232460	1235660	And then Copilot will generate all those test cases for you.
1235660	1237600	You execute those test cases.
1237600	1240880	Again, review the output that it generates.
1242020	1242760	Oh.
1243660	1244560	Did you get that?
1247310	1248190	Okay.
1248810	1250710	Snippet one.
1250890	1252110	What is this?
1252110	1253830	What is it vulnerable to?
1254350	1256250	Guesses, guesses.
1257370	1259110	Not buffer overflow.
1261290	1262610	Integer overflow.
1262610	1263250	Perfect.
1263490	1266170	So this is a Python program.
1266170	1270150	Python is used to handling really large numbers,
1270150	1273690	but you can make it vulnerable to integer overflow
1273690	1276290	by using C-compatible data types.
1276510	1278990	So what is basically happening in this program
1278990	1281330	is that if you give it big numbers,
1281330	1284370	for example, A is using the maximum size
1284370	1286530	of the 32-bit sign integer,
1286530	1288770	so 2x minus 31 minus 1,
1288770	1290330	and you give it B, 1,
1290330	1294910	you would expect it to be some positive big number, right?
1295050	1298710	But instead, since this program is vulnerable to integer overflow,
1298710	1300210	it's going to give you a negative value.
1300210	1301990	It's going to do a whole wraparound.
1301990	1304430	So that is how this program is vulnerable.
1305030	1308210	Now, let's go ahead to application.
1309670	1311190	So this is what I'm saying.
1311190	1314390	Hey, write unit tests for vulnerable addition function
1314390	1315750	and vulnerable pi.
1317130	1321590	Then it does a good job of suggesting two cases for me.
1321590	1322030	Or three.
1322030	1323890	It's breaking it down into two lines here.
1323890	1326230	Test one, normal addition.
1326370	1329810	Test two, it should raise an exception
1329810	1334670	when there's an overflow and underflow.
1334670	1337810	Then it proceeds to write,
1338330	1341250	one, normal addition, you're adding 1 and 2,
1341250	1342930	should return 3, straightforward.
1344090	1346890	Two, it should raise an overflow error
1346890	1349550	when you're adding a really big number,
1349550	1351230	2x minus 31 with 1,
1351230	1353010	should raise an overflow error.
1353010	1354450	Three, same thing.
1354450	1358350	You're adding the smallest value of the 32-bit sign to minus 1,
1358350	1360310	it should raise an overflow error.
1360310	1361150	Straightforward?
1361150	1363430	Are we following any questions so far?
1363590	1364890	Okay, perfect.
1367070	1372710	And it does, thankfully, generate two failures.
1372710	1374370	It says that vulnerable addition
1374370	1378030	is not generating overflow error for both the cases,
1378030	1380230	the maximum values and the minimum values.
1380230	1382130	So it works fine, right?
1382130	1384410	This is what we were expecting it to do.
1384410	1389910	So, this is the bad example.
1389930	1392290	Did we follow what I just spoke about?
1392330	1393310	Okay, great.
1393310	1395870	So now that we understand how it's generating unit test cases,
1395870	1397250	this is the second example.
1397470	1399250	What is the vulnerability?
1400850	1402270	We saw this.
1404940	1406980	SQL injection, right.
1407620	1409100	SQL injection, right.
1409100	1411580	So now, I go ahead and tell it,
1411580	1414440	generate security-focused test cases for me,
1414440	1416280	for the previous function,
1416280	1418580	and then it does this thing.
1418580	1420480	It does this really weird thing.
1421120	1424440	One, it looks, hey, is this generating,
1425520	1427840	is when I enter the correct username,
1427840	1431680	is it rendering, when I enter the correct combination
1431680	1434460	for the username and password, is it a success?
1434480	1437340	And if I go for a non-existent user,
1437340	1440360	is it going to raise an is-none, right?
1440360	1445060	But it is completely ignoring the SQL injection.
1445120	1450180	A good addition would have been this, right?
1450180	1452680	Give it a maliciously crafted input, right?
1452680	1455240	It completely ignores the fact
1455240	1460500	that there is SQL injection in here, right?
1460500	1463320	So these are the only two test cases it generates.
1463320	1467760	Okay, so now we've seen security-focused test cases.
1468260	1469820	Everybody follow so far?
1469820	1471060	Okay, great.
1473280	1475160	Another weird application.
1475160	1477060	How many of you all use CodeQL?
1478740	1480500	You find it annoying?
1481420	1483220	Sort of, but does the job? Yeah?
1483480	1484700	Yeah, okay.
1484700	1487140	So for people who don't know what CodeQL is,
1487860	1490620	it's an amazing tool that is generated by GitHub
1490620	1493940	to find out if there are any vulnerabilities
1493940	1496180	in the code that you've pushed, right?
1496240	1498440	So think of it as a SAST.
1498980	1500760	So now I ask...
1500760	1505340	this is the correct query that I have generated.
1505340	1507440	Say I want to find in my whole...
1508120	1510580	in the entire repository,
1510580	1512840	if I want to find all of the to-do comments,
1512840	1514700	this is the query you give CodeQL.
1515300	1516700	Okay? Looks beautiful.
1516780	1519180	Straightforward, it's like, hey, from a comment,
1519180	1521300	you get text, find this regex,
1521300	1523500	it should have to-do in it, right?
1523500	1524260	Okay.
1524800	1527360	When I ask the same question to ChatGPD,
1527360	1529420	it gives me this...
1530000	1530960	I don't know what this is.
1530960	1532260	It gives me this whole thing.
1532580	1534540	When I try to run it,
1534540	1535980	the results are so bad,
1535980	1537220	there's so many errors in it,
1537220	1539400	it doesn't even fit in a screenshot.
1539920	1541180	And this is what I felt.
1541260	1542100	This is what I felt.
1542100	1543820	I would have rather invested all my time
1543820	1546720	understanding how CodeQL works and making the query,
1546720	1549520	but instead, ChatGPD generated this query for me
1549520	1551240	for five hours, and then I was trying to debug it
1551240	1552360	and debug it and debug it,
1552360	1553720	and I invested my whole day,
1553720	1554700	and then I was irritated.
1554700	1560420	So, it's not a good CodeQL query developer.
1561380	1562620	Okay.
1562920	1563880	Praise Jesus!
1563880	1565620	Our jobs are safe, guys!
1568480	1570640	Our jobs are completely safe.
1570640	1573040	It hasn't taken over our job yet.
1573740	1575720	But, why?
1575720	1578020	So certain limitations that I found
1578020	1581280	throughout this whole research of mine,
1581280	1582860	I'm just gonna list them out.
1582860	1587200	One, ChatGPD, the cutoff data for it is September 2021,
1587200	1589420	so all of the vulnerabilities that came out after,
1589420	1591060	it has no idea about those.
1591060	1593580	If I even give it a requirements.txt
1593580	1596280	with a bunch of vulnerable packages,
1596280	1599200	it's not gonna identify which is vulnerable
1599200	1602180	because it does not have most updated knowledge on it.
1602420	1604440	Same thing with Copilot as well.
1604940	1608680	And dependence on prompts, like we saw.
1608680	1609960	The more context you give,
1609960	1612460	the more breakdown in your prompts that you give,
1612460	1617020	the less vagueness that your prompts have,
1617020	1622380	the better the output of ChatGPD and Copilot will be.
1622940	1625620	Intellectual property concerns.
1625620	1626280	Okay.
1626300	1630040	So, do not enter, for the love of God,
1630040	1631600	do not put business critical information
1631600	1633420	in any of these tools.
1633420	1635080	Please, please do not.
1635220	1638980	Do not even put your secret, your PIIs,
1638980	1640840	like your SSNs or something like that.
1640840	1642960	If you're trying to use,
1642960	1644360	if you're trying to make,
1644360	1645520	if you're trying to generate a response
1645520	1646700	that contains IP addresses,
1646700	1647820	give it some fake IP address
1647820	1649780	or give it some fake website name,
1649780	1652860	like this is www.example.com or something like that.
1653660	1656800	Basically, don't give anything that is intellectual property.
1657700	1660700	Since these tools are in the nascent phases of development,
1660700	1662440	they do have potential bias
1662440	1665140	and they are still prone to false positives and negatives,
1665140	1666440	like we saw throughout the presentation
1666440	1668220	in different examples.
1668220	1673120	And lastly, we saw in all of these examples,
1673120	1676500	it is introducing unintended, insecure consequences.
1676500	1679920	It is not developing more secure form of the code from get-go.
1680660	1683800	So, these are certain limitations that we have.
1684840	1687320	So, if there's one thing you can take home today
1687320	1688480	from your flight back,
1688480	1689520	from this whole talk,
1689520	1693080	would be please do not fall for the false sense of security
1693080	1695500	that these tools are providing, please.
1697300	1698940	And here's a fun meme for you.
1698940	1700060	Thanks, Drake.
1700300	1702500	Please do not blindly trust AI.
1702540	1706180	Assess and review the responses that are generated by AI.
1708620	1712220	And here is a chat GPT prompt engineering hack
1712220	1714260	that I stumbled upon during my research.
1714260	1718500	If you want to create more clearer prompts, use this.
1718780	1720740	It's basically an iterative process
1720740	1721980	where you tell chat GPT,
1721980	1723240	hey, this is my question.
1723240	1724780	It comes up with its own suggestions.
1724780	1727160	It comes up with a revised prompt,
1727160	1728840	more additional questions you should be asking
1728840	1730580	to generate a better prompt.
1730940	1733100	And, yeah, this whole iteration process continues
1733100	1734080	until you let it know that,
1734080	1735460	hey, I'm happy with my prompt.
1738460	1738860	And...
1742600	1743600	I'll wait for you.
1753070	1756650	And here's a slide of helpful links.
1756650	1758830	Thanks to Gavin Clonlight from AI Village.
1759550	1761570	He has done some really amazing research.
1761570	1763290	Feel free to check him out on Twitter.
1763290	1765430	Some really helpful links
1765430	1767710	if you want to learn prompt engineering,
1767710	1770030	different stances that OpenAI has,
1770030	1771910	things like that.
1776800	1777560	And...
1778560	1779500	Okay.
1779500	1781320	And that's it. Thank you.