start	end	text
4120	14440	Hi, everyone. Good morning. That's the kind of cheer we need. So thank you so much for
14440	21040	coming to our DEF CON this year, Recon Village. So let's give a big round of applause for
21040	31820	Recon Village. So happy to see so many people. It's like the room full and I can see some
31820	37540	people standing out there as well. We're going to be introducing our first speaker,
37540	41600	our keynote speaker, who's going to be talking about something very interesting, something
41600	51380	very hot right now. So who all have heard about AI? No? No? No? Yes? Maybe? So he's
51380	57400	going to be speaking about reshaping reconnaissance, AI's role in open source intelligence. So
57400	65000	let's welcome Matt Edmondson. And he's actually founder for Argilius Labs, hacker, forensicator,
65000	71100	science author, practical open source intelligence course, principal instructor, Black Hat speaker,
71100	76440	OSCP holder, featured in Wired, and so many more things to say about him. But what I'm
76440	82940	going to do is... I'm going to shut my mouth now and I'll give the stage to him. But let's
82940	93740	give him a round of applause. I met her half an hour ago and I'm already deeply afraid
93740	100260	of her. Also, the hardest part of this talk is going to be not moving for an hour with
100260	105780	the stationary mic. I literally might explode. So she did a good job introducing me. Almost
105780	110740	22 years as a federal agent, started up a lot of OSINT units with the government. Louder?
110740	115200	All right. All right. I got to get real close and intimate with this microphone. Thank you
115200	119660	for letting me know. I'm the author of the SANS SEC 497 practical OSINT course, Black
119660	123500	Hat speaker, featured in Wired, et cetera, et cetera. What are we going to talk about
123500	128340	today? A little bit of introduction. We'll talk about using AIML for profile creation,
128340	132680	see what that looks like real quick. For processing, which I think is honestly what we'll probably
132680	137440	end up spending most of our time, kind of processing, massaging data. Code analysis
137440	144740	and end it with a little bit of report writing. It's kind of weird. If we're being perfectly
144740	152660	honest, if we gave this talk a year ago, like these would just be buzzwords, right? I mean,
152660	156660	seriously, for most of us in here. AIML. I think I joked when I was speaking at the
156660	163040	2021 SANS Summit. I wrote a persistent monitoring tool. And I said, yeah, if I was trying to
163040	166900	get funding, if I was trying to sell it, I would say it uses AIML. It's just lists and
166900	172440	common sense and some shoddy Python code. Really it is. And all of a sudden a year later,
172440	177740	we actually have usable things that do produce some amazing, amazing capabilities, right?
178280	183440	And you're already starting to see some people think that this is just a fad. We live in an
183440	187660	industry full of fads, right? Next generation this. There's always some new next thing that's
187660	193680	going to be the savior. It never actually pans out. And so it's very, very easy to think that
193680	197820	it's going to be the same way with some of these capabilities. But I really don't think that's
197820	203900	going to be the case at all. I really, really don't. I originally started playing with this
203900	209740	tech late last year, like I think a lot of us probably did. And I was in curiosity and I just
209740	214600	started using it more and more. I was amazed by the results. And usually how I was using it was
214600	218380	to help me writing code, right? I'm not a professional programmer. I can write shoddy
218380	223600	Python code that helps me, my friends, my teammates and kind of gets the job done.
223600	228360	And all of a sudden with ChatGPT, I was writing that code a lot quicker, right? I was just really,
228360	234620	really enjoying it. And I saw someone on an OSINT Discord say, yeah, ChatGPT, it's neat,
234620	239400	but it's not really useful for OSINT. And I'm like, man, I'm using it every day, right? And
239400	245180	so that's when I decided to do a webinar on how ChatGPT for OSINT. And when I did it, I didn't
245180	250300	just want to talk about how I was using it. And so I actually forced myself to sit there and think,
250300	256040	how could we use it in every phase of the intelligence cycle? And as I started playing
256040	262180	with it more and more, frankly, I was more and more amazed. I just really, really truly was.
262980	270760	The good news is the hard stuff is getting more and more unnecessary every day, right? You don't
270760	276980	really have to understand the math behind it. The same way if we're being honest at a hardware level,
276980	280280	my phone does a lot of stuff that I don't understand exactly how it works.
280300	284740	But I use it every single day. And this technology is quickly getting there, very quickly.
284740	290800	If anyone's ever tried to play with the local models, right? Like Lama or some of those others.
290840	296560	If we think back even four or five months ago, what do these things want you to do? Oh, download
296560	301920	this. Download all these weights. Train the model. This is about it. You don't have to do that
301920	306880	anymore. It's like one click. I mean, seriously, right? It's like a pip install. And then you have
306880	311700	things running. And it's just going there. And I like that. And it's not just installing and using
311700	316000	local. I think it's kind of everything in this space. Right now for 2023, right? One of the hot
316000	321980	jobs has been prompt engineering. It's getting very, very good at asking these models questions
321980	326640	and getting the results you want. That's not going to be a job in like two years for the most part.
326640	330460	It's really, really not. Because they're getting better and better at knowing what they want,
330460	336600	right? When I was a little kid, I tried to draw. I can't draw, right? I can't draw a
336600	341360	straight line with a ruler. I'm horrible. But now, like if you look at all these graphics, I made
341360	346120	them myself using my words, right? Using mid journey. And you used to have to give mid journey
346120	350920	like these long, complicated commands to try to get decent output. Nowadays you can give it two
350920	355120	words and get pictures like this, right? So it's heading in the right direction. And I'm very happy
355120	361900	to see it. It's going to change the way that a lot of things are getting done, frankly. It just is,
361900	367480	right? And it's going to be important to be at least familiar and comfortable with it. There's
367480	371860	someone named Daniel Missler who's brilliant. He comes out with a newsletter each week on kind of
371860	376380	trends in technology. If you like his stuff, you should definitely check it out. But he's mentioned
376560	382160	a few times something called like a K-shaped recovery, right? And as we start to think of what
382160	385620	this is going to change and how it's going to affect our industry and how it's going to affect
385620	390080	our jobs, it kind of makes sense that it's probably going to be very, very good for some people and
390080	394140	very, very bad for some people, right? If I was an artist right now, I would be terrified.
394220	398280	And it sucks because I appreciate artists because I don't have that capability. So I can just sit
398280	402380	there for hours and watch. Like I can go down to the strip and see someone with cans of spray paint
402380	407420	making pictures. I can watch them for hours. I'm mesmerized because that's not a skill set I have.
407620	411920	But now I'm making pictures like this, right? So like I said, if I was an artist, I'd be terrifying.
411960	418620	And so what he means by a K-shaped recovery is just like a capital A, right? Are you utilizing AI ML?
420260	424880	Probably going to be good for you. Are you not? It may not be the best. So it's just something
424880	431200	to think about. And like I said, we live in an industry full of trends. I was a black hat
431200	435980	yesterday and I saw a booth where they had like set up a coffin and candles saying something was
435980	439660	dead. It was probably the tackiest booth I've ever seen in my life. If you work for that company, I
439660	444520	apologize. But like I said, when we live in this industry, right, where there's so many things
444520	449300	that are fads and trends, it's really not hyperbole to say that things that haven't
449300	455420	been possible before now are. And it's kind of a silly example that I use, but the example that I
455420	460060	love is sentiment, right? Has anyone in here ever tried in the past to do sentiment analysis at
460060	466240	scale? It is a dumpster fire, right? Those of you that haven't tried, no, but it is a dumpster fire.
466240	470240	It's one of those things. It's like three lines of Python to implement, right?
470240	476580	Import text blob, get your score, you're good to go. But the score is iffy at best. And when I
476580	480080	looked at this, this was, we used to have a site called Twitter. It was pretty cool. It's something
480080	486540	else now. But when I was looking at a site to analyze sentiment, this was classified as the
486540	493320	most negative tweet. And if you look at it, this is bad AF. Y'all killed it. That's clearly very
493320	500380	positive, right? But everything that we throw it at says it's extremely negative. But now I just gave
500380	504820	it to Google Bard, and Google Bard said it's positive and takes the time to sit there and
504820	510660	explain why it's positive. We couldn't do this a year ago, right? At least not publicly. And so
510660	514420	this is just one example of, like I said, it's not hyperbole. The things that weren't possible
515080	519580	are now possible. And right now, like, honestly, this is kind of a good for demonstration
520560	524860	operationally. What? I would want positive, negative, or mixed. But now think about that.
524860	530420	When you're trying to set up persistent monitoring efforts, right? Why is sentiment so hard? Language
530420	536360	changes, right? Bad is actually good now sometimes, depending on the context. Sarcasm. Hey, great job,
536360	541920	Salvin. Real good to see you, right? These are the reasons why it's hard, but now these models are
541920	546860	doing a really, really good job with it. Let's take a look at... we've got a couple of examples that
546860	549960	I've shown before, and they're in here because they're really good, but then we have a lot of
549960	555200	new stuff too. So one thing, right? How can we actually use this technology creating sock puppet
555200	562040	accounts? And so here I asked chat GPT, can you give me 10 fake names for a French woman in her
562040	569860	30s? And it spits them out. I picked number five, Chantelle Rousseau. I then asked, can you recommend
569860	577480	10 usernames for a woman named Chantelle Rousseau? And it does it. This to me is hurts. This slide
577480	587260	just hurts. Yeah, I'm tall and handsome and charming and modest, but honestly, clever, right? Clever
587900	591520	is what's kind of gotten me along all these years. And now I look at the usernames that
591520	596940	chat GPT came up with, and I'm like, dammit, chat GPT is more clever than me, right? Chantellicious,
597740	603900	the Chantress, Chantelle C. Rousseau, Chante Rousse. These are like, there's like six names
603900	609380	that I will freely admit are better than anything I could come up with. And that hurts. That's one
609380	615100	less thing I got going for me, right? And now that we have the name, and now that we have the username,
615100	620540	and I know this went to small, I apologize, but I say, can you please fill out all the profile
620540	627900	information? And it gives me a date of birth, a city, her interests, and then a short bio right
627900	634460	in there, right? It really, really kind of is crazy. And so this is push-button automated now
634460	638600	to just come up with an accurate, right? And I've showed this to people from France, and they're
638600	644760	like, no, that looks spot on. This is push-button capability. But what about the profile pictures,
644760	651400	right? We have these things now to detect that a picture isn't real. Except most of the tech
651400	657880	that's detecting if a picture isn't real, they don't know what everyone in the world looks like,
658440	664020	and long story short, GAN stands for a generative adversarial network. And this was some researchers
664020	668020	at NVIDIA, the company that their stock is skyrocketing right now, right, because of this
668020	672420	stuff. But they make graphics cards, and their researchers had to come up with an algorithm to
672420	677700	detect faces. And so they made an algorithm to create faces, and basically generative adversarial
677700	682200	network. Generative because it actually makes something, and adversarial network because you
682200	686800	have two computer programs that you're battling against each other. And so, for instance, if we
686800	691760	make a program that makes counterfeit currency, and another program that detects counterfeit
691760	697020	currency, and we battle them together thousands, millions of times, what's eventually going to
697020	701320	happen is the counterfeiter is going to get so good that it's almost never going to get detected,
701320	705340	right? And so they came up with this algorithm to make faces, but there are tells, right? The
705340	709780	most common one is, if you've ever played with that site, this person does not exist,
709780	714920	and you take your fingers, and you cover eyes, and you hit refresh, refresh, refresh, all these
714920	719080	new faces, right? Your fingers never move. You're still covering the eyes. That's kind of its north
719080	727900	star. And so a lot of these tools, a lot of these plugins, etc, oh, we detect fake faces with a 99.82%
727900	734420	accuracy. No, you don't. You detect if it was generated with a GAN, right? And so a lot of
734420	738360	people are familiar with ChatGPT, obviously, but a lot of people haven't ever played around on a
738360	743560	site called Hugging Face. You ever have like a friend that always likes to brag that they were
743560	748520	on a band before anyone else, right? I was watching Green Day back when they were playing in bars in
748520	754760	Hemet, right? If you want to be that person in the AI ML space, and who doesn't, Hugging Face is where
754760	759980	you should be playing out. It's a lot like GitHub, except they actually let you set up spaces where
759980	765160	you can test and play with these codes. And one of the code out there right now is a project called
765160	772000	Face Swap, and Face Swap is pretty cool. It lets me take a picture of myself, mix in that bottom
772000	776660	person at the left, the far less handsome person in the lower left corner. That's actually a GAN
776660	782040	generated image, right? You can tell with the soulless little eyes there. And it lets me now mix
782040	789200	in elements from that GAN onto my picture. So even though I kind of mixed in attributes from a GAN,
789200	795580	if you look at my picture, it still has the same basic shape, the same basic, right, kind of angle.
795580	801840	The eyes aren't in the exact same spot. And so all these tools that brag that they have a 99.82%
801840	807600	accuracy, this is just a rip in the right biome, right? And so literally right now, we have the capability
807600	816040	with an API to generate profiles and generate profile pictures in seconds, right? And for those
816040	820580	of us that have to create accounts at scale, this is glorious. For those of us that are trying to
820580	825300	deal with propaganda networks, right, and the things that a lot of us are dealing with, this
825300	831760	is terrifying, right? That is absolutely the scary part. And right now, I don't honestly have a lot of
831760	836640	answers, but I think those of us in this room and those of us at home, we're going to be on the hook
836640	843480	to solve this problem, right? We're using these capabilities to generate these things at scale.
843520	847560	We're going to have to figure out ways that we can use these capabilities to detect them at scale,
847560	851940	right? And we can. And like everything else, this is going to turn into a cat and mouse game,
851940	858660	right? This AIML, it's like, what is it going to be good at? Yes, right? Seriously, yes. We just have
858660	863640	to get clever and creative about how we're going to use it and how we're going to deploy it.
864800	871200	Let's talk for a little bit about processing. Throughout my entire career with the government,
871860	875440	I would hear people say collection and analysis, right? And I think if any of you were in the
875440	881140	Gubby space, you always hear collection analysis, collection analysis. You're missing a step,
881140	886740	right? There's a step in between there, and that step is honestly where I lived most of my career,
886740	893460	and that step is processing. Processing is taking the data and actually putting it in a
893460	898220	format where it's suitable to be analyzed, right? I always joke, like, everyone says they want all
898220	902920	the data until you give them all the data, right? Seriously, people are like, I want all the data,
902920	908760	and you hand them like a 16 gigabyte SQL database of breach data, and they're just staring at it
908940	913800	like a hog looking at a wristwatch, right? No idea what's going on. And it's actually being able to
913800	918680	process that data, massage that, and put that in a format where people can analyze it. It could
918680	922580	be something like just translating it to a native language, right? To a language that you speak,
922580	928640	or whatever it is, but processing. It's a critical step, like I said, because of my technical skills,
928640	932700	most of my career, this is where I lived, is kind of taking these things that people didn't know how
932700	937800	to deal with and making it to where they could deal with it. And I got to tell you, this tech
937800	945720	is absolutely ridiculous at doing these things. Another example that I've used, but it's just too
945720	950240	perfect not to use, Conti, right? A lot of us have heard of that. Ransomware gang, been observed
950240	955580	since 2020, 2022, right? Last year, the government offered a reward of 10 million, I'd like 10
955580	962420	million dollars. Conti came out, right? If you're like a threat actor operating in Russia, right?
962420	965520	What are the rules, right? You really, for the most part, don't mess with other Russians, right?
965520	969940	That's where the malware tries to detect the language, and if Russia asks you to do something,
969940	976460	you do it, right? That's kind of the rules. And so obviously, Conti came out and expressed
976460	981160	support for the invasion of Ukraine. Well, not everyone in the group was on board, and so someone
981160	988660	leaked 60,000 messages from their internal chat. This sounds amazing, this is like beautiful, let's
988660	995220	read them. Cool, here they are. Once again, the font is small, I know, but number one, they're in
995220	1001760	JSON. All right, the, you know, the addresses are a little weird, right? They end in a dot onion, got
1001760	1007240	some dark web stuff going on here, and the body, the text, right? It's all in Russian, right? It's all
1007240	1013100	in Cyrillic, it's all in Russian. We can do this, right? We have translate, we have human translators,
1013720	1018580	we have tech translator, translation engines are getting good, right? Translations are way better
1018580	1022940	than a lot of people think right now. We can make this happen, but it's going to be a little bit of
1023000	1030900	a pain in the butt, right? 60,000 messages, grab a Snickers. Here, I give the text to chat GPT and I
1030900	1035300	say, parse the body field in the JSON and tell me what the conversations are about.
1036740	1045200	It cannot be this easy. Damn it, it's this easy. Seriously, look at that. This person and this
1045200	1049280	person having a conversation about registering an external account for backup purposes,
1050000	1055240	these ones talking about their location, providing login details for several servers.
1056460	1061740	I didn't have to massage the JSON. I didn't have to translate the Russian
1061740	1069660	to English so I could understand it. I didn't have to do any of that. It just did it, right? Like I
1069660	1077100	said, it's to say that this is a fad, I think is crazy personally, because all of a sudden,
1077100	1084880	things that just were not possible at scale before now are. And this is one thing, too, where
1085880	1091260	it gets a little tricky, right? I was just, two weeks ago, I was teaching at a conference in New
1091260	1096180	York and all of the people for the conference were setting up, well, there were some union
1096180	1100220	workers that were just standing around watching, right? But they were getting paid. You had to
1100220	1103740	have the union workers there, right? And that's just, you know, there are some cities that are
1103740	1110120	like that. I'm not saying it's good or bad, but it's this, well, it's taking someone's job,
1110120	1113400	right? Like, no, you can't do that. I need to do that, because that's someone else's job, right?
1113400	1118920	I can't do that. That's someone else's. And so it's easy to think this and get afraid. Like I
1118920	1123280	said, if I was an artist, I'm going to be honest, I'd be terrified. But this tech isn't about
1123280	1130300	replacing people. It's about toll reduction, right? It's about taking the things that a human being
1130300	1136780	doesn't like and automating that and letting them focus on what we're good at, because there
1136780	1141900	are some things that humans are absolutely better at than this tech, especially right now. But if we
1141900	1148100	can use it so we can focus our time, I always think back to when I first ever used Python,
1148100	1154340	it was actually for GIS work, and I had one of my Intel analysts who every week had to generate a
1154340	1160040	report. She had to analyze certain activities that occurred in different regions and actually
1160040	1165600	come up with some stats for that, and this literally took her an entire day every week.
1165980	1169540	And I'm like, there's got to be a way to automate this, right, with some of these capabilities from
1169540	1173920	Esri. And I reached out to the company. I'm like, hey, is it possible to use these tools externally
1173920	1178780	and just tap the Python API? And they're like, I don't know. No one's ever asked me that. Cool.
1179300	1183540	Took me about two days, but I was able to cobble together some code and automate that and take
1183540	1188880	something that was literally taking her a full day every week and made a push button run in about ten
1188880	1197260	minutes, right? She wasn't out of a job. We didn't convert her to part-time. It let her get rid of
1197260	1201980	something that she hated doing every week and focus on things that she was actually good at
1201980	1206440	and enjoyed a little bit more, right? So that's definitely the mindset that I try to keep in mind
1206440	1211320	as I think about how to use this. Like I said, I really think it's about toll reduction, not about
1211320	1218920	replacement, at least for right now and hopefully forever. I personally, you see all these cool
1218920	1223600	little parlor tricks that people are doing. I think the killer app right now for these large
1223600	1229300	language models like ChatGPT and Google BARD, and we'll talk more about BARD later, I really think
1229300	1236920	that the killer app is summarization, right? I give you just a mountain of text and you summarize it
1236920	1243740	for me, right? Just like those conversations we just looked at. We saw the power now of taking
1243740	1249360	text, but what about other stuff, right? What about audio and video? And for that, let's look at
1249360	1253680	something called a Whisper. If you've never seen Whisper before, you are in for a treat. Whisper is
1254620	1259960	ridiculous and it's got everything going for it and it is completely free. It's made by OpenAI,
1259960	1265160	the same company that makes ChatGPT. And there's a lot of times for a lot of reasons where you have
1265160	1272060	audio and video files that you actually need to transcribe, right? We need text from this. Whisper
1272060	1278640	automates that task. Here, I took a video from my personal trainer on how to, kind of advice on how
1278640	1284660	to meet your protein goals, and I sicked Whisper on it. Whisper is two lines to install. You have
1284660	1290880	to install a FFmpeg, like an open source audio video processor, and then it's just a pip install
1290880	1297100	Whisper. It is seriously that easy. Once you do that here, you see I say Whisper. I give it the
1297100	1303160	name of the video. It takes about 30 seconds of audio to analyze the language and detect it,
1303700	1308020	and then it just starts spitting out. The blur there is he dropped an F-bomb, right? I didn't
1308020	1311920	want an F-bomb in my talk. If I'm going to drop an F-bomb, it's going to be me, not a text.
1312420	1318320	And I'll try not to, I promise. But no, I mean, if we're looking at that, it's spot on, right?
1318320	1324340	And he's got a fairly thick Irish accent, but he's easy to understand. So this is impressive,
1324340	1334320	but it's easy to understand, and it's in English. What other languages does it support? Yes,
1334320	1341840	right? And like I said, I know this font is small, but that's a lot of languages right there. Some
1341840	1345520	are going to be better than others, right? It's going to have better support for English, the
1345520	1350260	popular languages, German, French, Russian, than it will for some of the more obscure ones, I'm
1350260	1357040	sure, there. But still, that is a ridiculous level of support. So I gave it English. It did good,
1357040	1362520	but how would it do with a foreign language and one that's not the easiest to understand there,
1362520	1367560	right? So Juan Huyabora, a.k.a. Spanish Laughing Man. A lot of us have probably seen his meme
1367560	1372820	videos, right? The AWS Exhibimon is just the GOAT. I've watched that probably 20 times.
1373560	1378600	So let's see how it does with him. First things first, it actually detects the language slightly
1378600	1384220	off, right? After 30 seconds, it detects it as Galician, like a mix of Spanish and Portuguese,
1384220	1388980	which from Spain, like, this actually kind of makes sense. And so here I aborted, just
1388980	1395080	control seat out, and I did it again, but I actually forced Spanish. So you see their language
1395080	1404180	ES, and it starts having no problem transcribing what is being said in the video. And guess what?
1404180	1410620	I can also do tac-tac translate, and now instead of just transcribing, it also translates to
1410620	1420500	English. We've seen what it does. We've seen the languages it supports. Would you believe me
1420500	1429280	if I told you that this capability was all offline? Because it is. Because it is. I wrote
1429280	1433160	up a blog post on this, and someone's like, there's no way that's offline. I'm like, let's
1433160	1441580	test it. So I fired up a cheap little nook, put Ubuntu on it, I installed Whisper, and I ran
1441580	1445840	it one time, because the first time you run it, it downloads one of the models. It downloads
1445840	1452140	the medium model by default. I think it was 461 megs. It took, like, three seconds to download,
1452140	1459120	seriously. Once it downloaded, and it ran one time, I stopped it, I disconnected the network
1459120	1464720	cable, showing the device was not hooked up to the internet in any way, shape, or form. I then
1464720	1473220	tried a different video, and it crushed it. Once again, that was in English. Cool. Let's try a
1473220	1480780	Spanish one, giving it the task to translate, and it does that. Locally, we are taking massive
1481120	1488140	amounts of audio, video, whatever we want, sucking the text out of it, and even translating it with
1488140	1493080	no internet connectivity. And once again, if you've ever tried to do things like this at scale,
1493080	1499960	and you see the quality of the output that it's producing, it's, like I said, to say it's
1499960	1505080	revolutionary, I think, is even an understatement at some points, right? And this is it, too. It's
1505080	1509580	an important thing to think about, and we're going to talk about it here in a little bit,
1509580	1516460	but we're at the very early phases of this technology, right? People like to point out flaws,
1517020	1524200	go down to a car museum, and look at a Model T. That's a cool car. Would anyone here want to use
1524200	1528840	that as a daily driver if you couldn't sell it and had to use it? That would be your worst life
1528840	1535500	choice ever, right? What we're playing with right now is the Model T of this tech. It really, really
1535500	1541140	is, right? And think of how far we've come, and how fast we've come, and I have no reason to believe
1541140	1549600	that this is going to be any different. Nothing is perfect, right? Sometimes these large language
1549600	1557140	models make things up. I was talking to a buddy of mine, a fellow SANS instructor, Mick Douglas,
1557140	1561200	about hallucinations. That's the term for it, right? When these large language models just
1561200	1567460	make stuff up, hallucinations. He was having ChatGPT write some .NET code for him, and he's
1567460	1574140	like, man, this code is clean, this code is beautiful. I didn't even know that function existed. It didn't.
1578780	1584960	That wasn't the worst one. ChatGPT is actually really good at commenting code, right? If you
1584960	1589080	try to use ChatGPT to crank out some code, it's pretty good about commenting it and about letting
1589080	1592560	you know what it's doing, which, from a learning perspective, is fantastic.
1594020	1598780	He asked it to write some code, and in the comments, it said, yeah, this, you know,
1598780	1603300	right here, what this is doing, this is actually referenced in this blog post, and it gave a link
1603300	1609500	to the blog post in the comments. He's like, this is so cool. He went to read the blog post.
1609520	1617940	It didn't exist. And he could find, and this is not a dumb person, he couldn't find any evidence
1617940	1624680	the blog ever existed. It wasn't he couldn't find the blog post. He doesn't think the blog ever
1624680	1632860	existed. And you hear stories like that. And it's very, very easy to say that this tech is not ready
1632860	1638020	for primetime, that this is a parlor trick, right? This is just another in a long line of fads in
1638020	1641840	this industry, so let's just pour a drink and go on to the next one and get it over with.
1642440	1648100	But I really don't think that's the case. And in this case, I think if we just abandon this,
1648100	1652980	and we decide that it's not perfect, so we don't want to use it, we're making a horrible mistake.
1652980	1657260	We're throwing the baby out with the bathwater, right? Goes all the way back to Voltaire,
1657260	1662340	who said, do not let perfect be the enemy of good, right? That is a very important concept
1662340	1667720	in so many things, right? I got John Hubbard here, good friend of mine, defense in depth,
1667720	1671980	right? We don't rely on any one piece of technology to defend our networks,
1671980	1677440	right? We have a collection of imperfect solutions that when used together leave us
1677440	1682940	much better off than we were before. And so yeah, that does not mean they have no value.
1682980	1690240	They absolutely do. I spent most of my career as a federal agent in intelligence, right? Because
1690240	1696900	of my technical skills that I kind of developed, I got sucked into that very early. And good
1696900	1702620	intelligence people, whether you're analysts, agents, whatever you are, but good intelligence
1702620	1708120	people will always take what they're hearing, and they'll always go directly to the operators
1708120	1713960	for ground truth, right? The actual boots on the ground, is this what you're seeing?
1714140	1718640	Does this actually make sense? Sometimes they say yes, sometimes they say no, that's not what's
1718640	1723420	going on, this is actually what's going on, and now you understand better. And the further you
1723420	1729500	are from the action, the more important this is, right? Just 100%. I spent a few years in Washington
1729500	1734120	DC working in some very cool places, but there you're removed from the front lines, you're
1734120	1739920	removed from what's going on, and that's when it was so important to go to the actual operators
1739920	1747580	and try to secure this ground truth. And so sometimes when I say where I think things are
1747580	1754100	going, it's a prediction, and sometimes it's just wishful thinking, like I just watched some
1754100	1760120	YouTube videos on meditation, I'm trying to wish this into existence. But this year, large language
1760120	1767320	models like chat GPT, generative AIs like mid-journey, stable diffusion, that's been all the rage.
1768040	1773540	What I really hope is that next year in 2024, one of the hottest terms we hear is grounding,
1774100	1778560	right? And a lot of us probably haven't heard of that term yet, but it's basically taking the
1778560	1785880	results from these large language models and running them through some known accurate data
1785880	1792840	to double check, right? To really increase the confidence that we have in these findings,
1793780	1798680	right? That we feel a little bit more comfortable about using it in ways that we don't feel
1798680	1802860	comfortable in using it right now. So like I said, I don't know if this is an actual prediction
1802860	1808720	or just what I want. I think it's a prediction. I do, I honestly do. I think this is going to be
1808720	1813420	kind of the hot term next year, taking known good data and kind of double checking at that
1813420	1817200	and trying to incorporate that and kind of taking this to the next level. Because once again,
1817200	1821540	I think we're honestly playing with the Model T version of this tech.
1823480	1828600	For now, these large language models, they can still be incredibly useful.
1828940	1833200	But what you need to think through is the risks versus the rewards of possible hallucinations
1833200	1837840	when you're using this, right? If you're sitting there using large language models
1839720	1846600	to analyze things and make recommendations that you're like acting on, Lord help you, right?
1846600	1851680	Like that's risky. That's very, very risky right now. But if you've got a bunch of data
1851680	1857200	that you're not doing anything with, and you use large language models, be it logs,
1857200	1862200	be it something else you're collecting, and you're using large language models to churn through that
1862200	1867180	and try to identify things that maybe you should look at, if you're not looking at it right now,
1867180	1872820	what in the heck is the downside? Like seriously, what is the downside of using it in that way?
1872960	1877520	And so right now with the current state of the tech, if we think about implementing this,
1877520	1883540	I think that's what we have to think about. We know that hallucinations are real. We see it.
1883540	1888380	If you've played with this tech, you've experienced it too. But the capability is so powerful, right?
1888380	1892160	We just think of what are the downsides, where it is appropriate to use and where it isn't.
1893640	1900920	And obviously, it kind of goes without saying in this crowd, but we want to think about OPSEC, right?
1900920	1908360	We want to think about putting these things into ChatGPT, and we've already seen some instances of that, right?
1908380	1914340	I gave ChatGPT on purpose some API creds the other day. I was testing something and asked it to write
1914340	1919700	some Python codes using these API creds. It yelled at me for giving it the API creds, but then it
1919700	1925660	generated the code, right? So it tells you not to, but then it does what you ask anyway.
1926100	1933340	And we have these local language models that are unbelievably impressive. If you've played with Lama,
1933340	1937880	if you've played with some of the others, Vicuna, when that came out, or now the new Lama 2,
1938820	1945280	what are they saying? I think they said Vicuna was what, like 92% to 93% of the capability of ChatGPT.
1945280	1951400	To have that running on a local system is amazing. But if you played with this tech,
1951400	1959880	and you have, like, normal consumer-grade hardware, then you know how unbelievably slow it is, right?
1960160	1970320	Unbelievably slow. Like, just through molasses. Very, very difficult to use. Right now, the GPUs
1970320	1977260	that, like, professionals are using with this are ridiculously expensive, right? Like, look up,
1977260	1984340	like, the NVIDIA A100s, and you see, like, these $16,000 video cards with 80 gigs of RAM.
1984480	1991160	Did you ever think that we would live in a world where you would see $16,000 graphic cards that
1991160	1997800	weren't even designed to be hooked up to a monitor? It's disgusting, right? And that's what it is,
1997800	2002360	is you have to load these large language models kind of into the RAM of your GPU,
2002360	2007080	otherwise the performance is even worse. And so if you're trying to do that on, like, an 8-gig
2007080	2011880	GPU, it's brutal. You need that more memory. And so that's another thing, and it's, like,
2011880	2017000	I don't know how much prediction, how much wishful thinking. I'm really hoping between NVIDIA, AMD,
2017000	2020400	maybe some other dark horses, or I don't know, Microsoft, Facebook, a lot of other companies
2020400	2025480	are looking at developing their own hardware now. But I hope we start to see some graphics card
2025480	2031000	hardware that's designed for consumers and it's designed for this. Because right now, if you look
2031000	2037680	at video cards with a 4090, the NVIDIA 4090 with 24 gigs of RAM, I think that's the most RAM we're
2037680	2043540	getting right now on a graphics card that's designed for a consumer. But the RAM, as the
2043540	2048380	video cards get faster and faster, the RAM almost just kind of comes along with it for the ride.
2048380	2052960	I really hope we start to see a trend where we're seeing cards that maybe don't have this cutting
2052960	2057220	edge performance. We don't need it for playing video games or whatever else we're doing, but it
2057220	2061400	has more RAM. Right? We start to kind of see it on that. Just the same way with cryptocurrency, right?
2061400	2066420	It's not the exact same problem, obviously. It's not one specific algorithm. But in cryptocurrency,
2066420	2070880	right, everyone was using video cards. And then what happened? They came out with ASICs, right?
2070880	2075860	They came out with those specialized devices that were a lot better, that use less power. I'm hoping
2075860	2080940	we see a similar thing. I'm hoping we see consumer-grade cards that focus more on RAM so we
2080940	2090960	can use these language models than what we have now. If you pay attention to social media, Twitter,
2090960	2095820	X, whatever it's named next month, right, whatever you're on, you always see people now, Reddit, the
2095820	2102120	ChatGPT. I think half the posts on the Reddit ChatGPT subreddit are on how ChatGPT is getting
2102120	2112380	dumber, right? It's just getting dumber. It's getting dumber. Kind of. Let's be honest. When this tech
2112380	2120540	first came out late last year and into, I think, probably about March, it was a race to impress us.
2120860	2127340	And it worked. It worked, right? That race was canceled. I hate to break it to you. They didn't
2127340	2134340	send out a notice. The reason the race was canceled, it's a new race now. It's a race to limit liability.
2135660	2141760	Right? Think about this. The number two person at OpenAI went on a TED Talk earlier this year
2141760	2149500	and was telling stories of how ChatGPT has helped people. And he told a story about a guy who had a
2149500	2155560	dog that was sick and the guy couldn't afford a vet. So he used ChatGPT to diagnose it and was
2155560	2160500	literally able to save the dog's life. The number two person at OpenAI told that at a TED Talk
2160500	2167140	earlier this year. And now they're removing functionality like that. Right? And you see people
2167140	2172040	that were like, couldn't afford a therapist, that were using ChatGPT. And in their opinion,
2172040	2177860	really getting real benefit from it. You can't do that anymore. Right? And so, yeah, it really
2177860	2183060	has shifted. I made that statement last week in Austin and I had someone come up to me after my
2183060	2188960	talk. And I won't say who he is and I won't say what company he works for, but it's one of the
2188960	2194120	big ones in this space and he's like, you're absolutely right. He said, I'm in meetings
2194120	2200360	basically daily with that team and 90% of what we talk about is limiting liability. I get it.
2200360	2205400	I hate it. I absolutely hate it. Right? I want the functionality, but I get it.
2205600	2210760	But what is that going to do? That's just going to force us, right? Once we have these more
2210760	2216120	unrestricted models, as people start to basically have this capability with no restrictions
2216120	2221700	whatsoever, that's just YOLO. We're all going to switch to that. I mean, we just really,
2221700	2227720	truly are, right? And so it's weird. Sometimes I just want to grab a bucket of popcorn
2228720	2231220	and I'm being perfectly honest, a lot of whiskey.
2232900	2237180	Those of you that know me know that. And just kind of sit back and see how it goes. Because
2237180	2240100	I'm going to be fascinated to see how it plays out. Because like I said, all the major players
2240100	2244400	in this space focusing on limiting liability, that's not going to play well long-term with
2244400	2250940	the consumer. It's just absolutely not. So we'll see. We'll see where it goes. And I was like,
2250940	2255540	I just asked mid-journey for a race pick last night and it gave me this and I'm like,
2255540	2260300	well, that's going in the talk. The rabbit has no meaning. I just like the rabbit.
2261920	2266560	Let's look a little bit at code analysis, right? As far as things that we can do for
2266560	2273780	reconnaissance a few different ways. One thing that I do in my SANS class is I always stay
2273780	2277040	late on day one and try to help out people who have never used Linux before to get a little
2277040	2284600	bit more familiar with it. And a lot of people have like want to get better with Linux, right?
2284600	2287580	And OSINT, that's kind of one of the things, usually the two things that people really
2287580	2290920	want to focus on when they're getting started in the space. I want to get more comfortable
2290920	2294960	with Linux and I want to get more comfortable with Python, right? Or programming. If you want
2294960	2299700	to learn programming in some other language, switch to Python. Unless you're trying to bypass
2299700	2307100	A, B. No one here is, probably. But then you always have these people that give you horrible
2307100	2312140	advice. Like, really, why should I use Linux? Because then you can customize it like bro.
2312780	2316000	They're just trying to get started with the command line. You want to start making kernel
2316000	2321880	modifications? Like, calm down. Let's be a little bit realistic right here. I think that advice,
2321880	2327080	honestly, on why you should learn these things is sometimes disingenuous. Analyzing source code
2327080	2333600	isn't always easy. And this right now, once again, large language models are made to order
2333600	2341460	for this. There are some problems that these just eviscerate, right? I had some Python code for a
2341460	2347020	project on GitHub. I tried to give chat GPT. Chat GPT doesn't take a whole lot of information right
2347020	2352340	now. There's a very limited amount of tokens that you can only give it so much text to ask a
2352340	2360100	question. Google BARD has a much higher limit, right? I think most of us in here use Google as
2360100	2365080	our default search engine. Some use DuckDuckGo, some use other. It sounds weird. Hey, John,
2365080	2371280	just Google it. He's like, okay. If I said, hey, Zobin, just Bing it. Half of us would win,
2371280	2375140	half of us would last. It's a good search engine. It is, right? And let's be honest,
2375140	2379840	it has a lot of results in it that Google doesn't. There's a lot of content that maybe is a little
2379840	2384620	more shady, illegal that's in Bing that you won't find in Google. Not to get political or tinfoil
2384620	2390860	hattie, it just is. The same way a lot of people sleep on Bing do not sleep on Google BARD.
2390860	2396580	Google BARD when it first came out was getting laughed at. It has improved a lot, right? It
2396580	2404040	really has. And so here I took that GitHub code of a project and I gave it to Google BARD and asked
2404040	2411620	it to analyze the code and explain it to me. And it did it. And it did a really, really good job.
2412220	2418320	I was at a mixer last night with a bunch of cyber threat intelligence analysts sitting around.
2418780	2423600	And I had one guy who started talking to me and the subject of like AI came up.
2424080	2429240	And right now he's taking a lot of advanced math classes, right? Working on crypto.
2429790	2437220	And he said there was one problem that he did not understand and his professor could not explain it
2437220	2447040	to him. He took that problem to ChatGPT and it gave him a very clear concise explanation that
2447040	2454560	was accurate. That is insane, right? I had someone last week in Austin ask me if it's between ChatGPT
2454560	2463060	doing something for you and learning, which way should you go? Yes, right? Use ChatGPT to help you
2463060	2469400	learn. Don't just ask it to write the code. Ask it to explain it to you. You've got a Python project
2469400	2472900	you want to run, right? And once again, you always have those same people. Well, now you can look at
2472900	2476880	the code and understand what it's doing. Well, kind of, but that gets tough sometimes, right?
2476880	2481900	If you try to look at my code, you'll be able to understand it. No, because it's basically like
2481900	2485500	me building Lego blocks. It's pretty straightforward. You try to look at code
2485500	2491580	written by a professional developer. Nope. But here, it does an amazing job. And it's not just
2491580	2496460	about programs that you want to run to kind of get an assessment of what it does, how it works,
2496460	2503240	and if it's safe to run. But what about websites, right? What about analyzing JavaScript on websites
2503960	2508960	to see if it's tracking you? It's fantastic for that. So, if you ever look at JavaScript on
2508960	2514140	websites, it's not plain English, right? It's very, very obfuscated. They try to make it hard
2514140	2522240	to figure out what it's doing a lot of times on purpose. Here, once again, was a lot of JavaScript.
2522960	2527160	Why are you taking a screenshot of this? Come on, this is ugly JavaScript. I'm just kidding with
2527160	2533960	you. No, this was not the entire JavaScript. If I put the entire JavaScript on here, it probably
2533960	2542300	would have been about eight slides. Like, it was not a small function. I could probably figure that
2542300	2547260	out. I would not be sober by the end of it, so my results would slightly start to get worse and
2547260	2553660	worse, right? I've done this type of stuff before. I can do it. It kind of sucks, if I'm being honest.
2554100	2560840	I give this to ChatGPT. ChatGPT is like, that's way too much, dog. All right. I give it to Bard,
2560840	2569140	and Bard just does it. It sits there and explains what all the functions do.
2570800	2577080	And then at the very bottom, it actually gave me a link you'll see here in a little bit,
2577080	2582520	where it's like, hey, do you want all this in a table? Right? It's really, really kind of ridiculous.
2582520	2588600	But here, that JavaScript that was clearly tracking, designed to track website visitors,
2588600	2593180	it gives us a breakdown of exactly what it's tracking and exactly how it's doing it. Once
2593180	2598040	again, I could do this without it, but before I was finished, I might gouge my eyes out with
2598160	2603680	a rusty spoon. And so here, like I said, the code was clearly used for tracking visitors,
2604420	2609260	and there were some other sites, and I was trying to tell, is this the same entity? Is the same
2609260	2614820	entity that's tracking on this site also the one that's tracking on this site? Because at first
2614820	2626600	glance, the JavaScript looks roughly similar. I didn't even have to paste the JavaScript into
2626600	2632560	Bard, because what, two months ago now, more or less, ChatGPT killed the browsing functionality.
2632560	2639040	Bards is still alive and well. I gave Bard the link to those two JavaScript files on the two
2639040	2644020	different sites and asked it to compare the JavaScript to see how similar they were.
2645680	2652120	And here it is. It breaks down the key differences between the files, the way they handle errors,
2652120	2658040	the way they deal with variables, right? It gives the pros and the cons. It breaks down
2658040	2663780	the styles being different. The second library is more object-oriented, robust, and portable.
2663920	2668700	And then at the very bottom, thank you, here's a table that summarizes the key differences between
2668700	2675260	the two libraries. And it gives it to me in a table with a little button to export it out to
2675260	2683600	Sheets. Anyone who's done this manually, like seriously, right? This is the way. This is like
2683600	2690020	I said, oh, these things, they hallucinate. You can't use these, really. Because in ways like
2690020	2698120	this, this is it, right? This is absolutely it. My time is almost up, so let's talk about report
2698120	2702540	writing really quickly. Everyone's absolute favorite thing, right? And I used to tell my
2702540	2708120	analyst all the time, you can spend a week doing things that are clever and very, very impressive.
2708280	2712860	You're not being judged on any of it, right? The only thing you're being judged on
2712860	2716840	is what goes in that email that you push send on. So make it good.
2718840	2725600	Let's see how large language models can help us. First things first, right? You try to ask
2725600	2729540	chat GPT to write something for you, a lot of times it comes up with some good stuff, but it's
2729540	2734660	not nearly good enough to copy paste, right? You just basically, how I've been using it a lot of
2734660	2739280	times up until recently was asking it for a lot of ideas and then picking and choosing, mixing and
2739280	2744020	matching, copying and pasting, right? It is good with writer block. Let's look at a few things that
2744020	2749020	can help us though when it comes time to report writing. Number one is very simple. Identify a
2749020	2753780	writing sample that you think represents you well, right? It has your tone, it has the way you explain
2753780	2760040	things. Take that writing sample, save it in a text file. My buddy Josh has his saved in Obsidian
2760040	2764860	in there because that's where he's organizing everything. And every time he goes to ask chat GPT
2764860	2769620	to write something, he first gives it that writing sample. I'm getting ready to ask you to do
2769620	2774280	something. I want you to write it in this style. And it's just a copy and paste from, like I said,
2774380	2781480	a text file, Obsidian, wherever you want. And then you watch, watch how much better the output is.
2781480	2787100	It is dramatically better and it's in your style, right? So things that are your style, you just
2787100	2794660	tend to like more anyway. And for the final thing here, chat GPT isn't great at turning a six-word
2794660	2798920	prompt. Everyone's like chat GPT is getting dumber. You gave it six words and asked it to write your
2798920	2804880	thesis. Stop, right? It's not good at taking six words and turning that into some masterpiece.
2805160	2810160	But if you gave it a thousand words right now, now it's going to do good. But the problem is
2810160	2815380	I don't want to type that. Ain't no one got time for all that, right? So what we can do now is start
2815380	2821180	to combine some of these things. Record yourself explaining what you want. You can use an app on
2821180	2825380	your phone. You can use Whisper. I don't care what you use. Transcribe what you just explained into
2825380	2831800	text. Now give that text to chat GPT or BARD and now ask it to write what you want, be it a report
2831800	2837840	or something else. How many times do you go to send an email to a friend and you realize there's
2837840	2842120	no way I'm going to send this email. I'm just going to call them because this is going to take me 15
2842120	2847040	minutes to write the email or it's going to take me two minutes to explain on the phone. Why do we
2847040	2852020	think it would be any different with chat GPT, right? So if we can harness that, if we can take
2852020	2856880	our voice and explain what we want. I almost feel like I'm talking to a toddler, right? Use your words
2858740	2863640	and then ask it. Once again, a lot better results. So there's a lot of different ways we've talked
2863640	2868700	about here, right? Transcribing, translating, summarizes. And once again, as far as how we're
2868700	2872680	going to use this, it's on us. It's on the people that are in this room. It's on the people that are
2872680	2879080	watching online to think of how we can use this technology for good, right? If you want to get a
2879080	2883540	hold of me, I am the easiest person in the world to get a hold of, right? I shouldn't even insult
2883540	2891040	you. This is Reconnaissance Village. I am Matt0177 on what used to be Twitter, now X, whatever you
2891040	2896120	want, right? Matt at ArgeliusLabs.com. If you want to hang out and talk OSINT, I put some stickers on
2896220	2899320	a table over there. I don't know if they're still there. They're already gone. But we have a lot more
2899320	2903120	stickers. We have challenge coins. The room wasn't ready yet, but we're actually going to post up at
2903120	2907380	the hospitality suite at the top of the horseshoe. And I have stickers. I have coins. If you want to
2907380	2912360	hang out and talk OSINT, talk about the SANS class, whatever, let me know. And also, there's no way I
2912360	2918220	have a hospitality suite that is not fully stocked with alcohol. And when I say a case of bourbon,
2918220	2921880	I mean a case of bourbon. So like I said, I'm easy to get a hold of. If you want to hit me up,
2921880	2926380	hit me up. Slide into my DMs and let's chat. So thank you all for your time. I appreciate it.