start	end	text
1420	3440	Done. Okay, is that better?
4500	5200	Yes.
5200	5720	Nice.
5880	6780	That's good.
6780	10200	Nailed it. Thank you for hanging in there.
12620	15460	Okay, so I want to ask, what isn't machine learning?
15820	23760	So, as math underpins everything we do,
23760	28320	I think you get to a point where everything is machine learning.
28320	35660	So, if you think about when you're a child and you're trying to learn to walk,
35660	42280	it doesn't look that unfamiliar to when someone has reinforcement learning algorithms
42280	45160	that are teaching something to walk.
46160	49300	So from an offensive standpoint, I think it's an important distinction
49300	56000	because where offensive folks try to live in the realm of possibility
57360	64060	rather than in a box that we try to define.
64060	67900	So, maybe if you start thinking of everything as machine learning,
67900	71880	you're going to find more opportunities to attack machine learning models,
71880	75080	use them in your ops, whatever that might be.
75740	83280	But for this attack in particular, we're going to do a copycat model.
83280	88060	And I think this kind of concept will come out as we go through it.
88840	91940	So, I'm kind of tired of the name copycat model,
91940	94740	so I'm just going to call them Pink Panther attacks from now on.
95220	99780	One, it's more fun. Two, it's still kind of cat-like.
100120	105720	And three, I know Pink Panther involves a diamond that is always trying to be stolen.
106260	110020	And preconditions for a successful attack, and these are pretty loose,
110020	113060	are a representative data set.
113380	117240	So, for example, in the case of an AMSI provider,
117240	123900	we're looking for like PowerShell, VBA code,
124440	127280	a big representative data set that we want to model.
127620	131960	And we need the ability to get feedback from our target model.
131960	135760	And this doesn't have to be direct output.
135760	141840	So, if you imagine that a model doesn't give you back a score,
141840	144580	but it keeps it in a Windows event log,
144580	149000	you still get your hard label, hard label being a 0 or a 1.
149860	153860	So, even if the model doesn't give you back output,
153860	161700	there are still, just due to the telemetry nature of our networks,
161700	166900	it's more likely than not that somewhere output's being recorded.
167420	170840	So, rather than maybe going directly to the model,
170840	176220	try and think of some binary test that you can perform
177240	182760	that is potentially outside of the realm of a machine learning model.
183540	187600	So, it's a pretty simple attack once you kind of get into it.
187600	190960	Effectively, you grab a massive...
190960	195300	in this case, we found a big PowerShell corpus.
195640	199740	So, there's like 400,000 scripts.
200020	204760	And the workshop, the lab piece of it, we'll talk about that.
205020	210800	Effectively, the scripts get fed into an AMSI integration.
212080	214980	And this just came off of the Windows sample.
214980	219700	So, all it does is load AMSI.dll and instantiate the com object
220400	226600	and we feed it every script in that corpus.
227000	232940	And that gets passed to Defender or whatever AMSI provider that might be.
233280	236140	And it comes back with a score.
237120	241460	And then we turn around and we just collect those scores offline.
242400	248420	And once we have known inputs and we know what the output...
248420	253740	Defender thinks of the output, we can model our model.
254560	262240	And the hardest part about, I think, adversarial machine learning
262240	264280	isn't the machine learning piece.
264720	267940	Most of us don't have to invent math.
267940	274620	People much smarter than myself have already done that for me.
274620	277840	But what is difficult is the engineering piece.
277840	280420	It's getting access to the right data.
280720	286280	It's getting data in a way that isn't going to get you caught
286280	291300	or you're going to distribute your traffic accordingly.
291760	294700	And you can kind of fly under the radar.
294700	297800	In the future, currently attackers kind of have free reign
298440	302760	and get away with quite a lot of querying and noise in the network.
302760	304680	That's not always going to be the case.
305120	311380	And I think it is helpful for us to start thinking about limiting our queries,
311380	313900	thinking about information density of a command,
314760	319140	a ping versus a process list, for example.
320880	324260	Ping, you're going to get if a host is up or not.
324260	326780	Or you're going to get TTL, which is going to tell the OS.
327200	330360	But a process list, you're going to get that and more.
330920	333040	But it probably just depends on what you're after.
334840	339500	And from a machine learning perspective and an offensive perspective,
339500	341100	anything can be modeled.
341100	349120	So if defenders are using machine learning to model whatever logs they're looking for,
349120	354520	there's nothing to say that we as offensive folks can't also use machine learning
354520	357320	to say model CT traffic.
357320	361600	Or let's say you have semantic, you have some product that is blocking you
361600	365260	and you can't quite pinpoint it.
365880	368200	Machine learning will be able to do that for you
368200	371240	if you can set up the right experiment.
371300	378780	So even though Semantic is not using machine learning necessarily to look for your CT traffic,
378780	385360	you can still use it to find those relationships between whatever domain
385360	394820	or whatever callback interval that you're using against whatever product it might be.
395500	399340	So I guess I just come back to the question of what isn't machine learning.
401840	408280	And in my mind, everything is machine learning if you kind of go back to the math,
408280	410580	which is kind of fun.
410720	418240	Okay, so in the lab, the workshop, all the code is in there for you.
418240	426280	So the first half, you'll step through and it'll be very explanatory.
427380	432840	And then as we go through, it'll be less and less explanation.
432840	438800	So you'll be required to kind of look more, ask more questions, etc., etc.
439380	443360	Break stuff, like you're not going to break it, you can always reset it.
443360	445720	Google it, ask us for help.
445720	454640	AI Village has easily, probably the highest concentration of some of the smartest people in the industry.
454820	461600	And there's so many PhDs and super experienced people.
461600	466980	So, you know, we love talking. I like talking offensive ops, but you know,
467580	475000	a lot of people like talking stats and topology and homology and all this other stuff.
475600	479240	The other day I learned that there is more than one algebra.
481760	486000	But yeah, so this is the place to ask, so go ask for it.
486640	493380	As you go through, I would challenge those of you who are more experienced to produce a better and more efficient attack.
494320	499800	And in there, there is also a massive data set of VBA code that I found.
499800	501980	It's like nine gigs of VBA code.
502640	507580	So if you want, go through this code, it would work the same.
507580	515760	And you could get your name on that leaderboard for CVs for machine learning systems.
517280	520400	So if you want to do that, you can go for it.
522280	526680	So feel free to ask questions in Discord. There are no dumb questions.
527620	532440	And we have three TAs that are helping me out here.
532500	536100	So if you have specific questions, we can ask for it.
536260	540280	So hopefully you have the link.
541000	542540	You have everything.
543980	549100	So effectively, this is the... what do you call this?
549100	551080	This is the workshop.
553180	556440	And there are two files, two Python notebooks.
556440	561100	One is the solutions, and one is the workbook.
561960	563740	And you should start with the workbook.
563740	572940	If you do get stuck, or you need code help, or you just want to grab it and go, the solutions one is going to work for you.
572940	585220	This amsy.h file is if you do want to compile the amsy stream.exe so you can recreate the full attack path.
586000	588380	You're going to need this header file.
588900	593640	Collect.py is what we use to gather all the information.
593640	597580	And then this insights.xlsx, this Excel file.
597580	603860	This is the version 1 of the insights that we pulled out of Defender.
604520	609840	So this would be like that proof point style insight.
610580	615760	And it is our version 1, so this is it.
615760	619400	So obviously you can see there's a lot of binary blobs in here.
619480	622220	But these would be the most malicious tokens.
622860	625040	And this makes sense to me.
626600	633240	And at the bottom you have the least giant binary blob.
634900	643180	And at the bottom you have the least malicious tokens.
646080	648240	So that's free.
650560	654260	So you guys can get started.
654260	656400	Do you guys have any questions right away?
659220	661640	I haven't seen any questions so far.
663840	664880	Perfect.
670490	671790	That's good.
675890	678150	Is anyone else getting 403?
692220	694300	I have to say I haven't used Defender.
695440	697660	Is anyone else getting 403?
705650	709630	We are not seeing any chat on Twitch or Discord.
709630	714550	I just reopened the link and it's building the container for me.
714910	716030	Yeah, me too.
716030	717310	Okay, that's good.
719070	721030	I have my own copy.
721050	723770	I am in the Jupyter Notebook.
723850	724770	Nice.
725110	732630	Yeah, so the first little bit, I'll give you guys 10-15 minutes to rip through the first little bit.
732630	738850	And if you guys have questions, let me know.
741570	743510	Remote is always a bit weird.
753570	756610	But, you know what always gets me in the mood?
761760	762740	Can you guys hear that?
762740	764680	What always gets you in the mood?
765540	766380	Tron.
767140	770980	Yeah, I don't think it's a good idea to stream copyrighted music on here.
770980	772900	Oh, right, right, right.
774100	779860	Twitch doesn't really have a problem with that, but the YouTube VOD at the end will be removed.
779980	780920	That's fair.
781960	784600	Yeah, I don't spend that much time on the internet.
786380	790220	So, Will, I can send you the Notebook link again.
790220	793660	The one that myself and other people are using on the chat.
793820	796240	That's good. I have my own.
822000	822740	Excellent.
822740	826720	Okay, so I'm just going to talk.
826840	835260	Everything is pretty self-explanatory in terms of the instructions.
837820	841420	Can you zoom in a little bit? It's kind of hard to see on the screen.
845910	847070	Is that better?
849610	850830	One more.
854780	856080	That should be good.
857840	859800	Sorry, there's a one-year-old.
860320	862180	I can see it very clearly.
863980	868900	Excellent. So, I'm just going to talk. You guys feel free to run through.
870240	876880	So, sort of the basis of this attack is really the AnyMalware scan interface.
879340	885060	And Emsi is something they introduced in, I want to say, PowerShell 5.
885940	887720	A little sooner than that.
887920	902580	But effectively, it's just a DLL that gets called into PowerShell, UAC dialogs, any Jscript, VBScript, and now .NET 4.8 and beyond.
903500	917380	And inside, it has a number of blacklisted functions that whenever one of those functions is called, Emsi will get called to scan content.
918560	926020	And Emsi is not a security boundary. It's not a security product. It's an interface.
926020	940980	And so, all it does is collect whatever content is being put into the buffer or the string, and it passes it back to an Emsi provider.
941360	947920	And the Emsi provider then has an opportunity to scan the content and make some determination.
947920	951820	So, obviously, this workshop is about Windows Defender.
953280	962300	But there are several other providers, Emsi providers, that this attack would work against.
964180	971460	And that's, I suppose, kind of the nice part about machine learning currently is there's a lot of attack surface.
971460	981080	So, once an Emsi provider scans the content and decides what it is, there's a range of scores it can give.
981080	987160	Currently, it only gives back 0 or 1, 32, 768.
987540	992060	But in Microsoft documentation, they talk about a range of scores.
992060	1009080	So, if you think rather than a hard label as a 0 or a 1, eventually you'd get some sort of regression type continuous variable on the out.
1009080	1012700	And so, you'd get some heuristic, I suppose.
1016520	1025000	There's a great talk by, actually Microsoft, called Badly Behaving Scripts.
1025000	1028540	It's probably like an hour long.
1028600	1033520	But I think they gave it two years ago at Blue Hat IL.
1033920	1037760	And it's two engineers from the machine learning team.
1037760	1053140	And they're just discussing how Emsi works, the things that they're looking for, the kind of models that they're building, and so on.
1053140	1060640	And so, this is kind of our first indication, at least officially, that machine learning is being pushed onto the client.
1060640	1071460	And I think it would seem anyway that overtly malicious things get stopped on the client.
1071460	1078860	But anything that's kind of in between seems to go to the cloud.
1078860	1083540	One way to test this would be to have some sort of timing attack.
1083540	1090840	You could get the response times for submission to the reception of the score.
1091140	1102600	And you could maybe see if there's a significant difference between the worst scripts and the medium bad scripts.
1105310	1108330	So, moving on. You guys can obviously read.
1109910	1117370	So, the first thing we obviously needed to do to create a copycat is we needed a data set.
1117370	1127130	And there's a talk by Lee Holmes and Daniel Bohannon in 2017 called Revoke Obfuscation, Powershot Obfuscation Detection Using Science.
1128210	1139370	So, Daniel Bohannon wrote Invoke Obfuscation, which basically takes a script or any number of different languages
1139790	1143550	and it puts it through an obfuscation.
1143950	1148390	So, it breaks it up into a million different pieces.
1148390	1152990	So, it'll break those regexes, those brittle detections.
1152990	1155470	It's an awesome, awesome tool.
1155470	1157690	These guys wrote it, wrote the talk.
1157690	1163090	And part of that research, they collected like two gigs worth of PowerShell scripts.
1163550	1169570	And they labeled them both benign and malicious for us.
1170310	1173470	And so, they're already... you can actually... the link's here.
1173470	1176130	So, you can go pull it down and rip through it.
1177790	1186070	Once you have that, the rest of it is just getting target outputs from the target models.
1186110	1194810	So, if we want to create a copycat, we want to know what Defender thinks of each script that we give it.
1195090	1198110	And then, that is remodeling the model.
1198110	1200030	So, we have the inputs and we have the outputs.
1200030	1201770	We don't know what's in the middle.
1201830	1203210	We don't know that black box.
1203210	1208590	But we can infer it with our own model.
1208630	1211530	And you'll never get 100% of the model.
1211530	1224790	But you might get just enough to bypass Windows Defender for a month or just one time or whatever it might be.
1224790	1237290	But the nice part about machine learning and the struggle that machine learning has, I think, is it does introduce a probability into what used to be a static decision.
1240850	1245050	The large repository of something interesting is VBA.
1245050	1250730	So, that's just 9 gigs of Excel macros that you could do the same with.
1252850	1254610	Which would be pretty awesome.
1258830	1262750	We have... let's see, what is this?
1262750	1267370	This is the... you're not on Windows.
1268250	1273150	If this were like Lab Lab, we'd have Windows VMs and we'd do this for real.
1273390	1276310	This is just the output from the AMSI stream.
1276430	1282270	So, InvokeWMIBackdoor looks malicious, but Defender doesn't think it's malicious.
1282270	1292590	So, if you look down, you see ScanResult is 1, IsMalware is 0, IsMalware is the official feedback loop.
1293410	1299730	And even in that talk that is above, they discuss the fact that you can't trust filenames.
1299730	1303330	And that makes sense. You can't trust headers anymore.
1303970	1315550	There's some research that did a number of, like 3 years ago now, against mail filters where you would null out the first 2 bytes of a .m file.
1316150	1326430	And depending on how the mail filter decided what kind of file it was, it would either block it or let it through.
1326430	1329550	Because it can't read the magic bytes at the top.
1330330	1334310	And then, obviously the document is corrupt at that point.
1334310	1339930	But what Windows does is when you open the .m file, it would ask you to repair it.
1340410	1349790	And if the user clicked yes, I'd like to repair this document, your macro would live and you could get code execution.
1351790	1356510	Right, but the next bit is the provider display name.
1356510	1358990	So we have Microsoft Defender Antivirus.
1359850	1361790	If this is...
1367110	1374810	If Defender is turned off for whatever reason, or there is no AMZ provider, it will give you an error.
1375090	1377730	And you will not get a score back.
1378430	1381770	The next piece is just PowerView.
1381770	1385450	So PowerView is pretty well known script, and it is definitely...
1385450	1392810	Well, it's not explicitly malicious, but it is used by malicious people.
1394390	1396190	That's not fair to say.
1396250	1398250	It's used by attackers.
1398630	1401710	I'm sure there are some nice attackers.
1403910	1405980	So in this lab, there are...
1407290	1412410	Uploading 380,000 scripts to GitHub was not popular with GitHub.
1412990	1415610	So in this little bit, you only have 3,000.
1415630	1419010	But the collect.py has everything you need.
1420090	1424210	And when we were parsing, we just did...
1424210	1427390	In your data, you have data clean and dirty.
1427890	1430070	These would be your malicious scripts.
1430070	1436740	And clean scripts are actually clean.
1436740	1438120	There's like 3,000.
1438120	1444280	There are about 1,200 malicious scripts that you can look at.
1444740	1448640	Yeah, if you guys want to start ripping through this code.
1451000	1455080	As we were parsing, you can look through collect.py.
1455300	1458200	But I personally like to keep...
1458200	1461380	So there's a lot of moving parts in machine learning.
1461380	1463560	You kind of can't get away from it.
1464060	1468560	So whenever possible, I like to keep at each step.
1468560	1473060	I'll build a data structure and I'll keep the previous output.
1473880	1475320	And I think it works nicely.
1475320	1477360	Most of my datasets are pretty small anyway.
1477440	1479280	So it's manageable.
1479620	1482820	But you can kind of see the GetScreenshot.
1482820	1486800	So this is the file name, the original file name.
1486800	1489580	This is the hash, the MD5 hash of it.
1489580	1493760	And then you have the result, being its malware.
1494000	1499080	And then you have the base64 encoded text that you can look through.
1499080	1502720	I like this because if there's any weirdness,
1502720	1505960	you have everything you need right in front of you.
1506000	1509640	You don't have to start again at a particular point.
1509640	1512320	You don't have to go back to the beginning of the process.
1513500	1517580	Being able to debug all the way through your pipeline is...
1517580	1520500	It might cost you some speed.
1522280	1525240	But at least for me, it's fine.
1525240	1527460	I'm not dealing with billions of anything.
1531600	1534140	We just got discussed lists.
1534340	1537180	So if you're not familiar with a list,
1537180	1541280	which I think probably most of you are,
1541860	1544500	raise your hand if you're familiar with lists.
1546000	1548420	Nice, okay, so about 20% of you.
1552420	1554040	Can't hear any giggling.
1554840	1557600	Alright, so you have lists. Lists are really nice.
1557600	1560160	So you can just put them on any delimiter you want.
1562280	1564860	So quick question, Will.
1565020	1569380	This is from the Twitch chat.
1570200	1574020	Should he be able to view the PowerShell code?
1574020	1577320	When they click on a file with clean or dirty,
1577320	1579620	get a mess of characters but not a script.
1581160	1583760	Yes, so you...
1583760	1587340	Yeah, so in this little code block here,
1588000	1589200	you can open it.
1589200	1591380	And it is just a...
1591380	1593620	So we've already parsed it for you.
1593720	1595960	It was just going to take forever otherwise.
1596320	1600460	So what you're going to see is the script name.
1601080	1603680	So this is the original script name.
1603680	1607580	The MD5 hash of the script content.
1608550	1612030	The AMSI result, or I should say the Defender result.
1612300	1614760	So what Defender thinks of the script.
1614980	1619900	And then the Base64 encoded version of the script.
1623900	1627680	Can you turn lines on?
1635820	1637940	So this is what we're doing here.
1637940	1640380	So we're splitting a file.
1640380	1642380	So what we're doing is we're setting up...
1642380	1645900	Eventually we're going to rip through all these files
1645900	1648340	and we're going to build a big vocab.
1648700	1650360	So this is a...
1650360	1653060	I just pulled one out as an example.
1654840	1655920	So let's see.
1656820	1659000	So for example, if we want to reference...
1659000	1660740	This is just list stuff.
1660740	1662080	So obviously zero index.
1662080	1665140	So we can reference it however we want.
1666480	1669260	You can then reference with semicolon.
1669300	1672600	You know, from index one to the end.
1672600	1676480	Or from index one to the third.
1678740	1681260	Or you can even do minus.
1681260	1683720	So you can go to the end of the list.
1684360	1686740	And just get the script content.
1687560	1690740	But if we're going to look at the script content,
1690740	1693740	we kind of want to know...
1695580	1697880	It's interesting that getScreenshot
1699000	1701140	is counted as malicious.
1702120	1705120	So if we decode it,
1705120	1706820	we can have a look and maybe...
1706820	1708500	Maybe there's some knowledge that we have
1708500	1713800	that we can think about as to why it might be malicious.
1717780	1721160	So we're just using split to make it a little nicer.
1721160	1722740	And there's a typo right there.
1724980	1726580	It's a double typo now.
1728780	1734130	Alright, so we're just going to run through it.
1734190	1735230	Actually, no, it's not.
1737770	1739230	So to make it nice, actually,
1739230	1740730	this isn't the solutions one,
1740730	1742110	so you guys already know this.
1744330	1746430	So if you want to make it a little nicer,
1746430	1748310	we're just going to split it again.
1749670	1754930	And we're going to get a list of the things.
1754930	1756610	So, looking through this list,
1756610	1759650	in my mind, some of the more malicious things
1759650	1761770	would be the AddTypeAssembly.
1762590	1766530	It would be ConvertToInt32,
1768470	1769110	NewObjectSystemIOMemoryStream,
1769110	1771450	so everything like in-memory attacks,
1771450	1772950	any sort of memory streams.
1772950	1775270	And then at the bottom, you also have
1775270	1777790	two Base64 string.
1779370	1781730	I think Base64 is typically something
1781730	1784370	that gets picked up quite easily.
1786430	1787750	But if we wanted to figure out
1787750	1790690	how this was...
1790690	1792570	why this was malicious,
1792570	1796270	or why Defender thought this was malicious,
1796270	1797750	does anybody have any ideas
1798850	1800850	that they want to type out?
1800850	1813620	Is anybody typing? I can't see.
1819720	1821480	Yeah, it doesn't seem like there is
1821960	1824020	anybody typing on Discord right now.
1824020	1824740	Nice.
1827420	1829840	Yeah, so we're just going to go through the dictionary.
1830560	1833660	So, this is just what I came up with,
1833660	1835500	and it would be to try and determine
1835500	1839500	the commonality between all the malicious scripts
1839500	1843500	and see if... see what tokens...
1844760	1847180	and when I say tokens, I mean words...
1847720	1849640	words came to the top.
1850100	1851640	And to do this, we're going to use...
1851640	1853040	we're just going to do a dictionary.
1853840	1855440	And it's kept, you know,
1855440	1857960	key-value pair.
1858600	1860720	So I'll let you guys run through that.
1871360	1873780	And if you need to play around with dictionaries,
1873780	1875440	they're pretty awesome.
1875440	1878860	I like dictionaries,
1879440	1881220	but there is one other data structure
1881220	1885240	that I like even more than dictionaries.
1886660	1887640	Excellent.
1887640	1889700	So, we know what dictionaries are now.
1889700	1892120	And we're just going to decode the content,
1892120	1895660	and we're going to add it all to a dictionary.
1896180	1898700	We're only going to do one script.
1899100	1901660	And does anybody know why we might
1901660	1904920	just only want to do one for now?
1913660	1916460	Because it's good practice?
1916460	1917800	Yeah, it's good practice.
1917800	1921120	But when you are dealing with, I don't know,
1921120	1922980	thousands of scripts,
1922980	1925320	like half a million scripts,
1925320	1928680	if you have something that's going to mess up
1928680	1932960	halfway through, or you start down that path,
1935140	1938280	you're just going to waste a lot of time fixing errors.
1938280	1940400	So, if you can get it to work with one,
1940400	1944640	and then ten, and then fifty, and then a thousand,
1944640	1947220	I think that's a much better way to go about
1949960	1954300	processing large data, large data sets.
1954300	1956140	So, for this one we're just going to do one,
1956140	1959120	but we're going to split it twice,
1959120	1962300	and then we're going to just add everything
1962300	1965520	to the word index.
1965520	1969560	So, there's a fill-in-the-blank bit here.
1970120	1972020	So, if you guys want to go ahead and run that code,
1972020	1975000	and then tell me what the issue is.
1982290	1983430	And I'll give you clues.
1983430	1985630	It's on the line with all the question marks.
1987230	1989670	I didn't know that I needed to be this close to the microphone.
2001940	2004960	Or you guys are already way, way, way past this.
2020850	2024090	So, to get a proper count of the words
2024090	2026590	as we go through them, we need to just add one.
2027610	2029470	And you would have been able to tell
2030830	2032610	after you went through it,
2032610	2039950	because there's only one token listed for each.
2042940	2047400	But the script has at least more than one.
2047400	2049740	So, if we just fix that,
2054110	2056510	now we get a better representation
2056510	2058550	of what's out there.
2060030	2062770	Going one at a time, I guess I've found
2062770	2065870	I used to have a real issue with loops,
2065870	2070750	where I wouldn't break them up properly.
2071670	2074610	And so, doing it a little more slowly,
2074610	2076630	at least one at a time, has helped me.
2077470	2079710	And then we're just going to sort the dictionary
2079710	2081610	by token.
2082010	2086010	So now, this equals sign,
2086010	2088070	there's nine of them,
2088070	2091130	there's six curly brackets,
2091130	2093350	there's four new objects,
2095170	2099690	but nothing particularly malicious.
2100110	2104950	So there is a lot of punctuation in there,
2106210	2108970	but we can deal with that later.
2108970	2112010	So, now, because we're impatient,
2112010	2113950	and we just want to get to the machine learning bit,
2113950	2118090	we're going to run through the entire...
2119090	2121630	all of the malicious scripts.
2126890	2129670	And it might take a little bit on Binder.
2137010	2138890	Might even take a little bit here.
2140930	2142250	The... yeah.
2142630	2144630	I wish I... I tried to ship
2145150	2148130	all of the scripts to you.
2153180	2155700	Okay, so now we're just going to sort the words.
2155700	2157780	So this is all of the tokens
2157780	2160900	for all of the malicious scripts.
2165000	2167040	And it isn't ideal.
2173510	2175910	So there's a lot of numbers,
2175910	2177190	and this is fairly common
2177190	2179310	when you're tokenizing text.
2180070	2183050	And I would generally just say go a little slow.
2183050	2185710	But, nice bit is, we're seeing get proc address.
2186370	2189070	Does anybody know...
2189690	2191570	you know, in what operation
2191570	2193690	get proc address might be used?
2198070	2200490	Does anybody that works in an AV vendor
2200490	2203190	know what get proc address might be used for?
2213010	2215050	You can Google it if you want.
2223250	2224710	No? Nobody?
2225230	2226150	GT?
2227270	2230150	Any penetration testers?
2230950	2233450	Malware authors? Anybody?
2235640	2237460	I'm a data scientist.
2237700	2239680	That's fair.
2240120	2242300	I'll let you look that up.
2242380	2246120	But it's used typically when you're looking up functions
2246120	2250080	in other DLLs.
2251320	2252640	Most notably
2252640	2255460	I think used in
2257800	2259600	process injection.
2260280	2261480	Anyway.
2263760	2265620	Alright, so there's a lot of numbers.
2265620	2267020	It's fairly classic.
2267780	2270360	But we're seeing write bytes to memory,
2270360	2273960	memory address, a lot of malware type things.
2275140	2277160	Remote proc handle.
2277700	2279820	Obviously this is all PowerShell.
2280480	2282540	But the numbers are actually kind of annoying.
2283960	2284760	So...
2286920	2289620	Let's scroll through.
2289620	2292160	Are there any tokens in there
2292160	2295780	that are particularly interesting to anybody?
2309840	2314040	Rob, Logistic Aggression managed to answer your question.
2314040	2315780	Thanks, Rob.
2315780	2317260	Rob's the man.
2317260	2321020	What was it?
2321580	2325400	He said you get the address of a DLL function in memory.
2325400	2327660	Also heavily used in packers.
2328360	2329900	Yeah, exactly.
2332120	2334600	Okay, so the next bit.
2334740	2337920	We're just going to continue to filter down.
2338320	2340560	And I would say that's generally the case
2340560	2343060	with most of your data.
2343060	2345500	I think most of your time spent
2345840	2347200	when you're doing machine learning
2347200	2353020	isn't the math piece that you typically think of.
2353020	2355280	It's processing data,
2355280	2360180	it's filtering data,
2360180	2361780	it's making sure that
2361780	2364240	your data sets are balanced
2364240	2367400	or have a distribution that you want
2367400	2370100	or whatever it might be.
2370100	2373860	I said earlier that we're not inventing new math.
2373860	2377640	I'm not going to be the guy to invent new math.
2379920	2382520	The place that I can be most effective
2383100	2385720	is applying my domain knowledge
2385720	2388200	to what I know of machine learning
2388200	2392260	and being extra careful with my data.
2393300	2395000	Abraham Lincoln has a saying
2395000	2396720	about sharpening an ax.
2396720	2398760	I can't remember exactly what it is.
2400700	2402860	If I had 10 hours to cut down a tree
2402860	2405600	I would spend the first 8 sharpening my ax.
2406160	2407980	That sounds right, I could have made that up.
2408960	2411640	The same is true for machine learning
2411640	2413260	and I would say data science.
2414120	2415160	I don't know if
2417440	2418760	Comath, do you want to chime in there
2418760	2420780	as a data scientist?
2423100	2424760	It's true or not?
2429500	2434180	I tend to try something that works
2434180	2435740	and then go back and perfect it
2435740	2437180	to make sure I don't get little mistakes
2437180	2439580	because when you scale things up
2439580	2442380	it really screws you.
2442380	2445220	There's a bug 0.1% of the time
2445220	2447260	and now you have 100,000 things
2447260	2449780	the bug is guaranteed to happen.
2451020	2452360	It's better to clean those up
2452360	2455520	before you really...
2455520	2459920	If you're a data scientist
2460760	2462060	in the twitch chat
2462060	2464660	what do you think the percentage of cleaning
2464660	2466280	versus actual machine learning
2466280	2469540	data processing versus machine learning
2469540	2470960	I think that would be interesting
2472260	2474040	and if you're new to machine learning
2474040	2477540	I would say it's okay
2477540	2480740	to be impatient and go really fast
2480740	2482040	because you want to get the end result
2482040	2485520	but if you want to get quality results
2485520	2488320	it is better to go a little slower.
2490720	2492240	We're just going to create a new list
2492240	2495320	and the difference here is we're creating a find all
2495320	2497540	so we're going to start removing some punctuation
2497540	2500280	and numbers and then we're just going to add it
2500280	2502360	to a new list and then we're going to print
2502360	2505580	the top 100 tokens.
2508480	2510020	So all the numbers are gone
2510320	2511600	a lot of the punctuation is gone
2512120	2513800	I'd say that looks quite a bit better
2513800	2516300	than the other one.
2517080	2519560	We still have a getproc address
2521760	2524720	we're still seeing the malicious tokens
2524720	2525620	that we would expect
2525620	2526480	shellcode
2527420	2530580	so we haven't completely ruined our
2532320	2534900	I want to call them crispy bits
2534900	2537800	but we haven't ruined our
2537800	2540360	what we want to model effectively
2540360	2543500	so that's also there.
2546400	2546740	So this bit
2546740	2549920	I just said pick some tokens and go google them
2549920	2552320	and make sure that they're
2553460	2555140	see what comes up
2555740	2556960	that's kind of the fun bit
2558980	2561740	I suppose if we are live I can do that
2562460	2563340	let's see
2567760	2569740	that's what pehandle does
2573920	2575160	I'm using edge
2575160	2576540	that's embarrassing
2584400	2586700	Have you guys heard of Mimikatz?
2588420	2590240	Do you know if that one's malicious?
2594600	2596000	Yeah, I would say so.
2597100	2598260	PowerShell Type Loader?
2598260	2599680	Mimikatz is flagged as malicious, yeah.
2599740	2601760	Yeah, yeah, definitely malicious.
2604780	2612500	And I would say, as an operator, I think a lot of the industry uses a lot of the same scripts.
2612500	2619910	So, yeah.
2620910	2625890	In fact, the DLL injection, bookdllinjection, atgraver...
2626730	2630710	Yeah, so these are malicious type scripts.
2632330	2641140	Okay, so once we have that bit, you know, you could...
2641140	2642780	We could stop here, right?
2642780	2646900	So we have some malicious tokens that we know that, you know,
2646900	2653200	these are all tokens or words that were labeled by Defender as malicious.
2653260	2659980	So we could stop here, theoretically, and have malicious scripts that never, you know,
2659980	2661900	don't use any of these words.
2662100	2666040	Alternatively, you could do the same with the clean scripts
2666040	2670560	and only use, you know, words from those clean scripts.
2670560	2672280	Or, you know, you could do both.
2672280	2676200	But I think that would probably work.
2677540	2682840	But I think you would have a hard time making it repeatable.
2682920	2685410	And you wouldn't necessarily, you know, if something...
2685920	2691620	If it all of a sudden stopped working, I don't think you would have much recourse
2691620	2694200	in being able to figure out how or why.
2695320	2699980	And so I think this is kind of where machine learning shines for us.
2699980	2707440	And it's just the ability to iterate through massive amounts of data extremely quickly.
2707940	2714880	So, you know, while... I'm going to use the GPT-2 fishing analogy.
2714880	2725000	And it's like, you can spend an hour crafting, you know, one fish for five targets.
2725000	2734260	Or you could spend an hour correcting five unique fish generated out of GPT-2 for five targets.
2735580	2742320	GPT-2 language models aren't perfect, but they do help scale.
2743200	2746580	You know, so it's like you generate five of them and you can correct them,
2746580	2752160	make them seem, you know, realistic and not sound terrible.
2752840	2755740	In the same time you could to do one.
2755740	2761620	So, you know, in terms of pulling out insights from Windows Defender, I think this is also true.
2762200	2766580	All right. So we're getting to... we got to the machine learning bit.
2767360	2772460	And in this manual, you know, we referenced 380,000 scripts.
2772460	2778760	There are, I think, 410 in the whole piece.
2778760	2784080	So in that PowerShell link at the top, you know, it has that many.
2784740	2788620	We just use... I just chose the biggest folder that had them in there.
2790560	2793780	And then introduced them to Insight.
2795960	2799060	So we're going to get into some of the data representation.
2799060	2804300	I think data representation is probably my favorite part of machine learning.
2804300	2814260	It's really where you get to shape... kind of shape the output and shape...
2814260	2816840	not the model, because the model has its own architecture,
2816840	2824060	but you get to encode and embed your domain knowledge when you...
2824060	2829620	you know, I guess it would be called... it's fine if I'm wrong,
2829620	2832240	but you could call it feature engineering.
2833820	2837720	And that piece is extremely important.
2837720	2843580	So, you know, if you want to...
2843580	2852140	the outputs of your model are... the accuracy of your model is a direct representation,
2852140	2857260	correlation with the quality of your data representation and your feature engineering.
2858320	2860400	And it's... yeah, it's my favorite part.
2862020	2864880	Okay, so...
2866940	2872320	GT went through tokenization earlier today,
2872740	2875140	but we've already kind of played with tokenization,
2875140	2880140	but tokenization is effectively the process of splitting,
2880140	2885240	you know, the words into separate words.
2885240	2888440	So, you know, each...
2888440	2891420	I'm losing it.
2891800	2893240	Not into separate words.
2893240	2896560	You piece out sentences into a list.
2896560	2900360	So once we have a tokenizer, rather than...
2900360	2904460	we could write it ourself, like we did up here,
2904460	2908540	but I would imagine there's probably better developers out there,
2908540	2914860	and I trust them probably more than I trust my own code for some reason.
2915180	2916720	But if you imagine it as a corpus,
2916720	2921040	so if you imagine this corpus is actually just our big PowerShell list,
2921040	2924320	so each line is a PowerShell script,
2924320	2926320	you know, just a list of it,
2926320	2928240	and we're going to fit on text,
2928240	2930820	and this is just to create an index.
2931180	2933960	So we're just creating those data structures,
2933960	2941120	and you can see, you know, each word gets its own integer.
2941860	2945140	And, you know, in terms of sequential numbers,
2945140	2948140	like in a text prediction scenario,
2948140	2949960	like sequences are really nice,
2950560	2954520	because you can say, you know, what number comes after what,
2954520	2957680	and you can go back to your index and look it up.
2963220	2966980	Yeah, so if we can go, my cat likes mittens.
2969280	2972820	I don't have a cat, but I assume cats like mittens.
2974380	2976980	Or most cats are called mittens.
2978740	2982520	But we're just taking the first index and we're tokenizing it.
2989920	2992080	Yeah, six small cats.
2992540	2994300	Yeah, it's a word index.
2994460	2995900	It's fairly straightforward.
2997460	3001460	And there are a number of ways to represent text.
3001580	3003700	You have machine learning models that'll do it,
3003700	3009400	you have frequency models,
3009400	3012520	you have TF-IDF,
3012520	3018460	which is Term Frequency Inverse Document Frequency,
3018460	3021460	which is like a weighted frequency.
3021800	3023200	What else do you have?
3024260	3025740	What am I missing?
3028260	3031600	Obviously, one-hot encoding, that's where we're going to go next.
3033340	3039120	So one-hot encoding is, basically, it's a vector,
3039120	3042240	which is the length of your vocabulary.
3043200	3049640	And the presence of a token or a word is denoted by a 1.
3049640	3052660	And if it doesn't exist in a document, then it's a 0.
3053200	3055240	And so what you end up getting is...
3055240	3058280	Let's see how long these are...
3058280	3059400	Well, it's only 17.
3059400	3066960	So each vector is going to be 17 integers long.
3066960	3070660	And the array is going to look like this.
3070660	3073580	So this is just the first sentence.
3073580	3079960	But as we go through, you'll see as the words change,
3080520	3083480	you get different... what do you call it?
3083560	3085920	Not indications, activations?
3085920	3088460	That's probably incorrect.
3088960	3092020	You get different indications.
3094880	3098260	Hopefully that makes a little bit of sense.
3099520	3101380	And let's see...
3101380	3110960	The nice part about one-hot encoding is that you do get every token in.
3111880	3117420	Once they're well represented, you do get some sparsity.
3117420	3126180	So if you have a really big corpus or vocabulary, you get some sparsity in there.
3128040	3132060	But it's really nice if you only care about the presence of a word.
3132060	3140160	It doesn't necessarily keep the semantics or the syntax.
3143060	3146500	It's not very good for text prediction.
3146500	3149760	It is good for classification.
3150640	3152100	Which is kind of what we want to do.
3153120	3158580	Alright, so now that we have an idea of what tokenization does and the tokenization scheme that we're going to use,
3158580	3164240	we're going to get into my favorite data structure, which is named tuples.
3164980	3172480	Named tuples are immutable... well, they're tuples, but they're like immutable classes.
3172480	3177940	So you can kind of create them, you can set your variables to whatever you want,
3177940	3182440	and it's really nice because then you can turn around and reference a tuple,
3182440	3185900	or reference a variable inside of a tuple, by name,
3185900	3195780	which makes your code read a lot cleaner than if you were just going to use a tuple or a list or even a dictionary.
3195780	3200660	Dictionaries can be... actually, I don't know the speed difference between dictionaries and named tuples.
3200660	3202120	Does anybody know?
3203780	3204840	If there's any?
3206660	3207740	Maybe not.
3208300	3210140	But this is an example of a named tuple.
3211860	3216480	It's like when you find something you like and you just use them absolutely everywhere.
3216780	3218660	This is where I'm at with named tuples.
3220800	3222460	Just because they're super nice.
3224890	3225970	Alright.
3226390	3232250	So this code, we are going to kind of create our training structure.
3232350	3237430	So this is... you should be familiar with everything in this code block by now.
3237430	3243370	So we are... we have our filter.
3243550	3246430	The tokenizer has a filter attached to it.
3247390	3252870	If you want to use that one, we're going to create a primary list to hold all of our training data,
3252870	3257990	and then we're going to hold a... we're going to create a list that we're going to use for tokenizing.
3258630	3262850	And this is kind of... eventually, when you start building these small pieces,
3262850	3265970	eventually there's a point where they kind of all come together,
3265970	3269830	and you can transition from processing into learning.
3270110	3273290	And this script here is where we do that.
3273650	3278850	So, all of a sudden we can... I shouldn't say script, but...
3278850	3284370	So we've built all these tiny pieces out, and you can kind of see how they're coming together.
3284710	3290510	And there's two really important things happening in this little code block.
3291350	3302750	Obviously the tokenization is happening, and we are collecting all of the... all of the named tuples.
3302750	3303770	So I know named tuples.
3303770	3307050	So we're going to go through every malicious script.
3307050	3309570	We're going to decode it as we did before.
3309570	3311610	We're going to remove all the punctuation.
3311610	3312770	We're going to split it.
3312770	3314170	We're going to unique it.
3314790	3320050	I did this because I assume it makes tokenization faster, but I don't...
3321010	3322750	As I said, I don't have any evidence for that.
3322750	3324530	It just seemed to make sense.
3325990	3327870	Maybe you guys can try and let me know.
3329350	3335490	We are going to then rejoin the unique tokens, unique words in a sentence,
3335490	3343350	and we're just going to add them to a named tuple, and then add that named tuple to a list.
3343350	3347390	So the important piece here is the label.
3350410	3358410	For malware, the label is a 1, and then for clean, labels are a 0.
3358410	3362110	So this is where we're kind of creating our training structure.
3362930	3373110	The other piece you might notice is originally we had 380,000 scripts that we ran this on for this demonstration,
3373110	3376770	this workshop that GitHub didn't really like it.
3377370	3383610	So we only gave you 3,000, but that's still like a 2-to-1.
3383610	3391630	But when you're having 380,000 scripts versus almost 1,200 scripts, your data is very unbalanced.
3391630	3400070	So if you, for example, were to do a similar scheme as we did earlier with just the malicious scripts, we did it for both,
3400070	3407630	what might end up happening is your clean script words will start to...
3409510	3411770	The word isn't drown out.
3414430	3416290	What's the word, Eric?
3416390	3417190	Spin.
3417190	3418990	What's the technical term?
3424400	3425980	I'm going to say drown out.
3428220	3430820	Or average out, whatever it might be.
3430820	3435720	It's like if your GPA is really bad, eventually you just can't get it up.
3438420	3439120	Anyway.
3439200	3440940	Alright, so then at the end of this...
3440940	3450220	The outcome of this is we're going to get a list that has all of our named tuples that are ready to be fed into a model,
3450220	3453840	and then we're going to have all of our documents that are ready to be tokenized.
3453840	3457780	So at the end of this, we're just going to build the vocab, and we're going to have everything.
3458780	3460420	This is a little blurb.
3463060	3466000	So it takes a little bit to go through all of them.
3477480	3478940	It's because I opened this.
3483050	3484570	I don't want to tokenize that.
3486130	3495210	Also, you guys probably know this, but if you guys use Vim, it creates a swap file.
3495210	3510430	So I helped debug some issues where they were opening their data files inside of a folder and running Python out of the same folder.
3510430	3516030	And so their Python was trying to tokenize their text, and they couldn't figure out the error.
3516030	3519070	And it was just because they were trying to read a swap file.
3521370	3523950	So now we have our vocab.
3530570	3534050	And we can kind of see the tokens that came out.
3534050	3538230	These are the tokens after our filtering.
3538730	3550810	The closer you get to training, you want to be increasingly happy with what you're seeing in terms of the words or the tokens.
3551230	3555090	And I think that I'm more happy with this than when we started.
3555710	3556970	But definitely play by ear.
3556970	3560930	But it is useful to take a moment and see.
3561010	3562050	Just take a look.
3564290	3566110	Remote DLL handle.
3566110	3566810	Just take a look.
3568050	3569550	There's quite a lot of them.
3571110	3577730	So this is something I haven't dealt with, and you can see in the insights.xls.
3577730	3581910	But if you were going to deal with it, this is probably how you would do it.
3582570	3587590	So you just want to do a regex for this TVQQAAMA.
3587590	3589770	Does anybody know what that is?
3590450	3591590	I bet Rob does.
3605220	3607340	I see you're binging it.
3607340	3610740	Yeah, this is embarrassing.
3612480	3614360	Bing, the best way to Google.
3616360	3620620	But yeah, you'd want to remove those obviously.
3621560	3623100	I didn't.
3623820	3627100	It indicates that it is a PE file.
3631770	3637510	Compressed or embedded in some other medium.
3638190	3640790	Okay, so now we're at the point where we are...
3641310	3643770	Well, I already described this, but...
3643770	3647930	If you're not familiar with machine learning at all, whatsoever,
3649370	3651750	there's a little video series by...
3653010	3654510	What's his name? I don't know.
3654510	3657270	But his YouTube channel is 3Blue1Brown.
3657270	3659470	It's just like 4 videos an hour.
3659530	3661150	It is...
3662490	3663690	It's really good.
3663690	3668790	It's going to be way better than I will ever be able to explain machine learning to you.
3669430	3672950	So I recommend having a look through that.
3673670	3674630	Watch it a couple times.
3674630	3676850	And if you're really into math, he has a lot of cool stuff.
3678350	3681290	Yeah, his animations are really good.
3683290	3686770	Neural networks, we've kind of all seen this picture.
3687550	3690150	But our input layers represent our input data.
3690150	3692610	Or our one-hot encoded text.
3692630	3696070	Hidden layers represent an activation function.
3696610	3701170	And the output layer or node is the result of the network.
3701170	3703630	Or classification, prediction, whatever it might be.
3704910	3707290	There are a lot of...
3707290	3712670	Machine learning is more than this little picture that we see everywhere.
3714130	3722590	And I would say this is like the tiniest little bit of machine learning.
3722590	3730530	And even inside of this picture, the mechanics that are going on are quite extensive.
3731490	3736590	And this is just like the most vanilla machine learning implementation.
3736590	3750550	I think the nice part about Keras and the frameworks is they bring that kind of ability and power to non-mathematicians.
3752090	3755510	What do you call non-mathematicians, Sven?
3757050	3758530	I don't.
3758530	3760730	Is there an industry word for them?
3764710	3765590	Students?
3766550	3766990	What?
3767470	3768350	Students?
3769450	3770490	Yeah, something like that.
3770550	3771170	Normies.
3771370	3772750	Normies, yeah.
3773270	3777010	Honestly, we just don't think about non...
3777010	3778010	Yeah.
3778270	3780450	I don't want to dig myself into a hole here.
3780530	3782710	You've got too much math to think about.
3782790	3786110	As a non-mathematician, I take great offense to this.
3787010	3788290	Yeah, yeah.
3788290	3789810	Laypeople is right, yeah.
3789810	3791730	Rich has got it right. Laypeople.
3791990	3793270	Yeah, laypeople.
3793510	3800830	And if you are... I think the first book I read was Make Your Own Neural Network by Tariq Rashid.
3802370	3805790	And that was... it was just the most... it was like 120 pages.
3805790	3813830	It was just the most basic, most straightforward explanation of a neural network.
3813830	3828750	I think there's a bajillion tutorials out there, and I think they are okay, but they're always so quick to get into a framework without actually explaining what's going on underneath.
3830010	3832090	And that is definitely a hindrance.
3832090	3837890	You don't have to be a mathematician to use machine learning, but you should at least spend time learning the basics.
3840890	3841770	Okay.
3843010	3843770	So.
3844830	3845830	Let's see.
3845830	3847570	We have a little explanation of machine learning.
3847570	3849290	We have our training set.
3849290	3851250	We're happy with the tokenization.
3852370	3853690	To a point.
3853870	3855830	Happy enough for now, anyway.
3856270	3858370	You know, eventually...
3858370	3862690	I think you always want to be moving forward, so you always kind of want to be thinking in pipelines.
3862690	3877650	So I wouldn't spend too much time necessarily in one area, but when you build it, make it such that it is modular and can be put into a pipeline.
3877970	3878650	Let's see.
3878650	3883350	So now we need to tokenize all the documents.
3883350	3899130	And this will take a list, which I think docs is.
3901230	3902730	We'll find out.
3907320	3908040	Nice.
3908040	3910200	And then we can... let's just double check.
3916390	3917630	Yeah, that's good.
3921470	3922430	Alright.
3922530	3926290	And now we're going to create our score array.
3926290	3928270	So here we have...
3928270	3930630	So this is a named tuple that we're going to go through.
3930630	3936750	So we're referencing E, which is a terrible variable name, sorry.
3936890	3940870	But alt text is the list that's holding all of our named tuples.
3941430	3945090	And this is called list comprehension.
3947710	3949850	Another favorite of mine.
3950130	3952090	So it creates a list.
3952210	3956710	Put a function inside of square brackets and it will create you a list.
3956710	3958950	But you can see we're referencing the labels.
3958950	3960830	So we're just going to create a score matrix.
3960830	3963130	And actually you can see what that looks like.
3966010	3967910	But they're just giant arrays.
3971830	3984630	And the reason they're giant arrays is because mechanics of machine learning is kind of...
3984630	3987670	Would you say rooted in matrix multiplication?
3990170	3995830	No, because that ignores all of the decision trees and a bunch of other things.
3995830	3996490	Yeah.
4001030	4007270	So when I was talking about this picture being kind of dumb, it's the only picture we ever see.
4007350	4013050	And it's just one of the smallest pieces.
4013330	4017270	There's so much more out there that you should look at.
4017270	4020430	And I probably could have introduced you to.
4021050	4023170	So now we have our score matrix.
4024410	4026570	Actually no, I'm looking at this.
4037230	4038070	Anyway.
4040310	4042670	So these are all our labels.
4043030	4051310	So when the network is learning, it's going to calculate a loss.
4051310	4052710	So it's going to have an input.
4053410	4060150	And then at the end of that it's going to take whatever the label was and see how close it was to the label.
4060150	4063970	So a large loss is bad and a small loss is good.
4064750	4080950	And through gradient descent and back propagation, the network will update the weights such that the next time it sees or there's something labeled like that, it'll hopefully be closer.
4081290	4083030	And we'll get into that.
4086250	4089170	I like this. This is probably one of my favorites.
4095100	4101700	So TAS vs. SPACES, NANO vs. VSCODE, MESSI vs. that other guy.
4101940	4104360	Machine learning frameworks are no different.
4106020	4109000	So do you guys have preferences for machine learning frameworks?
4109840	4116800	Why do you choose the machine learning framework you do?
4119950	4126330	I use PyTorch because it tends to look more like real code than TensorFlow.
4126490	4129870	But Keras is super easy to get up and running.
4130090	4131430	So it's a good choice.
4134130	4140190	And then when you do the real web stuff, I reach into a compiled language like Rust or C.
4140970	4142530	Oh, C, that's brave.
4144470	4146610	Has anybody used ML.NET?
4149170	4151510	Did anybody know ML.NET existed?
4153130	4154090	No.
4155030	4156510	Depends on what you want to do.
4156510	4159590	I have a Win32, so DirectX.
4159630	4163330	There's a Win32 machine learning implementation.
4163330	4169430	Actually, it's just an interface for TensorFlow on X models.
4171290	4173070	Yeah, I prefer Keras.
4173070	4174050	It's simple.
4174170	4181310	If all the math is the same, then you're really just looking at what you prefer.
4184350	4185990	Keras is good.
4185990	4189290	To get off the ground, but it may limit your customization.
4191530	4194090	And I think that's exactly what an API...
4194090	4198730	It doesn't limit you, it abstracts complexity.
4198990	4203830	So I think the reason Keras is so easy to use is because it does a really good job of abstracting complexity.
4203830	4208670	But if you want that complexity you want to dig in, it's still there for you.
4208670	4211750	You just have to potentially dig a little deeper.
4211750	4215910	Versus something like PyTorch, which is...
4217210	4220610	pretty raw, I would say.
4222310	4224150	That's probably not true.
4224650	4228730	Some guy in the back is yelling about MATLAB right now.
4229510	4232250	Oh, MATLAB. That's a whole other
4232250	4236570	stack of words you'll have a little bit of PTSD about.
4236650	4239450	Yeah, don't ever say MATLAB.
4240410	4241410	Rich is saying
4243450	4246730	Keras is good for 99% of your projects, and then PyTorch
4246730	4250830	for when you're getting freaky. And I say PyTorch is good for 90% of your projects
4250830	4255690	and Rust for when you're getting freaky. There's also JAX, all sorts of stuff.
4255690	4258310	Oh yeah, JAX. Does anybody use JAX? Do they like it?
4258310	4260930	I know Jason. Is he in here?
4263330	4266130	Yeah, it's kind of the Wild West, I would say.
4267550	4270050	There's a bajillion different frameworks.
4270050	4272350	Everybody uses something different.
4275470	4278250	I like TensorFlow. It has TensorFlow
4278250	4282030	Serving. It seems to have a good ecosystem
4282030	4286750	around it. But you obviously make trade-offs, right?
4286750	4288850	But if your
4289810	4294330	verbiage and your fundamentals are good, then I think you could probably use just about any language
4294330	4295890	with enough practice.
4297650	4299910	We're going to create our model now.
4303850	4305770	It's the same as the picture
4306670	4310030	effectively, but we're just going to create it in code.
4310030	4313230	One thing we didn't talk about are activation functions.
4315090	4317190	Actually, we didn't talk about it a lot.
4320030	4322450	As your data traverses from weights
4322450	4326590	into hidden layers and hidden nodes,
4327150	4330710	weights, their job is to modulate
4330710	4337490	inputs based on
4337490	4339770	their inputs.
4340030	4343730	Such that the output is relative.
4351090	4351790	Sorry.
4351790	4353730	I just lost him in the chat.
4354630	4358170	We're just going to use Sigmoid. I think Sigmoid is super simple.
4358170	4359590	It's a good place to start.
4363030	4366350	What are some other favorites? I know Relu
4366350	4369770	has replaced Sigmoid.
4370510	4372810	Is there a good reason for this?
4372810	4378250	Does everybody go, oh, this is better, so we're going to use it all the time now?
4379510	4381070	The Relu paper
4384370	4385330	basically said
4385330	4388850	Relu gets better accuracy scores than Sigmoid.
4388850	4393030	They just compared that across a bunch of things. For images, Relu
4393030	4397150	tends to have a higher accuracy score
4398950	4400630	than Sigmoid does.
4400630	4404610	And that's basically why it won, because on images you can easily
4404610	4408750	show that it's better. I don't know whether it's concretely better
4408750	4411390	in all situations because of that.
4412130	4414730	You're showing your age with Sigmoid here.
4416350	4419250	Relu also handles the disappearing
4420990	4422110	differential.
4424050	4428690	Vanishing gradient. Where if you go too far into
4428690	4432570	the negatives or too far into the positives, you're never going to crawl yourself
4432570	4433630	back out.
4435990	4440190	And then there's Leaky Relu that tries to help that even more.
4440710	4444670	And there's Ellu who tries to say, hey guys, you could be
4444670	4447810	Relu, you could be Leaky Relu, let's just combine the two.
4447810	4452250	I think it never goes below negative one as a weight.
4453870	4455470	I think this is
4456350	4459470	actually, I like this discussion. I don't know how many
4459470	4463250	of you listening are machine learning people to begin with.
4465130	4467610	And this is why the tutorials are kind of
4467610	4470550	they're nice to get you off the ground, but I think they're limiting
4471050	4474150	in the fact that they're always using kind of the same
4475710	4479510	architectures. So they're never to say, oh this is
4479510	4483430	good for this and this is good for this, and that's true. And you might end up
4483430	4487810	there anyway like they are. Maybe there's just lessons they've learned that you haven't.
4488010	4490390	But I think it is important to explore
4492070	4494830	different architectures, different losses, just play with
4496350	4499270	numbers. I think machine learning is ultimately
4499270	4502550	about iteration and experimentation
4503150	4507930	at scale. And that includes
4507930	4511050	activation functions, that includes output nodes, that includes
4511050	4515070	any lever you can pull. You should pull it
4515070	4516810	and see what happens.
4518210	4523190	Also, as a data scientist, you
4523190	4527130	should pull the levers and see what happens. Because you might
4527130	4531170	find that on security data, sigmoid works better than relu in this
4531170	4535050	case. And if you never tried sigmoid because you just go with standard
4535050	4537970	relu, you'd never learn that.
4540970	4542230	Read everything,
4542230	4545030	try everything. Hang out
4545670	4550670	at AI village. Super smart people.
4550810	4553230	Just hang out at journal club and
4554710	4557490	be a little fly on the wall. That's what I do.
4558250	4562030	Okay, so we've got our model now. One thing we need to do is we need
4562030	4566310	to put our documents,
4566310	4570030	our vectors,
4570030	4574470	into our model. And when I was first starting
4574470	4578430	this was actually one of the harder pieces that I had to fit in my
4578430	4582010	brain. I think that's true. It's just the shape
4583610	4586270	of arrays and how they get introduced into
4586270	4590470	models. Does anyone in the chat want
4590470	4593410	to take a stab at
4594130	4596910	what these three question marks should be?
4616060	4618180	You can try it.
4619360	4621740	I'm hung up on the missing parentheses.
4624180	4625300	Hey, you're right.
4629030	4632530	Yeah, but Rich has got it right. The feature size.
4632970	4637430	Yeah, exactly. Let's see, what is that? That would be...
4650560	4652060	I don't know.
4682140	4685140	Pretty sure it's your matrix.shape
4686420	4687620	variable.
4698060	4700220	Text matrix shape.
4703250	4705730	Man, that is one thing.
4705730	4710530	What are the things that you guys google constantly?
4718700	4720060	Sorry, you guys must be
4721040	4723940	probably missing some imports.
4725020	4728160	I'll push a new version. Alright, so if you're missing those imports
4728160	4760940	you're going to want to add. Nice.
4760980	4765320	We created our model. It's super simple.
4766540	4770100	The text matrix, the input dim that we're
4770100	4772240	doing is an array of
4774500	4777220	let's see, probably like 80
4779940	4781100	86,000
4781100	4784480	tokens. So across this you're going to see
4788050	4791590	that's going to look obviously like this.
4791590	4794850	So we just have, I think, 1,200
4795450	4799230	samples times 80,000 tokens
4799230	4805440	long. And we're just going to feed it
4805440	4809000	in. Nice.
4809000	4812720	And then we're going to do a test train split.
4815960	4817080	And when you
4817080	4821080	build a model, you obviously have test data and you want to keep some test data out.
4821120	4825120	But the idea is eventually that your model will be deployed into the
4825120	4829060	real world where it's not going to be trained on, it's not going to be seeing test
4829060	4833080	data necessarily. So ideally you want to keep some data
4833080	4836260	away or out of your training set.
4836260	4838980	So when it sees real data
4839820	4842940	or when it sees data it hasn't seen before
4844040	4848620	it can make hopefully an accurate guess.
4849800	4851440	So splitting them out
4854520	4856420	for that reason.
4856620	4860000	I've heard you guys talk about training
4862080	4862800	leaking
4864860	4868060	into your training data during training.
4868060	4869720	How does that happen?
4876640	4879060	One of the ways it could happen is
4879060	4883660	you could have duplicate samples
4883660	4887320	that are, normally you just say, pick a random
4887320	4890460	set of indices or pick the last 20% of my indices
4891800	4895300	and that's my test set. Or you do a cross-validation
4895300	4898700	where you divide it up somehow.
4898900	4903380	If you had duplicate data, say the last 20% of your data was actually duplicated
4903380	4906880	with the first 20% of your data and you trained on the first
4907480	4910600	four fifths and then you tested on the last one fifth
4911240	4915700	well since it's duplicated it's included in your training set and you do really well.
4916100	4919240	So you have to make sure that you don't have that sort of issue. And then there's other little
4919240	4920720	things that you could have.
4922960	4925020	It's basically a data cleaning issue
4926560	4928860	a lot of the times. Sometimes it's a weird bug
4929520	4932700	in your code. Sometimes you can just
4932700	4937160	reverse the model itself and you'll say, oh, show me
4937160	4940620	what this looks like and then it'll just print out an example
4940620	4942940	from the training set. Nice.
4943760	4945540	So we have early stopping
4946380	4948000	twice. Early stopping
4949940	4953660	is super nice. So if you have a really long running task and your
4954600	4958060	model is not improving, early stopping will just stop your model.
4959280	4961580	Part of me doesn't like it. I just feel like
4961580	4965840	it could get better. But this is a tiny
4965840	4968460	model so we don't really mind.
4970200	4973740	Callbacks. So if you want to use TensorBoard, whatever.
4973740	4977660	Batch size. This would be the frequency
4977660	4982580	at which updates to the waits will happen.
4983040	4984700	Epochs. This is just time
4985580	4989600	that we're going to do. Sorry, the number of times that we're going to
4989600	4991120	run through.
4998000	5001780	We can train. It's obviously
5001780	5004320	seems pretty accurate. I don't know
5005480	5008340	I haven't actually tried to optimize this
5008340	5011440	at all. But what do you guys do when
5012420	5016560	a model trains really accurately at first? Do you think it's beneficial
5016560	5020420	that you would try and overfit a model
5020420	5024280	at first? Because that would at least indicate it
5024280	5025740	could learn something.
5028300	5031880	Rich and I both get very paranoid when the thing does too well.
5034280	5036220	You've got an accuracy of 100%
5036220	5040500	but something's wrong. Something must be wrong.
5040680	5042280	Yes, something is wrong.
5042820	5047540	There could be any number of things. This one says version 1
5047540	5051960	and I'm giving you
5051960	5056280	the code so you can go and recreate the model.
5056660	5059980	Do whatever you need to and I would love to see
5062200	5063520	I'd love to see
5063520	5068420	some blog post or some write up about everything I missed.
5069980	5072640	So this is a visualization of the training loss.
5072760	5076060	Obviously with the gradient going down you would like to see
5076060	5078460	this going down. If this were inverted
5080300	5084100	that would indicate that, well it could indicate a number of things
5084100	5088720	but either it's not learning or it's overfitting.
5088720	5091820	And it's stopped learning completely.
5092640	5095280	There are some evaluation metrics.
5096240	5100060	Obviously it's going to be like, yeah I did really well on this.
5101280	5104680	As we saw up here. So that's a little suspect.
5104900	5108760	But even after you have a model, I like
5108760	5112840	to pull out a couple of the best
5112840	5116980	and worst examples of a category.
5116980	5119140	So we're just taking the first
5122000	5122880	malicious document
5125840	5126720	because we
5129980	5132840	put them in a list such that they were malicious
5132840	5136120	and then non-malicious. So we're taking the first, we're taking the second
5139280	5141000	malicious document and then we're going to take the
5141000	5145420	last non-malicious and we're just going to try and predict and see what they are.
5146120	5148780	And yeah, they're
5150680	5153500	a little too accurate for my liking.
5153820	5157800	But yeah, that's kind of it.
5157860	5161260	Okay, so this is kind of where we leave you. So the first
5161260	5165280	Excel file that you're looking at here, when I did it on
5165280	5169800	all 380,000 scripts, it took me
5169800	5173680	well, and this would depend on whatever kind of potato you're running
5173680	5177320	but it took me 17 hours to do.
5177840	5181780	So we're probably not going to run it here. You might already be running it
5181780	5185500	but have them reset a few times because you saw your thing froze.
5186080	5189600	But effectively what we're doing is we're just toggling absolutely every
5191860	5193500	possible combination and then
5193500	5196520	making a base prediction and then a new prediction
5197700	5201660	and then we're keeping a cumulative sum of those. And what you
5201660	5205360	end up getting is a spread of scores
5205360	5209540	across a number of predictions
5210560	5213120	that will sort your tokens into
5213570	5217580	malicious and non-malicious. This is the same
5217580	5221760	code from the Proofpoint research. The Proofpoint research was
5221760	5225120	easier in this regard because they had a wider
5226780	5229600	range of 1
5229600	5233720	to 999, where these are hard labels and it's a 0
5233720	5237520	or a 1. As a first run through
5237520	5241140	looking at the most malicious and the least malicious tokens
5242240	5245520	without any optimization, if it were ever
5245520	5249860	going to work, I would expect to see
5249860	5252300	that at least they're being sorted
5253160	5257480	to some degree. Because now you can go back and you can start tweaking
5257480	5260000	the model or whatever it might be
5261360	5265480	to really pull out, really hone in on what's
5265480	5268200	accurate. I think a lot of times for attackers, this
5269180	5273380	first version would be just fine. But
5273380	5276000	as you go into the future
5277400	5280980	you could even do this. You could always be collecting data
5282260	5284020	and actually I remember
5284020	5288220	after my first BSides talk about machine learning
5288220	5292080	last year, Rob came up to me and was like, hey, you should
5292080	5296320	have a separate data gathering campaign
5297440	5300060	so you keep your ops and your
5300060	5304060	data collection separate, but your data collection can support
5304060	5307100	your ops and it doesn't necessarily
5307940	5312200	burn you because collecting data can be
5312200	5316280	noisy. So it's a bit over time, but do you guys have
5316280	5320580	any questions? Comments?
5320580	5324360	What I would love to see is someone to take
5324360	5328640	this Defender model and absolutely crush it.
5330560	5332320	I didn't... I mean
5332320	5336280	it's a tiny model, I'm not sure Microsoft would care that much. There's a VBA
5337000	5340320	dataset I have not touched. So if you
5340320	5344260	guys want to race to whatever, I'd love
5344260	5346760	to see what comes out of it.
5347680	5351940	I appreciate everybody. Thanks for coming.
5352740	5356080	Hit me up in the Slack or not Slack, Discord
5356080	5360260	if you have any questions or you just want to rag on my terrible code
5360260	5363000	that's fine too, but
5365480	5367980	I will end it there.
5369980	5372100	So I think this is the
5372100	5376100	last stream of the night, so we're closing out the
5376100	5380040	Twitch stream now. Tomorrow hopefully will go smoother.
5380040	5383080	We learned some things today, but I'll
5383880	5388120	see you guys all in the morning. Hopefully.
5388320	5389520	Hopefully.