start	end	text
7440	11400	Hello everyone, Aftab Paki here. We have another great speaker for you.
11640	18260	We have Mauricio and Olaf with a presentation of Purple on My Mind,
18260	22140	Cost-Effective Automated Adversary Simulation.
22280	26420	And with that, I will let them take it away. Thank you very much, guys.
27340	31100	All right, thank you. Hello, everyone. Welcome to our session.
31100	34980	Welcome to the Blue Team Village. We're really excited to be here.
34980	41780	This is our first time. Thanks to all the organizers for all the hard work.
41780	45360	My name is Mauricio. I'm Olaf Hartung.
46200	49800	And today we're streaming live from the East Coast.
49800	56260	Myself, 10.30 here, and Olaf here is in Europe where it's 4.30 a.m. right now.
56260	60140	So I hope that he doesn't fall asleep during our talk.
60140	68640	Today we're going to be talking about setting up a cost-effective adversary simulation program.
69280	75260	First, we're going to go over a quick introduction, define the problem statement.
75740	81480	Then we're going to talk about automated adversary simulation or Purple Teaming
81480	86680	and kind of describe a methodology and a framework that works for us,
86680	89640	that we've experienced and we use.
89960	97620	We're going to go over the available tooling to execute this type of engagement.
97700	102980	And finally, we're going to talk about, show you the open source tools that we've built
103680	114680	and now how we've integrated them as a POC really for this talk to set up a cost-effective adversary simulation program.
114680	120140	We have 20 minutes of demos of our tools being integrated, so I hope you enjoy it.
120140	126640	We had a lot of fun building the demos, so I hope you have fun and get something out of this.
127880	131720	Okay, so we'll start real quick with introductions.
133160	135020	My name is Olaf Hartong.
135680	141980	I'm based in the Netherlands where I founded FalconForce together with a couple of really smart people.
141980	146200	And it's a really purple company, so we do a lot of offensive and defensive work.
146200	148300	And I'm a defensive specialist there.
148680	154960	And I develop quite a lot of open source software, mostly on the defensive spectrum,
154960	158760	which you can find on my GitHub page, which is linked over here.
160160	163500	Thanks, Olaf. And my name is Mauricio Velasco.
163860	171160	I currently lead the threat management team at a financial services organization, a Fortune 500.
171160	180520	My team is responsible for, among many other things, detection, response, and executing simulations, purple team exercises.
180880	191140	In the past few years, I've done some research and developed some tools around threat hunting and detection and adversary simulation.
191140	195240	So, if you want to check out some of that content, you can do that on my GitHub.
195980	201780	Okay. All right. So, we'll start with the introduction.
202260	206920	So, let's talk about deploying an effective detection program.
206960	212220	It's definitely not an easy task, right? I know many of you have probably done it.
212220	217100	We could probably talk hours about setting up a detection program, right?
217100	227620	There's a lot of steps, right? You have to identify the relevant telemetry, the data sources that you think you need to ingest, right?
227620	238520	You need to instrument your environment to basically develop an event pipeline where you're grabbing all this telemetry from your network and your endpoints
238520	249560	and sending it to an event pipeline through an analytics engine where you can query this information and start detecting, right?
249560	257900	Start creating your own detection logic to look for bad, look for suspicious behavior happening on your environment.
257900	259700	And this is great.
259700	274680	Now, I was naive at some point after deploying one of these programs where I thought, we're great, we have a detection program, detections are working, we're creating our logic, the alerts are triggering, we're investigating them.
275040	289060	But I learned the hard way that after you deploy a detection program, which is a challenge by itself, another challenge starts and perhaps more difficult than the one deploying the program.
289060	297280	And that is, we need to maintain this complex balance of tools, process, technology working together.
297280	300120	And it's not easy. So many things can go wrong, right?
300120	310260	So we need to basically deploy controls to measure the effectiveness and verify the health status of a detection program.
310260	316040	We're not thinking about those metrics. We need to start thinking about these metrics.
318000	323380	Yeah, so I have a long history in consulting and I've built a lot of these detection programs in the past.
323380	327180	Usually they're great or even rocking the detection.
327180	329140	So they're building a lot of cool stuff.
329140	331940	They're responding to it, as Marisha also said.
332640	336840	And then some of the really mature teams also have a real good hunting practice.
336840	343000	They do their own research into all kinds of threats that are out there or new techniques that are being applied.
343000	346920	However, in most cases, these teams are quite reactive in nature.
346920	349680	So they only respond to what is new.
349900	354460	And they don't always look inward really well.
354500	359640	So the more mature teams actually monitor all their incoming log sources.
360020	362600	But it's not always at the granular level.
362600	369040	Usually it's on a system level and then maybe only after a couple of days that there's no logs incoming.
369040	371880	So never on a really short timeframe.
371880	380480	And as someone responsible like Marisha or you're a CISO or the lead of a detection team, you might get questions from your stakeholders.
380480	387620	Are we in control or an auditor starts asking questions, which may raise some uncertainties on your end, right?
389660	390440	Yeah.
390440	397980	So how do you know you're actually monitoring what you expect to be monitoring?
402150	408230	Yeah. So what are all the problems that could happen to a detection program, right?
408250	413590	Especially since environments are always under constant change.
413590	417270	How do you know if your event pipeline is working?
417410	426170	I'm sure many of you will relate to something in a scenario like this where you are starting a new threat hunt for a particular hypothesis.
426170	433950	Or maybe your team is investigating a particular alert that is taking you to look at some logs.
433950	440730	And all of a sudden you realize that your firewall logs stopped sending events 30 days before.
440730	443450	I'm sure many of you have experienced that.
444150	446530	And so many things could be wrong with that, right?
446530	452010	It could be a licensing issue. It could be a network issue. So many things.
452890	462210	Another example that I'm sure a lot of you can relate to is what a GPO management means in an enterprise environment.
462210	464890	It can be really cumbersome, right?
464890	470310	There's so many GPOs for all these different sites and OUs and companies.
471130	490410	So what if one of your changes, someone overwrites a GPO that is defining your advanced audit policy and you stop getting your Sysmon Event 1 event or Sysmon events in general in that small part of the network where an attacker may be hiding, right?
491290	498390	Or another example, maybe you did a red team engagement with an awesome consulting company.
498390	503790	They show you all these awesome ways that they can execute Kerberos in attack.
503870	509590	And as part of your assessment, you create a detection to detect this. Great.
510450	514690	Fast forward, three months later, is that logic still working?
515430	524670	What if something changed on your analytic engine, maybe the schema changed, and now you're looking for a field name that is no longer the right one?
524670	528090	What if that complex detection logic just stopped working?
528090	529730	How do you know that?
532930	534530	Well, it's gone from there.
534530	544790	So let's say you don't have your own detection capability and you outsource this to a very nice vendor that promised to take care of you, monitor everything.
544910	552090	But how do you actually know that they are actually monitoring the right things and responding in the right way?
552090	558110	I mean, maybe they pop up an alert and they don't know much about your environment.
558110	564830	They assume it's fine, and they basically close it as a false positive, while it might have actually been a true positive.
564930	567290	And you might actually be already hacked.
568590	582110	And from that end, your stakeholders, which might be your CFO or your internal audit or whoever it might be on your end, how do you convince them that you're actually in control?
582110	587590	You do have a lot of detection rules and you do have a lot of incidents, but is this everything?
587590	592290	And how can they measure if you're doing well or not?
592550	594310	And from a detection perspective as well.
594310	596490	So how do you know your detection is actually resilient?
596490	601890	You do have a lot of really good detection rules, but do they cover variants?
601890	605370	Do they cover only the one that you read up on?
605370	609030	Or do they cover a multitude of ways of executing that?
610010	613270	So you need a way to measure this.
617280	623080	Which comes to our very graphic approach.
623440	626100	So in the center, there's a couple of pillars.
626100	637240	And the risk and the controls one are probably the most well-known ones, where risk is usually determined by the business, which is augmented by threat intel.
637240	643520	And then there might be some other values that come to a certain risk posture.
644340	655440	And then there are your controls, which are obviously your detections, your mitigations, your preventative measures, your hardening baselines, and so forth.
655440	660780	These are usually based on a couple of reference models, like MITRE ATT&CK, of course, which we all love.
660960	663260	There might be a kill chain involved.
663260	670160	There might be some additional frameworks in terms of hardening your environment.
670440	683900	And then there's the effectiveness one that we actually want to add to this, which will support the other ones by providing meaningful, valuable data that you can use for your management, for your auditing parties, but also for your self-management.
683900	686960	So, you can actually know how you're doing.
687780	693680	So, then it a little bit depends on what you want to do with these pillars in which team you're at.
693680	703080	So, if you're in the blue team, on the risk side, there's the control effectiveness that you might want to report to your regulatory parties.
703080	713880	And on the red side, the risk might give you a lot of input for advanced simulations that you want to execute.
714280	722420	And these are usually manual and very creative exercises, but these require a lot of time and money and effort.
722420	724480	And you can't do this on a continuous basis, right?
724480	725940	So, we need something else.
726200	730860	And this is where the purple part on the right, on the effectiveness side, comes in.
730860	736140	So, you can do a normal purple team, of course, which is red and blue joined together.
736400	739900	It's usually based on a forecast or some hypotheses.
740640	749540	And the outcome is usually a really good detection engineering effort where you start building all kinds of new awesome detections, which is great.
749540	751960	And if you can do this, you should totally do this.
751960	754740	But this is always limited by time and effort.
754740	757500	Not everybody can do this.
757500	768100	You don't always have an internal red team or you don't have the means to do this on a monthly basis because it's quite costly to get consultants in there every time.
769460	772600	So, you need something else next to that.
772600	776880	And this is where the automated adversarial simulation comes in.
777100	784460	On top of the building new detections, you also can then augment that with all kinds of validations of all your controls.
784460	785980	So, you can do this on a continuous basis.
785980	787300	And this is why we're here.
787920	792860	So, it's built by red and blue together to get a maximum result on a continuous basis.
794440	796240	This is why we're here, exactly.
797180	804740	So, yeah, that takes us to our next section here, automated adversarial simulation.
804740	822020	We strongly believe that the challenges that we described, the potential issues that could affect a detection program can be approached by deploying automated adversarial simulation exercises on corporate environments.
822020	832480	So, in this section, we would like to share with you what we call adversarial simulation, how we understand it, automated adversarial simulation to be specific.
832480	842660	I think it's important to say that we don't want to claim this term because automated adversarial simulation could mean so many different things to different people.
842660	849480	But in this section, we want to describe to you how we understand this concept.
851240	860540	So, basically, automated attack simulations or adversarial simulation is an automated activity that has two sides.
860540	874620	We're running simulations and then we're validating that the controls that we have previously deployed in terms of detection or maybe even prevention controls are working as expected.
874620	878980	They're passing the test, they're passing the simulation test.
879980	898340	The challenge, as Olaf was saying, purple teaming is great, right? But we cannot execute purple team exercises with 15, 10 people in a room every week, right?
898340	913600	How many times can you do it a year? Every quarter? Maybe every month? But not many organizations have access to high-skilled red teamers that can execute these techniques.
915360	922420	And it's basically also a snapshot in time. When we're doing a purple team exercise, it's just you're looking at it as a snapshot in time.
922420	928600	But with automated adversarial simulation, we can continuously check if those controls are working.
929360	944300	It's basically also a way of generating telemetry, right? And we use this telemetry to validate the environment, to validate the health of a detection program on a continuous basis, as Olaf was saying.
944300	961640	So with this type of program, we can go over all the challenges that we described a few slides before. We can identify issues with the event pipeline. We can identify gaps in visibility, those misconfigured GPOs missing some events.
961660	970420	If we're running simulation exercises against those endpoints, we will know that the events are missing, that we have a gap, right?
973530	984930	And what is it not? It's definitely not a red team replacement. So this is really not a threat to any of your red teamer friends, or if you're a red teamer, welcome.
985710	995870	So you do need red team input if you can get it, because they usually have better techniques than what you can build yourself.
995870	1008310	So it's also not a pentesting engagement either. This is really meant to be a continuous validation or improvement phase.
1008350	1017310	And it's, of course, also not Skynet. It won't take over your company. And from that same aspect, it's not impacting continuity by design.
1017310	1026150	So it shouldn't break your systems unless you really, really want it to. And from that perspective, it should be run in production, of course.
1026930	1039070	And it's also not the new Holy Grail or the new buzzword that you should hear at RSA next year, but it should be something that you definitely need to be considering in order to be really strong.
1045100	1052880	And moving on from what I meant before, so you're not just running Benjamin Delphi's tools in your production environment, right?
1052880	1057980	You need some form of methodology to properly execute these.
1058240	1066060	So, of course, there is some governance involved. You need to establish a sort of white team, depending on how intrusive you want your tests to be.
1066060	1073540	You need to set some ground rules. There needs to be some escalation paths in case something actually goes wrong.
1073760	1084320	You need involvement from IT ops because they might need to accommodate some access to you or give you some means to take some steps.
1084580	1090120	Of course, you need a reference framework from that perspective as well. Of course, MITRE ATT&CK is there again.
1090120	1100620	But there might be some PCI controls that you want testing or COVID or HIPAA or whatever other fancy acronym out there.
1100620	1107160	You need to have some frameworks that can actually measure what you want to prove.
1108180	1112300	And then we move a little bit more towards the execution part.
1112300	1121300	So, we need to have a scope. And this scope selection phase has two crucial points that need to be in there.
1121300	1129000	So, you need to know which systems, and this can be, of course, in an office in Singapore or the US.
1129260	1132500	It can be workstation servers. Well, you can figure that out.
1132540	1142040	And then, of course, you also need to validate or to set the scope for which controls you want to test or to avoid from that end.
1142040	1156940	So, if you do know that you want to test A, B, and C, but you don't want to hit another one because then it might raise too many incidents or that system might be more volatile, you need to be aware of that.
1157440	1163940	And from these scope sessions, you can actually start designing a campaign.
1165220	1171420	And in this phase, you actually build the required objectives or techniques that you want to execute.
1171420	1177900	You add them to a playbook or a runbook or a campaign design or however you want to call it.
1178000	1182960	You test it, of course, first, and then you run it in production, obviously.
1184540	1189060	And one of the steps in here also is defining the cadence of testing.
1189060	1196940	So, do you want to do this hourly, daily, monthly, whatever your preference is, and the timing between all these techniques.
1196940	1202180	Do you want to run it within, I don't know, 10 minutes? Do you want it to take an hour?
1202760	1205560	This can also impact your detection rules, right?
1205560	1210360	So, you need to vary this a little bit on and off whenever you do this.
1210460	1215760	And then, of course, you need to report and validate all of your outcomes.
1215760	1218940	So, you want to know which test you ran.
1218940	1224280	You want to know the results of each individual test, whether it was successful or not.
1224280	1228540	You want to know the commands that were executed or the tools that were launched,
1228540	1234800	or all the relevant information that a blue team needs, including the time that it actually happened.
1234980	1240440	If they didn't detect it, they need to be improving their detection, so they need this data.
1240840	1247700	And, of course, ideally, you also want to have your alerts or your incident tickets or all kinds of telemetry
1247700	1252060	that you can combine in the report where you can show the full end-to-end cycle,
1252060	1262180	where you executed something, you did detect it, and it was actually also picked up by the team properly or not.
1262520	1269260	So, whether they actually flagged it as a true positive is also a very important measure.
1270240	1274040	Yep. I want to follow up on the scoping piece that you were mentioning,
1274040	1278380	because I feel when executing PRPL team exercises,
1278380	1282440	we really need to think about which parts of our network we're targeting,
1282440	1286520	because you're always running exercises on the same parts of the network.
1286680	1289700	Again, we're going to go back to the same issue as before.
1289700	1293980	How do you know if things are working end-to-end across your environment?
1293980	1300380	So, there needs to be a little bit of randomness on that scoping as well.
1303040	1306780	Totally. Yeah, you should also start thinking about,
1306780	1310420	how do I integrate this into my detection engineering process?
1310420	1313100	And this slide has been shown in multiple variants.
1313100	1318940	I think the first one was built by Jared Atkinson over at Spectrops,
1318940	1325980	and it's a really, really nice flow of building detection, new detection rules.
1325980	1332300	And I only highlighted a few of the steps that are interesting for our current talk,
1332300	1335180	because, of course, you need to be working on all these steps
1335180	1338460	that you can now probably hardly read by intention.
1338760	1341520	But in the first phase, when you start thinking of,
1341520	1344380	okay, what do I actually want to add as a detection?
1344680	1348400	You're already looking at threat intel, industry reports.
1348840	1353840	You might also have your internal red team or an external red team that you can consult.
1353840	1355880	So, how can I execute this stuff?
1356360	1359760	And in the next phase, when you start investigating,
1359760	1368060	you want to investigate as much ways of executing that specific technique that you want to detect.
1368560	1376180	And you also will want to be starting to develop scripts that you can then use in a later phase
1376180	1381360	to validate all of the detections that you are trying to build in the first place,
1381360	1387660	so that you already think a little bit about, how do I repeat this process later on?
1387660	1393380	And this is also where you start implementing it in phase four,
1393380	1398880	when you are building your detection rule and actually starting to implement it in production.
1398880	1405720	You also want to start implementing these validation scripts or tools or whatever you've created
1405720	1411780	into the same process, so that you can actually already, from the minute it goes live,
1411780	1415900	start measuring its efficiency, whether it's working correctly,
1415900	1420820	whether the outcome is actually as you expect it to be.
1420820	1425620	And this is, of course, a delicate process where you don't want to be creating detections
1425620	1428860	only tailored to your own validation scripts.
1428860	1433500	So, also these validation scripts need to be maintained, they need to be expanded.
1433860	1439980	You need to keep on researching whether it's resilient enough, of course.
1442840	1453980	Yeah, ultimately, as we are creating detections, each one of them should have its own unit test, right?
1453980	1461600	We see detection as an engineering process, we just need to test each one of our products.
1462180	1469140	And one big requirement for us is, we believe that it needs to be end-to-end.
1469140	1476200	And I think there's also something to bring up here where some challenges with other approaches,
1476200	1482320	where I would like to differentiate between unit testing and functional testing,
1482320	1491700	where in unit testing you can perhaps inject to the event pipeline a particular telemetry,
1491700	1497540	maybe generated by the execution of adversary techniques, to test the detection logic.
1497540	1500900	And that's great, we should do that to test the detection logic.
1500900	1508720	But that misses a point, because you're assuming that the event pipeline is working as expected.
1508980	1516160	So, I think we need to complement this type of testing with functional testing, end-to-end,
1516160	1521860	testing a technique on a specific host, so that you're only testing the engine,
1521860	1527140	but you're testing the data source generating telemetry.
1527140	1533540	So, it's proper configuration, the data event being sent through the event pipeline,
1533540	1537900	through the analytics engine, and ultimately generating that detection.
1537900	1541460	So, it needs to be end-to-end, right Olaf?
1542240	1548040	Definitely, yeah. And to the last two words in the sentence, it needs to be also in production.
1548040	1557760	Because I've seen labs and acceptance environments, and in my long experience as a consultant,
1557760	1560300	it's never been really lifelike.
1560300	1565260	It can be close, but it's only on maybe a handful of clients I've been to,
1565260	1568220	where acceptance actually comes close to production.
1568220	1570960	But still, production is always different.
1571200	1573860	And this is the stuff that you're trying to protect, right?
1573860	1578940	So, why would you be executing all this stuff in a lab and then hope it works in production?
1579120	1580480	You need to make sure.
1586080	1593100	So, what are some of the requirements if you want to run an automated adversarial simulation exercise?
1593100	1598000	I think similar to a PRPL team exercise, in this case, I think the requirements are similar.
1598380	1600280	You need a defensive capability.
1600280	1607900	It doesn't have to be cutting edge, like with the whole visibility and hundreds of detections.
1607900	1613140	You can have less mature, and still, it will help to test the controls that you have in place.
1613140	1617620	Of course, you have to tailor the simulation exercise to the controls that you have in place.
1617620	1621040	But you need to have some kind of visibility in a detection program, right?
1621040	1625620	Otherwise, you don't know which controls to test.
1626480	1631120	Ideally, and definitely, we need some knowledge of the threats that we are executing, right?
1631120	1635320	So, this is, again, where the PRPL comes in on the talk, right?
1635320	1642520	So, we as detection engineers, as a detection team, we need to be aware of these techniques.
1642520	1648820	Maybe not to the level of the Red Team does and execute them like a Red Team does,
1648820	1655640	but at least to the point where we can simulate them and confirm that our logic is working.
1655640	1657560	It's testing, our programs are working.
1658580	1663140	And, of course, we need tools, right, to simulate this.
1663140	1673580	We cannot be just downloading a VM and maintaining hundreds of different tools to execute all these techniques.
1673640	1682300	Ideally, we need one tool or a set of tools that will give us the most scope as possible in executing these techniques.
1682300	1686920	And these tools need to have certain features and capabilities, right?
1686920	1688620	They need to be able to be automated.
1688960	1703560	They need to generate logs, specific logs, to be able to compare the execution of techniques with the detection controls that are triggering, right?
1703560	1705640	So, tooling is really important.
1705640	1709200	And we'll talk about that on the next section.
1710680	1714600	And it also requires that you have a sort of an adversarial mindset.
1714600	1721940	You need to not only think about how do I defend this stuff, but also how can my company be attacked?
1721940	1728140	And sometimes it's a little bit hard, so this is also where tools might actually help you a little bit.
1728180	1733120	But you need to be a little bit aware of what can be done in my environment, what can't be done.
1733120	1736480	And then all of these things need to be validated, of course.
1736480	1749100	And in order to do this, usually in a more normal organization, you actually need some management buy-in, because they need to approve you spending time on this, since it will cost a little bit of money.
1749100	1754020	And if you want to buy one of the commercial tools, you of course need some more budget.
1755240	1764980	And if you're a little bit bold, as I've done in the past as well, you can also just do it and maybe say sorry later if they get upset.
1764980	1777340	The big upside to this, if they don't really understand the value of a program like this, is that you can immediately prove them the results with some more details.
1778280	1792340	But frankly, every type of management should be interested in this, whether they're very financially focused or not, because this will give you a lot of insight into how you're actually doing as a team.
1792340	1807900	And of course, if you have a security vendor that does monitoring for you, you need their involvement as well, because otherwise you'll spoil a good relationship and you need to actually be working together since they're doing part of your detection work.
1812770	1824090	Yep. So let's talk a little bit about the tools that are out there that can help us in executing automated adversarial simulation exercises.
1824090	1831850	So there's definitely a lot of tools out there, right? We're not reinventing the wheel here, really.
1832730	1840610	There's commercial tools. There's some open source tools. We're going to stay away from the commercial tools since that's not our focus today.
1840610	1855550	But let me tell you a little bit about my experience. So when we first started executing PRPL team exercises internally, our natural approach was to use red team tools, right?
1856370	1875770	Red team tools, like Metasploit, Empire, Parse2. And then we started validating controls, which worked great at the beginning when we were more on the form of a manual PRPL team exercise where maybe once a week we check a particular technique manually.
1875770	1897070	But that soon stopped being practical because as the number of detection grows and as our testing moves to a weekly cadence, it's not practical to have an engineer just getting all these shells from your C2s and then start running things manually.
1897070	1906830	Now, some of these tools do have APIs, and there's actually adversary simulation tools that leverage the APIs, so they can definitely be used.
1908630	1923510	Then some of these tools are also fully automated and allow you to create a stand-up infrastructure in an automated way where you can run simulations and hopefully your endpoint controls will catch them, right?
1923510	1933090	But I think a big challenge with that is that you are running these simulations from fixed infrastructure, from the same host every time.
1933090	1935510	And that takes me back to end-to-end, right?
1935510	1943510	Because with this type of exercise, you are not really confirming if your event pipeline is working or not.
1943510	1951350	You're just confirming if that host, which is built for that, is generating the telemetry and if your detection is triggering.
1951350	1957230	So, they're great, these tools are great, but they have some limitations.
1958410	1963870	Ultimately, I think some of these tools have been built to be run manually, right?
1963870	1966650	On a host, like on the screen, interactively.
1966650	1973510	And that's fine, they still have a value, but some of them have not really been built for automation.
1976290	1983130	The ones that do actually have some complicated ways of building scenarios or adding new techniques, it's not always easy.
1983130	1991870	And that's also the drawback with open source, probably also with my own tools, where somebody has a certain way of working in mind, it works for them.
1991870	1995950	But for a new adopter, it's actually quite hard to start working with it.
1995950	2007370	And from a detection perspective, not all these tools have a really verbose way of telling you what happened, when it happened, and so on.
2007370	2016710	So, the output is quite limited, which is hard then to match against your detections or put into a more measurement program.
2017370	2027770	And as Mauricio mentioned, some of them are very static in terms of where they run from, or they're not as realistic, where it's just a bunch of Python scripts, which are useful.
2027770	2031610	But it also is not always the way that an attacker would do it.
2031610	2036490	So, how realistic is it for a SOC then to be responding to this?
2036970	2041910	And lastly, not all of them are capable of doing stuff remotely.
2041910	2045670	So, it's usually from that same system, they do something on that system.
2045670	2047270	And then that's that.
2047270	2050390	Which is also not always the way an attacker would work, of course.
2053520	2056240	Yep. So, which takes us to our tooling, right?
2056240	2061520	So, having mentioned all these challenges and the approach,
2064180	2070620	Olav and I have both built tools that have different goals.
2070620	2078420	And we've integrated them to execute a POC for automated adversary simulation.
2078420	2083700	So, we want to share some of these tools in the next section.
2083700	2086400	So, I'll start on mine.
2086400	2094720	So, actually, yesterday at Black Hat Arsenal, I released my adversary simulation tool called PurpleSharp.
2094720	2098760	So, you can go to the GitHub page and you'll find all the information you need.
2098760	2108780	But basically, PurpleSharp is a tool in C Sharp that executes adversary techniques within Windows AD environments.
2108780	2111720	So, it's focused for AD environments.
2111720	2117520	It follows a MITRE ATT&CK framework as far as the techniques.
2117980	2132300	And the main goal of PurpleSharp is to help us execute some of these simulations to generate the telemetry that can help us build, test, and enhance detection controls, right?
2132300	2133540	That's what it does, basically.
2133540	2136920	It just executes techniques and will generate telemetry.
2136920	2149660	So, if you have a properly monitored Windows environment, Windows domain environment, this telemetry will make its way to the event pipeline, to your analytics engine.
2149660	2158080	And if you have coverage for the techniques that PurpleSharp supports, you should validate your controls, right?
2159840	2163200	So, what are some of the... why PurpleSharp?
2163200	2164160	Why is it different?
2164160	2167080	We've talked about some of the tools that exist already.
2167080	2168820	So, I want to mention a few points real quick.
2168820	2173160	I know we're not great on time, Olaf, so I'll go fast.
2173160	2177220	Simplicity, it's one simple .NET binary.
2177220	2186700	All you need is to compile a .NET binary and you can just grab it and copy it to a Windows endpoint and run simulations against remote hosts from that.
2186700	2187360	Really easy.
2187360	2191880	No C2, no C2 channels, no infrastructure, no VMs, one binary.
2191880	2203680	Verbose logging, as Olaf was saying, we need the audit trail of all these techniques that executed at what point, from which processes, from which process ID.
2203680	2213680	So, we can grab that data and maybe if our detection controls didn't trigger, we can at least find that telemetry and create a new detection or identify why the detection didn't trigger, right?
2214520	2216080	Remote simulations.
2216080	2223520	PurpleSharp is actually able to execute simulations or what I want to call deploy simulations on remote endpoints.
2223520	2232120	So, once again, the challenge of verifying that the event pipeline is working or not can be easily approached with PurpleSharp.
2232120	2245320	All you have to do is point your simulation to a host in your Singapore office and you'll confirm if that simulation will be detected as good as the one that in your headquarters, right?
2245320	2247240	Credible simulations.
2247240	2253500	I've used a couple of techniques with PurpleSharp to make sure that the simulation is credible.
2253720	2255620	I'll mention a couple of things.
2255620	2264160	You can go through the documentation to really understand, but one is process ID, parent process ID spoofing, PPID spoofing technique.
2264160	2274960	I use this technique to be able to deploy simulations on remote endpoints and run the simulations under the context of the user who's logged in on that host at that point, right?
2274960	2281900	So, no longer we have to run simulations from that same infrastructure, from that same user that our SOC already knows.
2281900	2289260	And it's just going to close that alert and you're not really going to train them unless these simulations run from real users on real computers.
2290380	2291500	Randomness.
2292140	2300540	PurpleSharp will leverage simple LDAP queries to query your environment and pick random hosts for the simulations.
2301360	2305160	For example, in a password spray attack, it's going to pick random users.
2305160	2308620	If it's going to do a network sharing iteration, it's going to pick random hosts.
2308720	2312660	So, that randomness, once again, gives us that end-to-end testing.
2313380	2314360	Attack variations.
2314360	2318800	The last one, one really important for me is what Olaf was saying before.
2318800	2324360	How do you know if your detection is resilient to a different way of executing the same attack technique?
2325420	2330180	Well, I've tried to execute the same technique in a couple of ways, at least.
2330180	2339100	For example, a simple example is running PowerShell, calling PowerShell.exe, or running it from .NET using System Automation Management, DLL.
2339100	2344180	So, one simple example, PurpleSharp can execute both for that technique.
2345700	2348620	Real quick, the architecture.
2349660	2352060	PurpleSharp consists of three modules.
2352060	2354940	The orchestrator, the scout, and the simulator.
2354940	2358200	The orchestrator will run on the operator endpoint.
2358640	2366160	And the role of the orchestrator is to interact with the simulation target to deploy the simulation.
2366280	2369140	The scout will run on the simulation target.
2369140	2373340	And the role of the scout is just to obtain information about that host.
2373340	2376340	For example, who's the user that's logged in?
2376340	2379100	What's the process ID of Explorer.exe?
2379100	2383800	So, we can use the parent process spoofing technique.
2383800	2386420	Parent process ID spoofing technique.
2386460	2389140	And the simulator, it just runs the simulation.
2389140	2395880	So, these three modules actually communicate with each other using name pipes so that they can orchestrate the simulation.
2395880	2399280	Again, if you want to learn more about this, go to the documentation.
2399540	2403020	But at a high level, this is how PurpleSharp works.
2403020	2411480	It's going to first upload... once you point it to a remote endpoint, it's going to upload the scout using SMB.
2411480	2413340	It's going to execute it using WMI.
2413340	2417020	Then the scout service starts on the remote endpoint.
2417020	2419920	And there's some communication on name pipes.
2419920	2425280	That's where the orchestrator instructs the scout to run a recon on this host.
2425280	2428300	And that's where we find out who's the user that's logged in.
2428300	2438680	Based on this information, the orchestrator will actually upload the simulator under the path of that user so that our simulation runs under the path of a real user and a real computer.
2438680	2440640	Once again, credible simulations.
2441370	2446820	And once the simulator has been uploaded, the orchestrator triggers the simulation.
2446820	2453140	The scout will execute the simulator using parent process ID spoofing.
2453140	2461460	That way, the simulator runs under the context of the user and also as a child process of Explorer.exe.
2461460	2464460	Once again, making the simulation credible.
2464860	2468720	This is just one example of an attack variation, as I was saying.
2468720	2484380	I can execute PowerShell just calling create process API, calling PowerShell.exe, which I could detect if I have some kind of EDR or this event ID being logged.
2484380	2487480	But on the second technique, we're using .NET.
2487660	2494880	So your EDR or your Sysmon or your event 4688 will not detect that.
2494880	2498400	But you need to have PowerShell logging enabled to detect that.
2498400	2502920	Once again, how resilient is your PowerShell detection technique?
2505420	2510140	We don't have time, so I'm not going to mention the other variations.
2510480	2511520	Go ahead, Olaf.
2512660	2516560	I've built an app for Splunk, which is called ThreatHunting.
2516560	2520140	Not the best name, but it's very descriptive, at least.
2520140	2535140	And it's a very graphically oriented tool that will run about, I think, 160 hunts or queries or triggers, as you want to call it, throughout your environment on a periodic basis.
2535140	2545500	And it will give you a lot of telemetry, which you can then start using for either moving your detections, or hunting, of course, which is the main intention of the tool.
2545500	2555440	But it can also provide you with a lot of visibility into all of the triggers that have been going on.
2555640	2556740	Thanks.
2556840	2560580	So, there's a lot of detections for all kinds of techniques on Windows.
2560600	2566540	It primarily utilizes Sysmon and the Windows native event logs right now.
2566560	2572860	It's heavily MITRE ATT&CK focused, because this is a framework that I really embraced.
2572860	2589780	And the primary goal was to provide an investigative workflow for hunters, but also for detection engineers to provide as much context as possible that is all related to the same events without having to click and copy paste a lot of times.
2589780	2596000	So, it's meant as an efficient way of going through your data based on triggers.
2597900	2598700	Yep.
2598700	2611980	So, the POC that we want to show you today is basically how we've integrated both our tools to use PurpleSharp as a simulation engine and the ThreatHunting app as a detection engine, so we can validate that these detections are being picked.
2611980	2614460	So, now we move to the demos, the fun part, guys.
2614460	2616320	You must be bored about us talking.
2616540	2618920	So, this is going to be the fun part.
2618920	2630780	So, in the first demo, we'll execute three execution, the execution tactic, MITRE tactic execution, three techniques, PowerShell, Windows Command Shell, and Jscript.
2630780	2632040	These are common techniques.
2632040	2640780	If you read most of the malware reports, you probably see that malware or most likely will use some of these techniques.
2640820	2645800	So, we'll move to the first demo, and hopefully this works.
2648920	2650040	Okay.
2653000	2655400	All right.
2656540	2661280	Let me see if I can undo full screen.
2662560	2665640	One second, of course.
2666280	2670140	Doing live, it's always...
2679090	2682970	What if you go out of full screen on the browser first?
2683770	2686110	Let's do that.
2689380	2690400	Thank you.
2690400	2691380	All right.
2692120	2698880	So, in this first demo, we have two endpoints.
2698880	2700180	Let me go back real quick.
2700180	2700900	Two endpoints.
2700900	2708720	We have PC Sam, where we're going to run PowerShell from, and we have another computer called PC Judy, where we're going to run the simulation against.
2708720	2709200	Okay?
2710360	2711040	All right.
2711040	2714220	So, we're going to use PowerShell command line in this case.
2714220	2716380	This case, we're going to...
2716380	2726520	If you want to learn more about the parameters, please go to the documentation, but basically we're running a simulation against PC Judy using a service account that needs to be an admin on PC Judy,
2726520	2734040	and we're going to run these three techniques, and we're going to slip five seconds between the execution of each one of these techniques.
2734380	2740880	So, we're going to pass the password of this service account that we're going to use to run the simulations against this host.
2741740	2751180	So, we see here that the scout gets uploaded, recon happens, and we identify that Judy Brody is logged in on this host.
2751180	2761360	So, now the simulator is uploaded to a path within the user's profile, making it look as if Judy downloaded a fake Firefox installer.
2761360	2772960	Then the simulation runs using parent process ID spoofing, so making Firefox installer a child process of Explorer running under the context of Judy.
2772960	2777780	And as I was mentioning before, this is the report that we get back.
2777780	2787760	So, we see all the details of when it ran, which technique, again, from which process ID, from which path, which command was executed.
2787760	2797700	In this case, we have a parser command, we have CMD, who am I, a simple who am I, and then we execute what looks to be a malicious Jscript file.
2797960	2804840	And all of them in the context of Judy, and then the PurpleShark will delete some of these logs files.
2804840	2811580	So, now we go to our Splunk environment to confirm these techniques were executed.
2813000	2826580	And as we can see here, we see the Scout running under the context of the PSharp service account and as a child process of WMIPRDSE.
2826720	2834900	And then we see the simulator, now this time running on the context of Explorer and running under the context...
2834900	2841140	sorry, as a child process of Explorer and running on the context of Judy, so the real user who's logged in on that host.
2841140	2850880	And then PowerShell is run, WhoAmI is run, and this suspicious Jscript is executed, right?
2851760	2863500	And if we go to the 4104 PowerShell event, we can see how the script was actually just asleep, 20 seconds asleep, right?
2863500	2871980	So, as a simple test, we confirm that this simulation run three techniques remotely, really easy with one command.
2871980	2873940	We can run with PurpleShark.
2874820	2878080	We'll go to the next one real quick.
2878500	2881620	Let's see, I'm just going to fast forward because of time here.
2881820	2884360	We're going to execute a different technique.
2885920	2889000	I honestly forgot what this technique is.
2889000	2894220	I don't speak ATT&CK, but we'll see what it is.
2894220	2894980	I forgot.
2895940	2897860	So, follow the same process.
2897860	2899880	I'm just going to fast forward real quick.
2901960	2909200	Okay, yes, in this case, it's actually the same technique, PowerShell, but this time running using system management automation, right?
2909540	2911700	So, a different way of executing PowerShell.
2911700	2914620	We're no longer using PowerShell.exe, right?
2914620	2920620	So, if we see on the logs, we don't see PowerShell running.
2920620	2929600	All we see is Firefox Installer because Firefox Installer, the simulator, executed PowerShell with .NET.
2929600	2941240	But if we look at event ID of module loads, I forgot what the event ID of that on this one is, but we see the system management automation being loaded by the simulator.
2941240	2945400	So, that's how we can catch that this guy is executing PowerShell techniques, right?
2945400	2951980	So, this is our first demo, really first way of just running a couple of simulations.
2952500	2959520	So, in the next demo, we'll run a few techniques on the persistent tactic.
2959520	2963120	We're going to use some race to keys, schedule task.
2963160	2966400	We're going to, on defensivation, we're going to use portable execution.
2966640	2969120	Sorry, portable executable injection.
2969200	2970720	So, PE injection.
2970940	2974680	And then we're going to dump LSAS using PurpleSharp.
2974680	2978360	So, Olav will take you on this demo.
2979100	2982120	All right, let me introduce some new stuff here.
2982120	2991320	So, as you can see, there's this JSON structure, which is called the playbook JSON structure, where you can define all the variants that you want to do.
2991320	2994440	So, here is where you schedule your scenarios, basically.
2994440	2996680	And you need to define some basic stuff.
2996680	3006700	So, username, domain, the domain controller, the sleep between all the techniques or the small playbooks that you want to execute, and then the playbooks themselves.
3007540	3009820	So, of course, you give them a name.
3009820	3011940	You give them a host that you want to target.
3011940	3015060	So, it can either be a host name or a random host.
3015060	3020660	And then, as Mauricio explained, it will do an LDAP query and find one itself.
3021120	3028260	You can here also define the path of the scout and the simulator where you want to be running it from.
3028260	3031500	So, you can actually make it even more realistic.
3032160	3036240	The PB sleep is the sleep between these playbook steps.
3036240	3045480	These two techniques for the registry and the scheduled tasks, they will be run 30 seconds in between.
3045860	3049080	And then the next playbook will start after the playbook sleep.
3049720	3052260	And over here, you can do the same thing.
3052260	3059040	So, the command line is a little bit easier here, where you just say PB for the playbook and the playbook name.
3059040	3061220	And you basically run it from there.
3063440	3067360	Once you, of course, provide a password, you could do this on a command line.
3067360	3071620	But I would advise against that, especially on a demo environment.
3073100	3076280	So, here it will show you that it will execute three playbooks.
3076280	3078300	And it will start with the first one.
3078360	3082620	It first does an LDAP query to find the targets that it can run it against.
3082620	3084060	It picked a random one.
3084060	3086720	In this case, it's PC Judy again.
3087220	3090060	And it's executing all these techniques on that machine.
3090060	3092360	Well, this takes a little bit of time.
3092500	3096920	So, we have to wait for all the techniques to be executed.
3097100	3100780	And this is where the pipe communication comes in.
3100780	3103300	So, it actually checks whether it's done.
3103380	3108480	And once it's done, it will also clean up after itself, so that at least you won't lose any traces.
3108600	3114680	All the binaries will be gone again, if your EDR didn't stop it already, of course.
3114740	3118300	Which is also a way of validating whether it worked or not.
3120120	3121140	And there you go.
3121140	3123320	So, they are executed.
3123320	3126980	So, you show it was started, where it was started from.
3127080	3129940	You see the key that was added to the registry.
3130400	3132560	It was successfully created.
3132780	3136020	Then it was deleted, because you don't want it to be there.
3136020	3137600	And the simulation is finished.
3137600	3149020	Same goes for this technique, where you see a different path and the same parent process ID that it actually executed from.
3149160	3159040	And you see the command that was executed in full detail, which you can then also use in your detection later on, to see whether you actually found it.
3159740	3166760	And in the meantime, it's starting to execute another one on a different machine.
3166760	3168280	And this is the PCMP.
3168280	3172520	It's a different simulator that you see running under a different user context.
3174140	3178900	And it's actually injecting into Notepad.
3178900	3180320	So, you see exactly what it did.
3180320	3183480	So, it does an open process for VirtualAlloc.
3183560	3190840	Then it writes to the process memory and creates a new thread from that Notepad.exe.
3191000	3194740	And then the last technique that we mentioned was running LSAS.
3194740	3197260	And you see that it actually ran.
3197260	3199520	It identified the LSAS process.
3199520	3201060	It tried to obtain a handle.
3201060	3204200	It called the mini dump.
3204980	3206860	It wrote it.
3206860	3208440	It dumped it there.
3208660	3215820	And it fortunately also deletes it, because you don't want to, again, leave stuff for real attackers.
3215820	3220860	And then when it's done, it actually writes a JSON file with all the results,
3220860	3226040	which you can then easily import into your analytics platform of choice,
3228040	3231700	which in our case will be Splunk.
3232340	3234380	But you can see here that the playbook is there.
3234380	3238080	And then there's a results JSON, which once you format it,
3238080	3242020	it shows you all the stuff that we actually just saw on the timeline,
3242020	3247220	but then in a nice JSON structure so that you can actually start importing them.
3248280	3254080	And then from the perspective of time, we might want to scroll through a little bit.
3254180	3255700	Yeah, makes sense.
3255700	3258660	So, ideally, this is the integration, right?
3258660	3262640	So, this is where you ingest this into your other platform.
3262640	3263400	So, go ahead.
3263740	3264260	Exactly.
3264260	3270000	So, I created a small POC dashboard, again, just to show what kind of data is in there.
3270000	3277220	So, you can see that the orchestrated PCs of SAM executed the techniques,
3277220	3282000	the techniques that you see on the right, on the machines that were involved.
3283300	3285280	And you can show these over time.
3285280	3290380	So, if you do a longer playbook than I did, you will see actually a longer scope, of course.
3290540	3293220	Here you see the playbooks that were executed.
3293860	3297380	And if you know my threat hunting app, you can click on everything.
3297380	3298600	So, it's the same over here.
3298600	3306620	So, once you click on one of those playbook names, you will get all the raw logs with all the details in the panel below,
3306620	3310280	which will show you all the results that you also saw on the command line.
3310280	3313820	So, once you click it, it will actually change to the proper playbook.
3314420	3317860	And the same will go for all the other fields.
3317860	3323680	So, once you click on a MITRE technique ID that was tied to one of those techniques...
3323680	3324940	Hey, the stream is paused.
3324940	3326020	Oh, there we go.
3326020	3332220	It will take you to my threat hunting app and will do a direct query for that technique being executed.
3332220	3337780	And you see that also it was detected by the logic in my app, fortunately.
3338040	3345260	And it will give you all the details that might help you contextualize it even more once you start playing with the app.
3350670	3356890	And the same can be done for clicking on a machine, which I don't do yet.
3356890	3364070	But I'll show you the raw logs first, where you can see all the results from all the techniques that were executed.
3364070	3368770	So, again, you can quickly spot that everything was executed.
3368770	3371870	So, you see the scout being run.
3372190	3375570	You see that then the WMI is being called.
3375730	3381630	The process is being ejected into and it starts executing the registry commands and so on.
3382190	3387570	And the same goes for the scheduled task logs, which isn't that properly formatted, thanks Microsoft.
3387570	3390650	But this is a quick POC.
3390650	3394750	You can obviously properly parse this neatly.
3394750	3399670	But it's quite easy to spot that some of the bad stuff actually was executed here.
3402670	3405210	And I think we can fast forward again.
3405350	3406010	Okay.
3407590	3410430	So, yeah, we're going to the threat hunting app.
3412030	3413170	Yeah, and this is the same.
3413170	3430670	So, once you would have clicked on a machine, you would have actually seen all the techniques that were executed on that system in the threat hunting app, which will give you a huge panel with all techniques, all the commands that were executed, all the parent processes, process GUIDs from a Sysmon perspective.
3431470	3439190	And it will try to at least map it to the proper MITRE tactics as well, as much as possible.
3439190	3447730	And here you can see that there were some DLLs loaded, there was some registry activity, and you still have access to all the raw data as well.
3448450	3450630	And that concludes the demo two.
3450670	3451750	Okay, cool.
3451750	3452850	Demo two.
3453630	3455410	Let's move on.
3456330	3459030	Okay, so I'll do this real quick.
3459030	3463490	Of course, ATT&CK MITRE framework is our framework of choosing.
3463490	3469170	And PurpleSharp does integrate with the ATT&CK Navigator to visualize these techniques.
3469170	3478710	So, with simply running just Navigator export, PurpleSharp will create a Navigator layer that you can open up and upload to the Navigator.
3478710	3483310	I'm going to fast forward here so that we make it on time.
3483310	3489470	Just upload that and you'll see visually all the techniques that PurpleSharp supports.
3490090	3498510	So, you can use it to pick the techniques that maybe you want to simulate on your automated adversary simulation exercise.
3498930	3504870	And in a similar way, I actually support importing Navigator layers.
3504870	3510850	So, in this case, I'm going to get a layer for APT, I think, 29.
3512050	3513110	Oh, sorry, FIN7.
3513110	3514490	Yeah, I picked FIN7.
3515970	3519690	On Navigator, I export that layer to a JSON file.
3520890	3522930	So, I'm going to fast forward here.
3523550	3527070	And then I move it to the folder where PurpleSharp is.
3527070	3531190	And now, in this case, I'm just going to import that Navigator layer.
3531190	3535670	And PurpleSharp will analyze the techniques that it does support and it doesn't support.
3535670	3537850	So, it will let you know.
3537850	3542970	But then it finally creates, in this case, it supports 17 techniques out of 29.
3542970	3548810	And it's going to create a JSON playbook like the one Olaf just run on demo number two.
3548810	3554630	And now you have a simulation playbook for FIN7 that you can run on your remote endpoints, right?
3554630	3556610	So, pretty cool.
3557310	3558350	There.
3560900	3562720	Okay, so...
3565200	3567940	Yeah, I think we only have one minute left, Marcia.
3567940	3571600	So, we have to publish this just online, I guess.
3572000	3573320	Yes, so...
3573320	3573900	Okay.
3573900	3574620	Sorry.
3574740	3575400	Yeah.
3575400	3582560	So, I've noticed on the Twitch that some of the demos were not showing great.
3582560	3586100	So, right after this, I'm going to upload them on YouTube also.
3586220	3587800	So, you see them.
3587800	3592680	But the last demo, yeah, it's also a cool one, but you'll see it on YouTube, okay?
3592680	3594300	It'll be uploaded there.
3596280	3600820	All right, you just want to take this last one, Olaf, since we're running out of time.
3600820	3601640	Yeah, sure.
3601880	3607740	In the last couple of seconds, I just want to highlight that this is a huge value in terms of control,
3607740	3613260	but also in terms of reportability and also ensuring that you're actually working on the right stuff.
3613260	3615940	So, it's also ease of mind, probably.
3615940	3620400	So, I strongly encourage everybody to just start with this.
3620480	3627700	And especially since there's a bunch of open source tools out there, including ours, it doesn't have to be very expensive.
3627700	3633000	So, you can start small, and then once you start maturing, you might want to look at a commercial tool or not.
3633780	3639620	And this should be really a nice way of implementing a way of testing all your detection engineering efforts,
3639620	3644500	so you actually know it's not in vain, you know it's actually doing something.
3644500	3649800	Because some of them probably won't trigger in production ever, hopefully, at least for you.
3649900	3652360	So, you at least know it still is working.
3652480	3655340	And it shouldn't be a grading mechanism for your team either.
3655340	3662200	It can be used that way, but you should see it as a constructive tool and not as a sort of slapping tool if you didn't do it well.
3662200	3670600	And of course, if you don't have a detection yet, you can of course use it also in a way of the purple exercise that we meant before,
3670600	3675540	where you can start building new detections based on the techniques that we execute,
3675540	3679320	but then make sure that you don't build tunnel vision detection.
3679680	3683040	Make sure it's still resilient and it's open for variants.
3684800	3689500	So, thanks everyone. That concludes our chat, our session today.
3690000	3694720	You have the links for our tools and I'll also be... check out our Twitter.
3694720	3701460	I'll also be releasing the links on the YouTube videos for the demo, so you can see them with a higher resolution there.
3701620	3708220	But yeah, thanks for coming to our session. I hope you guys had fun. And thank you.
3709160	3711120	Thanks, guys. Girls?
3713480	3715580	Thank you, both of you.
3715580	3719930	We really appreciate your demo and your presentation.
3720780	3732400	As a reminder to everyone, we encourage everyone to join the Blue Team Discord and head over to Text Talk Track 1.
3732720	3739400	And we can have the presenters answer some of your questions there as well.
3740180	3745860	Right off the bat, I did see a couple of questions. So, if you guys don't mind.
3747060	3747880	Okay.
3752030	3753510	Scroll up.
3757360	3764000	The question was, what are some of the major benefits of PurpleSharp versus Atomic Red Team?
3765520	3768900	Yeah, okay. That's a great question. So, I think there are different things.
3768900	3779600	Atomic Red Team is more a way of describing how to execute the techniques and not really an engine to execute the techniques themselves.
3779600	3790800	Although the Red Canary team is working on Invoke Atomic, which is a partial tool that will be able to execute Atomic Red files.
3790800	3804200	But if you ask me the difference between Invoke Atomic and PurpleSharp, I would say running the simulations under the context of a real user and as a child process of explorer.
3804200	3808660	So, creating that credible simulation, that's something PurpleSharp can do.
3808660	3822740	Another one is randomness, being able to run a technique against a random host to really identify if your detection controls are working end-to-end across your environment. That's the second one.
3822740	3833560	And I guess the third one would be just simplicity. Maybe you can run simple binary that you can just copy on a host and run simulations from.
3836230	3838650	Very good. Cool.
3840850	3844570	I think that's it for the questions for now.
3845390	3846650	Oh, here we go.
3847070	3851010	Did you guys teach a course on this during this week?
3852170	3856210	No, we didn't. But maybe that's a good idea. Maybe we can do that on the next one.
3856210	3858590	That will be fun. Yeah, definitely.
3859090	3861630	I know we had fun doing the demos.
3861910	3864230	Yeah, too much.
3864930	3865890	Too much.
3866090	3869270	Maybe something in the future that you can do a workshop.
3872350	3883150	Yeah, I think that's it for this. And if you guys don't mind sticking around for a little bit and let's talk track one.
3883150	3883770	Sure.
3883770	3885710	If people have questions.
3885710	3893410	So one last thing is just I'll be posting the links of all the demos right after this. So check out our Twitter.
3894930	3901030	And you'll be able to watch all the third last demo, which was also really cool. We didn't get a chance to.
3902210	3906070	Cool. Very good. Thank you guys again. I appreciate it.
3906450	3907450	Yeah, most welcome.
3907450	3909230	Thank you.