Hello everyone, Aftab Paki here. We have another great speaker for you.
We have Mauricio and Olaf with a presentation of Purple on My Mind,
Cost-Effective Automated Adversary Simulation.
And with that, I will let them take it away. Thank you very much, guys.
All right, thank you. Hello, everyone. Welcome to our session.
Welcome to the Blue Team Village. We're really excited to be here.
This is our first time. Thanks to all the organizers for all the hard work.
My name is Mauricio. I'm Olaf Hartung.
And today we're streaming live from the East Coast.
Myself, 10.30 here, and Olaf here is in Europe where it's 4.30 a.m. right now.
So I hope that he doesn't fall asleep during our talk.
Today we're going to be talking about setting up a cost-effective adversary simulation program.
First, we're going to go over a quick introduction, define the problem statement.
Then we're going to talk about automated adversary simulation or Purple Teaming
and kind of describe a methodology and a framework that works for us,
that we've experienced and we use.
We're going to go over the available tooling to execute this type of engagement.
And finally, we're going to talk about, show you the open source tools that we've built
and now how we've integrated them as a POC really for this talk to set up a cost-effective adversary simulation program.
We have 20 minutes of demos of our tools being integrated, so I hope you enjoy it.
We had a lot of fun building the demos, so I hope you have fun and get something out of this.
Okay, so we'll start real quick with introductions.
My name is Olaf Hartong.
I'm based in the Netherlands where I founded FalconForce together with a couple of really smart people.
And it's a really purple company, so we do a lot of offensive and defensive work.
And I'm a defensive specialist there.
And I develop quite a lot of open source software, mostly on the defensive spectrum,
which you can find on my GitHub page, which is linked over here.
Thanks, Olaf. And my name is Mauricio Velasco.
I currently lead the threat management team at a financial services organization, a Fortune 500.
My team is responsible for, among many other things, detection, response, and executing simulations, purple team exercises.
In the past few years, I've done some research and developed some tools around threat hunting and detection and adversary simulation.
So, if you want to check out some of that content, you can do that on my GitHub.
Okay. All right. So, we'll start with the introduction.
So, let's talk about deploying an effective detection program.
It's definitely not an easy task, right? I know many of you have probably done it.
We could probably talk hours about setting up a detection program, right?
There's a lot of steps, right? You have to identify the relevant telemetry, the data sources that you think you need to ingest, right?
You need to instrument your environment to basically develop an event pipeline where you're grabbing all this telemetry from your network and your endpoints
and sending it to an event pipeline through an analytics engine where you can query this information and start detecting, right?
Start creating your own detection logic to look for bad, look for suspicious behavior happening on your environment.
And this is great.
Now, I was naive at some point after deploying one of these programs where I thought, we're great, we have a detection program, detections are working, we're creating our logic, the alerts are triggering, we're investigating them.
But I learned the hard way that after you deploy a detection program, which is a challenge by itself, another challenge starts and perhaps more difficult than the one deploying the program.
And that is, we need to maintain this complex balance of tools, process, technology working together.
And it's not easy. So many things can go wrong, right?
So we need to basically deploy controls to measure the effectiveness and verify the health status of a detection program.
We're not thinking about those metrics. We need to start thinking about these metrics.
Yeah, so I have a long history in consulting and I've built a lot of these detection programs in the past.
Usually they're great or even rocking the detection.
So they're building a lot of cool stuff.
They're responding to it, as Marisha also said.
And then some of the really mature teams also have a real good hunting practice.
They do their own research into all kinds of threats that are out there or new techniques that are being applied.
However, in most cases, these teams are quite reactive in nature.
So they only respond to what is new.
And they don't always look inward really well.
So the more mature teams actually monitor all their incoming log sources.
But it's not always at the granular level.
Usually it's on a system level and then maybe only after a couple of days that there's no logs incoming.
So never on a really short timeframe.
And as someone responsible like Marisha or you're a CISO or the lead of a detection team, you might get questions from your stakeholders.
Are we in control or an auditor starts asking questions, which may raise some uncertainties on your end, right?
Yeah.
So how do you know you're actually monitoring what you expect to be monitoring?
Yeah. So what are all the problems that could happen to a detection program, right?
Especially since environments are always under constant change.
How do you know if your event pipeline is working?
I'm sure many of you will relate to something in a scenario like this where you are starting a new threat hunt for a particular hypothesis.
Or maybe your team is investigating a particular alert that is taking you to look at some logs.
And all of a sudden you realize that your firewall logs stopped sending events 30 days before.
I'm sure many of you have experienced that.
And so many things could be wrong with that, right?
It could be a licensing issue. It could be a network issue. So many things.
Another example that I'm sure a lot of you can relate to is what a GPO management means in an enterprise environment.
It can be really cumbersome, right?
There's so many GPOs for all these different sites and OUs and companies.
So what if one of your changes, someone overwrites a GPO that is defining your advanced audit policy and you stop getting your Sysmon Event 1 event or Sysmon events in general in that small part of the network where an attacker may be hiding, right?
Or another example, maybe you did a red team engagement with an awesome consulting company.
They show you all these awesome ways that they can execute Kerberos in attack.
And as part of your assessment, you create a detection to detect this. Great.
Fast forward, three months later, is that logic still working?
What if something changed on your analytic engine, maybe the schema changed, and now you're looking for a field name that is no longer the right one?
What if that complex detection logic just stopped working?
How do you know that?
Well, it's gone from there.
So let's say you don't have your own detection capability and you outsource this to a very nice vendor that promised to take care of you, monitor everything.
But how do you actually know that they are actually monitoring the right things and responding in the right way?
I mean, maybe they pop up an alert and they don't know much about your environment.
They assume it's fine, and they basically close it as a false positive, while it might have actually been a true positive.
And you might actually be already hacked.
And from that end, your stakeholders, which might be your CFO or your internal audit or whoever it might be on your end, how do you convince them that you're actually in control?
You do have a lot of detection rules and you do have a lot of incidents, but is this everything?
And how can they measure if you're doing well or not?
And from a detection perspective as well.
So how do you know your detection is actually resilient?
You do have a lot of really good detection rules, but do they cover variants?
Do they cover only the one that you read up on?
Or do they cover a multitude of ways of executing that?
So you need a way to measure this.
Which comes to our very graphic approach.
So in the center, there's a couple of pillars.
And the risk and the controls one are probably the most well-known ones, where risk is usually determined by the business, which is augmented by threat intel.
And then there might be some other values that come to a certain risk posture.
And then there are your controls, which are obviously your detections, your mitigations, your preventative measures, your hardening baselines, and so forth.
These are usually based on a couple of reference models, like MITRE ATT&CK, of course, which we all love.
There might be a kill chain involved.
There might be some additional frameworks in terms of hardening your environment.
And then there's the effectiveness one that we actually want to add to this, which will support the other ones by providing meaningful, valuable data that you can use for your management, for your auditing parties, but also for your self-management.
So, you can actually know how you're doing.
So, then it a little bit depends on what you want to do with these pillars in which team you're at.
So, if you're in the blue team, on the risk side, there's the control effectiveness that you might want to report to your regulatory parties.
And on the red side, the risk might give you a lot of input for advanced simulations that you want to execute.
And these are usually manual and very creative exercises, but these require a lot of time and money and effort.
And you can't do this on a continuous basis, right?
So, we need something else.
And this is where the purple part on the right, on the effectiveness side, comes in.
So, you can do a normal purple team, of course, which is red and blue joined together.
It's usually based on a forecast or some hypotheses.
And the outcome is usually a really good detection engineering effort where you start building all kinds of new awesome detections, which is great.
And if you can do this, you should totally do this.
But this is always limited by time and effort.
Not everybody can do this.
You don't always have an internal red team or you don't have the means to do this on a monthly basis because it's quite costly to get consultants in there every time.
So, you need something else next to that.
And this is where the automated adversarial simulation comes in.
On top of the building new detections, you also can then augment that with all kinds of validations of all your controls.
So, you can do this on a continuous basis.
And this is why we're here.
So, it's built by red and blue together to get a maximum result on a continuous basis.
This is why we're here, exactly.
So, yeah, that takes us to our next section here, automated adversarial simulation.
We strongly believe that the challenges that we described, the potential issues that could affect a detection program can be approached by deploying automated adversarial simulation exercises on corporate environments.
So, in this section, we would like to share with you what we call adversarial simulation, how we understand it, automated adversarial simulation to be specific.
I think it's important to say that we don't want to claim this term because automated adversarial simulation could mean so many different things to different people.
But in this section, we want to describe to you how we understand this concept.
So, basically, automated attack simulations or adversarial simulation is an automated activity that has two sides.
We're running simulations and then we're validating that the controls that we have previously deployed in terms of detection or maybe even prevention controls are working as expected.
They're passing the test, they're passing the simulation test.
The challenge, as Olaf was saying, purple teaming is great, right? But we cannot execute purple team exercises with 15, 10 people in a room every week, right?
How many times can you do it a year? Every quarter? Maybe every month? But not many organizations have access to high-skilled red teamers that can execute these techniques.
And it's basically also a snapshot in time. When we're doing a purple team exercise, it's just you're looking at it as a snapshot in time.
But with automated adversarial simulation, we can continuously check if those controls are working.
It's basically also a way of generating telemetry, right? And we use this telemetry to validate the environment, to validate the health of a detection program on a continuous basis, as Olaf was saying.
So with this type of program, we can go over all the challenges that we described a few slides before. We can identify issues with the event pipeline. We can identify gaps in visibility, those misconfigured GPOs missing some events.
If we're running simulation exercises against those endpoints, we will know that the events are missing, that we have a gap, right?
And what is it not? It's definitely not a red team replacement. So this is really not a threat to any of your red teamer friends, or if you're a red teamer, welcome.
So you do need red team input if you can get it, because they usually have better techniques than what you can build yourself.
So it's also not a pentesting engagement either. This is really meant to be a continuous validation or improvement phase.
And it's, of course, also not Skynet. It won't take over your company. And from that same aspect, it's not impacting continuity by design.
So it shouldn't break your systems unless you really, really want it to. And from that perspective, it should be run in production, of course.
And it's also not the new Holy Grail or the new buzzword that you should hear at RSA next year, but it should be something that you definitely need to be considering in order to be really strong.
And moving on from what I meant before, so you're not just running Benjamin Delphi's tools in your production environment, right?
You need some form of methodology to properly execute these.
So, of course, there is some governance involved. You need to establish a sort of white team, depending on how intrusive you want your tests to be.
You need to set some ground rules. There needs to be some escalation paths in case something actually goes wrong.
You need involvement from IT ops because they might need to accommodate some access to you or give you some means to take some steps.
Of course, you need a reference framework from that perspective as well. Of course, MITRE ATT&CK is there again.
But there might be some PCI controls that you want testing or COVID or HIPAA or whatever other fancy acronym out there.
You need to have some frameworks that can actually measure what you want to prove.
And then we move a little bit more towards the execution part.
So, we need to have a scope. And this scope selection phase has two crucial points that need to be in there.
So, you need to know which systems, and this can be, of course, in an office in Singapore or the US.
It can be workstation servers. Well, you can figure that out.
And then, of course, you also need to validate or to set the scope for which controls you want to test or to avoid from that end.
So, if you do know that you want to test A, B, and C, but you don't want to hit another one because then it might raise too many incidents or that system might be more volatile, you need to be aware of that.
And from these scope sessions, you can actually start designing a campaign.
And in this phase, you actually build the required objectives or techniques that you want to execute.
You add them to a playbook or a runbook or a campaign design or however you want to call it.
You test it, of course, first, and then you run it in production, obviously.
And one of the steps in here also is defining the cadence of testing.
So, do you want to do this hourly, daily, monthly, whatever your preference is, and the timing between all these techniques.
Do you want to run it within, I don't know, 10 minutes? Do you want it to take an hour?
This can also impact your detection rules, right?
So, you need to vary this a little bit on and off whenever you do this.
And then, of course, you need to report and validate all of your outcomes.
So, you want to know which test you ran.
You want to know the results of each individual test, whether it was successful or not.
You want to know the commands that were executed or the tools that were launched,
or all the relevant information that a blue team needs, including the time that it actually happened.
If they didn't detect it, they need to be improving their detection, so they need this data.
And, of course, ideally, you also want to have your alerts or your incident tickets or all kinds of telemetry
that you can combine in the report where you can show the full end-to-end cycle,
where you executed something, you did detect it, and it was actually also picked up by the team properly or not.
So, whether they actually flagged it as a true positive is also a very important measure.
Yep. I want to follow up on the scoping piece that you were mentioning,
because I feel when executing PRPL team exercises,
we really need to think about which parts of our network we're targeting,
because you're always running exercises on the same parts of the network.
Again, we're going to go back to the same issue as before.
How do you know if things are working end-to-end across your environment?
So, there needs to be a little bit of randomness on that scoping as well.
Totally. Yeah, you should also start thinking about,
how do I integrate this into my detection engineering process?
And this slide has been shown in multiple variants.
I think the first one was built by Jared Atkinson over at Spectrops,
and it's a really, really nice flow of building detection, new detection rules.
And I only highlighted a few of the steps that are interesting for our current talk,
because, of course, you need to be working on all these steps
that you can now probably hardly read by intention.
But in the first phase, when you start thinking of,
okay, what do I actually want to add as a detection?
You're already looking at threat intel, industry reports.
You might also have your internal red team or an external red team that you can consult.
So, how can I execute this stuff?
And in the next phase, when you start investigating,
you want to investigate as much ways of executing that specific technique that you want to detect.
And you also will want to be starting to develop scripts that you can then use in a later phase
to validate all of the detections that you are trying to build in the first place,
so that you already think a little bit about, how do I repeat this process later on?
And this is also where you start implementing it in phase four,
when you are building your detection rule and actually starting to implement it in production.
You also want to start implementing these validation scripts or tools or whatever you've created
into the same process, so that you can actually already, from the minute it goes live,
start measuring its efficiency, whether it's working correctly,
whether the outcome is actually as you expect it to be.
And this is, of course, a delicate process where you don't want to be creating detections
only tailored to your own validation scripts.
So, also these validation scripts need to be maintained, they need to be expanded.
You need to keep on researching whether it's resilient enough, of course.
Yeah, ultimately, as we are creating detections, each one of them should have its own unit test, right?
We see detection as an engineering process, we just need to test each one of our products.
And one big requirement for us is, we believe that it needs to be end-to-end.
And I think there's also something to bring up here where some challenges with other approaches,
where I would like to differentiate between unit testing and functional testing,
where in unit testing you can perhaps inject to the event pipeline a particular telemetry,
maybe generated by the execution of adversary techniques, to test the detection logic.
And that's great, we should do that to test the detection logic.
But that misses a point, because you're assuming that the event pipeline is working as expected.
So, I think we need to complement this type of testing with functional testing, end-to-end,
testing a technique on a specific host, so that you're only testing the engine,
but you're testing the data source generating telemetry.
So, it's proper configuration, the data event being sent through the event pipeline,
through the analytics engine, and ultimately generating that detection.
So, it needs to be end-to-end, right Olaf?
Definitely, yeah. And to the last two words in the sentence, it needs to be also in production.
Because I've seen labs and acceptance environments, and in my long experience as a consultant,
it's never been really lifelike.
It can be close, but it's only on maybe a handful of clients I've been to,
where acceptance actually comes close to production.
But still, production is always different.
And this is the stuff that you're trying to protect, right?
So, why would you be executing all this stuff in a lab and then hope it works in production?
You need to make sure.
So, what are some of the requirements if you want to run an automated adversarial simulation exercise?
I think similar to a PRPL team exercise, in this case, I think the requirements are similar.
You need a defensive capability.
It doesn't have to be cutting edge, like with the whole visibility and hundreds of detections.
You can have less mature, and still, it will help to test the controls that you have in place.
Of course, you have to tailor the simulation exercise to the controls that you have in place.
But you need to have some kind of visibility in a detection program, right?
Otherwise, you don't know which controls to test.
Ideally, and definitely, we need some knowledge of the threats that we are executing, right?
So, this is, again, where the PRPL comes in on the talk, right?
So, we as detection engineers, as a detection team, we need to be aware of these techniques.
Maybe not to the level of the Red Team does and execute them like a Red Team does,
but at least to the point where we can simulate them and confirm that our logic is working.
It's testing, our programs are working.
And, of course, we need tools, right, to simulate this.
We cannot be just downloading a VM and maintaining hundreds of different tools to execute all these techniques.
Ideally, we need one tool or a set of tools that will give us the most scope as possible in executing these techniques.
And these tools need to have certain features and capabilities, right?
They need to be able to be automated.
They need to generate logs, specific logs, to be able to compare the execution of techniques with the detection controls that are triggering, right?
So, tooling is really important.
And we'll talk about that on the next section.
And it also requires that you have a sort of an adversarial mindset.
You need to not only think about how do I defend this stuff, but also how can my company be attacked?
And sometimes it's a little bit hard, so this is also where tools might actually help you a little bit.
But you need to be a little bit aware of what can be done in my environment, what can't be done.
And then all of these things need to be validated, of course.
And in order to do this, usually in a more normal organization, you actually need some management buy-in, because they need to approve you spending time on this, since it will cost a little bit of money.
And if you want to buy one of the commercial tools, you of course need some more budget.
And if you're a little bit bold, as I've done in the past as well, you can also just do it and maybe say sorry later if they get upset.
The big upside to this, if they don't really understand the value of a program like this, is that you can immediately prove them the results with some more details.
But frankly, every type of management should be interested in this, whether they're very financially focused or not, because this will give you a lot of insight into how you're actually doing as a team.
And of course, if you have a security vendor that does monitoring for you, you need their involvement as well, because otherwise you'll spoil a good relationship and you need to actually be working together since they're doing part of your detection work.
Yep. So let's talk a little bit about the tools that are out there that can help us in executing automated adversarial simulation exercises.
So there's definitely a lot of tools out there, right? We're not reinventing the wheel here, really.
There's commercial tools. There's some open source tools. We're going to stay away from the commercial tools since that's not our focus today.
But let me tell you a little bit about my experience. So when we first started executing PRPL team exercises internally, our natural approach was to use red team tools, right?
Red team tools, like Metasploit, Empire, Parse2. And then we started validating controls, which worked great at the beginning when we were more on the form of a manual PRPL team exercise where maybe once a week we check a particular technique manually.
But that soon stopped being practical because as the number of detection grows and as our testing moves to a weekly cadence, it's not practical to have an engineer just getting all these shells from your C2s and then start running things manually.
Now, some of these tools do have APIs, and there's actually adversary simulation tools that leverage the APIs, so they can definitely be used.
Then some of these tools are also fully automated and allow you to create a stand-up infrastructure in an automated way where you can run simulations and hopefully your endpoint controls will catch them, right?
But I think a big challenge with that is that you are running these simulations from fixed infrastructure, from the same host every time.
And that takes me back to end-to-end, right?
Because with this type of exercise, you are not really confirming if your event pipeline is working or not.
You're just confirming if that host, which is built for that, is generating the telemetry and if your detection is triggering.
So, they're great, these tools are great, but they have some limitations.
Ultimately, I think some of these tools have been built to be run manually, right?
On a host, like on the screen, interactively.
And that's fine, they still have a value, but some of them have not really been built for automation.
The ones that do actually have some complicated ways of building scenarios or adding new techniques, it's not always easy.
And that's also the drawback with open source, probably also with my own tools, where somebody has a certain way of working in mind, it works for them.
But for a new adopter, it's actually quite hard to start working with it.
And from a detection perspective, not all these tools have a really verbose way of telling you what happened, when it happened, and so on.
So, the output is quite limited, which is hard then to match against your detections or put into a more measurement program.
And as Mauricio mentioned, some of them are very static in terms of where they run from, or they're not as realistic, where it's just a bunch of Python scripts, which are useful.
But it also is not always the way that an attacker would do it.
So, how realistic is it for a SOC then to be responding to this?
And lastly, not all of them are capable of doing stuff remotely.
So, it's usually from that same system, they do something on that system.
And then that's that.
Which is also not always the way an attacker would work, of course.
Yep. So, which takes us to our tooling, right?
So, having mentioned all these challenges and the approach,
Olav and I have both built tools that have different goals.
And we've integrated them to execute a POC for automated adversary simulation.
So, we want to share some of these tools in the next section.
So, I'll start on mine.
So, actually, yesterday at Black Hat Arsenal, I released my adversary simulation tool called PurpleSharp.
So, you can go to the GitHub page and you'll find all the information you need.
But basically, PurpleSharp is a tool in C Sharp that executes adversary techniques within Windows AD environments.
So, it's focused for AD environments.
It follows a MITRE ATT&CK framework as far as the techniques.
And the main goal of PurpleSharp is to help us execute some of these simulations to generate the telemetry that can help us build, test, and enhance detection controls, right?
That's what it does, basically.
It just executes techniques and will generate telemetry.
So, if you have a properly monitored Windows environment, Windows domain environment, this telemetry will make its way to the event pipeline, to your analytics engine.
And if you have coverage for the techniques that PurpleSharp supports, you should validate your controls, right?
So, what are some of the... why PurpleSharp?
Why is it different?
We've talked about some of the tools that exist already.
So, I want to mention a few points real quick.
I know we're not great on time, Olaf, so I'll go fast.
Simplicity, it's one simple .NET binary.
All you need is to compile a .NET binary and you can just grab it and copy it to a Windows endpoint and run simulations against remote hosts from that.
Really easy.
No C2, no C2 channels, no infrastructure, no VMs, one binary.
Verbose logging, as Olaf was saying, we need the audit trail of all these techniques that executed at what point, from which processes, from which process ID.
So, we can grab that data and maybe if our detection controls didn't trigger, we can at least find that telemetry and create a new detection or identify why the detection didn't trigger, right?
Remote simulations.
PurpleSharp is actually able to execute simulations or what I want to call deploy simulations on remote endpoints.
So, once again, the challenge of verifying that the event pipeline is working or not can be easily approached with PurpleSharp.
All you have to do is point your simulation to a host in your Singapore office and you'll confirm if that simulation will be detected as good as the one that in your headquarters, right?
Credible simulations.
I've used a couple of techniques with PurpleSharp to make sure that the simulation is credible.
I'll mention a couple of things.
You can go through the documentation to really understand, but one is process ID, parent process ID spoofing, PPID spoofing technique.
I use this technique to be able to deploy simulations on remote endpoints and run the simulations under the context of the user who's logged in on that host at that point, right?
So, no longer we have to run simulations from that same infrastructure, from that same user that our SOC already knows.
And it's just going to close that alert and you're not really going to train them unless these simulations run from real users on real computers.
Randomness.
PurpleSharp will leverage simple LDAP queries to query your environment and pick random hosts for the simulations.
For example, in a password spray attack, it's going to pick random users.
If it's going to do a network sharing iteration, it's going to pick random hosts.
So, that randomness, once again, gives us that end-to-end testing.
Attack variations.
The last one, one really important for me is what Olaf was saying before.
How do you know if your detection is resilient to a different way of executing the same attack technique?
Well, I've tried to execute the same technique in a couple of ways, at least.
For example, a simple example is running PowerShell, calling PowerShell.exe, or running it from .NET using System Automation Management, DLL.
So, one simple example, PurpleSharp can execute both for that technique.
Real quick, the architecture.
PurpleSharp consists of three modules.
The orchestrator, the scout, and the simulator.
The orchestrator will run on the operator endpoint.
And the role of the orchestrator is to interact with the simulation target to deploy the simulation.
The scout will run on the simulation target.
And the role of the scout is just to obtain information about that host.
For example, who's the user that's logged in?
What's the process ID of Explorer.exe?
So, we can use the parent process spoofing technique.
Parent process ID spoofing technique.
And the simulator, it just runs the simulation.
So, these three modules actually communicate with each other using name pipes so that they can orchestrate the simulation.
Again, if you want to learn more about this, go to the documentation.
But at a high level, this is how PurpleSharp works.
It's going to first upload... once you point it to a remote endpoint, it's going to upload the scout using SMB.
It's going to execute it using WMI.
Then the scout service starts on the remote endpoint.
And there's some communication on name pipes.
That's where the orchestrator instructs the scout to run a recon on this host.
And that's where we find out who's the user that's logged in.
Based on this information, the orchestrator will actually upload the simulator under the path of that user so that our simulation runs under the path of a real user and a real computer.
Once again, credible simulations.
And once the simulator has been uploaded, the orchestrator triggers the simulation.
The scout will execute the simulator using parent process ID spoofing.
That way, the simulator runs under the context of the user and also as a child process of Explorer.exe.
Once again, making the simulation credible.
This is just one example of an attack variation, as I was saying.
I can execute PowerShell just calling create process API, calling PowerShell.exe, which I could detect if I have some kind of EDR or this event ID being logged.
But on the second technique, we're using .NET.
So your EDR or your Sysmon or your event 4688 will not detect that.
But you need to have PowerShell logging enabled to detect that.
Once again, how resilient is your PowerShell detection technique?
We don't have time, so I'm not going to mention the other variations.
Go ahead, Olaf.
I've built an app for Splunk, which is called ThreatHunting.
Not the best name, but it's very descriptive, at least.
And it's a very graphically oriented tool that will run about, I think, 160 hunts or queries or triggers, as you want to call it, throughout your environment on a periodic basis.
And it will give you a lot of telemetry, which you can then start using for either moving your detections, or hunting, of course, which is the main intention of the tool.
But it can also provide you with a lot of visibility into all of the triggers that have been going on.
Thanks.
So, there's a lot of detections for all kinds of techniques on Windows.
It primarily utilizes Sysmon and the Windows native event logs right now.
It's heavily MITRE ATT&CK focused, because this is a framework that I really embraced.
And the primary goal was to provide an investigative workflow for hunters, but also for detection engineers to provide as much context as possible that is all related to the same events without having to click and copy paste a lot of times.
So, it's meant as an efficient way of going through your data based on triggers.
Yep.
So, the POC that we want to show you today is basically how we've integrated both our tools to use PurpleSharp as a simulation engine and the ThreatHunting app as a detection engine, so we can validate that these detections are being picked.
So, now we move to the demos, the fun part, guys.
You must be bored about us talking.
So, this is going to be the fun part.
So, in the first demo, we'll execute three execution, the execution tactic, MITRE tactic execution, three techniques, PowerShell, Windows Command Shell, and Jscript.
These are common techniques.
If you read most of the malware reports, you probably see that malware or most likely will use some of these techniques.
So, we'll move to the first demo, and hopefully this works.
Okay.
All right.
Let me see if I can undo full screen.
One second, of course.
Doing live, it's always...
What if you go out of full screen on the browser first?
Let's do that.
Thank you.
All right.
So, in this first demo, we have two endpoints.
Let me go back real quick.
Two endpoints.
We have PC Sam, where we're going to run PowerShell from, and we have another computer called PC Judy, where we're going to run the simulation against.
Okay?
All right.
So, we're going to use PowerShell command line in this case.
This case, we're going to...
If you want to learn more about the parameters, please go to the documentation, but basically we're running a simulation against PC Judy using a service account that needs to be an admin on PC Judy,
and we're going to run these three techniques, and we're going to slip five seconds between the execution of each one of these techniques.
So, we're going to pass the password of this service account that we're going to use to run the simulations against this host.
So, we see here that the scout gets uploaded, recon happens, and we identify that Judy Brody is logged in on this host.
So, now the simulator is uploaded to a path within the user's profile, making it look as if Judy downloaded a fake Firefox installer.
Then the simulation runs using parent process ID spoofing, so making Firefox installer a child process of Explorer running under the context of Judy.
And as I was mentioning before, this is the report that we get back.
So, we see all the details of when it ran, which technique, again, from which process ID, from which path, which command was executed.
In this case, we have a parser command, we have CMD, who am I, a simple who am I, and then we execute what looks to be a malicious Jscript file.
And all of them in the context of Judy, and then the PurpleShark will delete some of these logs files.
So, now we go to our Splunk environment to confirm these techniques were executed.
And as we can see here, we see the Scout running under the context of the PSharp service account and as a child process of WMIPRDSE.
And then we see the simulator, now this time running on the context of Explorer and running under the context...
sorry, as a child process of Explorer and running on the context of Judy, so the real user who's logged in on that host.
And then PowerShell is run, WhoAmI is run, and this suspicious Jscript is executed, right?
And if we go to the 4104 PowerShell event, we can see how the script was actually just asleep, 20 seconds asleep, right?
So, as a simple test, we confirm that this simulation run three techniques remotely, really easy with one command.
We can run with PurpleShark.
We'll go to the next one real quick.
Let's see, I'm just going to fast forward because of time here.
We're going to execute a different technique.
I honestly forgot what this technique is.
I don't speak ATT&CK, but we'll see what it is.
I forgot.
So, follow the same process.
I'm just going to fast forward real quick.
Okay, yes, in this case, it's actually the same technique, PowerShell, but this time running using system management automation, right?
So, a different way of executing PowerShell.
We're no longer using PowerShell.exe, right?
So, if we see on the logs, we don't see PowerShell running.
All we see is Firefox Installer because Firefox Installer, the simulator, executed PowerShell with .NET.
But if we look at event ID of module loads, I forgot what the event ID of that on this one is, but we see the system management automation being loaded by the simulator.
So, that's how we can catch that this guy is executing PowerShell techniques, right?
So, this is our first demo, really first way of just running a couple of simulations.
So, in the next demo, we'll run a few techniques on the persistent tactic.
We're going to use some race to keys, schedule task.
We're going to, on defensivation, we're going to use portable execution.
Sorry, portable executable injection.
So, PE injection.
And then we're going to dump LSAS using PurpleSharp.
So, Olav will take you on this demo.
All right, let me introduce some new stuff here.
So, as you can see, there's this JSON structure, which is called the playbook JSON structure, where you can define all the variants that you want to do.
So, here is where you schedule your scenarios, basically.
And you need to define some basic stuff.
So, username, domain, the domain controller, the sleep between all the techniques or the small playbooks that you want to execute, and then the playbooks themselves.
So, of course, you give them a name.
You give them a host that you want to target.
So, it can either be a host name or a random host.
And then, as Mauricio explained, it will do an LDAP query and find one itself.
You can here also define the path of the scout and the simulator where you want to be running it from.
So, you can actually make it even more realistic.
The PB sleep is the sleep between these playbook steps.
These two techniques for the registry and the scheduled tasks, they will be run 30 seconds in between.
And then the next playbook will start after the playbook sleep.
And over here, you can do the same thing.
So, the command line is a little bit easier here, where you just say PB for the playbook and the playbook name.
And you basically run it from there.
Once you, of course, provide a password, you could do this on a command line.
But I would advise against that, especially on a demo environment.
So, here it will show you that it will execute three playbooks.
And it will start with the first one.
It first does an LDAP query to find the targets that it can run it against.
It picked a random one.
In this case, it's PC Judy again.
And it's executing all these techniques on that machine.
Well, this takes a little bit of time.
So, we have to wait for all the techniques to be executed.
And this is where the pipe communication comes in.
So, it actually checks whether it's done.
And once it's done, it will also clean up after itself, so that at least you won't lose any traces.
All the binaries will be gone again, if your EDR didn't stop it already, of course.
Which is also a way of validating whether it worked or not.
And there you go.
So, they are executed.
So, you show it was started, where it was started from.
You see the key that was added to the registry.
It was successfully created.
Then it was deleted, because you don't want it to be there.
And the simulation is finished.
Same goes for this technique, where you see a different path and the same parent process ID that it actually executed from.
And you see the command that was executed in full detail, which you can then also use in your detection later on, to see whether you actually found it.
And in the meantime, it's starting to execute another one on a different machine.
And this is the PCMP.
It's a different simulator that you see running under a different user context.
And it's actually injecting into Notepad.
So, you see exactly what it did.
So, it does an open process for VirtualAlloc.
Then it writes to the process memory and creates a new thread from that Notepad.exe.
And then the last technique that we mentioned was running LSAS.
And you see that it actually ran.
It identified the LSAS process.
It tried to obtain a handle.
It called the mini dump.
It wrote it.
It dumped it there.
And it fortunately also deletes it, because you don't want to, again, leave stuff for real attackers.
And then when it's done, it actually writes a JSON file with all the results,
which you can then easily import into your analytics platform of choice,
which in our case will be Splunk.
But you can see here that the playbook is there.
And then there's a results JSON, which once you format it,
it shows you all the stuff that we actually just saw on the timeline,
but then in a nice JSON structure so that you can actually start importing them.
And then from the perspective of time, we might want to scroll through a little bit.
Yeah, makes sense.
So, ideally, this is the integration, right?
So, this is where you ingest this into your other platform.
So, go ahead.
Exactly.
So, I created a small POC dashboard, again, just to show what kind of data is in there.
So, you can see that the orchestrated PCs of SAM executed the techniques,
the techniques that you see on the right, on the machines that were involved.
And you can show these over time.
So, if you do a longer playbook than I did, you will see actually a longer scope, of course.
Here you see the playbooks that were executed.
And if you know my threat hunting app, you can click on everything.
So, it's the same over here.
So, once you click on one of those playbook names, you will get all the raw logs with all the details in the panel below,
which will show you all the results that you also saw on the command line.
So, once you click it, it will actually change to the proper playbook.
And the same will go for all the other fields.
So, once you click on a MITRE technique ID that was tied to one of those techniques...
Hey, the stream is paused.
Oh, there we go.
It will take you to my threat hunting app and will do a direct query for that technique being executed.
And you see that also it was detected by the logic in my app, fortunately.
And it will give you all the details that might help you contextualize it even more once you start playing with the app.
And the same can be done for clicking on a machine, which I don't do yet.
But I'll show you the raw logs first, where you can see all the results from all the techniques that were executed.
So, again, you can quickly spot that everything was executed.
So, you see the scout being run.
You see that then the WMI is being called.
The process is being ejected into and it starts executing the registry commands and so on.
And the same goes for the scheduled task logs, which isn't that properly formatted, thanks Microsoft.
But this is a quick POC.
You can obviously properly parse this neatly.
But it's quite easy to spot that some of the bad stuff actually was executed here.
And I think we can fast forward again.
Okay.
So, yeah, we're going to the threat hunting app.
Yeah, and this is the same.
So, once you would have clicked on a machine, you would have actually seen all the techniques that were executed on that system in the threat hunting app, which will give you a huge panel with all techniques, all the commands that were executed, all the parent processes, process GUIDs from a Sysmon perspective.
And it will try to at least map it to the proper MITRE tactics as well, as much as possible.
And here you can see that there were some DLLs loaded, there was some registry activity, and you still have access to all the raw data as well.
And that concludes the demo two.
Okay, cool.
Demo two.
Let's move on.
Okay, so I'll do this real quick.
Of course, ATT&CK MITRE framework is our framework of choosing.
And PurpleSharp does integrate with the ATT&CK Navigator to visualize these techniques.
So, with simply running just Navigator export, PurpleSharp will create a Navigator layer that you can open up and upload to the Navigator.
I'm going to fast forward here so that we make it on time.
Just upload that and you'll see visually all the techniques that PurpleSharp supports.
So, you can use it to pick the techniques that maybe you want to simulate on your automated adversary simulation exercise.
And in a similar way, I actually support importing Navigator layers.
So, in this case, I'm going to get a layer for APT, I think, 29.
Oh, sorry, FIN7.
Yeah, I picked FIN7.
On Navigator, I export that layer to a JSON file.
So, I'm going to fast forward here.
And then I move it to the folder where PurpleSharp is.
And now, in this case, I'm just going to import that Navigator layer.
And PurpleSharp will analyze the techniques that it does support and it doesn't support.
So, it will let you know.
But then it finally creates, in this case, it supports 17 techniques out of 29.
And it's going to create a JSON playbook like the one Olaf just run on demo number two.
And now you have a simulation playbook for FIN7 that you can run on your remote endpoints, right?
So, pretty cool.
There.
Okay, so...
Yeah, I think we only have one minute left, Marcia.
So, we have to publish this just online, I guess.
Yes, so...
Okay.
Sorry.
Yeah.
So, I've noticed on the Twitch that some of the demos were not showing great.
So, right after this, I'm going to upload them on YouTube also.
So, you see them.
But the last demo, yeah, it's also a cool one, but you'll see it on YouTube, okay?
It'll be uploaded there.
All right, you just want to take this last one, Olaf, since we're running out of time.
Yeah, sure.
In the last couple of seconds, I just want to highlight that this is a huge value in terms of control,
but also in terms of reportability and also ensuring that you're actually working on the right stuff.
So, it's also ease of mind, probably.
So, I strongly encourage everybody to just start with this.
And especially since there's a bunch of open source tools out there, including ours, it doesn't have to be very expensive.
So, you can start small, and then once you start maturing, you might want to look at a commercial tool or not.
And this should be really a nice way of implementing a way of testing all your detection engineering efforts,
so you actually know it's not in vain, you know it's actually doing something.
Because some of them probably won't trigger in production ever, hopefully, at least for you.
So, you at least know it still is working.
And it shouldn't be a grading mechanism for your team either.
It can be used that way, but you should see it as a constructive tool and not as a sort of slapping tool if you didn't do it well.
And of course, if you don't have a detection yet, you can of course use it also in a way of the purple exercise that we meant before,
where you can start building new detections based on the techniques that we execute,
but then make sure that you don't build tunnel vision detection.
Make sure it's still resilient and it's open for variants.
So, thanks everyone. That concludes our chat, our session today.
You have the links for our tools and I'll also be... check out our Twitter.
I'll also be releasing the links on the YouTube videos for the demo, so you can see them with a higher resolution there.
But yeah, thanks for coming to our session. I hope you guys had fun. And thank you.
Thanks, guys. Girls?
Thank you, both of you.
We really appreciate your demo and your presentation.
As a reminder to everyone, we encourage everyone to join the Blue Team Discord and head over to Text Talk Track 1.
And we can have the presenters answer some of your questions there as well.
Right off the bat, I did see a couple of questions. So, if you guys don't mind.
Okay.
Scroll up.
The question was, what are some of the major benefits of PurpleSharp versus Atomic Red Team?
Yeah, okay. That's a great question. So, I think there are different things.
Atomic Red Team is more a way of describing how to execute the techniques and not really an engine to execute the techniques themselves.
Although the Red Canary team is working on Invoke Atomic, which is a partial tool that will be able to execute Atomic Red files.
But if you ask me the difference between Invoke Atomic and PurpleSharp, I would say running the simulations under the context of a real user and as a child process of explorer.
So, creating that credible simulation, that's something PurpleSharp can do.
Another one is randomness, being able to run a technique against a random host to really identify if your detection controls are working end-to-end across your environment. That's the second one.
And I guess the third one would be just simplicity. Maybe you can run simple binary that you can just copy on a host and run simulations from.
Very good. Cool.
I think that's it for the questions for now.
Oh, here we go.
Did you guys teach a course on this during this week?
No, we didn't. But maybe that's a good idea. Maybe we can do that on the next one.
That will be fun. Yeah, definitely.
I know we had fun doing the demos.
Yeah, too much.
Too much.
Maybe something in the future that you can do a workshop.
Yeah, I think that's it for this. And if you guys don't mind sticking around for a little bit and let's talk track one.
Sure.
If people have questions.
So one last thing is just I'll be posting the links of all the demos right after this. So check out our Twitter.
And you'll be able to watch all the third last demo, which was also really cool. We didn't get a chance to.
Cool. Very good. Thank you guys again. I appreciate it.
Yeah, most welcome.
Thank you.