1
00:00:07,440 --> 00:00:09,320
Hello everyone, Aftab Paki here.

2
00:00:09,320 --> 00:00:11,400
We have another great speaker for you.

3
00:00:11,640 --> 00:00:22,140
We have Mauricio and Olaf with a presentation of Purple on My Mind, Cost-Effective Automated Adversary Simulation.

4
00:00:22,280 --> 00:00:25,000
And with that, I will let them take it away.

5
00:00:25,000 --> 00:00:26,420
Thank you very much, guys.

6
00:00:27,340 --> 00:00:28,740
All right, thank you.

7
00:00:28,760 --> 00:00:29,740
Hello, everyone.

8
00:00:29,740 --> 00:00:31,100
Welcome to our session.

9
00:00:31,100 --> 00:00:33,420
Welcome to the Blue Team Village.

10
00:00:33,420 --> 00:00:34,980
We're really excited to be here.

11
00:00:34,980 --> 00:00:36,500
This is our first time.

12
00:00:37,180 --> 00:00:41,780
Thanks to all the organizers for all the hard work.

13
00:00:41,780 --> 00:00:43,260
My name is Mauricio.

14
00:00:43,720 --> 00:00:45,360
I'm Olaf Hartung.

15
00:00:46,200 --> 00:00:49,800
And today we're streaming live from the East Coast.

16
00:00:49,800 --> 00:00:55,700
Myself, 10.30 here, and Olaf here is in Europe where it's 4.30 a.m.

17
00:00:55,700 --> 00:00:56,260
right now.

18
00:00:56,260 --> 00:01:00,140
So I hope that he doesn't fall asleep during our talk.

19
00:01:00,140 --> 00:01:08,640
Today we're going to be talking about setting up a cost-effective adversary simulation program.

20
00:01:09,280 --> 00:01:15,260
First, we're going to go over a quick introduction, define the problem statement.

21
00:01:15,740 --> 00:01:29,640
Then we're going to talk about automated adversary simulation or Purple Teaming and kind of describe a methodology and a framework that works for us, that we've experienced and we use.

22
00:01:29,960 --> 00:01:37,620
We're going to go over the available tooling to execute this type of engagement.

23
00:01:37,700 --> 00:01:54,680
And finally, we're going to talk about, show you the open source tools that we've built and now how we've integrated them as a POC really for this talk to set up a cost-effective adversary simulation program.

24
00:01:54,680 --> 00:02:00,140
We have 20 minutes of demos of our tools being integrated, so I hope you enjoy it.

25
00:02:00,140 --> 00:02:06,640
We had a lot of fun building the demos, so I hope you have fun and get something out of this.

26
00:02:07,880 --> 00:02:11,720
Okay, so we'll start real quick with introductions.

27
00:02:13,160 --> 00:02:15,020
My name is Olaf Hartong.

28
00:02:15,680 --> 00:02:21,980
I'm based in the Netherlands where I founded FalconForce together with a couple of really smart people.

29
00:02:21,980 --> 00:02:26,200
And it's a really purple company, so we do a lot of offensive and defensive work.

30
00:02:26,200 --> 00:02:28,300
And I'm a defensive specialist there.

31
00:02:28,680 --> 00:02:38,760
And I develop quite a lot of open source software, mostly on the defensive spectrum, which you can find on my GitHub page, which is linked over here.

32
00:02:40,160 --> 00:02:41,080
Thanks, Olaf.

33
00:02:41,080 --> 00:02:43,500
And my name is Mauricio Velasco.

34
00:02:43,860 --> 00:02:51,160
I currently lead the threat management team at a financial services organization, a Fortune 500.

35
00:02:51,160 --> 00:03:00,520
My team is responsible for, among many other things, detection, response, and executing simulations, purple team exercises.

36
00:03:00,880 --> 00:03:11,140
In the past few years, I've done some research and developed some tools around threat hunting and detection and adversary simulation.

37
00:03:11,140 --> 00:03:15,240
So, if you want to check out some of that content, you can do that on my GitHub.

38
00:03:15,980 --> 00:03:16,880
Okay.

39
00:03:17,320 --> 00:03:18,200
All right.

40
00:03:18,200 --> 00:03:21,780
So, we'll start with the introduction.

41
00:03:22,260 --> 00:03:26,920
So, let's talk about deploying an effective detection program.

42
00:03:26,960 --> 00:03:29,380
It's definitely not an easy task, right?

43
00:03:29,380 --> 00:03:32,220
I know many of you have probably done it.

44
00:03:32,220 --> 00:03:37,100
We could probably talk hours about setting up a detection program, right?

45
00:03:37,100 --> 00:03:38,700
There's a lot of steps, right?

46
00:03:38,700 --> 00:03:47,620
You have to identify the relevant telemetry, the data sources that you think you need to ingest, right?

47
00:03:47,620 --> 00:04:09,360
You need to instrument your environment to basically develop an event pipeline where you're grabbing all this telemetry from your network and your endpoints and sending it to an event pipeline through an analytics engine where you can query this information and start detecting,

48
00:04:09,360 --> 00:04:09,560
right?

49
00:04:09,560 --> 00:04:17,900
Start creating your own detection logic to look for bad, look for suspicious behavior happening on your environment.

50
00:04:17,900 --> 00:04:19,700
And this is great.

51
00:04:19,700 --> 00:04:34,680
Now, I was naive at some point after deploying one of these programs where I thought, we're great, we have a detection program, detections are working, we're creating our logic, the alerts are triggering, we're investigating them.

52
00:04:35,040 --> 00:04:49,060
But I learned the hard way that after you deploy a detection program, which is a challenge by itself, another challenge starts and perhaps more difficult than the one deploying the program.

53
00:04:49,060 --> 00:04:57,280
And that is, we need to maintain this complex balance of tools, process, technology working together.

54
00:04:57,280 --> 00:04:58,340
And it's not easy.

55
00:04:58,340 --> 00:05:00,120
So many things can go wrong, right?

56
00:05:00,120 --> 00:05:10,260
So we need to basically deploy controls to measure the effectiveness and verify the health status of a detection program.

57
00:05:10,260 --> 00:05:12,100
We're not thinking about those metrics.

58
00:05:12,100 --> 00:05:16,040
We need to start thinking about these metrics.

59
00:05:18,000 --> 00:05:23,380
Yeah, so I have a long history in consulting and I've built a lot of these detection programs in the past.

60
00:05:23,380 --> 00:05:27,180
Usually they're great or even rocking the detection.

61
00:05:27,180 --> 00:05:29,140
So they're building a lot of cool stuff.

62
00:05:29,140 --> 00:05:31,940
They're responding to it, as Marisha also said.

63
00:05:32,640 --> 00:05:36,840
And then some of the really mature teams also have a real good hunting practice.

64
00:05:36,840 --> 00:05:43,000
They do their own research into all kinds of threats that are out there or new techniques that are being applied.

65
00:05:43,000 --> 00:05:46,920
However, in most cases, these teams are quite reactive in nature.

66
00:05:46,920 --> 00:05:49,680
So they only respond to what is new.

67
00:05:49,900 --> 00:05:54,460
And they don't always look inward really well.

68
00:05:54,500 --> 00:05:59,640
So the more mature teams actually monitor all their incoming log sources.

69
00:06:00,020 --> 00:06:02,600
But it's not always at the granular level.

70
00:06:02,600 --> 00:06:09,040
Usually it's on a system level and then maybe only after a couple of days that there's no logs incoming.

71
00:06:09,040 --> 00:06:11,880
So never on a really short timeframe.

72
00:06:11,880 --> 00:06:20,480
And as someone responsible like Marisha or you're a CISO or the lead of a detection team, you might get questions from your stakeholders.

73
00:06:20,480 --> 00:06:27,620
Are we in control or an auditor starts asking questions, which may raise some uncertainties on your end, right?

74
00:06:29,660 --> 00:06:30,440
Yeah.

75
00:06:30,440 --> 00:06:37,980
So how do you know you're actually monitoring what you expect to be monitoring?

76
00:06:42,150 --> 00:06:42,570
Yeah.

77
00:06:42,570 --> 00:06:48,230
So what are all the problems that could happen to a detection program, right?

78
00:06:48,250 --> 00:06:53,590
Especially since environments are always under constant change.

79
00:06:53,590 --> 00:06:57,270
How do you know if your event pipeline is working?

80
00:06:57,410 --> 00:07:06,170
I'm sure many of you will relate to something in a scenario like this where you are starting a new threat hunt for a particular hypothesis.

81
00:07:06,170 --> 00:07:13,950
Or maybe your team is investigating a particular alert that is taking you to look at some logs.

82
00:07:13,950 --> 00:07:20,730
And all of a sudden you realize that your firewall logs stopped sending events 30 days before.

83
00:07:20,730 --> 00:07:23,450
I'm sure many of you have experienced that.

84
00:07:24,150 --> 00:07:26,530
And so many things could be wrong with that, right?

85
00:07:26,530 --> 00:07:28,970
It could be a licensing issue.

86
00:07:28,970 --> 00:07:30,490
It could be a network issue.

87
00:07:30,490 --> 00:07:32,010
So many things.

88
00:07:32,890 --> 00:07:42,210
Another example that I'm sure a lot of you can relate to is what a GPO management means in an enterprise environment.

89
00:07:42,210 --> 00:07:44,890
It can be really cumbersome, right?

90
00:07:44,890 --> 00:07:50,310
There's so many GPOs for all these different sites and OUs and companies.

91
00:07:51,130 --> 00:08:10,410
So what if one of your changes, someone overwrites a GPO that is defining your advanced audit policy and you stop getting your Sysmon Event 1 event or Sysmon events in general in that small part of the network where an attacker may be hiding, right?

92
00:08:11,290 --> 00:08:18,390
Or another example, maybe you did a red team engagement with an awesome consulting company.

93
00:08:18,390 --> 00:08:23,790
They show you all these awesome ways that they can execute Kerberos in attack.

94
00:08:23,870 --> 00:08:28,950
And as part of your assessment, you create a detection to detect this.

95
00:08:28,950 --> 00:08:29,590
Great.

96
00:08:30,450 --> 00:08:34,690
Fast forward, three months later, is that logic still working?

97
00:08:35,430 --> 00:08:44,670
What if something changed on your analytic engine, maybe the schema changed, and now you're looking for a field name that is no longer the right one?

98
00:08:44,670 --> 00:08:48,090
What if that complex detection logic just stopped working?

99
00:08:48,090 --> 00:08:49,730
How do you know that?

100
00:08:52,930 --> 00:08:54,530
Well, it's gone from there.

101
00:08:54,530 --> 00:09:04,790
So let's say you don't have your own detection capability and you outsource this to a very nice vendor that promised to take care of you, monitor everything.

102
00:09:04,910 --> 00:09:12,090
But how do you actually know that they are actually monitoring the right things and responding in the right way?

103
00:09:12,090 --> 00:09:18,110
I mean, maybe they pop up an alert and they don't know much about your environment.

104
00:09:18,110 --> 00:09:24,830
They assume it's fine, and they basically close it as a false positive, while it might have actually been a true positive.

105
00:09:24,930 --> 00:09:27,290
And you might actually be already hacked.

106
00:09:28,590 --> 00:09:42,110
And from that end, your stakeholders, which might be your CFO or your internal audit or whoever it might be on your end, how do you convince them that you're actually in control?

107
00:09:42,110 --> 00:09:47,590
You do have a lot of detection rules and you do have a lot of incidents, but is this everything?

108
00:09:47,590 --> 00:09:52,290
And how can they measure if you're doing well or not?

109
00:09:52,550 --> 00:09:54,310
And from a detection perspective as well.

110
00:09:54,310 --> 00:09:56,490
So how do you know your detection is actually resilient?

111
00:09:56,490 --> 00:10:01,890
You do have a lot of really good detection rules, but do they cover variants?

112
00:10:01,890 --> 00:10:05,370
Do they cover only the one that you read up on?

113
00:10:05,370 --> 00:10:09,030
Or do they cover a multitude of ways of executing that?

114
00:10:10,010 --> 00:10:13,270
So you need a way to measure this.

115
00:10:17,280 --> 00:10:23,080
Which comes to our very graphic approach.

116
00:10:23,440 --> 00:10:26,100
So in the center, there's a couple of pillars.

117
00:10:26,100 --> 00:10:37,240
And the risk and the controls one are probably the most well-known ones, where risk is usually determined by the business, which is augmented by threat intel.

118
00:10:37,240 --> 00:10:43,520
And then there might be some other values that come to a certain risk posture.

119
00:10:44,340 --> 00:10:55,440
And then there are your controls, which are obviously your detections, your mitigations, your preventative measures, your hardening baselines, and so forth.

120
00:10:55,440 --> 00:11:00,780
These are usually based on a couple of reference models, like MITRE ATT&CK, of course, which we all love.

121
00:11:00,960 --> 00:11:03,260
There might be a kill chain involved.

122
00:11:03,260 --> 00:11:10,160
There might be some additional frameworks in terms of hardening your environment.

123
00:11:10,440 --> 00:11:23,900
And then there's the effectiveness one that we actually want to add to this, which will support the other ones by providing meaningful, valuable data that you can use for your management, for your auditing parties, but also for your self-management.

124
00:11:23,900 --> 00:11:26,960
So, you can actually know how you're doing.

125
00:11:27,780 --> 00:11:33,680
So, then it a little bit depends on what you want to do with these pillars in which team you're at.

126
00:11:33,680 --> 00:11:43,080
So, if you're in the blue team, on the risk side, there's the control effectiveness that you might want to report to your regulatory parties.

127
00:11:43,080 --> 00:11:53,880
And on the red side, the risk might give you a lot of input for advanced simulations that you want to execute.

128
00:11:54,280 --> 00:12:02,420
And these are usually manual and very creative exercises, but these require a lot of time and money and effort.

129
00:12:02,420 --> 00:12:04,480
And you can't do this on a continuous basis, right?

130
00:12:04,480 --> 00:12:05,940
So, we need something else.

131
00:12:06,200 --> 00:12:10,860
And this is where the purple part on the right, on the effectiveness side, comes in.

132
00:12:10,860 --> 00:12:16,140
So, you can do a normal purple team, of course, which is red and blue joined together.

133
00:12:16,400 --> 00:12:19,900
It's usually based on a forecast or some hypotheses.

134
00:12:20,640 --> 00:12:29,540
And the outcome is usually a really good detection engineering effort where you start building all kinds of new awesome detections, which is great.

135
00:12:29,540 --> 00:12:31,960
And if you can do this, you should totally do this.

136
00:12:31,960 --> 00:12:34,740
But this is always limited by time and effort.

137
00:12:34,740 --> 00:12:37,500
Not everybody can do this.

138
00:12:37,500 --> 00:12:48,100
You don't always have an internal red team or you don't have the means to do this on a monthly basis because it's quite costly to get consultants in there every time.

139
00:12:49,460 --> 00:12:52,600
So, you need something else next to that.

140
00:12:52,600 --> 00:12:56,880
And this is where the automated adversarial simulation comes in.

141
00:12:57,100 --> 00:13:04,460
On top of the building new detections, you also can then augment that with all kinds of validations of all your controls.

142
00:13:04,460 --> 00:13:05,980
So, you can do this on a continuous basis.

143
00:13:05,980 --> 00:13:07,300
And this is why we're here.

144
00:13:07,920 --> 00:13:12,860
So, it's built by red and blue together to get a maximum result on a continuous basis.

145
00:13:14,440 --> 00:13:16,240
This is why we're here, exactly.

146
00:13:17,180 --> 00:13:24,740
So, yeah, that takes us to our next section here, automated adversarial simulation.

147
00:13:24,740 --> 00:13:42,020
We strongly believe that the challenges that we described, the potential issues that could affect a detection program can be approached by deploying automated adversarial simulation exercises on corporate environments.

148
00:13:42,020 --> 00:13:52,480
So, in this section, we would like to share with you what we call adversarial simulation, how we understand it, automated adversarial simulation to be specific.

149
00:13:52,480 --> 00:14:02,660
I think it's important to say that we don't want to claim this term because automated adversarial simulation could mean so many different things to different people.

150
00:14:02,660 --> 00:14:09,480
But in this section, we want to describe to you how we understand this concept.

151
00:14:11,240 --> 00:14:20,540
So, basically, automated attack simulations or adversarial simulation is an automated activity that has two sides.

152
00:14:20,540 --> 00:14:34,620
We're running simulations and then we're validating that the controls that we have previously deployed in terms of detection or maybe even prevention controls are working as expected.

153
00:14:34,620 --> 00:14:38,980
They're passing the test, they're passing the simulation test.

154
00:14:39,980 --> 00:14:47,720
The challenge, as Olaf was saying, purple teaming is great, right?

155
00:14:47,720 --> 00:14:58,340
But we cannot execute purple team exercises with 15, 10 people in a room every week, right?

156
00:14:58,340 --> 00:14:59,840
How many times can you do it a year?

157
00:14:59,840 --> 00:15:01,200
Every quarter?

158
00:15:02,100 --> 00:15:04,000
Maybe every month?

159
00:15:04,000 --> 00:15:13,600
But not many organizations have access to high-skilled red teamers that can execute these techniques.

160
00:15:15,360 --> 00:15:18,160
And it's basically also a snapshot in time.

161
00:15:18,160 --> 00:15:22,420
When we're doing a purple team exercise, it's just you're looking at it as a snapshot in time.

162
00:15:22,420 --> 00:15:28,600
But with automated adversarial simulation, we can continuously check if those controls are working.

163
00:15:29,360 --> 00:15:33,780
It's basically also a way of generating telemetry, right?

164
00:15:33,780 --> 00:15:44,300
And we use this telemetry to validate the environment, to validate the health of a detection program on a continuous basis, as Olaf was saying.

165
00:15:44,300 --> 00:15:51,880
So with this type of program, we can go over all the challenges that we described a few slides before.

166
00:15:51,880 --> 00:15:55,060
We can identify issues with the event pipeline.

167
00:15:55,060 --> 00:16:01,640
We can identify gaps in visibility, those misconfigured GPOs missing some events.

168
00:16:01,660 --> 00:16:10,420
If we're running simulation exercises against those endpoints, we will know that the events are missing, that we have a gap, right?

169
00:16:13,530 --> 00:16:15,150
And what is it not?

170
00:16:15,150 --> 00:16:17,770
It's definitely not a red team replacement.

171
00:16:17,770 --> 00:16:24,930
So this is really not a threat to any of your red teamer friends, or if you're a red teamer, welcome.

172
00:16:25,710 --> 00:16:35,870
So you do need red team input if you can get it, because they usually have better techniques than what you can build yourself.

173
00:16:35,870 --> 00:16:42,510
So it's also not a pentesting engagement either.

174
00:16:42,510 --> 00:16:48,310
This is really meant to be a continuous validation or improvement phase.

175
00:16:48,350 --> 00:16:50,650
And it's, of course, also not Skynet.

176
00:16:50,650 --> 00:16:52,570
It won't take over your company.

177
00:16:52,910 --> 00:16:57,310
And from that same aspect, it's not impacting continuity by design.

178
00:16:57,310 --> 00:17:01,950
So it shouldn't break your systems unless you really, really want it to.

179
00:17:02,550 --> 00:17:06,150
And from that perspective, it should be run in production, of course.

180
00:17:06,930 --> 00:17:19,070
And it's also not the new Holy Grail or the new buzzword that you should hear at RSA next year, but it should be something that you definitely need to be considering in order to be really strong.

181
00:17:25,100 --> 00:17:32,880
And moving on from what I meant before, so you're not just running Benjamin Delphi's tools in your production environment, right?

182
00:17:32,880 --> 00:17:37,980
You need some form of methodology to properly execute these.

183
00:17:38,240 --> 00:17:40,340
So, of course, there is some governance involved.

184
00:17:40,340 --> 00:17:46,060
You need to establish a sort of white team, depending on how intrusive you want your tests to be.

185
00:17:46,060 --> 00:17:48,600
You need to set some ground rules.

186
00:17:48,600 --> 00:17:53,540
There needs to be some escalation paths in case something actually goes wrong.

187
00:17:53,760 --> 00:18:04,320
You need involvement from IT ops because they might need to accommodate some access to you or give you some means to take some steps.

188
00:18:04,580 --> 00:18:07,700
Of course, you need a reference framework from that perspective as well.

189
00:18:07,700 --> 00:18:10,120
Of course, MITRE ATT&CK is there again.

190
00:18:10,120 --> 00:18:20,620
But there might be some PCI controls that you want testing or COVID or HIPAA or whatever other fancy acronym out there.

191
00:18:20,620 --> 00:18:27,160
You need to have some frameworks that can actually measure what you want to prove.

192
00:18:28,180 --> 00:18:32,300
And then we move a little bit more towards the execution part.

193
00:18:32,300 --> 00:18:34,100
So, we need to have a scope.

194
00:18:34,420 --> 00:18:41,300
And this scope selection phase has two crucial points that need to be in there.

195
00:18:41,300 --> 00:18:49,000
So, you need to know which systems, and this can be, of course, in an office in Singapore or the US.

196
00:18:49,260 --> 00:18:50,860
It can be workstation servers.

197
00:18:50,860 --> 00:18:52,500
Well, you can figure that out.

198
00:18:52,540 --> 00:19:02,040
And then, of course, you also need to validate or to set the scope for which controls you want to test or to avoid from that end.

199
00:19:02,040 --> 00:19:16,940
So, if you do know that you want to test A, B, and C, but you don't want to hit another one because then it might raise too many incidents or that system might be more volatile, you need to be aware of that.

200
00:19:17,440 --> 00:19:23,940
And from these scope sessions, you can actually start designing a campaign.

201
00:19:25,220 --> 00:19:31,420
And in this phase, you actually build the required objectives or techniques that you want to execute.

202
00:19:31,420 --> 00:19:37,900
You add them to a playbook or a runbook or a campaign design or however you want to call it.

203
00:19:38,000 --> 00:19:42,960
You test it, of course, first, and then you run it in production, obviously.

204
00:19:44,540 --> 00:19:49,060
And one of the steps in here also is defining the cadence of testing.

205
00:19:49,060 --> 00:19:56,940
So, do you want to do this hourly, daily, monthly, whatever your preference is, and the timing between all these techniques.

206
00:19:56,940 --> 00:20:00,180
Do you want to run it within, I don't know, 10 minutes?

207
00:20:00,180 --> 00:20:02,180
Do you want it to take an hour?

208
00:20:02,760 --> 00:20:05,560
This can also impact your detection rules, right?

209
00:20:05,560 --> 00:20:10,360
So, you need to vary this a little bit on and off whenever you do this.

210
00:20:10,460 --> 00:20:15,760
And then, of course, you need to report and validate all of your outcomes.

211
00:20:15,760 --> 00:20:18,940
So, you want to know which test you ran.

212
00:20:18,940 --> 00:20:24,280
You want to know the results of each individual test, whether it was successful or not.

213
00:20:24,280 --> 00:20:34,800
You want to know the commands that were executed or the tools that were launched, or all the relevant information that a blue team needs, including the time that it actually happened.

214
00:20:34,980 --> 00:20:40,440
If they didn't detect it, they need to be improving their detection, so they need this data.

215
00:20:40,840 --> 00:21:02,180
And, of course, ideally, you also want to have your alerts or your incident tickets or all kinds of telemetry that you can combine in the report where you can show the full end-to-end cycle, where you executed something, you did detect it, and it was actually also picked up by the team properly or not.

216
00:21:02,520 --> 00:21:09,260
So, whether they actually flagged it as a true positive is also a very important measure.

217
00:21:10,240 --> 00:21:11,160
Yep.

218
00:21:11,160 --> 00:21:26,520
I want to follow up on the scoping piece that you were mentioning, because I feel when executing PRPL team exercises, we really need to think about which parts of our network we're targeting, because you're always running exercises on the same parts of the network.

219
00:21:26,680 --> 00:21:29,700
Again, we're going to go back to the same issue as before.

220
00:21:29,700 --> 00:21:33,980
How do you know if things are working end-to-end across your environment?

221
00:21:33,980 --> 00:21:40,380
So, there needs to be a little bit of randomness on that scoping as well.

222
00:21:43,040 --> 00:21:44,320
Totally.

223
00:21:44,520 --> 00:21:50,420
Yeah, you should also start thinking about, how do I integrate this into my detection engineering process?

224
00:21:50,420 --> 00:21:53,100
And this slide has been shown in multiple variants.

225
00:21:53,100 --> 00:22:05,980
I think the first one was built by Jared Atkinson over at Spectrops, and it's a really, really nice flow of building detection, new detection rules.

226
00:22:05,980 --> 00:22:18,460
And I only highlighted a few of the steps that are interesting for our current talk, because , of course, you need to be working on all these steps that you can now probably hardly read by intention.

227
00:22:18,760 --> 00:22:24,380
But in the first phase, when you start thinking of, okay, what do I actually want to add as a detection?

228
00:22:24,680 --> 00:22:28,400
You're already looking at threat intel, industry reports.

229
00:22:28,840 --> 00:22:33,840
You might also have your internal red team or an external red team that you can consult.

230
00:22:33,840 --> 00:22:35,880
So, how can I execute this stuff?

231
00:22:36,360 --> 00:22:48,060
And in the next phase, when you start investigating, you want to investigate as much ways of executing that specific technique that you want to detect.

232
00:22:48,560 --> 00:23:07,660
And you also will want to be starting to develop scripts that you can then use in a later phase to validate all of the detections that you are trying to build in the first place, so that you already think a little bit about, how do I repeat this process later on?

233
00:23:07,660 --> 00:23:18,880
And this is also where you start implementing it in phase four, when you are building your detection rule and actually starting to implement it in production.

234
00:23:18,880 --> 00:23:40,820
You also want to start implementing these validation scripts or tools or whatever you've created into the same process, so that you can actually already, from the minute it goes live, start measuring its efficiency, whether it's working correctly, whether the outcome is actually as you expect it to be.

235
00:23:40,820 --> 00:23:48,860
And this is, of course, a delicate process where you don't want to be creating detections only tailored to your own validation scripts.

236
00:23:48,860 --> 00:23:53,500
So, also these validation scripts need to be maintained, they need to be expanded.

237
00:23:53,860 --> 00:23:59,980
You need to keep on researching whether it's resilient enough, of course.

238
00:24:02,840 --> 00:24:13,980
Yeah, ultimately, as we are creating detections, each one of them should have its own unit test, right?

239
00:24:13,980 --> 00:24:21,600
We see detection as an engineering process, we just need to test each one of our products.

240
00:24:22,180 --> 00:24:29,140
And one big requirement for us is, we believe that it needs to be end-to-end.

241
00:24:29,140 --> 00:24:51,700
And I think there's also something to bring up here where some challenges with other approaches, where I would like to differentiate between unit testing and functional testing, where in unit testing you can perhaps inject to the event pipeline a particular telemetry,

242
00:24:51,700 --> 00:24:57,540
maybe generated by the execution of adversary techniques, to test the detection logic.

243
00:24:57,540 --> 00:25:00,900
And that's great, we should do that to test the detection logic.

244
00:25:00,900 --> 00:25:08,720
But that misses a point, because you're assuming that the event pipeline is working as expected.

245
00:25:08,980 --> 00:25:27,140
So, I think we need to complement this type of testing with functional testing, end-to-end, testing a technique on a specific host, so that you're only testing the engine, but you're testing the data source generating telemetry.

246
00:25:27,140 --> 00:25:37,900
So, it's proper configuration, the data event being sent through the event pipeline, through the analytics engine, and ultimately generating that detection.

247
00:25:37,900 --> 00:25:41,460
So, it needs to be end-to-end, right Olaf?

248
00:25:42,240 --> 00:25:43,560
Definitely, yeah.

249
00:25:43,560 --> 00:25:48,040
And to the last two words in the sentence, it needs to be also in production.

250
00:25:48,040 --> 00:26:00,300
Because I've seen labs and acceptance environments, and in my long experience as a consultant, it's never been really lifelike.

251
00:26:00,300 --> 00:26:08,220
It can be close, but it's only on maybe a handful of clients I've been to, where acceptance actually comes close to production.

252
00:26:08,220 --> 00:26:10,960
But still, production is always different.

253
00:26:11,200 --> 00:26:13,860
And this is the stuff that you're trying to protect, right?

254
00:26:13,860 --> 00:26:18,940
So, why would you be executing all this stuff in a lab and then hope it works in production?

255
00:26:19,120 --> 00:26:20,480
You need to make sure.

256
00:26:26,080 --> 00:26:33,100
So, what are some of the requirements if you want to run an automated adversarial simulation exercise?

257
00:26:33,100 --> 00:26:38,000
I think similar to a PRPL team exercise, in this case, I think the requirements are similar.

258
00:26:38,380 --> 00:26:40,280
You need a defensive capability.

259
00:26:40,280 --> 00:26:47,900
It doesn't have to be cutting edge, like with the whole visibility and hundreds of detections.

260
00:26:47,900 --> 00:26:53,140
You can have less mature, and still, it will help to test the controls that you have in place.

261
00:26:53,140 --> 00:26:57,620
Of course, you have to tailor the simulation exercise to the controls that you have in place.

262
00:26:57,620 --> 00:27:01,040
But you need to have some kind of visibility in a detection program, right?

263
00:27:01,040 --> 00:27:05,620
Otherwise, you don't know which controls to test.

264
00:27:06,480 --> 00:27:11,120
Ideally, and definitely, we need some knowledge of the threats that we are executing, right?

265
00:27:11,120 --> 00:27:15,320
So, this is, again, where the PRPL comes in on the talk, right?

266
00:27:15,320 --> 00:27:22,520
So, we as detection engineers, as a detection team, we need to be aware of these techniques.

267
00:27:22,520 --> 00:27:35,640
Maybe not to the level of the Red Team does and execute them like a Red Team does, but at least to the point where we can simulate them and confirm that our logic is working.

268
00:27:35,640 --> 00:27:37,560
It's testing, our programs are working.

269
00:27:38,580 --> 00:27:43,140
And, of course, we need tools, right, to simulate this.

270
00:27:43,140 --> 00:27:53,580
We cannot be just downloading a VM and maintaining hundreds of different tools to execute all these techniques.

271
00:27:53,640 --> 00:28:02,300
Ideally, we need one tool or a set of tools that will give us the most scope as possible in executing these techniques.

272
00:28:02,300 --> 00:28:06,920
And these tools need to have certain features and capabilities, right?

273
00:28:06,920 --> 00:28:08,620
They need to be able to be automated.

274
00:28:08,960 --> 00:28:23,560
They need to generate logs, specific logs, to be able to compare the execution of techniques with the detection controls that are triggering, right?

275
00:28:23,560 --> 00:28:25,640
So, tooling is really important.

276
00:28:25,640 --> 00:28:29,200
And we'll talk about that on the next section.

277
00:28:30,680 --> 00:28:34,600
And it also requires that you have a sort of an adversarial mindset.

278
00:28:34,600 --> 00:28:41,940
You need to not only think about how do I defend this stuff, but also how can my company be attacked?

279
00:28:41,940 --> 00:28:48,140
And sometimes it's a little bit hard, so this is also where tools might actually help you a little bit.

280
00:28:48,180 --> 00:28:53,120
But you need to be a little bit aware of what can be done in my environment, what can't be done.

281
00:28:53,120 --> 00:28:56,480
And then all of these things need to be validated, of course.

282
00:28:56,480 --> 00:29:09,100
And in order to do this, usually in a more normal organization, you actually need some management buy-in, because they need to approve you spending time on this, since it will cost a little bit of money.

283
00:29:09,100 --> 00:29:14,020
And if you want to buy one of the commercial tools, you of course need some more budget.

284
00:29:15,240 --> 00:29:24,980
And if you're a little bit bold, as I've done in the past as well, you can also just do it and maybe say sorry later if they get upset.

285
00:29:24,980 --> 00:29:37,340
The big upside to this, if they don't really understand the value of a program like this, is that you can immediately prove them the results with some more details.

286
00:29:38,280 --> 00:29:52,340
But frankly, every type of management should be interested in this, whether they're very financially focused or not, because this will give you a lot of insight into how you're actually doing as a team.

287
00:29:52,340 --> 00:30:07,900
And of course, if you have a security vendor that does monitoring for you, you need their involvement as well, because otherwise you'll spoil a good relationship and you need to actually be working together since they're doing part of your detection work.

288
00:30:12,770 --> 00:30:13,410
Yep.

289
00:30:13,410 --> 00:30:24,090
So let's talk a little bit about the tools that are out there that can help us in executing automated adversarial simulation exercises.

290
00:30:24,090 --> 00:30:27,770
So there's definitely a lot of tools out there, right?

291
00:30:27,770 --> 00:30:31,850
We're not reinventing the wheel here, really.

292
00:30:32,730 --> 00:30:34,430
There's commercial tools.

293
00:30:34,430 --> 00:30:36,310
There's some open source tools.

294
00:30:36,310 --> 00:30:40,610
We're going to stay away from the commercial tools since that's not our focus today.

295
00:30:40,610 --> 00:30:44,930
But let me tell you a little bit about my experience.

296
00:30:44,930 --> 00:30:55,550
So when we first started executing PRPL team exercises internally, our natural approach was to use red team tools, right?

297
00:30:56,370 --> 00:31:00,550
Red team tools, like Metasploit, Empire, Parse2.

298
00:31:00,590 --> 00:31:15,770
And then we started validating controls, which worked great at the beginning when we were more on the form of a manual PRPL team exercise where maybe once a week we check a particular technique manually.

299
00:31:15,770 --> 00:31:37,070
But that soon stopped being practical because as the number of detection grows and as our testing moves to a weekly cadence, it's not practical to have an engineer just getting all these shells from your C2s and then start running things manually.

300
00:31:37,070 --> 00:31:46,830
Now, some of these tools do have APIs, and there's actually adversary simulation tools that leverage the APIs, so they can definitely be used.

301
00:31:48,630 --> 00:32:03,510
Then some of these tools are also fully automated and allow you to create a stand-up infrastructure in an automated way where you can run simulations and hopefully your endpoint controls will catch them, right?

302
00:32:03,510 --> 00:32:13,090
But I think a big challenge with that is that you are running these simulations from fixed infrastructure, from the same host every time.

303
00:32:13,090 --> 00:32:15,510
And that takes me back to end-to-end, right?

304
00:32:15,510 --> 00:32:23,510
Because with this type of exercise, you are not really confirming if your event pipeline is working or not.

305
00:32:23,510 --> 00:32:31,350
You're just confirming if that host, which is built for that, is generating the telemetry and if your detection is triggering.

306
00:32:31,350 --> 00:32:37,230
So, they're great, these tools are great, but they have some limitations.

307
00:32:38,410 --> 00:32:43,870
Ultimately, I think some of these tools have been built to be run manually, right?

308
00:32:43,870 --> 00:32:46,650
On a host, like on the screen, interactively.

309
00:32:46,650 --> 00:32:53,510
And that's fine, they still have a value, but some of them have not really been built for automation.

310
00:32:56,290 --> 00:33:03,130
The ones that do actually have some complicated ways of building scenarios or adding new techniques, it's not always easy.

311
00:33:03,130 --> 00:33:11,870
And that's also the drawback with open source, probably also with my own tools, where somebody has a certain way of working in mind, it works for them.

312
00:33:11,870 --> 00:33:15,950
But for a new adopter, it's actually quite hard to start working with it.

313
00:33:15,950 --> 00:33:27,370
And from a detection perspective, not all these tools have a really verbose way of telling you what happened, when it happened, and so on.

314
00:33:27,370 --> 00:33:36,710
So, the output is quite limited, which is hard then to match against your detections or put into a more measurement program.

315
00:33:37,370 --> 00:33:47,770
And as Mauricio mentioned, some of them are very static in terms of where they run from, or they're not as realistic, where it's just a bunch of Python scripts, which are useful.

316
00:33:47,770 --> 00:33:51,610
But it also is not always the way that an attacker would do it.

317
00:33:51,610 --> 00:33:56,490
So, how realistic is it for a SOC then to be responding to this?

318
00:33:56,970 --> 00:34:01,910
And lastly, not all of them are capable of doing stuff remotely.

319
00:34:01,910 --> 00:34:05,670
So, it's usually from that same system, they do something on that system.

320
00:34:05,670 --> 00:34:07,270
And then that's that.

321
00:34:07,270 --> 00:34:10,390
Which is also not always the way an attacker would work, of course.

322
00:34:13,520 --> 00:34:13,980
Yep.

323
00:34:13,980 --> 00:34:16,240
So, which takes us to our tooling, right?

324
00:34:16,240 --> 00:34:30,620
So, having mentioned all these challenges and the approach, Olav and I have both built tools that have different goals.

325
00:34:30,620 --> 00:34:38,420
And we've integrated them to execute a POC for automated adversary simulation.

326
00:34:38,420 --> 00:34:43,700
So , we want to share some of these tools in the next section.

327
00:34:43,700 --> 00:34:46,400
So , I'll start on mine.

328
00:34:46,400 --> 00:34:54,720
So, actually, yesterday at Black Hat Arsenal, I released my adversary simulation tool called PurpleSharp.

329
00:34:54,720 --> 00:34:58,760
So, you can go to the GitHub page and you'll find all the information you need.

330
00:34:58,760 --> 00:35:08,780
But basically, PurpleSharp is a tool in C Sharp that executes adversary techniques within Windows AD environments.

331
00:35:08,780 --> 00:35:11,720
So, it's focused for AD environments.

332
00:35:11,720 --> 00:35:17,520
It follows a MITRE ATT&CK framework as far as the techniques.

333
00:35:17,980 --> 00:35:32,300
And the main goal of PurpleSharp is to help us execute some of these simulations to generate the telemetry that can help us build, test, and enhance detection controls, right?

334
00:35:32,300 --> 00:35:33,540
That's what it does, basically.

335
00:35:33,540 --> 00:35:36,920
It just executes techniques and will generate telemetry.

336
00:35:36,920 --> 00:35:49,660
So, if you have a properly monitored Windows environment, Windows domain environment, this telemetry will make its way to the event pipeline, to your analytics engine.

337
00:35:49,660 --> 00:35:58,080
And if you have coverage for the techniques that PurpleSharp supports, you should validate your controls, right?

338
00:35:59,840 --> 00:36:03,200
So, what are some of the... why PurpleSharp?

339
00:36:03,200 --> 00:36:04,160
Why is it different?

340
00:36:04,160 --> 00:36:07,080
We've talked about some of the tools that exist already.

341
00:36:07,080 --> 00:36:08,820
So, I want to mention a few points real quick.

342
00:36:08,820 --> 00:36:13,160
I know we're not great on time, Olaf, so I'll go fast.

343
00:36:13,160 --> 00:36:17,220
Simplicity, it's one simple .NET binary.

344
00:36:17,220 --> 00:36:26,700
All you need is to compile a .NET binary and you can just grab it and copy it to a Windows endpoint and run simulations against remote hosts from that.

345
00:36:26,700 --> 00:36:27,360
Really easy.

346
00:36:27,360 --> 00:36:31,880
No C2, no C2 channels, no infrastructure, no VMs, one binary.

347
00:36:31,880 --> 00:36:43,680
Verbose logging, as Olaf was saying, we need the audit trail of all these techniques that executed at what point, from which processes, from which process ID.

348
00:36:43,680 --> 00:36:53,680
So, we can grab that data and maybe if our detection controls didn't trigger, we can at least find that telemetry and create a new detection or identify why the detection didn't trigger, right?

349
00:36:54,520 --> 00:36:56,080
Remote simulations.

350
00:36:56,080 --> 00:37:03,520
PurpleSharp is actually able to execute simulations or what I want to call deploy simulations on remote endpoints.

351
00:37:03,520 --> 00:37:12,120
So, once again, the challenge of verifying that the event pipeline is working or not can be easily approached with PurpleSharp.

352
00:37:12,120 --> 00:37:25,320
All you have to do is point your simulation to a host in your Singapore office and you'll confirm if that simulation will be detected as good as the one that in your headquarters, right?

353
00:37:25,320 --> 00:37:27,240
Credible simulations.

354
00:37:27,240 --> 00:37:33,500
I've used a couple of techniques with PurpleSharp to make sure that the simulation is credible.

355
00:37:33,720 --> 00:37:35,620
I'll mention a couple of things.

356
00:37:35,620 --> 00:37:44,160
You can go through the documentation to really understand, but one is process ID, parent process ID spoofing, PPID spoofing technique.

357
00:37:44,160 --> 00:37:54,960
I use this technique to be able to deploy simulations on remote endpoints and run the simulations under the context of the user who's logged in on that host at that point, right?

358
00:37:54,960 --> 00:38:01,900
So, no longer we have to run simulations from that same infrastructure, from that same user that our SOC already knows.

359
00:38:01,900 --> 00:38:09,260
And it's just going to close that alert and you're not really going to train them unless these simulations run from real users on real computers.

360
00:38:10,380 --> 00:38:11,500
Randomness.

361
00:38:12,140 --> 00:38:20,540
PurpleSharp will leverage simple LDAP queries to query your environment and pick random hosts for the simulations.

362
00:38:21,360 --> 00:38:25,160
For example, in a password spray attack, it's going to pick random users.

363
00:38:25,160 --> 00:38:28,620
If it's going to do a network sharing iteration, it's going to pick random hosts.

364
00:38:28,720 --> 00:38:32,660
So, that randomness, once again, gives us that end-to-end testing.

365
00:38:33,380 --> 00:38:34,360
Attack variations.

366
00:38:34,360 --> 00:38:38,800
The last one, one really important for me is what Olaf was saying before.

367
00:38:38,800 --> 00:38:44,360
How do you know if your detection is resilient to a different way of executing the same attack technique?

368
00:38:45,420 --> 00:38:50,180
Well, I've tried to execute the same technique in a couple of ways, at least.

369
00:38:50,180 --> 00:38:59,100
For example, a simple example is running PowerShell, calling PowerShell.exe, or running it from .NET using System Automation Management, DLL.

370
00:38:59,100 --> 00:39:04,180
So, one simple example, PurpleSharp can execute both for that technique.

371
00:39:05,700 --> 00:39:08,620
Real quick, the architecture.

372
00:39:09,660 --> 00:39:12,060
PurpleSharp consists of three modules.

373
00:39:12,060 --> 00:39:14,940
The orchestrator, the scout, and the simulator.

374
00:39:14,940 --> 00:39:18,200
The orchestrator will run on the operator endpoint.

375
00:39:18,640 --> 00:39:26,160
And the role of the orchestrator is to interact with the simulation target to deploy the simulation.

376
00:39:26,280 --> 00:39:29,140
The scout will run on the simulation target.

377
00:39:29,140 --> 00:39:33,340
And the role of the scout is just to obtain information about that host.

378
00:39:33,340 --> 00:39:36,340
For example, who's the user that's logged in?

379
00:39:36,340 --> 00:39:39,100
What's the process ID of Explorer.exe?

380
00:39:39,100 --> 00:39:43,800
So, we can use the parent process spoofing technique.

381
00:39:43,800 --> 00:39:46,420
Parent process ID spoofing technique.

382
00:39:46,460 --> 00:39:49,140
And the simulator, it just runs the simulation.

383
00:39:49,140 --> 00:39:55,880
So, these three modules actually communicate with each other using name pipes so that they can orchestrate the simulation.

384
00:39:55,880 --> 00:39:59,280
Again, if you want to learn more about this, go to the documentation.

385
00:39:59,540 --> 00:40:03,020
But at a high level, this is how PurpleSharp works.

386
00:40:03,020 --> 00:40:11,480
It's going to first upload... once you point it to a remote endpoint, it's going to upload the scout using SMB.

387
00:40:11,480 --> 00:40:13,340
It's going to execute it using WMI.

388
00:40:13,340 --> 00:40:17,020
Then the scout service starts on the remote endpoint.

389
00:40:17,020 --> 00:40:19,920
And there's some communication on name pipes.

390
00:40:19,920 --> 00:40:25,280
That's where the orchestrator instructs the scout to run a recon on this host.

391
00:40:25,280 --> 00:40:28,300
And that's where we find out who's the user that's logged in.

392
00:40:28,300 --> 00:40:38,680
Based on this information, the orchestrator will actually upload the simulator under the path of that user so that our simulation runs under the path of a real user and a real computer.

393
00:40:38,680 --> 00:40:40,640
Once again, credible simulations.

394
00:40:41,370 --> 00:40:46,820
And once the simulator has been uploaded, the orchestrator triggers the simulation.

395
00:40:46,820 --> 00:40:53,140
The scout will execute the simulator using parent process ID spoofing.

396
00:40:53,140 --> 00:41:01,460
That way, the simulator runs under the context of the user and also as a child process of Explorer.exe.

397
00:41:01,460 --> 00:41:04,460
Once again, making the simulation credible.

398
00:41:04,860 --> 00:41:08,720
This is just one example of an attack variation, as I was saying.

399
00:41:08,720 --> 00:41:24,380
I can execute PowerShell just calling create process API, calling PowerShell.exe, which I could detect if I have some kind of EDR or this event ID being logged.

400
00:41:24,380 --> 00:41:27,480
But on the second technique, we're using .NET.

401
00:41:27,660 --> 00:41:34,880
So your EDR or your Sysmon or your event 4688 will not detect that.

402
00:41:34,880 --> 00:41:38,400
But you need to have PowerShell logging enabled to detect that.

403
00:41:38,400 --> 00:41:42,920
Once again, how resilient is your PowerShell detection technique?

404
00:41:45,420 --> 00:41:50,140
We don't have time, so I'm not going to mention the other variations.

405
00:41:50,480 --> 00:41:51,520
Go ahead, Olaf.

406
00:41:52,660 --> 00:41:56,560
I've built an app for Splunk, which is called ThreatHunting.

407
00:41:56,560 --> 00:42:00,140
Not the best name, but it's very descriptive, at least.

408
00:42:00,140 --> 00:42:15,140
And it's a very graphically oriented tool that will run about, I think, 160 hunts or queries or triggers, as you want to call it, throughout your environment on a periodic basis.

409
00:42:15,140 --> 00:42:25,500
And it will give you a lot of telemetry, which you can then start using for either moving your detections, or hunting, of course, which is the main intention of the tool.

410
00:42:25,500 --> 00:42:35,440
But it can also provide you with a lot of visibility into all of the triggers that have been going on.

411
00:42:35,640 --> 00:42:36,740
Thanks.

412
00:42:36,840 --> 00:42:40,580
So, there's a lot of detections for all kinds of techniques on Windows.

413
00:42:40,600 --> 00:42:46,540
It primarily utilizes Sysmon and the Windows native event logs right now.

414
00:42:46,560 --> 00:42:52,860
It's heavily MITRE ATT&CK focused, because this is a framework that I really embraced.

415
00:42:52,860 --> 00:43:09,780
And the primary goal was to provide an investigative workflow for hunters, but also for detection engineers to provide as much context as possible that is all related to the same events without having to click and copy paste a lot of times.

416
00:43:09,780 --> 00:43:16,000
So, it's meant as an efficient way of going through your data based on triggers.

417
00:43:17,900 --> 00:43:18,700
Yep.

418
00:43:18,700 --> 00:43:31,980
So, the POC that we want to show you today is basically how we've integrated both our tools to use PurpleSharp as a simulation engine and the ThreatHunting app as a detection engine, so we can validate that these detections are being picked.

419
00:43:31,980 --> 00:43:34,460
So, now we move to the demos, the fun part, guys.

420
00:43:34,460 --> 00:43:36,320
You must be bored about us talking.

421
00:43:36,540 --> 00:43:38,920
So, this is going to be the fun part.

422
00:43:38,920 --> 00:43:50,780
So, in the first demo, we'll execute three execution, the execution tactic, MITRE tactic execution, three techniques, PowerShell, Windows Command Shell, and Jscript.

423
00:43:50,780 --> 00:43:52,040
These are common techniques.

424
00:43:52,040 --> 00:44:00,780
If you read most of the malware reports, you probably see that malware or most likely will use some of these techniques.

425
00:44:00,820 --> 00:44:05,800
So, we'll move to the first demo, and hopefully this works.

426
00:44:08,920 --> 00:44:10,040
Okay.

427
00:44:13,000 --> 00:44:15,400
All right.

428
00:44:16,540 --> 00:44:21,280
Let me see if I can undo full screen.

429
00:44:22,560 --> 00:44:25,640
One second, of course.

430
00:44:26,280 --> 00:44:30,140
Doing live, it's always...

431
00:44:39,090 --> 00:44:42,970
What if you go out of full screen on the browser first?

432
00:44:43,770 --> 00:44:46,110
Let's do that.

433
00:44:49,380 --> 00:44:50,400
Thank you.

434
00:44:50,400 --> 00:44:51,380
All right.

435
00:44:52,120 --> 00:44:58,880
So, in this first demo, we have two endpoints.

436
00:44:58,880 --> 00:45:00,180
Let me go back real quick.

437
00:45:00,180 --> 00:45:00,900
Two endpoints.

438
00:45:00,900 --> 00:45:08,720
We have PC Sam, where we're going to run PowerShell from, and we have another computer called PC Judy, where we're going to run the simulation against.

439
00:45:08,720 --> 00:45:09,200
Okay ?

440
00:45:10,360 --> 00:45:11,040
All right.

441
00:45:11,040 --> 00:45:14,220
So, we're going to use PowerShell command line in this case.

442
00:45:14,220 --> 00:45:16,380
This case, we're going to...

443
00:45:16,380 --> 00:45:34,040
If you want to learn more about the parameters, please go to the documentation, but basically we're running a simulation against PC Judy using a service account that needs to be an admin on PC Judy, and we're going to run these three techniques, and we're going to slip five seconds between the execution of each one of these techniques.

444
00:45:34,380 --> 00:45:40,880
So, we're going to pass the password of this service account that we're going to use to run the simulations against this host.

445
00:45:41,740 --> 00:45:51,180
So, we see here that the scout gets uploaded, recon happens, and we identify that Judy Brody is logged in on this host.

446
00:45:51,180 --> 00:46:01,360
So, now the simulator is uploaded to a path within the user's profile, making it look as if Judy downloaded a fake Firefox installer.

447
00:46:01,360 --> 00:46:12,960
Then the simulation runs using parent process ID spoofing, so making Firefox installer a child process of Explorer running under the context of Judy.

448
00:46:12,960 --> 00:46:17,780
And as I was mentioning before, this is the report that we get back.

449
00:46:17,780 --> 00:46:27,760
So, we see all the details of when it ran, which technique, again, from which process ID, from which path, which command was executed.

450
00:46:27,760 --> 00:46:37,700
In this case, we have a parser command, we have CMD, who am I, a simple who am I, and then we execute what looks to be a malicious Jscript file.

451
00:46:37,960 --> 00:46:44,840
And all of them in the context of Judy, and then the PurpleShark will delete some of these logs files.

452
00:46:44,840 --> 00:46:51,580
So, now we go to our Splunk environment to confirm these techniques were executed.

453
00:46:53,000 --> 00:47:06,580
And as we can see here, we see the Scout running under the context of the PSharp service account and as a child process of WMIPRDSE.

454
00:47:06,720 --> 00:47:14,900
And then we see the simulator, now this time running on the context of Explorer and running under the context...

455
00:47:14,900 --> 00:47:21,140
sorry, as a child process of Explorer and running on the context of Judy, so the real user who's logged in on that host.

456
00:47:21,140 --> 00:47:30,880
And then PowerShell is run, WhoAmI is run, and this suspicious Jscript is executed, right?

457
00:47:31,760 --> 00:47:43,500
And if we go to the 4104 PowerShell event, we can see how the script was actually just asleep, 20 seconds asleep, right?

458
00:47:43,500 --> 00:47:51,980
So, as a simple test, we confirm that this simulation run three techniques remotely, really easy with one command.

459
00:47:51,980 --> 00:47:53,940
We can run with PurpleShark.

460
00:47:54,820 --> 00:47:58,080
We'll go to the next one real quick.

461
00:47:58,500 --> 00:48:01,620
Let's see, I'm just going to fast forward because of time here.

462
00:48:01,820 --> 00:48:04,360
We're going to execute a different technique.

463
00:48:05,920 --> 00:48:09,000
I honestly forgot what this technique is.

464
00:48:09,000 --> 00:48:14,220
I don't speak ATT&CK, but we'll see what it is.

465
00:48:14,220 --> 00:48:14,980
I forgot.

466
00:48:15,940 --> 00:48:17,860
So, follow the same process.

467
00:48:17,860 --> 00:48:19,880
I'm just going to fast forward real quick.

468
00:48:21,960 --> 00:48:29,200
Okay, yes, in this case, it's actually the same technique, PowerShell, but this time running using system management automation, right?

469
00:48:29,540 --> 00:48:31,700
So, a different way of executing PowerShell.

470
00:48:31,700 --> 00:48:34,620
We're no longer using PowerShell.exe, right?

471
00:48:34,620 --> 00:48:40,620
So, if we see on the logs, we don't see PowerShell running.

472
00:48:40,620 --> 00:48:49,600
All we see is Firefox Installer because Firefox Installer, the simulator, executed PowerShell with .NET.

473
00:48:49,600 --> 00:49:01,240
But if we look at event ID of module loads, I forgot what the event ID of that on this one is, but we see the system management automation being loaded by the simulator.

474
00:49:01,240 --> 00:49:05,400
So, that's how we can catch that this guy is executing PowerShell techniques, right?

475
00:49:05,400 --> 00:49:11,980
So, this is our first demo, really first way of just running a couple of simulations.

476
00:49:12,500 --> 00:49:19,520
So, in the next demo, we'll run a few techniques on the persistent tactic.

477
00:49:19,520 --> 00:49:23,120
We're going to use some race to keys, schedule task.

478
00:49:23,160 --> 00:49:26,400
We're going to, on defensivation, we're going to use portable execution.

479
00:49:26,640 --> 00:49:29,120
Sorry, portable executable injection.

480
00:49:29,200 --> 00:49:30,720
So, PE injection.

481
00:49:30,940 --> 00:49:34,680
And then we're going to dump LSAS using PurpleSharp.

482
00:49:34,680 --> 00:49:38,360
So, Olav will take you on this demo.

483
00:49:39,100 --> 00:49:42,120
All right, let me introduce some new stuff here.

484
00:49:42,120 --> 00:49:51,320
So, as you can see, there's this JSON structure, which is called the playbook JSON structure, where you can define all the variants that you want to do.

485
00:49:51,320 --> 00:49:54,440
So, here is where you schedule your scenarios, basically.

486
00:49:54,440 --> 00:49:56,680
And you need to define some basic stuff.

487
00:49:56,680 --> 00:50:06,700
So , username, domain, the domain controller, the sleep between all the techniques or the small playbooks that you want to execute, and then the playbooks themselves.

488
00:50:07,540 --> 00:50:09,820
So, of course, you give them a name.

489
00:50:09,820 --> 00:50:11,940
You give them a host that you want to target.

490
00:50:11,940 --> 00:50:15,060
So, it can either be a host name or a random host.

491
00:50:15,060 --> 00:50:20,660
And then, as Mauricio explained, it will do an LDAP query and find one itself.

492
00:50:21,120 --> 00:50:28,260
You can here also define the path of the scout and the simulator where you want to be running it from.

493
00:50:28,260 --> 00:50:31,500
So, you can actually make it even more realistic.

494
00:50:32,160 --> 00:50:36,240
The PB sleep is the sleep between these playbook steps.

495
00:50:36,240 --> 00:50:45,480
These two techniques for the registry and the scheduled tasks, they will be run 30 seconds in between.

496
00:50:45,860 --> 00:50:49,080
And then the next playbook will start after the playbook sleep.

497
00:50:49,720 --> 00:50:52,260
And over here, you can do the same thing.

498
00:50:52,260 --> 00:50:59,040
So, the command line is a little bit easier here, where you just say PB for the playbook and the playbook name.

499
00:50:59,040 --> 00:51:01,220
And you basically run it from there.

500
00:51:03,440 --> 00:51:07,360
Once you, of course, provide a password, you could do this on a command line.

501
00:51:07,360 --> 00:51:11,620
But I would advise against that, especially on a demo environment.

502
00:51:13,100 --> 00:51:16,280
So, here it will show you that it will execute three playbooks.

503
00:51:16,280 --> 00:51:18,300
And it will start with the first one.

504
00:51:18,360 --> 00:51:22,620
It first does an LDAP query to find the targets that it can run it against.

505
00:51:22,620 --> 00:51:24,060
It picked a random one.

506
00:51:24,060 --> 00:51:26,720
In this case, it's PC Judy again.

507
00:51:27,220 --> 00:51:30,060
And it's executing all these techniques on that machine.

508
00:51:30,060 --> 00:51:32,360
Well, this takes a little bit of time.

509
00:51:32,500 --> 00:51:36,920
So, we have to wait for all the techniques to be executed.

510
00:51:37,100 --> 00:51:40,780
And this is where the pipe communication comes in.

511
00:51:40,780 --> 00:51:43,300
So, it actually checks whether it's done.

512
00:51:43,380 --> 00:51:48,480
And once it's done, it will also clean up after itself, so that at least you won't lose any traces.

513
00:51:48,600 --> 00:51:54,680
All the binaries will be gone again, if your EDR didn't stop it already, of course.

514
00:51:54,740 --> 00:51:58,300
Which is also a way of validating whether it worked or not.

515
00:52:00,120 --> 00:52:01,140
And there you go.

516
00:52:01,140 --> 00:52:03,320
So, they are executed.

517
00:52:03,320 --> 00:52:06,980
So, you show it was started, where it was started from.

518
00:52:07,080 --> 00:52:09,940
You see the key that was added to the registry.

519
00:52:10,400 --> 00:52:12,560
It was successfully created.

520
00:52:12,780 --> 00:52:16,020
Then it was deleted, because you don't want it to be there.

521
00:52:16,020 --> 00:52:17,600
And the simulation is finished.

522
00:52:17,600 --> 00:52:29,020
Same goes for this technique, where you see a different path and the same parent process ID that it actually executed from.

523
00:52:29,160 --> 00:52:39,040
And you see the command that was executed in full detail, which you can then also use in your detection later on, to see whether you actually found it.

524
00:52:39,740 --> 00:52:46,760
And in the meantime, it's starting to execute another one on a different machine.

525
00:52:46,760 --> 00:52:48,280
And this is the PCMP.

526
00:52:48,280 --> 00:52:52,520
It's a different simulator that you see running under a different user context.

527
00:52:54,140 --> 00:52:58,900
And it's actually injecting into Notepad.

528
00:52:58,900 --> 00:53:00,320
So, you see exactly what it did.

529
00:53:00,320 --> 00:53:03,480
So, it does an open process for VirtualAlloc.

530
00:53:03,560 --> 00:53:10,840
Then it writes to the process memory and creates a new thread from that Notepad.exe.

531
00:53:11,000 --> 00:53:14,740
And then the last technique that we mentioned was running LSAS.

532
00:53:14,740 --> 00:53:17,260
And you see that it actually ran.

533
00:53:17,260 --> 00:53:19,520
It identified the LSAS process.

534
00:53:19,520 --> 00:53:21,060
It tried to obtain a handle.

535
00:53:21,060 --> 00:53:24,200
It called the mini dump.

536
00:53:24,980 --> 00:53:26,860
It wrote it.

537
00:53:26,860 --> 00:53:28,440
It dumped it there.

538
00:53:28,660 --> 00:53:35,820
And it fortunately also deletes it, because you don't want to, again, leave stuff for real attackers.

539
00:53:35,820 --> 00:53:51,700
And then when it's done, it actually writes a JSON file with all the results, which you can then easily import into your analytics platform of choice, which in our case will be Splunk.

540
00:53:52,340 --> 00:53:54,380
But you can see here that the playbook is there.

541
00:53:54,380 --> 00:54:07,220
And then there's a results JSON, which once you format it, it shows you all the stuff that we actually just saw on the timeline, but then in a nice JSON structure so that you can actually start importing them.

542
00:54:08,280 --> 00:54:14,080
And then from the perspective of time, we might want to scroll through a little bit.

543
00:54:14,180 --> 00:54:15,700
Yeah , makes sense.

544
00:54:15,700 --> 00:54:18,660
So, ideally, this is the integration, right?

545
00:54:18,660 --> 00:54:22,640
So, this is where you ingest this into your other platform.

546
00:54:22,640 --> 00:54:23,400
So , go ahead.

547
00:54:23,740 --> 00:54:24,260
Exactly.

548
00:54:24,260 --> 00:54:30,000
So, I created a small POC dashboard, again, just to show what kind of data is in there.

549
00:54:30,000 --> 00:54:42,000
So, you can see that the orchestrated PCs of SAM executed the techniques, the techniques that you see on the right, on the machines that were involved.

550
00:54:43,300 --> 00:54:45,280
And you can show these over time.

551
00:54:45,280 --> 00:54:50,380
So, if you do a longer playbook than I did, you will see actually a longer scope, of course.

552
00:54:50,540 --> 00:54:53,220
Here you see the playbooks that were executed.

553
00:54:53,860 --> 00:54:57,380
And if you know my threat hunting app, you can click on everything.

554
00:54:57,380 --> 00:54:58,600
So, it's the same over here.

555
00:54:58,600 --> 00:55:10,280
So, once you click on one of those playbook names, you will get all the raw logs with all the details in the panel below, which will show you all the results that you also saw on the command line.

556
00:55:10,280 --> 00:55:13,820
So, once you click it, it will actually change to the proper playbook.

557
00:55:14,420 --> 00:55:17,860
And the same will go for all the other fields.

558
00:55:17,860 --> 00:55:23,680
So, once you click on a MITRE technique ID that was tied to one of those techniques...

559
00:55:23,680 --> 00:55:24,940
Hey, the stream is paused.

560
00:55:24,940 --> 00:55:26,020
Oh, there we go.

561
00:55:26,020 --> 00:55:32,220
It will take you to my threat hunting app and will do a direct query for that technique being executed.

562
00:55:32,220 --> 00:55:37,780
And you see that also it was detected by the logic in my app, fortunately.

563
00:55:38,040 --> 00:55:45,260
And it will give you all the details that might help you contextualize it even more once you start playing with the app.

564
00:55:50,670 --> 00:55:56,890
And the same can be done for clicking on a machine, which I don't do yet.

565
00:55:56,890 --> 00:56:04,070
But I'll show you the raw logs first, where you can see all the results from all the techniques that were executed.

566
00:56:04,070 --> 00:56:08,770
So, again, you can quickly spot that everything was executed.

567
00:56:08,770 --> 00:56:11,870
So, you see the scout being run.

568
00:56:12,190 --> 00:56:15,570
You see that then the WMI is being called.

569
00:56:15,730 --> 00:56:21,630
The process is being ejected into and it starts executing the registry commands and so on.

570
00:56:22,190 --> 00:56:27,570
And the same goes for the scheduled task logs, which isn't that properly formatted, thanks Microsoft.

571
00:56:27,570 --> 00:56:30,650
But this is a quick POC.

572
00:56:30,650 --> 00:56:34,750
You can obviously properly parse this neatly.

573
00:56:34,750 --> 00:56:39,670
But it's quite easy to spot that some of the bad stuff actually was executed here.

574
00:56:42,670 --> 00:56:45,210
And I think we can fast forward again.

575
00:56:45,350 --> 00:56:46,010
Okay.

576
00:56:47,590 --> 00:56:50,430
So, yeah, we're going to the threat hunting app.

577
00:56:52,030 --> 00:56:53,170
Yeah, and this is the same.

578
00:56:53,170 --> 00:57:07,830
So, once you would have clicked on a machine, you would have actually seen all the techniques that were executed on that system in the threat hunting app, which will give you a huge panel with all techniques, all the commands that were executed, all the parent processes,

579
00:57:07,830 --> 00:57:10,670
process GUIDs from a Sysmon perspective.

580
00:57:11,470 --> 00:57:19,190
And it will try to at least map it to the proper MITRE tactics as well, as much as possible.

581
00:57:19,190 --> 00:57:27,730
And here you can see that there were some DLLs loaded, there was some registry activity, and you still have access to all the raw data as well.

582
00:57:28,450 --> 00:57:30,630
And that concludes the demo two.

583
00:57:30,670 --> 00:57:31,750
Okay, cool.

584
00:57:31,750 --> 00:57:32,850
Demo two.

585
00:57:33,630 --> 00:57:35,410
Let's move on.

586
00:57:36,330 --> 00:57:39,030
Okay, so I'll do this real quick.

587
00:57:39,030 --> 00:57:43,490
Of course, ATT&CK MITRE framework is our framework of choosing.

588
00:57:43,490 --> 00:57:49,170
And PurpleSharp does integrate with the ATT&CK Navigator to visualize these techniques.

589
00:57:49,170 --> 00:57:58,710
So, with simply running just Navigator export, PurpleSharp will create a Navigator layer that you can open up and upload to the Navigator.

590
00:57:58,710 --> 00:58:03,310
I'm going to fast forward here so that we make it on time.

591
00:58:03,310 --> 00:58:09,470
Just upload that and you'll see visually all the techniques that PurpleSharp supports.

592
00:58:10,090 --> 00:58:18,510
So, you can use it to pick the techniques that maybe you want to simulate on your automated adversary simulation exercise.

593
00:58:18,930 --> 00:58:24,870
And in a similar way, I actually support importing Navigator layers.

594
00:58:24,870 --> 00:58:30,850
So, in this case, I'm going to get a layer for APT, I think, 29.

595
00:58:32,050 --> 00:58:33,110
Oh, sorry, FIN7.

596
00:58:33,110 --> 00:58:34,490
Yeah, I picked FIN7.

597
00:58:35,970 --> 00:58:39,690
On Navigator, I export that layer to a JSON file.

598
00:58:40,890 --> 00:58:42,930
So, I'm going to fast forward here.

599
00:58:43,550 --> 00:58:47,070
And then I move it to the folder where PurpleSharp is.

600
00:58:47,070 --> 00:58:51,190
And now, in this case, I'm just going to import that Navigator layer.

601
00:58:51,190 --> 00:58:55,670
And PurpleSharp will analyze the techniques that it does support and it doesn't support.

602
00:58:55,670 --> 00:58:57,850
So, it will let you know.

603
00:58:57,850 --> 00:59:02,970
But then it finally creates, in this case, it supports 17 techniques out of 29.

604
00:59:02,970 --> 00:59:08,810
And it's going to create a JSON playbook like the one Olaf just run on demo number two.

605
00:59:08,810 --> 00:59:14,630
And now you have a simulation playbook for FIN7 that you can run on your remote endpoints, right?

606
00:59:14,630 --> 00:59:16,610
So, pretty cool.

607
00:59:17,310 --> 00:59:18,350
There.

608
00:59:20,900 --> 00:59:22,720
Okay, so...

609
00:59:25,200 --> 00:59:27,940
Yeah, I think we only have one minute left, Marcia.

610
00:59:27,940 --> 00:59:31,600
So, we have to publish this just online, I guess.

611
00:59:32,000 --> 00:59:33,320
Yes, so...

612
00:59:33,320 --> 00:59:33,900
Okay.

613
00:59:33,900 --> 00:59:34,620
Sorry.

614
00:59:34,740 --> 00:59:35,400
Yeah.

615
00:59:35,400 --> 00:59:42,560
So, I've noticed on the Twitch that some of the demos were not showing great.

616
00:59:42,560 --> 00:59:46,100
So, right after this, I'm going to upload them on YouTube also.

617
00:59:46,220 --> 00:59:47,800
So, you see them.

618
00:59:47,800 --> 00:59:52,680
But the last demo, yeah, it's also a cool one, but you'll see it on YouTube, okay?

619
00:59:52,680 --> 00:59:54,300
It'll be uploaded there.

620
00:59:56,280 --> 01:00:00,820
All right, you just want to take this last one, Olaf, since we're running out of time.

621
01:00:00,820 --> 01:00:01,640
Yeah, sure.

622
01:00:01,880 --> 01:00:13,260
In the last couple of seconds, I just want to highlight that this is a huge value in terms of control, but also in terms of reportability and also ensuring that you're actually working on the right stuff.

623
01:00:13,260 --> 01:00:15,940
So, it's also ease of mind, probably.

624
01:00:15,940 --> 01:00:20,400
So, I strongly encourage everybody to just start with this.

625
01:00:20,480 --> 01:00:27,700
And especially since there's a bunch of open source tools out there, including ours, it doesn't have to be very expensive.

626
01:00:27,700 --> 01:00:33,000
So, you can start small, and then once you start maturing, you might want to look at a commercial tool or not.

627
01:00:33,780 --> 01:00:44,500
And this should be really a nice way of implementing a way of testing all your detection engineering efforts, so you actually know it's not in vain, you know it's actually doing something.

628
01:00:44,500 --> 01:00:49,800
Because some of them probably won't trigger in production ever, hopefully, at least for you.

629
01:00:49,900 --> 01:00:52,360
So, you at least know it still is working.

630
01:00:52,480 --> 01:00:55,340
And it shouldn't be a grading mechanism for your team either.

631
01:00:55,340 --> 01:01:02,200
It can be used that way, but you should see it as a constructive tool and not as a sort of slapping tool if you didn't do it well.

632
01:01:02,200 --> 01:01:19,320
And of course, if you don't have a detection yet, you can of course use it also in a way of the purple exercise that we meant before, where you can start building new detections based on the techniques that we execute, but then make sure that you don't build tunnel vision detection.

633
01:01:19,680 --> 01:01:23,040
Make sure it's still resilient and it's open for variants.

634
01:01:24,800 --> 01:01:26,560
So, thanks everyone.

635
01:01:26,560 --> 01:01:29,500
That concludes our chat, our session today.

636
01:01:30,000 --> 01:01:34,720
You have the links for our tools and I'll also be... check out our Twitter.

637
01:01:34,720 --> 01:01:41,460
I'll also be releasing the links on the YouTube videos for the demo, so you can see them with a higher resolution there.

638
01:01:41,620 --> 01:01:44,020
But yeah, thanks for coming to our session.

639
01:01:44,020 --> 01:01:46,680
I hope you guys had fun.

640
01:01:46,780 --> 01:01:48,220
And thank you.

641
01:01:49,160 --> 01:01:50,360
Thanks, guys.

642
01:01:50,460 --> 01:01:51,120
Girls?

643
01:01:53,480 --> 01:01:55,580
Thank you, both of you.

644
01:01:55,580 --> 01:01:59,930
We really appreciate your demo and your presentation.

645
01:02:00,780 --> 01:02:12,400
As a reminder to everyone, we encourage everyone to join the Blue Team Discord and head over to Text Talk Track 1.

646
01:02:12,720 --> 01:02:19,400
And we can have the presenters answer some of your questions there as well.

647
01:02:20,180 --> 01:02:23,420
Right off the bat, I did see a couple of questions.

648
01:02:23,420 --> 01:02:25,860
So, if you guys don't mind.

649
01:02:27,060 --> 01:02:27,880
Okay.

650
01:02:32,030 --> 01:02:33,510
Scroll up.

651
01:02:37,360 --> 01:02:44,000
The question was, what are some of the major benefits of PurpleSharp versus Atomic Red Team?

652
01:02:45,520 --> 01:02:46,440
Yeah, okay.

653
01:02:46,460 --> 01:02:47,580
That's a great question.

654
01:02:47,580 --> 01:02:48,900
So, I think there are different things.

655
01:02:48,900 --> 01:02:59,600
Atomic Red Team is more a way of describing how to execute the techniques and not really an engine to execute the techniques themselves.

656
01:02:59,600 --> 01:03:10,800
Although the Red Canary team is working on Invoke Atomic, which is a partial tool that will be able to execute Atomic Red files.

657
01:03:10,800 --> 01:03:24,200
But if you ask me the difference between Invoke Atomic and PurpleSharp, I would say running the simulations under the context of a real user and as a child process of explorer.

658
01:03:24,200 --> 01:03:28,660
So, creating that credible simulation, that's something PurpleSharp can do.

659
01:03:28,660 --> 01:03:41,400
Another one is randomness, being able to run a technique against a random host to really identify if your detection controls are working end-to-end across your environment.

660
01:03:41,400 --> 01:03:42,740
That's the second one.

661
01:03:42,740 --> 01:03:46,520
And I guess the third one would be just simplicity.

662
01:03:46,520 --> 01:03:53,560
Maybe you can run simple binary that you can just copy on a host and run simulations from.

663
01:03:56,230 --> 01:03:57,550
Very good.

664
01:03:57,570 --> 01:03:58,650
Cool.

665
01:04:00,850 --> 01:04:04,570
I think that's it for the questions for now.

666
01:04:05,390 --> 01:04:06,650
Oh, here we go.

667
01:04:07,070 --> 01:04:11,010
Did you guys teach a course on this during this week?

668
01:04:12,170 --> 01:04:13,250
No, we didn't.

669
01:04:13,250 --> 01:04:14,430
But maybe that's a good idea.

670
01:04:14,430 --> 01:04:16,210
Maybe we can do that on the next one.

671
01:04:16,210 --> 01:04:17,690
That will be fun.

672
01:04:17,690 --> 01:04:18,590
Yeah, definitely.

673
01:04:19,090 --> 01:04:21,630
I know we had fun doing the demos.

674
01:04:21,910 --> 01:04:24,230
Yeah, too much.

675
01:04:24,930 --> 01:04:25,890
Too much.

676
01:04:26,090 --> 01:04:29,270
Maybe something in the future that you can do a workshop.

677
01:04:32,350 --> 01:04:35,570
Yeah, I think that's it for this.

678
01:04:35,570 --> 01:04:43,150
And if you guys don't mind sticking around for a little bit and let's talk track one.

679
01:04:43,150 --> 01:04:43,770
Sure .

680
01:04:43,770 --> 01:04:45,710
If people have questions.

681
01:04:45,710 --> 01:04:51,750
So one last thing is just I'll be posting the links of all the demos right after this.

682
01:04:51,750 --> 01:04:53,410
So check out our Twitter.

683
01:04:54,930 --> 01:04:59,730
And you'll be able to watch all the third last demo, which was also really cool.

684
01:04:59,730 --> 01:05:01,030
We didn't get a chance to.

685
01:05:02,210 --> 01:05:02,810
Cool.

686
01:05:02,810 --> 01:05:03,430
Very good.

687
01:05:03,430 --> 01:05:04,690
Thank you guys again.

688
01:05:04,690 --> 01:05:06,070
I appreciate it.

689
01:05:06,450 --> 01:05:07,450
Yeah, most welcome.

690
01:05:07,450 --> 01:05:09,230
Thank you.