1
00:00:00,000 --> 00:00:03,160
And anyway, I'm not going to waste any time. Eric Capuano, everybody.

2
00:00:14,560 --> 00:00:19,640
Okay, I appreciate you guys. Hey, so thanks a lot for coming to my talk, especially being, you

3
00:00:19,640 --> 00:00:23,480
know, noon on Sunday, final day of con. I know that a lot of people are already on planes

4
00:00:23,480 --> 00:00:28,540
heading home. So the fact that you guys are here with me is very humbling. So thank you. I'll

5
00:00:28,540 --> 00:00:32,660
just go straight into it. So obviously you guys know you're here because we all know that

6
00:00:32,660 --> 00:00:40,320
tabletop scenarios are not exactly the key way to train for incidents, right? But what do you do

7
00:00:40,320 --> 00:00:44,900
about that? If that's all you know, then what other option do you have? So what I'm going to

8
00:00:44,900 --> 00:00:48,280
talk to you guys about is something that I've actually done personally with my incident

9
00:00:48,280 --> 00:00:52,860
response team because I recognized there was a gap that needed to be filled. Tabletop is not

10
00:00:52,860 --> 00:00:58,520
it. And so I built an incident response simulation platform. Now, just so you guys know, you

11
00:00:58,520 --> 00:01:02,800
probably saw in the summary of the talk, this isn't going to be like just, you know, waves and

12
00:01:02,800 --> 00:01:06,080
waves and waves of data and things that you've never heard of before. A lot of you are probably

13
00:01:06,080 --> 00:01:10,140
familiar with virtual environments and things of that nature. But I am going to go a little bit

14
00:01:10,140 --> 00:01:16,820
deeper and go over some of the tips and tricks that I learned in my painstaking process to build

15
00:01:16,820 --> 00:01:22,480
this environment to achieve high fidelity training, right? There's a difference between just

16
00:01:22,480 --> 00:01:28,500
building a bunch of VMs and hacking them. No. In order to really get the most value out of it, you

17
00:01:28,520 --> 00:01:33,860
know, there's some things that you want to do to as closely simulate your enterprise environment

18
00:01:33,860 --> 00:01:45,180
as possible. So quick introduction. Should be next. So a little bit about me. I am the SOC lead

19
00:01:45,180 --> 00:01:51,540
for the Texas Department of Public Safety. We have 12,000 endpoints, over 467 remote locations

20
00:01:51,540 --> 00:01:57,560
across the entire state of Texas. So it's a very large network. We are the state police, the Texas

21
00:01:57,560 --> 00:01:58,500
Rangers, Department of Public Safety, the Department of Public Safety, the Department of Public

22
00:01:58,500 --> 00:02:01,100
Safety, the Department of Emergency Management, driver's license. I mean, we've got a lot of

23
00:02:01,100 --> 00:02:05,920
stuff. So that's a busy job. I'm also a cyber warfare operator for the Texas Air National Guard.

24
00:02:06,140 --> 00:02:12,780
Cyber, cyber. In my spare time that doesn't exist, I'm also a private consultant. I do security,

25
00:02:13,080 --> 00:02:17,620
information security for small and medium businesses in central Texas and surrounding areas.

26
00:02:18,500 --> 00:02:22,540
I'm also a very proud member of the packet hacking village where I run the emerging threats,

27
00:02:22,720 --> 00:02:27,000
honeypots and deception system. So if anybody here got to play in the honeypot challenges, that's

28
00:02:27,000 --> 00:02:28,480
mine. I built that. I built it. I built it. I built it. I built it. I built it. I built it. I built it. I

29
00:02:28,480 --> 00:02:28,720
built it. I built it. I built it. I built it. I built it. That's enough. That's me. With the

30
00:02:28,720 --> 00:02:33,980
help of some really awesome people. I'm a cyber patriot instructor because I believe that anyone in

31
00:02:33,980 --> 00:02:39,280
this field, IT, security, doesn't matter, pay it forward because those of us were once in junior

32
00:02:39,280 --> 00:02:44,120
high in high school didn't have somebody to come and teach us this awesome stuff. We had to

33
00:02:44,120 --> 00:02:49,640
figure it out. Well now you can be the one teaching the next generation of professionals. I'm a

34
00:02:49,640 --> 00:02:53,760
part-time blog author. I don't write nearly as much but security against obscurity is my blog.

35
00:02:53,980 --> 00:02:58,080
You see the link there. And lastly and most importantly I'm just a hobbyist, teacher,

36
00:02:58,180 --> 00:02:58,460
student, librarian, بعah, politician. Thank yous. I'm the post graduate here in Massachusetts. And I've been

37
00:02:58,460 --> 00:03:05,620
just like the rest of you. And as you can see, everything revolves around caffeine. So ask

38
00:03:05,620 --> 00:03:10,720
yourself this. And be honest. Don't tell me, but tell yourself the truth. Is your IR team ready

39
00:03:10,720 --> 00:03:18,220
for a worst case scenario? How would you know? You know, how often do you encounter worst case

40
00:03:18,220 --> 00:03:23,240
scenario? We're not talking about, you know, macros and viruses and e-mails. We're talking

41
00:03:23,240 --> 00:03:30,560
about APT on the network, ex-filling sensitive data. What do you do, right? You know, does your

42
00:03:30,560 --> 00:03:35,140
team have the skill sets that are required to operate in an environment in a situation like

43
00:03:35,140 --> 00:03:42,340
that? More importantly than the skill set, but do they know how to work together as a team when

44
00:03:42,340 --> 00:03:48,240
the shit hits the fan? Or do they all run off into separate rabbit holes and stop talking? Which

45
00:03:48,240 --> 00:03:53,220
is probably more likely. That kind of rolls into the next one. Do they communicate? Do they

46
00:03:53,240 --> 00:03:57,840
communicate? Because I know how my team is. Well, the multiple teams I'm on, when they get to

47
00:03:57,840 --> 00:04:02,880
chasing something, they go quiet. They go silent. Verbally and, you know, if you're in Slack or some

48
00:04:02,880 --> 00:04:07,680
kind of messaging platform, quiet. We're in the middle of a breach. Why are we quiet? We should

49
00:04:07,680 --> 00:04:11,680
be talking. We should be sharing information. I know you're seeing things. I'm seeing things.

50
00:04:11,800 --> 00:04:16,580
He's seeing things. Why aren't we talking about it? So how well are they communicating? And then

51
00:04:16,580 --> 00:04:21,060
lastly, organization, right? And I'm talking, yeah, the boring stuff, the administrative, the

52
00:04:21,060 --> 00:04:23,220
hierarchy of the incident response team. Is there a way to communicate? Is there a way to

53
00:04:23,240 --> 00:04:27,060
communicate? Is there a leader? Does that leader know his role? Do the followers know their

54
00:04:27,060 --> 00:04:32,080
roles? And are they properly executing? Or is everybody just kind of free for all and I hope you

55
00:04:32,080 --> 00:04:36,220
find something. Hope I find something. So those are the things you should be asking yourself.

56
00:04:36,380 --> 00:04:40,800
Because look, even if your daily job is working in a sock, you're like, oh, I'm not new to this. I

57
00:04:40,800 --> 00:04:45,860
respond to incidents all day long. Yeah, you're responding to those routine, mundane incidents

58
00:04:45,860 --> 00:04:51,680
that you can do with your eyes closed, right? But it's a totally different situation when you

59
00:04:51,680 --> 00:04:53,220
find out that there is a way to communicate.

60
00:04:53,240 --> 00:04:57,080
It is a persistent threat on the network. Maybe it's been here for three months and you just found

61
00:04:57,080 --> 00:05:01,960
it. That's not mundane. That's not routine. And neither should your response to such an

62
00:05:01,960 --> 00:05:08,400
incident. So you need to ask yourself, is your team ready for that, oh, shit, the data center's

63
00:05:08,400 --> 00:05:18,180
on fire situation? Obviously, the point that I'm making here is that a tabletop scenario cannot

64
00:05:18,180 --> 00:05:23,220
prepare you or your team for this. Now, it's better than doing nothing. It's better than doing

65
00:05:23,240 --> 00:05:27,260
nothing. I will give you that. To have a sit down and talk to your team and say, okay, guys, here's

66
00:05:27,260 --> 00:05:32,740
what's happened. There's a hacker on the network right now. And he's attacked our SQL database. And

67
00:05:32,740 --> 00:05:39,100
we're pretty sure he's exfilling records right now over DNS. What would you do? We'd block him.

68
00:05:39,200 --> 00:05:46,700
We'd stop him. We'd put in a firewall rule and we'd stop him. Okay. Cool. Awesome. Prove it.

69
00:05:46,960 --> 00:05:52,840
Like, you know, how would you, one, even have detected this to begin with? Two, how would you

70
00:05:53,240 --> 00:05:58,440
identify what is even happening? You know, like, okay, great, we see a huge, you know, influx of

71
00:05:58,440 --> 00:06:03,840
DNS activity. But we don't know what it is. Hackers aren't using clear text. Wall of sheep victims

72
00:06:03,840 --> 00:06:07,760
use clear text. But, you know, how are you going to figure out what's really happening and

73
00:06:07,760 --> 00:06:13,080
respond accurately? Because in a business environment, you can't walk in, you know, the bull in

74
00:06:13,080 --> 00:06:18,060
a china shop mentality and just start throwing firewall rules in. You stop business, you're in

75
00:06:18,060 --> 00:06:23,220
trouble, right? Now you've got two incidents on your hands. So, you know, tabletop is a

76
00:06:23,240 --> 00:06:27,820
great, it's a good start. It can help you identify weaknesses and gaps in your capabilities or

77
00:06:27,820 --> 00:06:33,000
your technologies. But it's not enough to really validate. Can you function with a sense of

78
00:06:33,000 --> 00:06:40,220
urgency when it has hit the fan? Are you validating the procedures that you rely on so closely?

79
00:06:40,380 --> 00:06:44,460
Because that's another thing I hate about tabletops is, okay, what would you do? You're getting

80
00:06:44,460 --> 00:06:49,340
DDoSed. What would you do? Well, we'd follow the procedure. When's the last time that procedure's

81
00:06:49,340 --> 00:06:53,220
ever been put to the test in a real DDoS? No way.

82
00:06:53,240 --> 00:06:58,700
Oh, some CISSP wrote it, hoping it was going to be effective. It's never been tested. So

83
00:06:58,700 --> 00:07:02,980
you're waiting for the day that you actually get DDoSed to find out if your procedures are

84
00:07:02,980 --> 00:07:09,100
going to work. Skills, training, right? I have a sock full of really sharp analysts. I mean, I'm

85
00:07:09,100 --> 00:07:13,620
so lucky. I was really, really lucky to have the team that I have. But let me tell you something.

86
00:07:13,780 --> 00:07:19,440
Even with the experience that all the people on my team have, years and years of experience, the

87
00:07:19,440 --> 00:07:23,220
first time we went through our scenario, the first time we sat in the simulation, the first time we

88
00:07:23,240 --> 00:07:28,780
had a breach, no kidding, this network's being hacked, you'd think they were all beginners.

89
00:07:29,440 --> 00:07:34,200
Because all those years of experience and knowledge and all that stuff, for some reason it almost

90
00:07:34,200 --> 00:07:39,160
goes out the window and all of a sudden you're panicking. Oh, my gosh. Like, stop this attack.

91
00:07:39,400 --> 00:07:42,960
Like, yes, they're exfilling records. Even though it's a simulation, it's funny because it has the

92
00:07:42,960 --> 00:07:48,480
same effect. You put a time constraint on this, be like, you have six hours. Stop the breach. It

93
00:07:48,480 --> 00:07:53,220
really does something to the heart rate, you know? And so my team that's, you know, we've got a lot of

94
00:07:53,240 --> 00:07:57,400
experience, not new to this, responds to incidents all day every day, all of a sudden couldn't

95
00:07:57,400 --> 00:08:02,900
remember that Palo Alto has many different tabs to click on to see traffic logs, threat logs, URL

96
00:08:02,900 --> 00:08:10,060
filter logs. They forgot all of that. And they did not find the exfil. So validation of your

97
00:08:10,060 --> 00:08:14,340
skills and your training, you know, tabletop is not going to cover that. And then lastly, right,

98
00:08:15,060 --> 00:08:19,680
tabletop, since you're not validating or actually doing anything, it's not going to highlight where

99
00:08:19,680 --> 00:08:23,220
your technology might fail you. It's not going to highlight where your personnel might fail you.

100
00:08:23,240 --> 00:08:28,160
Might not have enough coverage. You've got one guy on your incident response team, sure, he's

101
00:08:28,160 --> 00:08:32,900
going to do great at a tabletop. He just has to answer questions. And I'll throw him into the mix

102
00:08:32,900 --> 00:08:37,140
with a real incident and see how many hats he's going to have to wear and how quickly he's going

103
00:08:37,140 --> 00:08:47,260
to quit. It doesn't work. And, you know, one quick point. Think of this, the tabletop, why it's

104
00:08:47,260 --> 00:08:53,220
inadequate. Let's take something like paramedics, for instance, first responders. You know,

105
00:08:53,240 --> 00:08:59,500
imagine if first responders trained to respond to life-threatening situations simply by sitting

106
00:08:59,500 --> 00:09:04,420
down with colleagues and talking about it. You know, hey, you drive up on an accident and there's

107
00:09:04,420 --> 00:09:08,700
someone that's been ejected from a vehicle. They're bleeding profusely. What would you do? I

108
00:09:08,700 --> 00:09:13,000
don't know. I'd put on a bunch of bandages. Okay. Cool. Awesome. That's what we'll do. No, that's

109
00:09:13,000 --> 00:09:19,540
not how it goes. They go and they train. Is anyone here CPR qualified? Awesome. What did you have

110
00:09:19,540 --> 00:09:23,140
to do in your training to become CPR qualified? A lot of fun.

111
00:09:23,240 --> 00:09:29,660
Physical testing. On a dummy, right? Why couldn't you have just watched some videos?

112
00:09:29,700 --> 00:09:40,840
Taken a CBT. Muscle memory. Absolutely. Because there is no CBT, there's no video that's going to give you

113
00:09:40,840 --> 00:09:45,740
that sense of feel. What does it feel like to do chest compressions, right? What does it feel like to

114
00:09:45,740 --> 00:09:51,040
breathe air into an unresponsive person? If you've never felt that, you've never done that, you're not

115
00:09:51,040 --> 00:09:53,220
going to be ready when someone's dying. You're not going to be ready when someone's dying. You're not

116
00:09:53,240 --> 00:09:55,760
going to be ready when someone's dying in front of you. Your heart's going to be racing. You're

117
00:09:55,760 --> 00:09:59,200
going to be panicking. You're going to forget the video you watched. But you know what you won't

118
00:09:59,200 --> 00:10:04,220
forget is that muscle memory. That's going to come in handy. That's going to come in handy. So

119
00:10:04,220 --> 00:10:08,260
it's the same concept. It's the exact same concept. Firefighters don't learn to fight fire by

120
00:10:08,260 --> 00:10:15,160
watching videos on YouTube. Because when they run into a burning building, fear sets in. Panic.

121
00:10:15,180 --> 00:10:19,380
You know? So that training has to be muscle memory. It has to be instinctive. Incident response is

122
00:10:19,380 --> 00:10:23,220
the same way. And no, I'm not saying that incident responders, you know, are paramedics or

123
00:10:23,240 --> 00:10:28,240
firefighters know lots of respect for what those people do. But incident response is very similar

124
00:10:28,240 --> 00:10:32,800
in the nature that if you don't know how to do it in the calm, you're definitely not going to know

125
00:10:32,800 --> 00:10:39,560
how to do it in the chaos. So let's get to the good part. And just so you know, this is not a

126
00:10:39,560 --> 00:10:44,420
very slide-heavy presentation. I didn't want it to be. So I'm going to pull up some diagrams and

127
00:10:44,420 --> 00:10:50,500
I'm going to talk about them for a while. So I hope you all are good with that. So my advice for

128
00:10:50,500 --> 00:10:53,220
how to effectively train for digital forensics initiatives is that you have to train for

129
00:10:53,240 --> 00:10:58,680
incident response. Is attack your own network. Sort of. Obviously we don't want to do this in

130
00:10:58,680 --> 00:11:05,800
production. So we build it. Right? We build an environment. And let me tell you something. I was

131
00:11:05,800 --> 00:11:11,140
able to build a pretty sizable environment on two bare metal ES6I servers. We're not talking

132
00:11:11,140 --> 00:11:16,180
about a cluster. I'm not talking about a full rack in a data center. I'm talking about two one

133
00:11:16,180 --> 00:11:23,220
use servers. I was able to build a very, very high fidelity and elaborate environment. So this

134
00:11:23,220 --> 00:11:26,740
is something you can do on a budget. This is something you can do with yesteryear's hardware that

135
00:11:26,740 --> 00:11:30,920
you've replaced, you know, with your latest refresh and it's just sitting in a closet somewhere. Grab

136
00:11:30,920 --> 00:11:35,300
that stuff. Bring it into an area and start building out an environment. And I'm going to go over

137
00:11:35,300 --> 00:11:38,420
some of the things that you're going to want to think about when you're doing this because, yes,

138
00:11:38,500 --> 00:11:44,260
there's a right and a wrong way to go about it. So I'm going to kind of start from the left and work

139
00:11:44,260 --> 00:11:52,780
over. So my decision was to actually have the ability to connect this range up to the no kidding

140
00:11:53,220 --> 00:11:57,480
net. And the reason that I wanted to do that is because that makes it way easier for me to have

141
00:11:57,480 --> 00:12:02,640
organic traffic generation. Because think about it, right? If you're responding to an incident and,

142
00:12:02,800 --> 00:12:07,300
you know, you're combing through firewall logs and all kinds of good stuff, it would be really easy

143
00:12:07,300 --> 00:12:12,640
to find a bad guy if there was no web traffic. No activity at all. You just sit there and just

144
00:12:12,640 --> 00:12:16,840
wait for the first log to roll by and, oh, there it is. We got it. No. For a high fidelity

145
00:12:16,840 --> 00:12:22,600
environment, you need noise just like in the real world. Who here has ever tailed the logs of their

146
00:12:23,020 --> 00:12:29,480
firewall? Maybe once, right? Never did it again because you're not going to find anything.

147
00:12:29,640 --> 00:12:35,020
Because there's just going to be millions of lines just screaming by. So you need to reproduce that

148
00:12:35,020 --> 00:12:40,360
or else you're not doing good training. So I decided to have, okay, I wanted the ability to connect

149
00:12:40,360 --> 00:12:44,120
this range up to the Internet because when I get to the part later about how we generate traffic,

150
00:12:44,620 --> 00:12:49,480
you'll see that Internet access is kind of required. However, I'll give you a trick here in a minute

151
00:12:49,480 --> 00:12:53,200
just in case you don't have that Internet access. It can still be done. So I'm going to go ahead and

152
00:12:53,220 --> 00:12:59,540
do that. And also I wanted to share a quick resource. So while building this, I had to solve some

153
00:12:59,540 --> 00:13:05,420
problems like how do I generate traffic. I wrote a Python script. It's in my GitHub. If you can just

154
00:13:05,420 --> 00:13:10,820
remember my first initial E and my last name, which is almost impossible to remember, you can find

155
00:13:10,820 --> 00:13:16,100
me on GitHub, web traffic generator. It's just a little simple Python script that you can feed a seed

156
00:13:16,100 --> 00:13:22,660
of URLs to. And it will literally headlessly browse the Internet. And it will go a specific depth. So say

157
00:13:23,220 --> 00:13:28,380
start at these five websites and go to a depth of ten. So it will start at yahoo.com and it's going to

158
00:13:28,380 --> 00:13:34,160
click ten links in. And so it's a really cool way to generate organic web traffic. Because, guys, you

159
00:13:34,160 --> 00:13:41,600
cannot leverage canned TCP dump like playbacks. And let me tell you why. Because what we're using

160
00:13:41,600 --> 00:13:48,020
here, just like in our real environment, are stateful layer seven firewalls, right? Layer seven

161
00:13:48,020 --> 00:13:52,900
firewalls, well, actually, any stateful firewall is not going to play nicely with your canned TCP

162
00:13:53,220 --> 00:13:58,080
dump traffic. You're not going to just be able to replay PCAPs and have these firewalls pass them.

163
00:13:58,180 --> 00:14:02,620
It's not going to work. So unless you've got a breaking point, which I usually don't drop brand

164
00:14:02,620 --> 00:14:07,240
names, sorry, but unless you've got a very expensive system that was built to do that, you're not

165
00:14:07,240 --> 00:14:11,820
going to be able to. So what you need is you need real traffic. You're not going to be able to fake

166
00:14:11,820 --> 00:14:16,360
this because your firewalls are not going to care. So anyway, so traffic generation, and I have

167
00:14:16,360 --> 00:14:22,860
another tip for you guys on that one in a minute that's even easier than my script. The next

168
00:14:22,860 --> 00:14:32,580
component in this mix would be what would be your external and your external router, but this

169
00:14:32,580 --> 00:14:39,220
dotted line right here indicates kind of where this range starts and stops from the perspective of your

170
00:14:39,220 --> 00:14:44,300
C-cert responders. They are not aware of anything to the left of this dotted line. So this router

171
00:14:44,300 --> 00:14:49,160
here serves a very unique purpose. You're using that, and that could be anything. That could be a

172
00:14:49,160 --> 00:14:55,100
Vios router, previously, or Viata. That could also be a PFSense firewall. You don't need any special

173
00:14:55,100 --> 00:14:59,060
gadgetry here, but you do need something that can route because what you're doing there is route

174
00:14:59,060 --> 00:15:05,280
manipulation. So again, right, the effectiveness of this training is going to be how well can we

175
00:15:05,280 --> 00:15:09,280
emulate the things that you're going to see in the real world. Well, it's really hard to emulate C2

176
00:15:09,700 --> 00:15:18,200
traffic to Russia unless you have a VPS in Russia. So what we can do is in this router here that

177
00:15:18,200 --> 00:15:19,140
you're, you're, you're, you're, you're, you're, you're, you're, you're, you're, you're, you're, you're

178
00:15:19,160 --> 00:15:22,480
you're, you're participants aren't even aware of, in this router we're setting up some

179
00:15:22,480 --> 00:15:30,740
destination NATs, right? You can NAT an entire slash 16, if you wanted to, of, you know, a known

180
00:15:30,740 --> 00:15:36,200
Russian IP space. You can just NAT it to a server that you've got sitting up here in your fake

181
00:15:36,200 --> 00:15:41,180
Internet, you know, which is just another V-switch, fake Internet. And so what I can do then is

182
00:15:41,180 --> 00:15:46,840
I can have C2 of any of these systems here, going to a no kidding Russian IP. But it's not even

183
00:15:46,840 --> 00:15:49,140
leaving my range. It's just once it hits this location. You've got a different IP network for

184
00:15:49,160 --> 00:15:54,460
router it's getting destination added to a server that I've got listening up here. Why is that

185
00:15:54,460 --> 00:16:00,080
good to do? Why is that useful? Well, because we want to generate the same or similar types of

186
00:16:00,080 --> 00:16:04,720
artifacts that an incident response team should be looking for. So if you can't simulate traffic

187
00:16:04,720 --> 00:16:11,980
to Russia, well, I mean, you know, that's definitely something that you're, that would add a

188
00:16:11,980 --> 00:16:16,660
lot of value to the training. So, but again, I mean, Russia, that was kind of an obvious

189
00:16:16,660 --> 00:16:20,840
point, but maybe something less obvious, you know. But the point is, if your organization

190
00:16:21,420 --> 00:16:26,080
doesn't do business outside of the country, and you throw something in there, even something

191
00:16:26,080 --> 00:16:33,760
simple, you know, Great Britain, UK, France, whatever, it should still be anomalous. If your

192
00:16:34,320 --> 00:16:38,180
incident response team is looking at it, hey, why are we talking to this country that we don't

193
00:16:38,180 --> 00:16:45,920
even do business with? So anyway, so the other thing that we're leveraging this router here for

194
00:16:45,920 --> 00:16:52,780
is you can hijack DNS. So if for whatever reason you want to stand up, you know, a site, like a

195
00:16:52,780 --> 00:16:59,660
paste bin clone, which actually we did in our scenario, we created ponbin.ru, which our red team

196
00:17:00,260 --> 00:17:05,800
infiltrators use to exfil data. Well, to create that record outside of this environment, because

197
00:17:05,800 --> 00:17:09,060
you don't want your incident response team to log into the firewall and be like, oh, look, there's

198
00:17:09,060 --> 00:17:13,740
a static DNS entry for ponbin.ru. I bet that's going to be used for something. You create it out

199
00:17:13,740 --> 00:17:18,260
here. You hijack DNS out here. Maybe you get a little DNS server up here that's responding to

200
00:17:18,260 --> 00:17:22,920
those queries. I could hijack Google. I could hijack Yahoo. I could hijack anything I want. So

201
00:17:22,920 --> 00:17:28,040
now I can further manipulate what's going on in this environment to make it do what I want it to,

202
00:17:28,180 --> 00:17:34,680
to achieve my training objectives. The next thing, oh, and also I wanted to highlight some of

203
00:17:34,680 --> 00:17:38,500
the resources that I'm using in my environment. So obviously this is also where your attackers

204
00:17:38,500 --> 00:17:43,720
live, right? So you're going to need someone to red team this engagement. So if you actually have a

205
00:17:43,720 --> 00:17:47,680
So your red team guy literally could have one Metasploit box,

206
00:17:47,820 --> 00:17:50,540
one Cali system sitting up here in the fake internet and he's

207
00:17:50,540 --> 00:17:55,500
got a handful of interpreters on it and all the C2 and all the

208
00:17:55,500 --> 00:17:58,240
bad stuff that's going on in here is literally, it could

209
00:17:58,240 --> 00:18:00,900
appear to be going to 100 different IP addresses but it's

210
00:18:00,900 --> 00:18:03,680
all getting added to this one guy up here and one dude is red

211
00:18:03,680 --> 00:18:07,220
teaming this entire engagement which is also a ton of fun. But

212
00:18:07,220 --> 00:18:09,240
you know Apache web server running a bunch of virtual

213
00:18:09,240 --> 00:18:12,460
servers for your special sites and services, you want to fake

214
00:18:12,460 --> 00:18:14,840
Twitter, set up a fake Twitter up here. You want to fake

215
00:18:14,840 --> 00:18:17,580
Facebook, you know you can do all of that up here using very

216
00:18:17,580 --> 00:18:20,260
simple things. Another thing that I'm doing in my fake

217
00:18:20,260 --> 00:18:25,900
internet is, so to really add additional realism to this

218
00:18:25,900 --> 00:18:30,800
environment, email, right? Email is the number one vector. It is,

219
00:18:30,800 --> 00:18:34,900
that's how they get in. Social engineering, someone got an

220
00:18:34,900 --> 00:18:37,220
email that they trusted, they believed in, they popped it

221
00:18:37,220 --> 00:18:39,180
open. So you need the ability to send email into this

222
00:18:39,180 --> 00:18:42,360
environment. So we've got a Python script literally that

223
00:18:42,460 --> 00:18:45,880
is just sending lorem ipsum every minute to every single user

224
00:18:45,880 --> 00:18:49,380
in our fake environment. That way when we send that one

225
00:18:49,380 --> 00:18:52,440
malicious email through, it just blends in. It just blends

226
00:18:52,440 --> 00:18:54,600
right in with all the other noise. So, you know, we're

227
00:18:54,600 --> 00:19:01,660
leveraging some pretty simple concepts here. Let's see. Okay,

228
00:19:01,760 --> 00:19:06,960
so the next piece in this puzzle is your perimeter firewall. Now

229
00:19:06,960 --> 00:19:08,960
here's the thing. I meant to touch on this a little earlier

230
00:19:08,960 --> 00:19:11,320
but, so here's the thing. When you're approaching this and

231
00:19:11,320 --> 00:19:12,440
you're going to build an environment, you're going to

232
00:19:12,460 --> 00:19:15,160
need to figure, there's a component you need to figure out

233
00:19:15,160 --> 00:19:18,680
first. If you're building this on the cheap, obviously you're

234
00:19:18,680 --> 00:19:21,040
going to lean more towards open source solutions, which if you've

235
00:19:21,040 --> 00:19:24,040
been to my previous talk, I'm all about. Open source is

236
00:19:24,040 --> 00:19:27,200
awesome, right? A lot of really powerful commercial tools run on

237
00:19:27,200 --> 00:19:30,240
open source engines, so what's the difference? However, know

238
00:19:30,240 --> 00:19:34,960
that, you know, the effectiveness of the training is

239
00:19:34,960 --> 00:19:38,840
going to be tied to how closely does this mimic your production

240
00:19:38,840 --> 00:19:42,440
environment. So there's a lot of value in trying to reproduce. So

241
00:19:42,460 --> 00:19:46,340
for instance, if you use, you know, FortiGate firewalls, it would

242
00:19:46,340 --> 00:19:49,580
probably be a good idea to try to get FortiGate virtual machines in

243
00:19:49,580 --> 00:19:53,620
here and use those because that is another way to, one, train your

244
00:19:53,620 --> 00:19:57,440
incident response team, to, two, validate how good are my FortiGates

245
00:19:57,440 --> 00:20:00,400
anyway? Have you ever really put them to the test? Do it in

246
00:20:00,400 --> 00:20:04,320
here where the stakes are not high. Well, know that you don't

247
00:20:04,320 --> 00:20:06,760
necessarily need to go out and buy licenses for all this stuff.

248
00:20:06,900 --> 00:20:12,360
Tell your vendor what you're doing. Hey, John, FortiGate, listen, man, I'm building a training environment.

249
00:20:12,360 --> 00:20:15,880
To make my guys better at what they do. Can I get a complimentary

250
00:20:15,880 --> 00:20:18,900
license? I'm going to tell you right now, guys, I've probably, I've

251
00:20:18,900 --> 00:20:23,800
probably been given $200,000, $300,000 worth of free licenses for my

252
00:20:23,800 --> 00:20:27,600
training environment. Vendors don't care. You tell them it's a lab, it's

253
00:20:27,600 --> 00:20:30,580
offline, it's not production, you're just training with it, they will

254
00:20:30,580 --> 00:20:34,200
throw product at you. So don't be afraid to ask, okay? So when you're

255
00:20:34,200 --> 00:20:38,200
approaching this and you're deciding, do I want to use, you know, Palo Alto, FortiGate,

256
00:20:38,280 --> 00:20:42,240
you know, ESAs, whatever, talk to your vendors. You're more than likely going

257
00:20:42,360 --> 00:20:45,700
to be given some leeway there. But if you do go the open source route, I'm going to tell you

258
00:20:45,700 --> 00:20:48,920
right now, you can achieve everything that I'm talking about here with open source solutions.

259
00:20:49,360 --> 00:20:53,360
Short of maybe your Windows workstations that you're going to want to break into over here

260
00:20:53,360 --> 00:20:59,800
eventually. So moving on. So perimeter firewall, you can, you can totally roll like a

261
00:20:59,800 --> 00:21:04,700
PFSense if you want to, or you can throw, like I said, a virtual appliance of, or even a

262
00:21:04,700 --> 00:21:08,360
physical appliance. If you have a little, you know, lab unit sitting around, fork that into

263
00:21:08,360 --> 00:21:12,280
this environment and use that. But what that perimeter firewall is

264
00:21:12,360 --> 00:21:15,260
going to do is just like in your production environment, it's going to have a DMZ probably

265
00:21:15,260 --> 00:21:19,180
with some web applications, some web servers, things like that. Maybe hang a male security

266
00:21:19,180 --> 00:21:23,380
appliance. Whatever you're doing in production, you need to do here. That's, it's really the

267
00:21:23,380 --> 00:21:29,420
extent of it. And hopefully just like in production, you're running IDS IPS here, right? To

268
00:21:29,420 --> 00:21:34,920
simplify, the next, the next piece of it is really just like one giant core router to rule them

269
00:21:34,920 --> 00:21:41,560
all. And, and it could be, it could be, you know, a copy of the same device you've got here or it

270
00:21:41,560 --> 00:21:44,820
could be a different device. It doesn't really matter. But that's where you're going to create all

271
00:21:44,820 --> 00:21:49,080
your different virtual networks to mimic, you know, your different branch offices or divisions or

272
00:21:49,080 --> 00:21:53,200
whatever, whatever subsections you have in your organization. And the reason you want to do this

273
00:21:53,200 --> 00:21:57,820
is because, you know, remember, during our incident response training, we want to create

274
00:21:57,820 --> 00:22:03,040
anomalies, right? We want to create things that are out of the ordinary. Like someone in HR, you

275
00:22:03,040 --> 00:22:08,780
know, hammering a SQL server. Well, in order to mimic that, we need to have an HR subnet, right?

276
00:22:08,780 --> 00:22:16,220
And then the C-cert needs to understand that that is, you know, what that is. On your core

277
00:22:16,220 --> 00:22:21,980
router, it also would be ideal, hopefully, just like in production, to have IDS IPS here so that

278
00:22:21,980 --> 00:22:26,640
you've got visibility on not only your north-south traffic coming into and leaving the

279
00:22:26,640 --> 00:22:33,060
environment, but east-west visibility on HR talking to the server farm, you know, or IT talking

280
00:22:33,060 --> 00:22:38,080
to marketing and sales, right? Now, again, this is cool because when you're architecting this,

281
00:22:38,780 --> 00:22:44,220
this very simple network, it starts to prompt questions in your mind of, well, do we really have

282
00:22:44,220 --> 00:22:49,200
that visibility in our environment? I don't think we do. You know, we've got a giant, you know,

283
00:22:50,320 --> 00:22:55,740
Cisco Nexus, you know, big iron router in our core, and it's not inspecting anything. It's just

284
00:22:55,740 --> 00:22:59,940
routing. So we wouldn't know if HR was beating up a SQL server. So this is cool because you're

285
00:22:59,940 --> 00:23:04,860
going to accidentally start to think about the gaps you've got in your own environment as you're

286
00:23:04,860 --> 00:23:08,760
thinking about how to build this one. But for the fidelity of the training, of course, you're

287
00:23:08,780 --> 00:23:12,940
going to want to have these sensors deployed so that they're inspecting everything because we

288
00:23:12,940 --> 00:23:19,220
want to have that flood of information for our C cert to have to sift through to look for evil. So

289
00:23:19,220 --> 00:23:23,280
server farm, obvious, I mean, I don't really need to go into detail to you guys about what needs

290
00:23:23,280 --> 00:23:27,040
to be in a server farm in a corporate environment, but you just want to have those important items

291
00:23:27,040 --> 00:23:32,220
like a domain controller, exchange slash mail server, right? A file server of some kind,

292
00:23:32,380 --> 00:23:38,140
CIFS, NFS, Samba, whatever. And, you know, in my environment, we rolled a SQL server, you know,

293
00:23:38,140 --> 00:23:44,240
because most organizations have large databases, and so we wanted to have one, too. Branch

294
00:23:44,240 --> 00:23:49,320
offices, self-explanatory, but I'm going to go over some cool tips and tricks here. So back to

295
00:23:49,320 --> 00:23:54,540
that organic traffic generation, right? So first and foremost, you know, what do you put out

296
00:23:54,540 --> 00:23:59,060
here? What do you put in these subnets? Well, ideally, you're going to take a copy of the golden

297
00:23:59,060 --> 00:24:03,400
image that your organization uses. You know, IT has probably some image that they use to image new

298
00:24:03,400 --> 00:24:08,120
systems. Get a copy of that. Use that one. And put it in here. Because now, you're going to have a

299
00:24:08,120 --> 00:24:11,340
copy of that image. Now, during your training, you're going to be validating the security of that

300
00:24:11,340 --> 00:24:15,840
image. How secure is that image? Have you ever really put it to the test? Have you attacked the

301
00:24:15,840 --> 00:24:20,360
crap out of it with Metasploit? Well, you're going to get to in this environment. So when you get

302
00:24:20,360 --> 00:24:24,820
all these workstations set up, Windows boxes, you know, whatever your folks are using, I'm going

303
00:24:24,820 --> 00:24:29,040
to share something really cool with you, save you some time. There's a Chrome extension called

304
00:24:29,040 --> 00:24:34,620
Chaff. And this thing is awesome. And it literally, it, you know, my traffic generation script,

305
00:24:34,800 --> 00:24:38,100
it does that. It does it actually even better. And, of course, it's a little bit more complex.

306
00:24:38,120 --> 00:24:42,140
Of course, I didn't find Chaff until after I'd spent, you know, like seven hours writing mine.

307
00:24:42,340 --> 00:24:47,200
But Chaff is actually really cool because you can install Chrome on each of these workstations. You

308
00:24:47,200 --> 00:24:51,880
can tell Chaff, you can tell Chrome to start at boot, auto log in, start at boot, and Chaff will

309
00:24:51,880 --> 00:24:56,840
start automatically. And Chaff will literally do everything a user will do. Browse the Internet,

310
00:24:57,040 --> 00:25:01,740
download dumb files, go to YouTube, stream some stuff. I've seen Chaff end up in some pretty

311
00:25:01,740 --> 00:25:07,300
risky places of the Internet, which is perfect. Because during your incident response simulation,

312
00:25:07,300 --> 00:25:12,200
you're going to see IPS signatures firing. You're like, I don't know what that is. It's one of the

313
00:25:12,200 --> 00:25:16,920
Chaff users. And that's perfect because that's what real users do. They do dumb stuff and they're

314
00:25:16,920 --> 00:25:20,880
going to go to weird websites and it's going to make your firewall upset and that's going to make

315
00:25:20,880 --> 00:25:25,040
your incident responders have to sort through the noise, right, which is what it takes in order to be

316
00:25:25,040 --> 00:25:29,920
effective at incident response. There's got to be a lot of noise. So Chaff, that's one of my little

317
00:25:29,920 --> 00:25:36,780
secrets there. But also it's important that whatever endpoint agents you're using, if you've got an AV,

318
00:25:37,300 --> 00:25:42,640
IPS, HIDS, whatever, a log forwarder, you want to get that on this image as well, obviously,

319
00:25:42,760 --> 00:25:47,120
because you're going to need data. You're going to need information off these workstations going to

320
00:25:47,120 --> 00:25:51,380
your aggregation, your SIM or whatever. We'll cover that in a second, too. So, you know, get

321
00:25:51,380 --> 00:25:55,480
those endpoints configured. You're going to need a mail client on them because, remember, mail.

322
00:25:55,680 --> 00:25:58,900
Got to have it. You're going to use it in these simulations, the scenarios that we're going to go

323
00:25:58,900 --> 00:26:07,280
over in a minute. And then obviously you want all these guys joined to a domain. So you're going to

324
00:26:13,200 --> 00:26:21,940
have to build out your subnet for your incident responders. And so typically, you know, I hang

325
00:26:21,940 --> 00:26:25,800
that off the same core router, but it is a totally separate subnet. That way, you know, the

326
00:26:25,800 --> 00:26:29,620
ACLs, it's permit, permit, you know, any, any, all that good stuff, no IDS, no IPS, because you

327
00:26:29,620 --> 00:26:33,260
just want your guys to be able to have a straight shot into the environment. Because later on, as

328
00:26:33,260 --> 00:26:37,280
you mature here, right, after you get past the stage of detection, you're going to be able to

329
00:26:37,300 --> 00:26:42,840
find the evil. Can you find the evil? Okay, great. You found it. Now go ahead and, you know,

330
00:26:42,840 --> 00:26:47,800
leverage your Wemmick, your PowerShell, whatever. Reach into that box. Let's get volatile data.

331
00:26:47,960 --> 00:26:51,680
Let's get event logs. Let's get a memory image. Let's, you know, then you can actually start

332
00:26:51,680 --> 00:26:56,820
getting into active response on these systems. And so it's good to have them on a separate subnet

333
00:26:56,820 --> 00:27:02,660
so that you can, you can permit them to do all of these things. So I'm going to go over a couple

334
00:27:02,660 --> 00:27:07,280
more, a couple more solutions that you guys will want to look into for people that are interested.

335
00:27:07,300 --> 00:27:10,000
Building out your environment. And actually, to be honest with you, these are solutions you can

336
00:27:10,000 --> 00:27:14,440
look into putting into your production environment. So if anyone here is familiar with Elastic

337
00:27:14,440 --> 00:27:20,880
Search, right, Elastic Search Logstash has a really cool log shipper called Beats. And there's

338
00:27:20,880 --> 00:27:25,620
two flavors of it, actually. So there's FileBeats, which is just an agent you can put on a

339
00:27:25,620 --> 00:27:31,040
system. And you can point it at any log that you want. It could be a custom log that you wrote for

340
00:27:31,040 --> 00:27:35,040
custom application. Or it could be a Windows event log. It could be syslog. It could be auth. It

341
00:27:35,040 --> 00:27:37,280
could be any log you want. And FileBeats can help you get that log out of your system. And so

342
00:27:37,280 --> 00:27:43,860
you can actually ship those logs to a central repository using TCP with TLS. Like all kinds of

343
00:27:43,860 --> 00:27:48,380
cool stuff. So it's kind of a step up from your simple rsyslog forwarder, right? Which is UDP.

344
00:27:48,620 --> 00:27:54,020
It's plain text. It's, you know, what happens if your syslog server goes down? Well, syslog

345
00:27:54,020 --> 00:27:57,520
keeps forwarding. It doesn't care. It's just going to keep forwarding. Well, Beats is a little

346
00:27:57,520 --> 00:28:01,480
smarter than that. If your syslog server went down, Beats is going to sit there and it's going to

347
00:28:01,480 --> 00:28:06,280
store those logs and wait for it to come back up. So really cool tool. And it integrates really

348
00:28:06,280 --> 00:28:12,100
nicely into Elasticsearch Logstash. Clam AV, which if most of you guys are familiar with

349
00:28:12,100 --> 00:28:19,260
Sourcefire, it's Clam AV under the hood. Who knew? Osec, which is an open source HIDs agent. Very,

350
00:28:19,360 --> 00:28:24,740
very, very, very powerful. This is an awesome tool. Millions of customization options. You put

351
00:28:24,740 --> 00:28:29,380
it on your host and it's looking for rootkits. It's looking for evil. And it's reporting back to

352
00:28:29,380 --> 00:28:32,840
a central server. So I'm just going over some of the things that you can look at using in your

353
00:28:32,840 --> 00:28:34,420
environment that cost nothing.

354
00:28:36,760 --> 00:28:41,900
How about your incident response team, right? Maybe you're in an organization with a ton of money

355
00:28:41,900 --> 00:28:50,580
and you've got a really cool SIM. Maybe you're not. The good news is you're not out of luck.

356
00:28:51,580 --> 00:28:57,780
Greylog. Very, very powerful tool. I mean, it is very close to an open source Splunk. Not to be

357
00:28:57,780 --> 00:29:03,740
confused, Splunk is awesome. Splunk is extremely powerful. But Greylog comes pretty close and the

358
00:29:03,740 --> 00:29:05,820
price is right. $300,000.

359
00:29:07,680 --> 00:29:14,580
Alien vault OSIM. Anyone familiar with OSIM? Extremely powerful SIM. It's got vulnerability

360
00:29:14,580 --> 00:29:19,000
scanning, passive asset detection, all kinds of good stuff. You could stand that up in this

361
00:29:19,000 --> 00:29:24,140
environment for free. The Hive. You saw my talk yesterday. I covered the Hive. Really awesome

362
00:29:24,140 --> 00:29:28,320
incident response collaboration tool. So if you've got a team of analysts that are responding to an

363
00:29:28,320 --> 00:29:33,340
incident, the Hive is an awesome open source platform for them to collaborate on. Check it out.

364
00:29:33,340 --> 00:29:38,880
So I just wanted to share a couple tools that you guys could use if you didn't already have these

365
00:29:38,880 --> 00:29:46,960
systems in place. Okay. So now that you have your environment, now you have to start scripting your

366
00:29:46,960 --> 00:29:51,920
scenarios. And this is one of those things where, you know, it sucks because you might have a team

367
00:29:51,920 --> 00:29:56,300
of ten. But you or someone you designate that's going to have to be the lone wolf that's gonna

368
00:29:56,300 --> 00:30:00,140
have to come up with these scenarios because, obviously, your incident response team can't know

369
00:30:00,140 --> 00:30:03,320
about it. Right? It's got to be a surprise. So I actually have a tourney where, you know, we're going to be

370
00:30:03,340 --> 00:30:09,040
had my Intel analyst draft up these Intel reports. They're totally bogus. I mean, it's like

371
00:30:09,040 --> 00:30:17,760
95% fluff and about 5% actual Intel, just like in real life. Right? So we dreamt up this

372
00:30:17,760 --> 00:30:22,560
scenario. And it's important that you pick one that's applicable to your organization, right?

373
00:30:22,700 --> 00:30:26,660
Like, so you know what kind of threats your organization faces. I'm a law enforcement

374
00:30:26,660 --> 00:30:30,500
agency. We deal with hacktivists. You know, people that don't like law enforcement

375
00:30:30,500 --> 00:30:38,600
agencies. And so we dreamt up a hacktivist scenario where there's this, you know, this group,

376
00:30:38,800 --> 00:30:43,040
you know, like anonymous or something that really wants to embarrass large law enforcement

377
00:30:43,040 --> 00:30:48,980
organizations. And we got all their TTPs and some observables that have been discovered in the

378
00:30:48,980 --> 00:30:54,760
wild and all this stuff. And in all reality, we really are only going to use one or two of the

379
00:30:54,760 --> 00:30:58,580
indicators that we provide to these guys. Because just like in the real world, I mean, these

380
00:30:58,580 --> 00:31:00,480
Intel reports, they're not smoking guns. They're not using a gun. They're not using a gun. They're

381
00:31:00,500 --> 00:31:04,800
not going to give you all the information you need. But this is to test your team. Are they

382
00:31:04,800 --> 00:31:10,020
paying attention? Are they looking at the relevant Intel that's available for a particular

383
00:31:10,020 --> 00:31:14,140
incident? Or are they like, oh, yeah, great, yeah, Intel. Okay, cool. No, I can find bad. I've

384
00:31:14,140 --> 00:31:19,400
done this before. I'll find it. So this is a test on are they ingesting the information that's

385
00:31:19,400 --> 00:31:23,000
available to them to make them more effective at finding a particular threat to the

386
00:31:23,000 --> 00:31:30,480
organization? Yes. And we'll go over in one of the scenarios ways that you can do that. So the

387
00:31:30,500 --> 00:31:35,080
first one is you can really weigh in on, you can test whether or not they're paying attention to

388
00:31:35,080 --> 00:31:39,440
things like this. So now I kind of want to dive into a couple actual scenarios that I've

389
00:31:39,440 --> 00:31:44,840
developed that were really successful in training our team. So first and foremost, though, I want

390
00:31:44,840 --> 00:31:49,520
to preface with this. Your red team injects. So the attacks that you're going to carry out on

391
00:31:49,520 --> 00:31:54,100
your environment, it can't be cowboy. You can't just, you know, wing it. It needs to be

392
00:31:54,100 --> 00:32:00,420
scripted. You need to have a playbook that has repeatable and measurable impacts because

393
00:32:00,500 --> 00:32:05,380
for it to be an effective training solution, for every red team inject, there needs to be a blue

394
00:32:05,380 --> 00:32:10,520
team observable. So as you're going through and you're mapping out your scenario, you literally

395
00:32:10,520 --> 00:32:16,180
need to wear both hats. So, okay, I'm going to try this attack on that system. What did I see on

396
00:32:16,180 --> 00:32:20,420
this system when I did that? Oh, okay. Or what did I see in my sim? What did I see in my logs?

397
00:32:20,520 --> 00:32:24,820
What did I see here and there? So you need to make sure that everything that you're scripting for

398
00:32:24,820 --> 00:32:30,440
your injects is something that your blue team can't even find. If they can't find it, why not? You

399
00:32:30,440 --> 00:32:37,160
know, that's a gap in technology. So another thing, you know, if you go off script during a red

400
00:32:37,160 --> 00:32:41,800
team engagement on, you know, in an environment like this, well, now your blue team, like, you're

401
00:32:41,800 --> 00:32:44,720
not going to know if they found the bad stuff or not because you don't even know what artifacts

402
00:32:44,720 --> 00:32:48,460
you're generating. So that's why it's important to script this out. So that's what we have. We've

403
00:32:48,460 --> 00:32:52,360
got playbooks for different scenarios that we're going to run through. So, you know, for

404
00:32:52,360 --> 00:32:57,220
instance, here's a, you know, and I'll just kind of blaze through these, but one scenario that we

405
00:32:57,220 --> 00:33:00,420
have is we're going to kick off our traffic generation. So we've got a ton of noise and we're

406
00:33:00,440 --> 00:33:04,100
going to get emails flooding into the network every minute. I mean, hundreds of emails. We've

407
00:33:04,100 --> 00:33:10,820
got traffic generation. And then we're going to socially engineer a DBA. We've got a database

408
00:33:10,820 --> 00:33:18,160
administrator, poor Regina Buckley. She's been with the organization for years. And, you know,

409
00:33:18,160 --> 00:33:21,620
not the most security aware, but she does all right. Well, we're going to phish her because

410
00:33:21,620 --> 00:33:26,640
she likes talking about what she does on LinkedIn and Twitter and Facebook. And so we found her.

411
00:33:26,700 --> 00:33:30,420
We're going to spear phish her. And we're going to send her an e-mail. We're going to send her an

412
00:33:30,440 --> 00:33:37,160
email that very urgently tells her to install a Windows update from vendosupdate.com, which,

413
00:33:37,220 --> 00:33:41,640
it's funny, you can, you know, when you're manipulating DNS, you can create any type of DNS

414
00:33:41,640 --> 00:33:48,100
record that you want. So VVIendosupdate.com plays really well into your scenario because a lazy

415
00:33:48,100 --> 00:33:52,480
analyst that's just kind of skimming through logs doesn't see the VV, right? The simple things

416
00:33:52,480 --> 00:33:56,880
that attackers do. They're playing on your lack of attention to detail. So when your red team

417
00:33:56,880 --> 00:34:00,420
injects, you want to do the same thing. Do things that are going to force your analyst to do the

418
00:34:00,420 --> 00:34:03,600
wrong thing. You want your analyst to pay attention to what they're looking at, not to glaze

419
00:34:03,600 --> 00:34:08,660
over it. So she downloads this update, which actually, of course, is a reverse shell that's

420
00:34:08,660 --> 00:34:14,180
going to give our red team attacker a shell on her system. So she's here. We're here. We now

421
00:34:14,180 --> 00:34:19,640
have a shell on her system. Escalate privileges. Get her credentials. We already know she's a

422
00:34:19,640 --> 00:34:24,660
database admin, so we already know what accesses she has. So what we're going to do next is pretty

423
00:34:24,660 --> 00:34:28,460
obvious. We're going to start enumerating the network. Now, remember, this is why it was

424
00:34:28,460 --> 00:34:33,400
important that we have east-west inspection because if we just had a direct link to the SQL

425
00:34:33,400 --> 00:34:37,760
server and we started scanning, our blue team's not going to see that, right? So we have

426
00:34:37,760 --> 00:34:41,960
inspection for east-west, and we're intentionally going to generate some noise. We're going to

427
00:34:41,960 --> 00:34:46,440
scan. We're scanning the server farm for the presence of a SQL server. That's a blue team

428
00:34:46,440 --> 00:34:53,080
observable. You know, port scan, 3306. They should find that. They should see that. Well, we

429
00:34:53,080 --> 00:34:57,900
found it. We found our SQL server. So from Regina Buckley's system, we now have our creds. We're

430
00:34:57,900 --> 00:35:01,640
just going to go ahead and log right into that SQL server, and we're going to start exfilling

431
00:35:01,640 --> 00:35:06,420
records. Now, here's the thing. Anybody here, especially red teamers, there are intentional

432
00:35:07,300 --> 00:35:12,040
dumbing down of red team activities because, again, when you're first getting started with

433
00:35:12,040 --> 00:35:17,260
those guys, your team doesn't know what's going on, okay? So if you go in all stealth, you're

434
00:35:17,260 --> 00:35:22,120
using an RC4 encrypted payload, and you're exfilling over encrypted channels, yes, of course,

435
00:35:22,160 --> 00:35:27,660
your team's not going to find it. So you need to do some things earlier on to enable them, to

436
00:35:27,660 --> 00:35:32,480
build up their confidence, to help them find, to know what to look for. So what we're going to do

437
00:35:32,480 --> 00:35:35,580
is we're going to grab all the SQL records, and we're just going to pull them over to Regina

438
00:35:35,580 --> 00:35:38,580
Buckley's system, even though we could have just pulled them right through our meterpreter shell

439
00:35:38,580 --> 00:35:42,980
and been gone. We're going to pull them to Regina Buckley's system. Through our meterpreter

440
00:35:42,980 --> 00:35:48,280
shell, we're going to drop curl onto her box. And remember, in our fake Internet here, we stood

441
00:35:48,280 --> 00:35:57,300
up a pastebin clone called ponbin.ru. From Regina's box, we are going to, via the API for

442
00:35:57,660 --> 00:36:02,840
ponbin, we're just going to post 5,000 sensitive records right to ponbin. Because remember, I'm a

443
00:36:02,840 --> 00:36:05,720
hacktivist. I'm not in this for the money. I just want to embarrass the hell out of your

444
00:36:05,720 --> 00:36:10,800
organization. So I'm just going to post 5,000 sensitive records right to ponbin.ru. So what

445
00:36:10,800 --> 00:36:15,140
additional observables have we generated? Well, we dropped a binary on the system in a protected

446
00:36:15,140 --> 00:36:20,480
directory. When we exfilled data from the SQL server through the network to Regina Buckley's

447
00:36:20,480 --> 00:36:26,320
box, if you've got a halfway decent IDS, it picked up on the fact that 5,000 PII records just

448
00:36:26,320 --> 00:36:33,260
traversed the network in the clear. That should be noticeable. Then we did it again. We curled in

449
00:36:33,260 --> 00:36:39,600
clear text from Regina's system via an API. We exfilled all 5,000 of those records. Now, did we

450
00:36:39,600 --> 00:36:44,260
do it the same way an attacker would? No, of course not. An attacker would not have used any of

451
00:36:44,260 --> 00:36:49,900
those methods. But remember, to start, your team, you're lucky if your team is going to see any of

452
00:36:49,900 --> 00:36:54,580
that. If your traffic generation is good. If all the noise is there. If your team sees the

453
00:36:55,040 --> 00:36:56,300
scenario I just described. If your team sees the scenario I just described, if your team sees the

454
00:36:56,300 --> 00:37:00,640
scenario I just described, you're better than most. So you've got to practice with the

455
00:37:00,640 --> 00:37:08,300
intentionally obvious scenarios first. Let your team get a feel for what does abnormal stuff look

456
00:37:08,300 --> 00:37:14,780
like. Then you upgrade to the more, you know, sophisticated attacks. Which, if you're not

457
00:37:14,780 --> 00:37:19,800
gathering this already, this is a ton of fun for the red team, too. So, you know, you build this to

458
00:37:19,800 --> 00:37:23,580
train a blue team, but that means that some lucky person gets to be the guy attacking these

459
00:37:23,580 --> 00:37:26,280
systems. And that's a ton of fun, too. So, you know, you build this to train a blue team, but that

460
00:37:26,280 --> 00:37:34,000
is one scenario that we've had. Yes. Absolutely, step by step. Literally copy and paste into the

461
00:37:34,000 --> 00:37:39,260
meta split console. We even pre-built meta split resource files to automate a lot of the things

462
00:37:39,260 --> 00:37:46,880
like... Hey, enumerate SQL database. Well, you want that to be 100 percent repeatable. So I might

463
00:37:46,880 --> 00:37:51,200
have one way of enumerating a network. You might have another. But we can't have those

464
00:37:51,200 --> 00:37:56,080
variances when we have a scoreboard that's expecting a specific answer. How many

465
00:37:56,280 --> 00:38:01,980
systems did the attacker scan while enumerating the network. That's why it needs to be 100%

466
00:38:01,980 --> 00:38:09,220
repeatable. So yes, the playbook is extremely detailed, down to the command that the red team

467
00:38:09,220 --> 00:38:12,840
is going to run for the inject so that we know exactly what the blue team should have seen.

468
00:38:15,720 --> 00:38:26,260
So as we start to migrate to more, I guess, more complex scenarios, excuse me, we're

469
00:38:26,760 --> 00:38:31,960
dehydrated here. So or I wouldn't say complex but different, right? A lot of organizations

470
00:38:31,960 --> 00:38:40,380
have, like I said, web servers sitting out here in the DMZ. Well, how sensitive are the systems

471
00:38:40,380 --> 00:38:45,180
that are in front of those web servers? If you've got F5s, if you've got IDS tipping points,

472
00:38:45,260 --> 00:38:49,900
whatever, are they properly tuned? Are they configured to look for the types of attacks that

473
00:38:49,900 --> 00:38:54,600
your systems are most vulnerable to? Well, try it here. So you get a web server, right,

474
00:38:54,700 --> 00:38:56,260
that's facing the Internet. But it has to be a web server. It has to be a web server that's

475
00:38:56,280 --> 00:39:00,420
a backend connection to the SQL server. Well, maybe that's another scenario where you get an

476
00:39:00,420 --> 00:39:04,200
attacker that's beating up this web server, running SQL injects against it in order to do the

477
00:39:04,200 --> 00:39:09,760
same thing, to exfil data, to exfil data. Would your team know what to look for, know how to

478
00:39:09,760 --> 00:39:15,740
respond to that type of situation? Now this is where things can get a little more complicated,

479
00:39:15,880 --> 00:39:21,540
right? When you want to start really upgrading to more sophisticated scenarios to put your

480
00:39:21,540 --> 00:39:25,940
team to a tougher test, that's where you do start using things like RC4 and other things like

481
00:39:25,940 --> 00:39:26,260
tougher test, that's where you do start using things like RC4 and other things like that. So if you

482
00:39:26,260 --> 00:39:29,720
have more encrypted payloads that are not going to set off IDS IPS, that's where you start using

483
00:39:29,720 --> 00:39:35,060
encrypted channels over, you know, normal web ports, things like that. Things that are not going

484
00:39:35,060 --> 00:39:38,620
to stand out, they're not going to be obvious, you're not going to be scanning the entire

485
00:39:38,620 --> 00:39:42,560
network, making a ton of noise, you're going to be super quiet, super covert, just like a real

486
00:39:42,560 --> 00:39:49,080
attacker would. But then how do you throw your team a bone to see how closely they've been paying

487
00:39:49,080 --> 00:39:55,300
attention, right? Maybe in a previous scenario, the Intel report had a bunch of IP addresses and

488
00:39:55,300 --> 00:40:01,420
none of them were relevant at the time, just like in the real world, right? But two scenarios

489
00:40:01,420 --> 00:40:08,100
later, one of them is. But they haven't even looked at that Intel report in hours. It's in the back

490
00:40:08,100 --> 00:40:13,320
of their mind, they're not even thinking about it. But all along, you've had a really slow and

491
00:40:13,320 --> 00:40:22,040
quiet beacon sitting on a system on the network. And while your team is scrambling, looking for

492
00:40:22,040 --> 00:40:25,280
things that don't exist, because, remember, we're on a stealthier engagement this time, you're

493
00:40:25,980 --> 00:40:33,480
sitting there doing its thing and you've got an attacker doing some really dirty things. You

494
00:40:33,480 --> 00:40:39,040
name it. Your imagination, the sky's the limit. Does your team ever connect the dots to say, wait a

495
00:40:39,040 --> 00:40:44,840
second, the threat Intel we got six hours ago that wasn't relevant then is relevant now. We've

496
00:40:44,840 --> 00:40:51,640
had C2 on this system for all day long. Connecting dots like that, that is analysis. Paying

497
00:40:51,640 --> 00:40:55,260
attention to the details. Not forgetting about previous indicators just because maybe the

498
00:40:55,260 --> 00:40:59,780
weren't relevant at the time. That's another effective way to test your team to pay attention.

499
00:41:00,020 --> 00:41:04,320
This was a really fun one, actually. This one I'm describing. What we did was we wrote ransomware

500
00:41:04,940 --> 00:41:12,440
and we had a persistent beacon on one system. We dropped the ransomware executable onto the box.

501
00:41:12,880 --> 00:41:17,020
We waited until things got really quiet. The incident response team was distracted looking for

502
00:41:17,620 --> 00:41:21,900
rabbits in the rabbit hole. And then we went ahead and we hit the file server and we encrypted

503
00:41:21,900 --> 00:41:28,960
every single file on it. And here's the thing. We had given them Intel of exactly what we were

504
00:41:28,960 --> 00:41:34,980
gonna do. Obviously you saw the Intel document. It didn't look like a really friendly, straight to

505
00:41:34,980 --> 00:41:39,860
the point manual because they're not. But we gave them an Intel document that said exactly what

506
00:41:39,860 --> 00:41:45,020
we were gonna do. They didn't see it coming. What if I walked up to you and said, hey, I'm about

507
00:41:45,020 --> 00:41:51,560
to punch you in the face. You have no right to be surprised if I then punch you in the face. I told you

508
00:41:51,560 --> 00:41:54,540
you I was going to do it. But if you're not listening to me when I tell you, if you're

509
00:41:54,540 --> 00:41:57,840
not paying attention, if you're ignoring the signs, you're ignoring the intel, you're not

510
00:41:57,840 --> 00:42:02,760
diving deeper into the information you have, you are to blame. And so test your team. Are

511
00:42:02,760 --> 00:42:07,200
they using the information that they're given? Are they looking out to gather additional

512
00:42:07,200 --> 00:42:13,520
information? You could go so far to use real indicators. So go to OTX, Open Threat Exchange,

513
00:42:13,840 --> 00:42:19,560
and find some IP addresses that are tied to malicious activity. It won't be hard. And

514
00:42:19,560 --> 00:42:23,360
then use those IP addresses in your environment. Now, make sure that your blue team knows

515
00:42:23,360 --> 00:42:28,600
that, hey, guys, just so you know, the indicators that we're using are real world. So now they

516
00:42:28,600 --> 00:42:36,000
know that they have an option to go and research on OSINT sources. Okay, I think this activity

517
00:42:36,000 --> 00:42:40,500
is malicious. I'm not sure. Let's go check virus total. Let's go check OTX. Let's go

518
00:42:40,500 --> 00:42:44,780
check, you know, Threat Connect, whatever. And now they can actually enrich the intel

519
00:42:44,780 --> 00:42:49,340
because you're using real indicators. So, I mean, you know, put your folks to the test

520
00:42:49,340 --> 00:42:49,540
on the internet.

521
00:42:49,540 --> 00:42:53,260
And then you can go and check on some of the more advanced capabilities that you need

522
00:42:53,260 --> 00:42:57,860
on an incident response team. Because just simply playing IP address whack-a-mole and

523
00:42:57,860 --> 00:43:10,800
blacklisting things all day, that's not effective. That's not solving problems. So

524
00:43:11,640 --> 00:43:16,860
you get these scenarios written and, you know, you've tested them, you've validated

525
00:43:16,860 --> 00:43:19,520
them. Like I said, for every red team inject, they're

526
00:43:19,520 --> 00:43:23,820
going to need to be a blue team observable. How do you measure? How do you validate? And

527
00:43:23,820 --> 00:43:28,320
then at the end of the day, how do you make this fun, right? Because if it's not fun,

528
00:43:28,920 --> 00:43:32,920
no one's going to want to do it. You're not going to want to run it. Guys, this is

529
00:43:32,920 --> 00:43:37,940
a ton of fun. So when I built this, I built it actually just for my own incident response

530
00:43:37,940 --> 00:43:44,960
team. Then B-Sides Austin reached out and said, hey, we want you guys to bring this

531
00:43:44,960 --> 00:43:48,800
platform to B-Sides Austin. I'm like, okay, yeah, sure, I'm not going to say no to that.

532
00:43:49,520 --> 00:43:53,400
We brought it to B-Sides Austin, and we had over 40 teams come and play through our

533
00:43:53,400 --> 00:44:00,400
scenarios. We've had rapid seven guys play through our scenarios. And they loved it. We've got

534
00:44:00,400 --> 00:44:05,740
overwhelming feedback about it because, one, they said, hey, this is awesome because this is

535
00:44:05,740 --> 00:44:10,160
real world. Like this is exactly what it's like, you know, being in a SOC, looking at incidents

536
00:44:10,560 --> 00:44:15,060
versus a CTF, which, don't get me wrong, CTFs are a blast, but, you know, a lot of CTFs, you

537
00:44:15,060 --> 00:44:18,820
know, puzzles, the games, the challenges, the decrypt, you know, all that good stuff, that's fun.

538
00:44:19,520 --> 00:44:24,460
Not directly applicable to what we do on a day-to-day basis. Whereas this challenge was

539
00:44:24,460 --> 00:44:29,140
meant to be exactly that. It's meant to be directly applicable. But we still want it to be

540
00:44:29,140 --> 00:44:34,780
fun, enjoyable. We gamify it a little bit. You pull down an open source CTF scoreboard, like

541
00:44:34,780 --> 00:44:40,780
this one here, this is actually Google's. This is the Google CTF framework. And you gamify it.

542
00:44:40,880 --> 00:44:45,880
You create a Jeopardy style, you know, question and answer board where you break out the

543
00:44:45,880 --> 00:44:49,500
categories and, like, you know, and I hate using this word, but you basically, you know, you

544
00:44:49,520 --> 00:44:54,380
basically create your own kill chain. You say, okay, you know, the first phase of the attack was

545
00:44:54,380 --> 00:44:59,340
the infiltration, you know, then there was the enumeration, then there was the attack, or, well,

546
00:44:59,460 --> 00:45:04,400
actually, attack came before that. But then the exfiltration, right? So you break it out into

547
00:45:04,400 --> 00:45:08,820
those categories and you make them run through the entire scenario. And you give them little

548
00:45:08,820 --> 00:45:13,060
breadcrumbs, right? Because, again, it's all about enabling them. You're not going to get any

549
00:45:13,060 --> 00:45:17,040
value throwing your team into a scenario that you know is over their head and giving them

550
00:45:17,040 --> 00:45:19,500
nothing. So when they're learning how to do it, they're going to learn how to do it. They're

551
00:45:19,520 --> 00:45:24,700
learning how to walk, you show them how to crawl, point them in the right direction, and then

552
00:45:24,700 --> 00:45:28,900
get progressively more difficult as you go on. But, like I said, the CTF scoreboard, something

553
00:45:28,900 --> 00:45:33,000
like that, it's going to gamify, it's going to make it a little more fun, but it also, as someone

554
00:45:33,000 --> 00:45:38,180
leading the training, it's going to give you a true measurement of how well is your team doing,

555
00:45:38,340 --> 00:45:41,440
how far are they getting. If they're doing it as individuals or if they're doing it as a team,

556
00:45:41,540 --> 00:45:48,880
you need to be able to measure the impact that you're making. And that is all I have.

557
00:45:49,520 --> 00:45:54,320
So I will open it up to questions. I know that there's a lot of technical information in here, and

558
00:45:54,320 --> 00:45:59,000
in the essence of time, I have to kind of skim over a lot of things that were actually six months'

559
00:45:59,160 --> 00:46:03,960
worth of development and implementation. So I'm perfectly happy to have a conversation with

560
00:46:03,960 --> 00:46:08,120
anyone that wants to know more about how to build this, how to run this. I'll even share my

561
00:46:08,120 --> 00:46:13,260
playbooks with you so you don't have to reinvent the wheel. Just let me know. But any questions?

562
00:46:14,120 --> 00:46:14,520
Yes?

563
00:46:25,200 --> 00:46:30,820
So are you talking about, like, communication channels to management of the organization, or?

564
00:46:35,600 --> 00:46:39,680
Oh. Okay. Absolutely. Absolutely. So, no, his question is good. It's like, are you including

565
00:46:39,680 --> 00:46:44,200
management in these scenarios? So if it's an incident response, like, of a large scale,

566
00:46:44,700 --> 00:46:47,880
obviously there's more than just the incident response team that's involved at this point, right?

567
00:46:48,060 --> 00:46:49,500
You're dealing with the executive team. Yeah.

568
00:46:49,520 --> 00:46:51,880
You're dealing with executives. You're dealing with legal. You're dealing with HR and so on and so

569
00:46:51,880 --> 00:46:58,120
forth. So, honestly, that's not something that I've baked into our simulations yet, but 100%

570
00:46:58,120 --> 00:47:03,060
should. Absolutely. This is something my team does on a quarterly basis, but we've only been

571
00:47:03,060 --> 00:47:07,440
doing it for three quarters. So we're still growing. We're still evolving. But you're 100%

572
00:47:07,440 --> 00:47:13,220
right that to really test, like, what would we do in this scenario, you're going to be including

573
00:47:13,220 --> 00:47:16,880
your executives. You're going to be up-channeling. So that's kind of part of the organization

574
00:47:16,880 --> 00:47:19,500
piece. You know, this isn't just about, you know, you're going to be up-channeling, you're going to be

575
00:47:19,520 --> 00:47:23,620
testing. This isn't just a skills test, guys, because I don't doubt the skills of anyone in this

576
00:47:23,620 --> 00:47:29,060
room or any of the people you probably work with. This is a test on when it hits the fan, can you

577
00:47:29,060 --> 00:47:36,600
remember the basics? Like, I don't know, talk to someone. Document things, right? Or do we all

578
00:47:36,600 --> 00:47:43,160
just fall into that silo and that rabbit hole of, oh, got to look at the logs. Oh, I think that's

579
00:47:43,160 --> 00:47:46,100
bad. Oh, I think that's bad. Oh, I think that's bad. But I haven't written down anything. I

580
00:47:46,100 --> 00:47:49,080
haven't documented anything. I haven't told, you know, other analysts what I'm looking at, what

581
00:47:49,080 --> 00:47:49,400
I'm doing.

582
00:47:49,520 --> 00:47:54,220
And I haven't up-channeled to the incident response lead. Hey, you know, I think I see

583
00:47:54,220 --> 00:48:00,020
indicators pointing this way. And I might not be on task from what the incident response lead asked

584
00:48:00,020 --> 00:48:04,940
me to do. So, yes, you've got to factor those things in to really test how effective an IR team

585
00:48:04,940 --> 00:48:09,040
is in the midst of it. Yes?

586
00:48:19,520 --> 00:48:20,600
So that's something that we're

587
00:48:20,600 --> 00:48:30,120
still measuring. Obviously the major investment here for me was time, right? To build this, yes, it was a

588
00:48:30,120 --> 00:48:34,020
sizeable investment of time. I'd say there were about three of us working on it. It took us about six

589
00:48:34,020 --> 00:48:39,000
months. We were working on it obviously on the side, right? We have daily operation to maintain.

590
00:48:39,240 --> 00:48:43,740
So, you know, for about six months we had to build this out. We had to learn a lot of hard lessons.

591
00:48:43,940 --> 00:48:49,020
You know, we had to, you know, a lot of troubleshooting. Once it was up and running, it actually became

592
00:48:49,020 --> 00:48:54,060
came very easy. Because now all we're doing is we're changing the scenario. Hey, instead of a

593
00:48:54,060 --> 00:48:59,780
DNSX fill, next time we're going to do this, we're going to do that. So writing the scenarios now is

594
00:48:59,780 --> 00:49:05,360
the upkeep. And that's actually probably the most fun and one of the easier pieces of it. So the

595
00:49:05,360 --> 00:49:10,920
upkeep element hasn't really been an issue. Like I said, the hardware, I'm using recycled power

596
00:49:10,920 --> 00:49:16,440
edges. These things are nothing special. So it was a very small investment in hardware. And like

597
00:49:16,440 --> 00:49:22,060
I said, if you have a good relationship with your vendors, you can go and you can get OVA's or

598
00:49:22,060 --> 00:49:27,780
probably even bare metal like lab units of the appliances that you're using in production and get

599
00:49:27,780 --> 00:49:32,800
those in the environment for no cost at all. You know, get a perpetual lab license. So I mean you

600
00:49:32,800 --> 00:49:37,400
can do this with a relatively small investment. The real kicker is going to be the time to build

601
00:49:37,400 --> 00:49:43,120
the environment. If you don't have the resources on your team, you know, with solid networking

602
00:49:43,120 --> 00:49:46,420
skills, solid sys admin skills and things like that, okay, you might have some resources. You

603
00:49:46,420 --> 00:49:50,600
have a tough time. I'll be honest with you. You're going to have a tough time. But I think I

604
00:49:50,600 --> 00:49:57,320
could say pretty safely that a lot of security folks came from other areas, right? Sys admin,

605
00:49:57,580 --> 00:50:01,820
network engineering, things like that. And so, again, I'm fortunate. I have a pretty, you

606
00:50:01,820 --> 00:50:06,920
know, wide skill set across my team. So I have network engineers. I have sys admins. And so it

607
00:50:06,920 --> 00:50:14,120
enabled us to build this without any outside help at all. But your mileage may vary. Yes?

608
00:50:27,320 --> 00:50:27,840
.

609
00:50:27,840 --> 00:50:28,500
.

610
00:50:28,500 --> 00:50:33,820
Okay. So that would be tough for me because I have 12,000 users. I don't have those types of

611
00:50:33,820 --> 00:50:39,160
resources for this environment, right? I cannot simulate 12,000 people. So no, obviously I'm

612
00:50:39,160 --> 00:50:45,320
definitely scaling down a lot. But I have found that as long as your traffic generation is organic

613
00:50:45,320 --> 00:50:50,420
and it's good. Put in some other things. I think in addition to

614
00:50:50,420 --> 00:50:53,840
the HTTP traffic, we also put some Python scripts that are

615
00:50:53,840 --> 00:50:59,860
doing all kinds of other weird stuff. Pings, ICMP. Remember we

616
00:50:59,860 --> 00:51:01,680
had a file share in the environment, right? So one of

617
00:51:01,680 --> 00:51:04,280
the GPOs that we put in the domain was at boot all those

618
00:51:04,280 --> 00:51:07,420
systems mount the file share. So now you've got a lot of east

619
00:51:07,420 --> 00:51:11,860
west SMB traffic. So you want to make sure that you're

620
00:51:11,860 --> 00:51:15,200
generating as much normal activity as possible. And that

621
00:51:15,200 --> 00:51:18,060
takes a little bit of ingenuity. Because if you can do that,

622
00:51:18,200 --> 00:51:21,660
you'll find that at least we found that it made plenty of

623
00:51:21,660 --> 00:51:27,160
noise. And I think we emulate about 35 or 40 user work

624
00:51:27,160 --> 00:51:30,780
stations. It makes plenty of noise. So it's effective for what

625
00:51:30,780 --> 00:51:34,160
we're doing. Because if you can effectively filter out 40 active

626
00:51:34,160 --> 00:51:38,500
users, it's the same concept to filter out 10,000. It's the

627
00:51:38,500 --> 00:51:41,280
same filtering principles, right? So we found it to be

628
00:51:41,280 --> 00:51:41,600
effective.

629
00:51:47,920 --> 00:51:50,560
What time it is.

630
00:51:56,100 --> 00:52:01,120
Oh, absolutely. Yeah. So from just a good security posture, you

631
00:52:01,120 --> 00:52:06,080
should have sensors that are detecting user behavior, you

632
00:52:06,080 --> 00:52:09,320
know, abnormalities, right? So, you know, Regina Buckley, she's

633
00:52:09,320 --> 00:52:11,840
my DBA I like to pick on. You know, she's the one that's

634
00:52:11,840 --> 00:52:13,560
doing it. So you should absolutely maybe script a

635
00:52:13,560 --> 00:52:17,180
scenario where you're setting the time. Okay, guys, it's 1 a.m.

636
00:52:17,620 --> 00:52:20,360
Regina Buckley is beating up the database. Now, Regina Buckley

637
00:52:20,360 --> 00:52:23,720
is the database administrator. So at first glance, that might

638
00:52:23,720 --> 00:52:26,780
be normal. But not at 1 a.m., right? So absolutely. Factor

639
00:52:26,780 --> 00:52:28,980
these things in. The things that should be unusual, that

640
00:52:28,980 --> 00:52:31,460
should be anomalous, that's what you're testing your team on

641
00:52:31,460 --> 00:52:35,440
detecting, right? Another one that I like to use. So we have

642
00:52:35,440 --> 00:52:39,900
a scenario that we called no worries AV caught it. I love this

643
00:52:39,900 --> 00:52:43,680
one. I love this one because here's what we did. We sent, like,

644
00:52:43,720 --> 00:52:46,840
a spam e-mail campaign to every user in the environment. And it

645
00:52:46,840 --> 00:52:50,080
had a malicious attachment, like a gray ware, you know? And we

646
00:52:50,080 --> 00:52:53,160
made sure that it was going to trigger AV. As soon as it hit

647
00:52:53,160 --> 00:52:56,100
the box, AV was going to squelch and be like, no, you're not

648
00:52:56,100 --> 00:53:00,000
going to run. What is your, in your sock today, or your IT

649
00:53:00,000 --> 00:53:02,440
security folks or whomever is watching these types of systems,

650
00:53:02,520 --> 00:53:05,680
what happens when you see something like that? No worries.

651
00:53:05,900 --> 00:53:09,880
AV caught it, right? Who cares? Moving on. So, I'm going to do a

652
00:53:09,900 --> 00:53:11,460
couple of things here. But let me tell you what the second half

653
00:53:11,460 --> 00:53:16,020
of that scenario is. If you would have taken the sample from

654
00:53:16,020 --> 00:53:19,380
AV that it caught and mitigated, if you would have taken that

655
00:53:19,380 --> 00:53:22,560
sample, it was just visual basic, you know? You know, you

656
00:53:22,560 --> 00:53:25,260
would de-obfuscate it. You would see that it had hard-coded C2

657
00:53:25,260 --> 00:53:29,960
beacons in it, hard-coded URLs for C2 beacons. Those beacons

658
00:53:29,960 --> 00:53:34,900
had been in use for multiple scenarios undetected. So what we

659
00:53:34,900 --> 00:53:39,220
staged was as if an APT had already had a foothold, but they

660
00:53:39,220 --> 00:53:39,880
were trying to increase the number of beacons that were in

661
00:53:39,900 --> 00:53:43,120
use to expand it, maybe to another branch office. But they

662
00:53:43,120 --> 00:53:46,060
got sloppy. They reused a payload that there was already a

663
00:53:46,060 --> 00:53:49,680
signature for, et cetera. These things happen. I'm testing my

664
00:53:49,680 --> 00:53:54,160
team to see how quickly do they disregard something because, oh,

665
00:53:54,200 --> 00:53:57,280
it appeared to have been mitigated. Yes, that was

666
00:53:57,280 --> 00:54:01,780
mitigated. But you failed to look deeper to see if you could

667
00:54:01,780 --> 00:54:04,540
learn something from it that might help you find something

668
00:54:04,540 --> 00:54:08,320
that wasn't mitigated, right? So, I mean, guys, the sky is the

669
00:54:08,320 --> 00:54:09,880
limit. I probably could have just talked to you about the

670
00:54:09,880 --> 00:54:11,860
different types of scenarios that you could do in this

671
00:54:11,860 --> 00:54:14,340
environment. But, I mean, I'll leave it to your imagination. But

672
00:54:14,800 --> 00:54:17,820
these are the things you have to think about. What would our team

673
00:54:17,820 --> 00:54:21,780
do in a really freaky situation like that? Like, would we be

674
00:54:21,780 --> 00:54:26,200
prepared? You won't know until you try it. You won't know until

675
00:54:26,200 --> 00:54:28,920
you test it. You won't know until you've got the muscle

676
00:54:28,920 --> 00:54:32,100
memory that it takes to do these types of things that don't happen

677
00:54:32,100 --> 00:54:37,420
every day. Unless you're RSA. Just kidding. Kidding. Yes.

678
00:54:40,820 --> 00:54:47,320
How do you handle the endpoints? How do I handle opening up?

679
00:54:47,640 --> 00:54:50,420
Users on the endpoints opening up. Okay. That's a great

680
00:54:50,420 --> 00:54:54,600
question. So, now, there are technologies that can do this

681
00:54:54,600 --> 00:54:57,000
that can interact with, like, the DOM and, like, click on

682
00:54:57,000 --> 00:55:00,420
things automatically. I'm not that cool. I don't have anything

683
00:55:00,420 --> 00:55:03,380
like that. So what we do is what we call it's a white cell

684
00:55:03,380 --> 00:55:07,200
task. So it usually falls on the red team guy. Or maybe if

685
00:55:07,200 --> 00:55:09,860
you've got a sys admin that's helping run the range during the

686
00:55:09,880 --> 00:55:12,460
environment, during the exercise, it would be a task item

687
00:55:12,460 --> 00:55:16,540
for them. Hey, we just fished Regina Buckley. Log on to that

688
00:55:16,540 --> 00:55:19,900
system. Open the e-mail. Click the link. Run the executable.

689
00:55:20,080 --> 00:55:24,180
There are ways to automate that. I just I'm not there yet. There

690
00:55:24,180 --> 00:55:26,640
are really expensive technologies and platforms that

691
00:55:26,640 --> 00:55:29,300
can do this. Some of you may be familiar. I'm very familiar. So

692
00:55:29,300 --> 00:55:32,140
my history I didn't really go into a good bit of history on

693
00:55:32,140 --> 00:55:35,360
this. But my time in the air national guard was spent largely

694
00:55:35,360 --> 00:55:39,460
in part going through simulations. Once a year I go up

695
00:55:39,880 --> 00:55:43,160
to Virginia and run through an international cyber warfare

696
00:55:43,160 --> 00:55:47,020
simulation. And the things I've seen there are what prompted me

697
00:55:47,020 --> 00:55:50,100
to build this. Because I got there and I noticed that, hey,

698
00:55:50,120 --> 00:55:52,900
what do you know? We're a bunch of people, very skilled. We

699
00:55:52,900 --> 00:55:55,660
know what we're doing. But we don't do this but once a year.

700
00:55:55,920 --> 00:56:00,760
And it shows. I want to do this more often. We need to get

701
00:56:00,760 --> 00:56:03,760
better. This needs to be more, you know, of a comfortable

702
00:56:03,760 --> 00:56:06,900
feeling for us. And so that's why I built it. But

703
00:56:06,900 --> 00:56:09,640
environments like that, they've got all those cool tools. Larry

704
00:56:09,880 --> 00:56:12,400
and I get, you know, there's all kinds of really cool user

705
00:56:12,400 --> 00:56:16,040
automation to simulate these things. But see, I'm a state

706
00:56:16,040 --> 00:56:19,140
agency. I've got to figure out how to do it on a budget. So

707
00:56:19,140 --> 00:56:24,140
chaff. Chrome extension. Python. Gets it done. Gets it

708
00:56:24,140 --> 00:56:26,100
done. Trust me. And then, you know, when you do have to do

709
00:56:26,100 --> 00:56:28,520
that one thing like open outlook to click an e-mail and open

710
00:56:28,520 --> 00:56:31,140
an attachment, that's a 30 second inject for your white cell

711
00:56:31,140 --> 00:56:36,420
team. Yeah. Oh, but, you know, that reminded me of one point

712
00:56:36,420 --> 00:56:39,860
though. There's one rule that I stick to when I'm writing a

713
00:56:39,880 --> 00:56:45,380
scenario, and I would advise you to do the same. Do not give

714
00:56:45,380 --> 00:56:49,880
in to the temptation to game the system a little bit. Everything

715
00:56:49,880 --> 00:56:53,060
that you do should be organic. You should be using the same

716
00:56:53,060 --> 00:56:56,260
tactics attackers use. Because if you do something out of the

717
00:56:56,260 --> 00:56:59,500
ordinary, you know, and, you know, you might have a technical

718
00:56:59,500 --> 00:57:01,960
limitation that means, oh, I need to side channel. I just

719
00:57:01,960 --> 00:57:04,980
need to, like, load a CD-ROM on the virtual machine of this

720
00:57:04,980 --> 00:57:08,060
workstation so I can put this binary on it undetected. That's

721
00:57:08,060 --> 00:57:09,860
unfair. It's unfair to your instant response team. It's

722
00:57:09,880 --> 00:57:11,400
unfair to your team. You didn't even give them the opportunity.

723
00:57:11,820 --> 00:57:14,760
So you need to up your skills and how are you going to deliver

724
00:57:14,760 --> 00:57:17,800
this attack. Because if you start side channeling and gaming

725
00:57:17,800 --> 00:57:20,180
the system, it's going to drive down the value of the training.

726
00:57:20,400 --> 00:57:22,140
It's going to demotivate your team. Because they're going to

727
00:57:22,140 --> 00:57:24,740
say, really? You did it that way? We were never going to find

728
00:57:24,740 --> 00:57:26,880
that. That's dumb. And they're not going to want to play next

729
00:57:26,880 --> 00:57:30,260
time. So keep it real, you know. Keep your scenarios real.

730
00:57:30,440 --> 00:57:34,960
But Ming's coming to kick me off stage. One more question before

731
00:57:34,960 --> 00:57:38,060
we have a conversation out there. Let's do it. All right.

732
00:57:38,160 --> 00:57:39,860
Thank you, sir. Hey, everyone. Bob Neumann.

733
00:57:39,880 --> 00:57:41,480
Newhart, everybody. Bob Newhart.