1
00:00:00,000 --> 00:00:01,876
Let's get started.

2
00:00:02,042 --> 00:00:04,834
I do want to give you one final warning.

3
00:00:04,876 --> 00:00:08,334
This is a talk about defending.

4
00:00:08,334 --> 00:00:10,334
I'm not breaking anything; you might have stumbled

5
00:00:10,334 --> 00:00:12,459
into the wrong room.

6
00:00:12,584 --> 00:00:17,542
I am trying to build something here, not like an android malware thing

7
00:00:17,542 --> 00:00:20,459
people will patch tomorrow.

8
00:00:20,626 --> 00:00:25,999
This may potentially change the game and help some problems dear

9
00:00:25,999 --> 00:00:31,999
to my heart when it comes to defending Blue Teams and things.

10
00:00:31,999 --> 00:00:34,667
Also, there is a lot of math involved.

11
00:00:34,667 --> 00:00:38,334
It's not like people will start a riot or throwing things,

12
00:00:38,334 --> 00:00:43,125
but if you haven't had lunch, you may experience difficulties

13
00:00:43,125 --> 00:00:48,501
because it is not good to do math on an empty stomach.

14
00:00:51,042 --> 00:00:56,667
I just want to give a quick note to where I am coming from.

15
00:00:56,999 --> 00:00:59,334
A bit ever this presentation will be a little randy

16
00:00:59,334 --> 00:01:01,876
because of the experience.

17
00:01:01,999 --> 00:01:06,999
I have been in security for 12 years, and why you have never heard of me,

18
00:01:06,999 --> 00:01:10,083
I did most of this in Brazil.

19
00:01:10,125 --> 00:01:12,999
I used to work with one of the largest consulting

20
00:01:12,999 --> 00:01:14,999
companies there.

21
00:01:16,999 --> 00:01:20,667
What I did the most was actually leading SOX,

22
00:01:20,667 --> 00:01:26,417
putting operation centers together and building log management systems

23
00:01:26,417 --> 00:01:29,876
either inside of a company, or inside one

24
00:01:29,876 --> 00:01:34,417
of the huge massive SOX which you have seen.

25
00:01:35,083 --> 00:01:41,584
This is trying to make day to day management and defending

26
00:01:41,584 --> 00:01:47,334
work, and then I got into machine learning.

27
00:01:47,334 --> 00:01:51,292
I thought it would be a very interesting subject to help

28
00:01:51,292 --> 00:01:55,083
the problem we have been facing.

29
00:01:56,083 --> 00:01:58,542
This is my first presentation.

30
00:01:58,792 --> 00:02:01,876
I do want my "shot" perks of the job.

31
00:02:07,918 --> 00:02:10,918
Don't tell my employer, but I used to drink all the time

32
00:02:10,918 --> 00:02:13,083
before my presentations!

33
00:02:18,459 --> 00:02:19,999
Excuse me.

34
00:02:25,501 --> 00:02:26,959
Anyway.

35
00:02:28,375 --> 00:02:30,999
Here is an overall structure.

36
00:02:30,999 --> 00:02:34,999
I will talk about the problem and about machine learning

37
00:02:34,999 --> 00:02:40,876
to make sure we are on the same level, not to dazzle you with math shit

38
00:02:40,876 --> 00:02:43,626
and just say it works.

39
00:02:44,584 --> 00:02:47,792
I want you to get the basic feeling, and this is not introduction

40
00:02:47,792 --> 00:02:50,334
to machine learning 101, I don't have four hours, and

41
00:02:50,334 --> 00:02:52,709
besides it isn't that cool.

42
00:02:56,083 --> 00:02:57,999
I just want to make sure we understand where

43
00:02:57,999 --> 00:03:00,834
machine learning is coming from and the special considerations

44
00:03:00,834 --> 00:03:03,584
for security learning which is important.

45
00:03:06,876 --> 00:03:10,167
Most experts or data scientists will not understand this point, and

46
00:03:10,167 --> 00:03:12,709
a bit of the case study goes through the process

47
00:03:12,709 --> 00:03:14,999
of an algorithm I designed.

48
00:03:16,083 --> 00:03:18,417
So let's get moving.

49
00:03:19,999 --> 00:03:23,250
First, you all learned of log management.

50
00:03:23,459 --> 00:03:27,584
You have a lot of logs, certifications, best practices; it is everywhere

51
00:03:27,584 --> 00:03:30,584
and you have to deal with it because everything

52
00:03:30,584 --> 00:03:33,250
is generating logs for you.

53
00:03:33,999 --> 00:03:39,459
The problem I have, having worked with a lot

54
00:03:39,459 --> 00:03:46,709
of SIM and management solutions, nobody is happy.

55
00:03:46,999 --> 00:03:51,209
Everyone is deploying this, doing everything they can

56
00:03:51,209 --> 00:03:56,667
with the state of the art stuff, and you can quote Gartner that 93%

57
00:03:56,667 --> 00:04:02,209
of the organizations couldn't catch the breeches they had.

58
00:04:05,709 --> 00:04:09,250
It can't be all that bad, can it?

59
00:04:09,501 --> 00:04:11,501
That is going on here?

60
00:04:11,626 --> 00:04:16,999
I got this great graph from SANs, and I have no idea what

61
00:04:16,999 --> 00:04:23,292
the horizontal markings are, but the first one seems big so I

62
00:04:23,292 --> 00:04:27,792
will just take their word for it.

63
00:04:27,792 --> 00:04:31,999
But what really caught my eye here, what is the biggest problem

64
00:04:31,999 --> 00:04:34,542
in SIM deployments?

65
00:04:34,999 --> 00:04:39,834
It seems you can't identify key events from normal behavior.

66
00:04:42,417 --> 00:04:47,834
Maybe it is a pronunciation issue, but security information

67
00:04:47,834 --> 00:04:51,584
and event management solutions.

68
00:04:51,999 --> 00:04:55,209
There are rules doing fancy shit, putting graphics on and trying

69
00:04:55,209 --> 00:04:59,751
to pretend you understand that the logs are talking about.

70
00:05:00,417 --> 00:05:03,125
But the point here, my biggest problem

71
00:05:03,125 --> 00:05:08,083
is that I cannot tell key events from normal behavior.

72
00:05:08,083 --> 00:05:11,209
It is like you buy a car and the main complaint

73
00:05:11,209 --> 00:05:14,459
is you can't get from point A to point B.

74
00:05:14,459 --> 00:05:19,459
The car doesn't start, I can't find the accelerator.

75
00:05:19,459 --> 00:05:23,584
That is why we buy these things, and we're not able to make them work.

76
00:05:24,292 --> 00:05:29,999
One reason I think we can't make them work, how do you use these things?

77
00:05:29,999 --> 00:05:33,083
You are coming up with magic numbers where I want

78
00:05:33,083 --> 00:05:36,999
to be a learned, if somebody hits me like five times

79
00:05:36,999 --> 00:05:41,999
or maybe I am home free if I only hit you four times.

80
00:05:42,501 --> 00:05:45,334
I am trying to make up arbitrary rules where

81
00:05:45,334 --> 00:05:48,999
if something happens and then other things happen, that

82
00:05:48,999 --> 00:05:52,959
is an alert that should be brought up to me.

83
00:05:53,834 --> 00:06:00,167
In a way, all the permeations of the rules people put together,

84
00:06:00,167 --> 00:06:05,999
and when you are on an engage the, you are just iterating

85
00:06:05,999 --> 00:06:12,999
until the customer is satisfied or until you run out of money, which

86
00:06:12,999 --> 00:06:15,584
is more likely.

87
00:06:17,584 --> 00:06:21,542
It sounds like a scheme to me, like it was built

88
00:06:21,542 --> 00:06:26,167
to make sure we generate the most consulting out of that,

89
00:06:26,167 --> 00:06:32,417
and I don't claim that SIM solutions are the largest offenders.

90
00:06:32,959 --> 00:06:35,999
If you have worked with identity management, these guys,

91
00:06:35,999 --> 00:06:38,167
they take the crown.

92
00:06:41,709 --> 00:06:44,459
The point I want to make, one of these vendors has

93
00:06:44,459 --> 00:06:46,459
a huge curriculum.

94
00:06:46,834 --> 00:06:54,501
You get one month of training to be an expert with the solution; four weeks,

95
00:06:54,501 --> 00:07:00,999
20 days, and not one single hour of the training explains to you

96
00:07:00,999 --> 00:07:04,709
the content it comes with.

97
00:07:04,709 --> 00:07:10,417
For weeks it will teach you how to build new rules and create new

98
00:07:10,417 --> 00:07:16,209
dashboards and new things, but how do I use these things

99
00:07:16,209 --> 00:07:20,999
and apply them to what I need to do?

100
00:07:20,999 --> 00:07:22,709
So it seems like things are harder than

101
00:07:22,709 --> 00:07:24,709
they should be.

102
00:07:25,375 --> 00:07:29,999
I do want to touch on behavior rules, as I will talk about them a bit

103
00:07:29,999 --> 00:07:34,792
in the future as they help a bit with the configuration.

104
00:07:34,792 --> 00:07:41,626
So I'm not ready to bash the tool, but some people are very, very good

105
00:07:41,626 --> 00:07:47,250
at log management, but there are very few.

106
00:07:47,999 --> 00:07:51,083
I have worked with teams which are very good.

107
00:07:51,083 --> 00:07:54,999
You give them six months and they would make this thing sing.

108
00:07:55,083 --> 00:07:58,083
But the problem I have, this does not scale;

109
00:07:58,083 --> 00:08:03,542
we do not have enough people who are good at this, at understanding

110
00:08:03,542 --> 00:08:05,501
all of that.

111
00:08:05,626 --> 00:08:07,876
Those are fantastic tools if you want to build something,

112
00:08:07,876 --> 00:08:09,999
but they are not ready.

113
00:08:10,709 --> 00:08:14,292
Most people buy them thinking they will be ready and

114
00:08:14,292 --> 00:08:19,834
will solve a problem, but it's not there, definitely not there.

115
00:08:20,083 --> 00:08:23,999
What really got me scared was Big Data.

116
00:08:27,459 --> 00:08:32,876
And the point is, we have this smallish database,

117
00:08:32,876 --> 00:08:39,083
so thinking about SIM solutions, this is nothing more than

118
00:08:39,083 --> 00:08:47,334
a highly vertical business data warehousing solution from the '90s.

119
00:08:49,501 --> 00:08:54,959
But now they are catching up to have better data by day,

120
00:08:54,959 --> 00:08:58,083
but who will analyze it?

121
00:09:02,959 --> 00:09:05,751
Do we have to create all the rules ourselves?

122
00:09:06,459 --> 00:09:09,542
Usually when you handle this kind of data, you need to work

123
00:09:09,542 --> 00:09:11,959
with statistical analysis.

124
00:09:12,125 --> 00:09:17,999
It is really is whole different discipline from your traditional information

125
00:09:17,999 --> 00:09:22,417
security analysis, and I have met people who are very

126
00:09:22,417 --> 00:09:27,501
good at both, but it makes the bowl even smaller.

127
00:09:29,918 --> 00:09:35,542
Given all this, the only solution at this stage I can tell you about,

128
00:09:35,542 --> 00:09:37,959
we need an army.

129
00:09:39,501 --> 00:09:42,250
Let the rule bots talk to themselves.

130
00:09:42,501 --> 00:09:46,292
If there is a machine that is generating the data,

131
00:09:46,292 --> 00:09:49,999
why shouldn't the machines read it?

132
00:09:49,999 --> 00:09:52,626
We can't keep up; there's just no way.

133
00:09:53,542 --> 00:09:57,501
That is when we talk about machine learning.

134
00:09:58,125 --> 00:10:01,501
I want to get the basics right so that we can

135
00:10:01,501 --> 00:10:05,959
understand a lot of what we can and cannot do.

136
00:10:05,959 --> 00:10:10,042
We really haven't got to this stage yet, but I have met people who are intent

137
00:10:10,042 --> 00:10:14,083
on building this, yes, put the machine learning thing together

138
00:10:14,083 --> 00:10:19,375
and it will be awesome, but I'm just not exactly sure about that.

139
00:10:19,876 --> 00:10:21,876
When you think about machine learning,

140
00:10:21,876 --> 00:10:25,792
you really aren't writing the code for the decision making;

141
00:10:25,792 --> 00:10:29,751
you are writing a code that will identify some data for you,

142
00:10:29,751 --> 00:10:34,083
and then start to make inferences based on the data.

143
00:10:34,626 --> 00:10:38,292
So pretty much like going to the computer and I say: Computer,

144
00:10:38,292 --> 00:10:40,375
this is a chair.

145
00:10:44,250 --> 00:10:49,042
The computer will look at the chair, okay, it has metal and some stuffing,

146
00:10:49,042 --> 00:10:51,501
got it, it's a chair.

147
00:10:51,501 --> 00:10:53,459
But this is another chair.

148
00:10:53,459 --> 00:10:59,375
Oh, that is a bit different, but I think I have the idea.

149
00:10:59,459 --> 00:11:01,999
As you show the computer enough chairs,

150
00:11:01,999 --> 00:11:06,999
it eventually begins to generalize that a chair is.

151
00:11:07,459 --> 00:11:13,417
The secret is all in how you tell the computer that a chair is,

152
00:11:13,417 --> 00:11:19,125
the science slash art around this, how you build the features

153
00:11:19,125 --> 00:11:23,125
for the algorithms to be able to take this

154
00:11:23,125 --> 00:11:28,709
in and make decisions based on what they saw, if something

155
00:11:28,709 --> 00:11:31,125
is this or that.

156
00:11:31,375 --> 00:11:34,250
I will give many more examples.

157
00:11:35,959 --> 00:11:40,999
This is everywhere, absolutely everywhere.

158
00:11:41,167 --> 00:11:43,292
I think it is a shame we don't use information

159
00:11:43,292 --> 00:11:45,083
security more.

160
00:11:45,626 --> 00:11:48,542
Everyone is selling you shit out of this.

161
00:11:48,542 --> 00:11:53,083
You go to Amazon, one of the big examples here.

162
00:11:53,667 --> 00:11:57,501
They use this technique called "collaborative filtering" which pretty

163
00:11:57,501 --> 00:12:00,250
much says okay, if you buy these things with a lot

164
00:12:00,250 --> 00:12:03,292
of other people just like you who bought these things,

165
00:12:03,292 --> 00:12:07,250
but if they bought something else, I will probably suggest that you are

166
00:12:07,250 --> 00:12:09,083
pretty similar.

167
00:12:09,125 --> 00:12:11,999
So it will make decisions.

168
00:12:12,083 --> 00:12:16,709
If you look at the math, there is a billion matrix column and

169
00:12:16,709 --> 00:12:20,250
they multiplicate the hell out of that shit,

170
00:12:20,250 --> 00:12:25,501
and you should read this book; it is quite awesome.

171
00:12:26,083 --> 00:12:29,959
This is something that is very well understood

172
00:12:29,959 --> 00:12:33,999
by the marketing and sales community.

173
00:12:34,125 --> 00:12:38,667
When you talk about trading, and this is a very good picture, but it

174
00:12:38,667 --> 00:12:40,999
is a cautionary tale.

175
00:12:41,083 --> 00:12:43,834
This was a flash crash that happened

176
00:12:43,834 --> 00:12:49,667
a couple years ago where most of the high frequency trading run, it

177
00:12:49,667 --> 00:12:54,083
is just algorithms talking to each other and buying

178
00:12:54,083 --> 00:12:58,584
and selling based on what is happening.

179
00:12:58,584 --> 00:13:00,375
And then some of them got confused

180
00:13:00,375 --> 00:13:03,083
and almost crashed the whole community,

181
00:13:03,083 --> 00:13:08,083
but luckily somebody was watching and said let's back up a little bit,

182
00:13:08,083 --> 00:13:12,125
you guys play nice now, let's separate this.

183
00:13:12,250 --> 00:13:19,709
But it is everywhere, and it is a very sensitive place.

184
00:13:20,667 --> 00:13:24,876
On the other hand you have people doing very cool stuff with image

185
00:13:24,876 --> 00:13:27,083
and voice recognition.

186
00:13:27,667 --> 00:13:41,999
(Applause) Should I continue?

187
00:13:54,876 --> 00:13:58,501
Evidently he is the only one who does not know

188
00:13:58,501 --> 00:14:02,334
why we are here or what we are called.

189
00:14:04,709 --> 00:14:07,834
Is this your first time speaking at DEF CON?

190
00:14:07,834 --> 00:14:09,209
ALEXANDRE PINTO: Yes, sir.

191
00:14:09,209 --> 00:14:10,209
Welcome.

192
00:14:10,209 --> 00:14:16,918
We would like to shove some alcohol

193
00:14:16,918 --> 00:14:21,834
down your throat.

194
00:14:21,999 --> 00:14:24,083
Raise your hand if you are a first timer.

195
00:14:24,083 --> 00:14:25,125
You in the blue shirt.

196
00:14:25,334 --> 00:14:27,501
You were faster than everybody else.

197
00:14:27,501 --> 00:14:30,709
Sunday morning at DEF CON; gotta love it!

198
00:14:35,542 --> 00:14:37,999
What is the shit on the screen?

199
00:14:40,626 --> 00:14:43,999
My God, that is really cool.

200
00:14:44,459 --> 00:14:47,459
Data visualization of some kind.

201
00:14:47,459 --> 00:14:49,876
To all first timers at DEF CON, cheers!

202
00:14:59,125 --> 00:15:02,250
(Applause)    We will see you soon.

203
00:15:02,250 --> 00:15:04,083
ALEXANDRE PINTO: Now I am happy.

204
00:15:04,083 --> 00:15:05,250
I guess we can continue.

205
00:15:07,250 --> 00:15:12,999
Anyway, the guys from Google set up like 16,000 cluster machine,

206
00:15:12,999 --> 00:15:16,999
telling the machine to find me pictures of cats

207
00:15:16,999 --> 00:15:19,459
on the Internet.

208
00:15:19,626 --> 00:15:22,083
Yes, they really know their audience.

209
00:15:35,834 --> 00:15:40,125
(Applause) So they use a technique called "deep learning"

210
00:15:40,125 --> 00:15:46,999
to actually create an arbitrary, deep and narrow network out of the blue.

211
00:15:47,083 --> 00:15:50,667
I don't claim to understand it, but it is mighty fancy.

212
00:15:51,083 --> 00:15:55,542
This is a visualization of that a computer thinks is cat is.

213
00:15:55,542 --> 00:15:56,999
That is awesome.

214
00:15:56,999 --> 00:15:57,999
I can see a cat there.

215
00:15:59,375 --> 00:16:03,292
This is the future, man; the computer is watching cat videos now.

216
00:16:03,292 --> 00:16:10,999
(Laughter and applause) But on a more serious note, what

217
00:16:10,999 --> 00:16:15,501
is this being used for?

218
00:16:15,501 --> 00:16:16,501
Security.

219
00:16:17,083 --> 00:16:21,584
Fraud detection systems, the most basic technique they use

220
00:16:21,584 --> 00:16:26,125
is called clustering which I will talk more about, but trying

221
00:16:26,125 --> 00:16:29,667
to find deviations on a pattern.

222
00:16:29,709 --> 00:16:33,918
If they create a baseline and identify where you are

223
00:16:33,918 --> 00:16:39,834
with a group of customers, if you are not "you" in the dimensions,

224
00:16:39,834 --> 00:16:42,834
they can look at that.

225
00:16:42,834 --> 00:16:44,709
If you used to do this kind of shit with a credit card

226
00:16:44,709 --> 00:16:48,999
and you do something very different, that is probably a flag.

227
00:16:50,999 --> 00:16:53,999
And behavior monitoring, no matter what people say, it

228
00:16:53,999 --> 00:16:56,334
is not machine learning.

229
00:17:02,417 --> 00:17:10,334
The HP12 can do rolling averages, very easy to do.

230
00:17:11,999 --> 00:17:17,999
Statistical analysis is helpful, and it is the first step in understanding,

231
00:17:17,999 --> 00:17:23,417
but it isn't machine learning in any way or shape or form.

232
00:17:23,417 --> 00:17:27,709
Finally, spam filters, the heros of some machine learning.

233
00:17:27,999 --> 00:17:30,584
You remember the Beijing filters?

234
00:17:31,626 --> 00:17:35,459
That is actually the algorithm they use.

235
00:17:35,959 --> 00:17:40,292
The point I am trying to make, how many talks did you see this year

236
00:17:40,292 --> 00:17:43,584
or the year before about spam?

237
00:17:44,375 --> 00:17:47,292
Nobody is researching that anymore.

238
00:17:47,751 --> 00:17:54,209
I know I have my gmail account I opened in 2004, a long time ago,

239
00:17:54,209 --> 00:18:02,876
and I am sure every single spammer on the Internet has hold of my account.

240
00:18:03,667 --> 00:18:05,542
I don't get spam.

241
00:18:05,542 --> 00:18:07,709
Do you get spam on gmail?

242
00:18:12,999 --> 00:18:18,209
I got phishing e mails which are crafted to look like a normal e mail

243
00:18:18,209 --> 00:18:20,999
to get passed all this.

244
00:18:21,083 --> 00:18:24,999
And this is a problem we really don't look at anymore.

245
00:18:26,584 --> 00:18:31,751
10 or 15 years ago I did research for a talk, a paper

246
00:18:31,751 --> 00:18:36,999
from 1998 talked about using Beijing filters for spam,

247
00:18:36,999 --> 00:18:42,792
and it seems like the problem is solved in a way.

248
00:18:45,167 --> 00:18:47,834
That is a message I want to leave there.

249
00:18:48,876 --> 00:18:52,999
If we start doing this work now, we will get very good at picking

250
00:18:52,999 --> 00:18:56,083
up this stuff in five or ten years.

251
00:18:58,167 --> 00:19:01,626
Once you have enough data, and I think Google would be

252
00:19:01,626 --> 00:19:06,375
the one who has the most data; we can probably agree on that.

253
00:19:06,375 --> 00:19:08,250
You can get pretty good at this.

254
00:19:08,292 --> 00:19:14,792
Anyway, now we start getting a little bit more technical.

255
00:19:14,792 --> 00:19:19,334
The idea, there are two big kinds of machine learning: Supervised

256
00:19:19,334 --> 00:19:23,999
helping where you tell the computer that these things are,

257
00:19:23,999 --> 00:19:27,501
and there are two major groups.

258
00:19:31,209 --> 00:19:34,999
Classification is like this is a chair, this is a table, budge of chairs,

259
00:19:34,999 --> 00:19:36,959
bunch of tables.

260
00:19:36,959 --> 00:19:41,292
Okay, here is an object, what is it, a chair or a table?

261
00:19:41,709 --> 00:19:45,999
You pretty much give it data to train on, and you give it labels, this

262
00:19:45,999 --> 00:19:48,999
is a chair and this is a table.

263
00:19:48,999 --> 00:19:51,334
Then for it to predict, you present new data.

264
00:19:57,459 --> 00:20:01,792
When we think about regression, we aren't actually looking

265
00:20:01,792 --> 00:20:05,584
for a binary answer, but we are looking for how much

266
00:20:05,584 --> 00:20:09,792
of this is a table and how much is a chair?

267
00:20:09,792 --> 00:20:15,083
I know it doesn't Sunday like it makes sense, but if you look

268
00:20:15,083 --> 00:20:20,584
at a stool or a piano bench that looks like a table,

269
00:20:20,584 --> 00:20:27,999
the computer would get confused, it is a dot four chair, and I know that

270
00:20:27,999 --> 00:20:31,167
is a general example.

271
00:20:31,792 --> 00:20:36,918
But trying to generalize where it lies, and based on that and this

272
00:20:36,918 --> 00:20:41,167
is definitely a chair or a table, we can use it for that,

273
00:20:41,167 --> 00:20:45,083
or we can use it to make the humans better informed

274
00:20:45,083 --> 00:20:48,959
on what decisions they should take.

275
00:20:49,250 --> 00:20:52,334
Then there is unsupervised learning which is okay,

276
00:20:52,334 --> 00:20:57,999
I don't know anything about this, look at this and tell me what you can find.

277
00:20:57,999 --> 00:21:01,334
There are also to big groups.

278
00:21:07,918 --> 00:21:12,626
There is clustering which I mentioned before with regard to fraud solutions,

279
00:21:12,626 --> 00:21:14,999
by far the most abused machine learning

280
00:21:14,999 --> 00:21:19,292
technique for anything ever, and it is not always applicable and has

281
00:21:19,292 --> 00:21:21,459
a very fatal flaw.

282
00:21:21,709 --> 00:21:24,751
You have to actually tell the computer how many clusters you

283
00:21:24,751 --> 00:21:27,999
are looking for so it can be able to guess which would be

284
00:21:27,999 --> 00:21:31,542
the separation of the data you are sending.

285
00:21:31,999 --> 00:21:34,209
Of course, there are techniques to discover that,

286
00:21:34,209 --> 00:21:36,709
but then it starts getting very complicated and most

287
00:21:36,709 --> 00:21:41,083
of the people don't actually do all the legwork and it can get a bit fuzzy.

288
00:21:41,417 --> 00:21:43,834
Finally, the composition.

289
00:21:43,999 --> 00:21:47,834
In a way, it is a tool for you to design better algorithms,

290
00:21:47,834 --> 00:21:49,751
in a nutshell.

291
00:21:51,999 --> 00:21:54,918
So I need to tell the computer what a share

292
00:21:54,918 --> 00:21:58,792
is and I have all the different things to tell the computer,

293
00:21:58,792 --> 00:22:01,626
height and weight and what it is made of, so

294
00:22:01,626 --> 00:22:04,918
like a hundred possible variables.

295
00:22:05,125 --> 00:22:08,999
And I am trying to find which ones matter the most

296
00:22:08,999 --> 00:22:13,542
in deciding if something is a chair or a table.

297
00:22:13,542 --> 00:22:16,999
Maybe it can be argued for chair and table, it can be the height.

298
00:22:18,292 --> 00:22:21,083
Probably if I ran the chair table model

299
00:22:21,083 --> 00:22:25,999
on a composition thing to tell me the principle components analysis

300
00:22:25,999 --> 00:22:31,501
of what is really making a difference, it would probably say height should be

301
00:22:31,501 --> 00:22:34,083
chosen for the model because it makes

302
00:22:34,083 --> 00:22:36,834
an awesome difference.

303
00:22:36,999 --> 00:22:41,999
By the way, if you like this tutorial, it is awesome.

304
00:22:44,999 --> 00:22:52,999
So it is not magic; you still have to train the computer and give it data.

305
00:22:55,125 --> 00:22:58,292
But if you design your algorithm and your model well,

306
00:22:58,292 --> 00:23:01,999
with more data it will generally get better.

307
00:23:03,501 --> 00:23:07,751
So there is a lot of mathematical proofs that you can

308
00:23:07,751 --> 00:23:14,083
show that the more data you have and this is the drawing on the left.

309
00:23:14,083 --> 00:23:18,667
Your E in which is the arrow inside the training model,

310
00:23:18,667 --> 00:23:23,751
the data you are using to train, and both of the E which

311
00:23:23,751 --> 00:23:29,876
is the stuff you have never seen before in your life, they converge

312
00:23:29,876 --> 00:23:32,751
to an expected error.

313
00:23:32,751 --> 00:23:36,999
Don't let anybody tell you there is a hundred percent working machine

314
00:23:36,999 --> 00:23:39,999
learning model; no such thing.

315
00:23:39,999 --> 00:23:42,375
It will make mistakes, as do we.

316
00:23:43,792 --> 00:23:47,667
It is all part of the way we make sense of reality,

317
00:23:47,667 --> 00:23:51,501
trying to emulate it into a computer.

318
00:23:51,501 --> 00:23:53,876
There will always be errors and you can all hope your model

319
00:23:53,876 --> 00:23:56,792
will be as error free as possible.

320
00:24:02,959 --> 00:24:07,876
The point is, you must be careful about the data you take in.

321
00:24:08,083 --> 00:24:14,167
For example, the SANS D shield data firewall you get

322
00:24:14,167 --> 00:24:22,751
from them, they wouldn't tell me but I assume it is pretty U.S.

323
00:24:24,334 --> 00:24:27,209
centric, putting a bias on the results.

324
00:24:27,999 --> 00:24:33,417
But you always have to be aware of that and to look out for adversaries,

325
00:24:33,417 --> 00:24:37,709
which is a point I want to make in my talk.

326
00:24:37,999 --> 00:24:40,501
Everybody who is into building these models, doing

327
00:24:40,501 --> 00:24:43,959
the machine learning, they don't understand that people

328
00:24:43,959 --> 00:24:48,542
would like to really fuck with them, which we do understand.

329
00:24:51,792 --> 00:24:55,751
And one of the first things that came to my mind, even

330
00:24:55,751 --> 00:24:59,834
if I build something here, how can I potentially exploit

331
00:24:59,834 --> 00:25:04,626
or send random noise to render the computer useless?

332
00:25:04,999 --> 00:25:08,417
And I will take a bit about that as a weakness

333
00:25:08,417 --> 00:25:14,501
of this specific model that I built, but it is there; people will try

334
00:25:14,501 --> 00:25:16,999
to mess with you.

335
00:25:19,709 --> 00:25:23,417
If this becomes a valid method of actually defending

336
00:25:23,417 --> 00:25:28,292
and helping defense, there will be a lot of talks in this conference

337
00:25:28,292 --> 00:25:32,876
with some really crazy math guys on how they will defeat this,

338
00:25:32,876 --> 00:25:35,792
crypto wars all over again.

339
00:25:38,709 --> 00:25:42,792
But the point is, remember spam engines?

340
00:25:43,250 --> 00:25:46,792
When we started getting this Beijing thing going on,

341
00:25:46,792 --> 00:25:52,292
the spam just started posting whole sonnets of Shakespeare at the end.

342
00:25:52,542 --> 00:25:56,083
And then the model will think hmm, I haven't seen many spams

343
00:25:56,083 --> 00:26:01,167
with the word "thou" so it is probably a legitimate e mail.

344
00:26:01,959 --> 00:26:04,959
But then people refined it and evolved the model to get

345
00:26:04,959 --> 00:26:06,999
to the level where we.

346
00:26:11,334 --> 00:26:15,626
But people always find a way to break things, and especially

347
00:26:15,626 --> 00:26:20,459
on these kinds of applications, this is something we need to be very,

348
00:26:20,459 --> 00:26:22,876
very careful about.

349
00:26:22,876 --> 00:26:23,999
Enough introductions.

350
00:26:23,999 --> 00:26:26,751
Let's get to it and to chewing on the logs.

351
00:26:29,459 --> 00:26:32,167
Most of the talk is about the feature engineering,

352
00:26:32,167 --> 00:26:34,999
the really important part here.

353
00:26:35,876 --> 00:26:38,584
On that slide about the kinds of machine learning are names

354
00:26:38,584 --> 00:26:41,999
of algorithms, and there are a whole bunch of them.

355
00:26:41,999 --> 00:26:44,083
You just try them all, you create a process

356
00:26:44,083 --> 00:26:48,334
to see which works best because there is no one size fits all,

357
00:26:48,334 --> 00:26:52,709
but the problem is what data you are feeding it.

358
00:26:54,083 --> 00:26:58,792
And everyone will tell you that is the real hard work about this.

359
00:26:58,792 --> 00:27:02,667
So I was telling you about this shield.

360
00:27:05,792 --> 00:27:11,334
I have been collecting their bulk logs, and on the website you see

361
00:27:11,334 --> 00:27:17,709
the top ten places where bad people come from on the Internet.

362
00:27:18,083 --> 00:27:22,167
But if you ask nicely and they were very nice about it, I thank

363
00:27:22,167 --> 00:27:26,999
the SANS institute for their help, and you can get the bulk data

364
00:27:26,999 --> 00:27:30,918
and I just started mining it in January.

365
00:27:34,125 --> 00:27:39,083
This is very basic stuff, which is one of the points I am trying to make.

366
00:27:39,083 --> 00:27:44,167
It is firewall data that is blocked and summarized.

367
00:27:44,375 --> 00:27:48,083
I get to know kind of how many blocks we got for each,

368
00:27:48,083 --> 00:27:53,959
which I use as a way to decide what to select for the model.

369
00:27:53,999 --> 00:27:56,792
But that is all there is, which you know.

370
00:27:56,792 --> 00:27:59,999
For the group of people submitting the log files, these are

371
00:27:59,999 --> 00:28:04,459
the guys who were potentially attacking them; right?

372
00:28:04,751 --> 00:28:10,918
You always start with a port scan to see which machines were up.

373
00:28:10,918 --> 00:28:14,375
If people were hitting them on the firewall

374
00:28:14,375 --> 00:28:20,083
on machines that didn't have a port open, maybe if they hit

375
00:28:20,083 --> 00:28:26,709
the write board, it would be something worth looking at.

376
00:28:28,250 --> 00:28:31,626
Just a summary here of the amount of data.

377
00:28:33,250 --> 00:28:39,667
Per day I got a million observations, which is pretty much this IP address

378
00:28:39,667 --> 00:28:42,209
attacked this port.

379
00:28:42,459 --> 00:28:46,792
From that, I would select the behaviors I was looking for.

380
00:28:47,999 --> 00:28:52,083
When you decompose that and I didn't get the decomposed logs,

381
00:28:52,083 --> 00:28:55,667
only a number, you would get roughly 30 million log

382
00:28:55,667 --> 00:28:57,709
events per day.

383
00:29:01,417 --> 00:29:05,876
One thing to point out here, this is not Big Data.

384
00:29:06,501 --> 00:29:10,876
I am running it on my laptop, and it's not even a good one.

385
00:29:11,751 --> 00:29:16,584
So don't let anybody tell you my God, it's the Cloud.

386
00:29:20,375 --> 00:29:24,083
I am really trying to bring to a tangible level, which

387
00:29:24,083 --> 00:29:27,167
is one objective of this talk.

388
00:29:27,167 --> 00:29:32,167
I am getting some data, doing mining with it on my own laptop.

389
00:29:33,167 --> 00:29:40,542
So one of the intuitions here of this model, it is the proximity.

390
00:29:40,626 --> 00:29:46,709
Anyone who has done this, they have an instinct like okay,

391
00:29:46,709 --> 00:29:53,167
I have seen this IP address before or I have seen people coming

392
00:29:53,167 --> 00:29:59,375
from "that side of the woods" before, and it is interesting

393
00:29:59,375 --> 00:30:07,125
because I have seen a lot of things being caught out of mistakes.

394
00:30:07,250 --> 00:30:12,584
People would look at IP addresses and say oh, I remember this guy,

395
00:30:12,584 --> 00:30:17,501
let's look at this, but it was an IP address that was similar,

396
00:30:17,501 --> 00:30:23,042
had never appeared, and there is actually something there.

397
00:30:25,999 --> 00:30:29,999
I started doing research, and there is some statistical analysis facts

398
00:30:29,999 --> 00:30:31,626
about this.

399
00:30:35,417 --> 00:30:40,751
I point out the spam house cyberbunker thing, I don't care about that,

400
00:30:40,751 --> 00:30:44,999
but I care that spam house said to forget about the ISP,

401
00:30:44,999 --> 00:30:51,167
anything that comes from there, it is spam, just block the whole thing.

402
00:30:53,584 --> 00:30:59,959
That is an interesting conclusion, and they really fight to be level headed.

403
00:30:59,959 --> 00:31:03,999
(Comment from audience member)

404
00:31:03,999 --> 00:31:11,167
ALEXANDRE PINTO: I don't know anybody from there; I

405
00:31:11,167 --> 00:31:14,999
will take your word.

406
00:31:17,876 --> 00:31:20,834
Then we got the Google malware report that said

407
00:31:20,834 --> 00:31:25,083
there are more places that are likely to have malware.

408
00:31:29,999 --> 00:31:32,959
And then a researcher did a statistical analysis

409
00:31:32,959 --> 00:31:36,999
for the past seven years of logs from the Brazilian research networks

410
00:31:36,999 --> 00:31:39,792
that connects the universities.

411
00:31:40,083 --> 00:31:44,999
He was proving that shit statistically, and it's not random at all.

412
00:31:45,292 --> 00:31:47,918
There is some information you can potentially

413
00:31:47,918 --> 00:31:49,876
extract from that.

414
00:31:51,292 --> 00:31:55,083
To create my features, I started grouping those logs

415
00:31:55,083 --> 00:31:58,876
by arbitrary net blogs, so let's pick some things

416
00:31:58,876 --> 00:32:03,083
and choose and see if it comes out all right.

417
00:32:04,626 --> 00:32:07,167
And by ASN, which is what really shines,

418
00:32:07,167 --> 00:32:12,083
the synchronous or automatic system where it is coming from on the Internet,

419
00:32:12,083 --> 00:32:17,125
and for that I use this awesome service for pulling that data.

420
00:32:24,709 --> 00:32:31,542
Here is a visualization of that, and I know it seems like a lot to take in,

421
00:32:31,542 --> 00:32:37,167
which is understandable because that is the Internet, but this

422
00:32:37,167 --> 00:32:43,083
is a projection that tries to maximize the proximity as you draw

423
00:32:43,083 --> 00:32:45,501
the IP address.

424
00:32:45,876 --> 00:32:50,292
This is called a Hubert Curve, and this is the transformation,

425
00:32:50,292 --> 00:32:54,375
thinking of the IP address like in a straight line

426
00:32:54,375 --> 00:32:58,667
from zero zero zero to 2555 blah blah blah, twisting

427
00:32:58,667 --> 00:33:03,709
to make sure the IP addresses are as close as possible to that

428
00:33:03,709 --> 00:33:06,083
of their neighbor.

429
00:33:06,250 --> 00:33:12,626
And this is accumulated data up to July 20th of people trying

430
00:33:12,626 --> 00:33:20,999
to attack the D shield group on 4/22, and I don't think that is random.

431
00:33:24,999 --> 00:33:28,250
Some places on this map, by nature they are dark.

432
00:33:28,999 --> 00:33:34,125
The DOD is really not doing anything there, but they have like eight blocks.

433
00:33:35,375 --> 00:33:39,959
IBM has 8 dot something, and they don't use it.

434
00:33:40,417 --> 00:33:45,709
You can see there is some density going on, and

435
00:33:45,709 --> 00:33:52,667
if you are in the DEF CON network, if you look at this cluster,

436
00:33:52,667 --> 00:33:57,834
you can start to jump to conclusions.

437
00:33:57,876 --> 00:34:03,375
Oh, my God, look at this guy, he's obviously up to something.

438
00:34:03,834 --> 00:34:08,792
Then you start doing more mining, and oh my God, they are definitely

439
00:34:08,792 --> 00:34:10,999
up to something.

440
00:34:12,584 --> 00:34:14,959
This is data, okay?

441
00:34:14,959 --> 00:34:19,292
But the point I am making, it is very easy to start jumping

442
00:34:19,292 --> 00:34:24,999
to conclusions, but we need to see this to fruition at the end.

443
00:34:25,125 --> 00:34:29,999
If you look at it, you see it is us, pretty much beating the shit

444
00:34:29,999 --> 00:34:33,209
out of each other all the time.

445
00:34:33,250 --> 00:34:35,999
So I make the point that if you are just blocking

446
00:34:35,999 --> 00:34:41,167
for I hate this country, I won't let it in, it won't mean anything; you just need

447
00:34:41,167 --> 00:34:43,959
to be careful about that.

448
00:34:45,792 --> 00:34:47,918
So let's get moving.

449
00:34:47,918 --> 00:34:51,999
So we get this, there is proximity, but I want to be able

450
00:34:51,999 --> 00:34:55,083
to decay it because the neighborhoods

451
00:34:55,083 --> 00:34:57,125
may renovate.

452
00:34:59,918 --> 00:35:03,417
If people change ISPs, I want to accompany that; otherwise, it

453
00:35:03,417 --> 00:35:06,709
is just a massive black list that will eventually block

454
00:35:06,709 --> 00:35:08,999
off the whole Internet.

455
00:35:09,167 --> 00:35:12,375
So as time passes I want to forget these things happen,

456
00:35:12,375 --> 00:35:16,083
and there is some exponential decay here.

457
00:35:16,918 --> 00:35:20,918
You choose a metric, and after three to four months,

458
00:35:20,918 --> 00:35:25,501
it really doesn't happen anymore, you forget that people were

459
00:35:25,501 --> 00:35:27,626
attacking you.

460
00:35:28,918 --> 00:35:31,751
And that is just like an analyst, like if you go

461
00:35:31,751 --> 00:35:35,751
to a party and something happened a month ago, you aren't as likely

462
00:35:35,751 --> 00:35:40,083
to remember it as you were if it happened just yesterday.

463
00:35:40,751 --> 00:35:46,209
So given the two intuitions, we have to start calculating this.

464
00:35:46,501 --> 00:35:50,083
The point is, we crate rankings which will be

465
00:35:50,083 --> 00:35:56,083
the features to the model by IP address, by some net blocks you choose and

466
00:35:56,083 --> 00:35:58,083
by the SANS.

467
00:35:58,167 --> 00:36:00,918
If you are missing data, I take a shortcut,

468
00:36:00,918 --> 00:36:04,667
if I can't discover your ASN, you are a bad man and I

469
00:36:04,667 --> 00:36:07,584
will give you a high score.

470
00:36:07,999 --> 00:36:12,167
It is a shortcut but it makes sure we don't ignore these

471
00:36:12,167 --> 00:36:17,542
people which might be bad in a scenario like this.

472
00:36:18,999 --> 00:36:22,999
The point is that for each day that happened based

473
00:36:22,999 --> 00:36:29,083
on the time series you saw, we are going to have a calculation.

474
00:36:29,083 --> 00:36:31,083
So this is what happened on this day.

475
00:36:31,083 --> 00:36:33,999
Then the following day we will actually decay this by the function,

476
00:36:33,999 --> 00:36:36,334
the exponential function.

477
00:36:36,334 --> 00:36:37,999
And then we add what happened the following day,

478
00:36:37,999 --> 00:36:40,083
pretty much adding one.

479
00:36:40,083 --> 00:36:41,667
This happened this day, this happened that day,

480
00:36:41,667 --> 00:36:43,999
and we go decaying the rest.

481
00:36:44,334 --> 00:36:49,083
The importance of having this history, if we're using log data to detect stuff,

482
00:36:49,083 --> 00:36:51,959
data logs will have dates.

483
00:36:52,125 --> 00:36:56,417
I can't use future calculations with things from yesterday;

484
00:36:56,417 --> 00:37:01,709
it doesn't make sense and the model won't make sense.

485
00:37:01,876 --> 00:37:07,876
It will tend to survivorship, which is a routine thing.

486
00:37:11,417 --> 00:37:13,999
So you have to make sure you are pairing

487
00:37:13,999 --> 00:37:17,834
the data you have with the features you have.

488
00:37:21,167 --> 00:37:23,999
The visual scale is actually a log scale.

489
00:37:24,501 --> 00:37:29,459
When you see "6" there, it is a million.

490
00:37:30,999 --> 00:37:34,709
If you get 1 on this score, the horizontal access,

491
00:37:34,709 --> 00:37:39,417
it means that you were hitting me every single day.

492
00:37:41,209 --> 00:37:46,999
I found it pretty interesting that you could get at least 10,000 guys,

493
00:37:46,999 --> 00:37:51,709
and this is RDP hitting me every single day.

494
00:37:51,709 --> 00:37:54,584
And even if I was using that as a black list, which

495
00:37:54,584 --> 00:37:58,626
is a perfectly valid way, look at all the guys I am ignoring,

496
00:37:58,626 --> 00:38:03,292
which doesn't mean anything compared to the data I have.

497
00:38:03,292 --> 00:38:04,292
And on No.

498
00:38:04,292 --> 00:38:07,417
22, you can see the similar behavior.

499
00:38:07,459 --> 00:38:10,751
There will be guys out to get you, but if you only look

500
00:38:10,751 --> 00:38:14,667
at the daily black lists, there is so much information that you are

501
00:38:14,667 --> 00:38:18,125
leaving behind which can potentially be leveraged to help you

502
00:38:18,125 --> 00:38:20,667
with that you are doing.

503
00:38:20,918 --> 00:38:22,375
Are we good?

504
00:38:22,999 --> 00:38:23,999
Too much math?

505
00:38:23,999 --> 00:38:27,584
We can fix that.

506
00:38:27,584 --> 00:38:37,709
(Applause and Laughter)    ALEXANDRE PINTO:

507
00:38:37,709 --> 00:38:43,125
I'll never finish.

508
00:38:43,125 --> 00:38:45,999
Enough with the math bullshit; time to drink!

509
00:38:47,667 --> 00:38:49,250
Wait a second.

510
00:38:49,250 --> 00:38:51,918
Didn't we already do a shot with you this morning?

511
00:38:51,918 --> 00:38:55,834
ALEXANDRE PINTO: Not this morning; you just came

512
00:38:55,834 --> 00:38:59,999
to this room a few minutes ago.

513
00:38:59,999 --> 00:39:02,083
Is your accent Russian?

514
00:39:04,999 --> 00:39:11,083
ALEXANDRE PINTO: No, Brazilian.

515
00:39:11,083 --> 00:39:13,999
Maybe we just want to drink!

516
00:39:22,918 --> 00:39:30,459
(Applause)    Is this your first time?

517
00:39:31,542 --> 00:39:33,209
You did a shot?

518
00:39:33,209 --> 00:39:34,501
ALEXANDRE PINTO: Yes.

519
00:39:34,501 --> 00:39:34,501
Fuckin A.

520
00:39:34,501 --> 00:39:35,501
All right.

521
00:39:35,501 --> 00:39:40,999
We've clearly lost track at this point.

522
00:39:40,999 --> 00:39:41,999
What's your name?

523
00:39:46,167 --> 00:39:47,792
Steve.

524
00:39:47,918 --> 00:39:52,459
Steve, this is everyone.

525
00:39:52,459 --> 00:39:53,626
STEVE: Hi everyone.

526
00:39:55,626 --> 00:40:00,584
Here is to Steve, a first time attendee.

527
00:40:00,918 --> 00:40:06,375
(Applause)    I guess we will be back in five minutes.

528
00:40:08,999 --> 00:40:12,999
ALEXANDRE PINTO: Let's try to get close to finishing.

529
00:40:13,167 --> 00:40:16,626
The point is, we have a bunch of numbers.

530
00:40:16,959 --> 00:40:20,667
But we know with this bunch of numbers    Back to the math.

531
00:40:20,667 --> 00:40:22,250
ALEXANDRE PINTO: Sorry, guys.

532
00:40:22,501 --> 00:40:24,459
We should do a drinking talk.

533
00:40:32,626 --> 00:40:35,250
So we get the features calculated, and we pair them

534
00:40:35,250 --> 00:40:37,834
with the data that we have.

535
00:40:37,834 --> 00:40:41,417
The assumption is if somebody hits the firewall, they are out to get you

536
00:40:41,417 --> 00:40:44,584
and they should be considered bad.

537
00:40:44,667 --> 00:40:46,999
So you put these guys on your training model,

538
00:40:46,999 --> 00:40:52,209
so pretty much the IP address, the features that you calculated.

539
00:40:52,999 --> 00:40:56,250
And then you feed that to the model.

540
00:40:56,626 --> 00:41:00,751
Usually you need more than one day, and that is why separating

541
00:41:00,751 --> 00:41:03,083
the ranks makes sense on the database

542
00:41:03,083 --> 00:41:09,083
to make sure you pair the right day with the right data that you are using.

543
00:41:09,375 --> 00:41:13,501
So rule of thumb and it all depends on the algorithm that you are using.

544
00:41:13,501 --> 00:41:15,999
But the point is, you can't always have malicious data;

545
00:41:15,999 --> 00:41:18,334
you need good data too.

546
00:41:18,375 --> 00:41:22,083
Otherwise, the computer will say: Yay, everything is a chair.

547
00:41:27,626 --> 00:41:32,792
So I took a bunch of IP addresses from Alexa and Chromium

548
00:41:32,792 --> 00:41:38,999
to make sure we got to paired up, and the last was 50/50.

549
00:41:41,999 --> 00:41:46,459
I won't go into the algorithms, but support vector machines are

550
00:41:46,459 --> 00:41:50,999
awesome and do math people don't even understand.

551
00:41:50,999 --> 00:41:52,999
It is like the mathematicians saying I think there

552
00:41:52,999 --> 00:41:56,375
is an infinite dimension that makes sense, but I don't care,

553
00:41:56,375 --> 00:41:59,584
I just calculate it and it makes sense.

554
00:42:00,999 --> 00:42:04,417
That is scary; scary as fuck.

555
00:42:07,083 --> 00:42:08,999
Here is the point.

556
00:42:09,375 --> 00:42:11,584
So we train this every day.

557
00:42:11,999 --> 00:42:13,459
I have a bunch of ports.

558
00:42:13,542 --> 00:42:19,542
What is happening on Port 22 and 338 and Port 25?

559
00:42:19,667 --> 00:42:25,125
And I would get something around 83 95% training courtesy,

560
00:42:25,125 --> 00:42:28,999
which is what I get right.

561
00:42:28,999 --> 00:42:32,999
So bad is bad, good is good, and then divided by the number

562
00:42:32,999 --> 00:42:36,417
of things I gave it to process.

563
00:42:36,501 --> 00:42:39,501
And this is good, but it is not really accurate because,

564
00:42:39,501 --> 00:42:42,542
especially if you are using time sensitive data,

565
00:42:42,542 --> 00:42:45,167
this is a technique called cross validation

566
00:42:45,167 --> 00:42:48,626
which doesn't work very well for that.

567
00:42:50,959 --> 00:42:53,667
So I looked at the following day.

568
00:42:54,083 --> 00:42:55,751
And this is the point.

569
00:42:55,834 --> 00:42:58,999
I did all this calculation for today, who are the guys most likely

570
00:42:58,999 --> 00:43:02,999
to be attacking me tomorrow, based on all this data?

571
00:43:02,999 --> 00:43:07,999
And I would get something from 75 85% which I

572
00:43:07,999 --> 00:43:15,334
will break down so that you can really understand what 79 95% man,

573
00:43:15,334 --> 00:43:17,876
this is hard.

574
00:43:19,083 --> 00:43:23,375
Anyway, just an idea of the progression.

575
00:43:23,542 --> 00:43:27,250
You have to run of course, if you have a model from February

576
00:43:27,250 --> 00:43:30,999
and you try to run it from something that happened in July,

577
00:43:30,999 --> 00:43:34,959
if you do that, you will have a terrible time.

578
00:43:34,959 --> 00:43:36,999
That is because a lot of things have happened,

579
00:43:36,999 --> 00:43:39,999
and it also shows a bit about the "moving about"

580
00:43:39,999 --> 00:43:44,999
of the environment, and blah blah and blah blah, yes, same shit.

581
00:43:45,709 --> 00:43:53,918
The month I am trying to make, it is 79 to 95, okay.

582
00:43:53,999 --> 00:43:57,626
What is the bad stuff I got bad and the good stuff I got good,

583
00:43:57,626 --> 00:44:00,667
so true positive and true negative.

584
00:44:00,709 --> 00:44:02,918
So the numbers are a bit slow.

585
00:44:02,918 --> 00:44:05,667
As you see, the true negative was really bringing it up.

586
00:44:05,667 --> 00:44:09,375
I think once I get different data, it should even it out.

587
00:44:09,709 --> 00:44:14,083
But the point is, if you calculate this, given the error rate,

588
00:44:14,083 --> 00:44:18,709
something this picks up in your logs and says you should

589
00:44:18,709 --> 00:44:25,501
look at, it is about 18 time more likely to attack you than all the rest.

590
00:44:25,999 --> 00:44:30,918
So wouldn't you think you should look at that first?

591
00:44:32,751 --> 00:44:36,125
This is not catching everything, and neither are the analysts,

592
00:44:36,125 --> 00:44:38,584
you know: Spoiler alert!

593
00:44:39,999 --> 00:44:45,083
But if you have time constraints, and people have so much time

594
00:44:45,083 --> 00:44:51,292
because they have to eat and sleep, it is terrible, labor laws.

595
00:44:51,292 --> 00:44:56,083
But let's try to make this a bit easier.

596
00:44:56,751 --> 00:44:59,542
And this is where it is coming from.

597
00:45:00,417 --> 00:45:02,834
This is an idea of prediction.

598
00:45:02,999 --> 00:45:06,709
So I am just it is the same curve that you saw before.

599
00:45:06,709 --> 00:45:07,999
I am sampling a hundred KIP addresses

600
00:45:07,999 --> 00:45:09,792
from each place.

601
00:45:10,999 --> 00:45:13,167
So the brighter the tile, the more likely the people

602
00:45:13,167 --> 00:45:15,083
will be attacking you.

603
00:45:15,501 --> 00:45:21,876
This is a logarithm rate, so it is go like tens of thousands, and this

604
00:45:21,876 --> 00:45:24,417
is the deal, guys.

605
00:45:24,417 --> 00:45:28,626
So you see, it is everywhere.

606
00:45:28,626 --> 00:45:29,626
Challenges.

607
00:45:29,626 --> 00:45:33,083
IP addresses are bad, the worst thing to use for instant response.

608
00:45:34,999 --> 00:45:38,792
The point I am trying to make, even with the real shitty stuff,

609
00:45:38,792 --> 00:45:41,999
we can get some interesting results.

610
00:45:46,626 --> 00:45:49,999
If you are coming from TOR with a lot of clustering, it messes it up.

611
00:45:50,125 --> 00:45:52,167
If I change the IP address every 30 minutes,

612
00:45:52,167 --> 00:45:53,834
fuck you.

613
00:45:56,542 --> 00:45:59,250
But I believe if you can reduce the cycle, so it

614
00:45:59,250 --> 00:46:03,667
is a daily resolution which you can begin to get smarter.

615
00:46:03,999 --> 00:46:07,999
So I wouldn't just go on IP addresses; I would go on different stuff too.

616
00:46:08,083 --> 00:46:15,334
So what I am trying to say in it is potentially help security analysts,

617
00:46:15,334 --> 00:46:20,999
bringing a new dimension for software, and these are

618
00:46:20,999 --> 00:46:26,834
the guys most likely who are out to get me.

619
00:46:28,083 --> 00:46:30,999
So let's invest some time in that.

620
00:46:30,999 --> 00:46:34,959
What is cool, I created a model for firewalls, and I could do

621
00:46:34,959 --> 00:46:38,709
the exact same thing for IPSs and WAFs.

622
00:46:39,999 --> 00:46:43,667
And if I take the inference each individual gives me

623
00:46:43,667 --> 00:46:45,999
on the IP addresses.

624
00:46:45,999 --> 00:46:49,999
Then I combine them, I can say this guy is 200 more times

625
00:46:49,999 --> 00:46:55,626
as likely to attack you; then I would block this fucker.

626
00:46:57,501 --> 00:47:01,999
This is the kind of confidence you are trying to build.

627
00:47:01,999 --> 00:47:02,999
"

628
00:47:03,125 --> 00:47:05,250
Blocking" is a bad word in information security, especially

629
00:47:05,250 --> 00:47:07,083
if you are defending, but maybe we can get

630
00:47:07,083 --> 00:47:08,999
to that confidence.

631
00:47:09,167 --> 00:47:12,876
And the way I am trying to do this, there is actually a project.

632
00:47:14,751 --> 00:47:19,250
So you send me data, and I send you a report.

633
00:47:20,083 --> 00:47:25,292
And the point is, the more data I have, I can start to fight the bias I got

634
00:47:25,292 --> 00:47:27,999
from the SANS database.

635
00:47:28,542 --> 00:47:31,501
There are URLs, there is a Twitter feed.

636
00:47:32,334 --> 00:47:37,292
You can look it up, but I will be around.

637
00:47:38,542 --> 00:47:40,083
Take aways.

638
00:47:41,999 --> 00:47:45,584
It is not a monster.

639
00:47:45,999 --> 00:47:49,083
Of course there is marketing hype, but it can help.

640
00:47:49,209 --> 00:47:52,334
I think 13 to 18 is pretty good.

641
00:47:52,999 --> 00:47:54,501
Thanks.

642
00:47:54,501 --> 00:47:54,501
(Applause)    ALEXANDRE PINTO: I will be at the Chill Out Lounge

643
00:47:54,501 --> 00:47:55,999
if you want to ask questions.

644
00:47:55,999 --> 00:47:57,209
Thank you very, very much.