1
00:00:15,040 --> 00:00:18,440
We have another great speaker for you today, Dr.

2
00:00:19,080 --> 00:00:24,340
Pinky, presenting Discovering ELK, the First Time Lessons Learned Over Two Years.

3
00:00:24,660 --> 00:00:27,040
And with that, I will let Dr.

4
00:00:27,300 --> 00:00:28,740
Pinky take it away.

5
00:00:29,440 --> 00:00:31,920
Hey guys, thank you so much for joining me today.

6
00:00:31,920 --> 00:00:33,800
I know it's a little late in the evening.

7
00:00:34,160 --> 00:00:39,280
I know things start a little bit late with Pacific, but we're kind of good in the afternoon crash, evening crash, depending on your time zone.

8
00:00:39,280 --> 00:00:42,060
So hang in with me and let's have some fun, OK?

9
00:00:42,840 --> 00:00:46,220
So about me real fast, other than the fact that I have pink hair, obviously.

10
00:00:46,740 --> 00:00:48,760
I'm a Linux and Windows threat hunter.

11
00:00:48,760 --> 00:00:51,980
I prefer Linux, but for some reason, Windows seems to have the marketplace.

12
00:00:51,980 --> 00:00:52,560
Don't know why.

13
00:00:52,560 --> 00:00:54,200
Can't be its usability or anything.

14
00:00:54,480 --> 00:00:56,480
So I end up there most of the time.

15
00:00:56,780 --> 00:00:59,040
I'm a huge fan of scotch, whiskey, and bourbon.

16
00:00:59,040 --> 00:01:00,600
You know, the main things you actually care.

17
00:01:01,200 --> 00:01:08,340
I'm over here in San Antonio, Texas, and as a result, I help coordinate various groups here, including B-Side San Antonio and Saha.

18
00:01:08,780 --> 00:01:16,580
And of course, the mandatory disclaimer, all views, thoughts, opinions, anything else I say here are my own and are representative of my employer.

19
00:01:17,080 --> 00:01:21,340
With that out of the way, let's go into what we're talking about today.

20
00:01:21,500 --> 00:01:26,080
First off, I'll just lay out some quick expectations, just so we're all on the same page.

21
00:01:26,420 --> 00:01:35,820
And I know most of you, since we're blue teamers for the most part here, are probably familiar with Elastic Stack, but I'm going to get into it just so we're using all the same terminology with the same understanding.

22
00:01:36,640 --> 00:01:41,300
And then we'll go into four fun lessons I learned the hard way these last two years.

23
00:01:41,300 --> 00:01:43,740
And I'll admit, some of them are pretty embarrassing.

24
00:01:43,740 --> 00:01:46,020
So hopefully you get a few good laughs out of this.

25
00:01:46,500 --> 00:01:48,640
So with that, let's get into it.

26
00:01:49,800 --> 00:01:53,720
So after two years, you may think, hey, I should be an expert by now.

27
00:01:53,720 --> 00:01:55,180
But no way.

28
00:01:55,340 --> 00:01:57,580
This keeps changing every day, it feels like.

29
00:01:57,580 --> 00:02:01,220
And every time I think I get a grasp of something, it changes.

30
00:02:01,220 --> 00:02:03,600
So that's the fun thing about our community, right?

31
00:02:04,300 --> 00:02:09,880
This talk is geared toward analysts, threat hunters, things of that nature, not toward building or deploying stacks.

32
00:02:09,880 --> 00:02:16,680
So anything I say here could, you know, crash your product or your, you know, stack and product.

33
00:02:16,980 --> 00:02:18,260
Sorry, life stack.

34
00:02:18,260 --> 00:02:18,860
Don't do it.

35
00:02:18,860 --> 00:02:20,960
Play first, then deploy.

36
00:02:21,640 --> 00:02:24,440
This is aimed at giving you a starting point to research and play.

37
00:02:24,440 --> 00:02:28,640
Again, please don't go, hey, let's go live with this right away.

38
00:02:29,300 --> 00:02:33,800
And again, this assumes basic familiarity with the Elastic Stack.

39
00:02:33,840 --> 00:02:37,540
So with that out of the way, what is the Elastic Stack?

40
00:02:37,600 --> 00:02:51,800
The most basic definition you could give is that the Elastic Stack consists of multiple open source projects, which, when used together, takes data from any source and enables search, analysis, and the creation of visualizations and dashboards.

41
00:02:51,800 --> 00:02:56,260
It was originally called the Elk Stack, and I've noticed it's still constantly used today.

42
00:02:56,640 --> 00:03:03,320
And it was called that way because it was originally just Elastic, Logstash, and Kibana for those open source projects.

43
00:03:03,720 --> 00:03:08,020
However, back in version 5, they introduced Beats.

44
00:03:08,020 --> 00:03:11,140
And Beats is a fantastic thing we'll get into briefly in this talk.

45
00:03:11,140 --> 00:03:15,120
But in summary, it's just a way to gather your data and ship it for you.

46
00:03:15,120 --> 00:03:16,620
So you don't have to worry about that.

47
00:03:16,820 --> 00:03:20,840
And for some reason, I guess I didn't like the sound of elk or anything like that.

48
00:03:20,840 --> 00:03:22,520
So it got called the Elastic Stack.

49
00:03:24,820 --> 00:03:28,980
So what is the Elastic Stack and its most basic components today?

50
00:03:29,340 --> 00:03:32,020
Note that I'm just going over the four primary ones.

51
00:03:32,020 --> 00:03:34,620
You can add many more things to this.

52
00:03:34,760 --> 00:03:38,660
But again, this is just so we can understand the basic terms.

53
00:03:39,160 --> 00:03:44,100
You have Kibana, which is what analysts, threat hunters, whatever, are going to be the most used to.

54
00:03:44,100 --> 00:03:45,260
It's the web interface.

55
00:03:45,260 --> 00:03:49,160
It's your visualization tool that lets you, again, visualize, search, and navigate.

56
00:03:49,160 --> 00:03:51,220
Just a handy little web GUI.

57
00:03:51,420 --> 00:03:54,060
And all this data is actually stored in Elastic.

58
00:03:54,320 --> 00:03:55,540
Elastic is the heart of the stack.

59
00:03:55,540 --> 00:03:56,560
Stores the logs.

60
00:03:56,560 --> 00:03:59,800
Actually does the really sexy indexing in the background.

61
00:03:59,840 --> 00:04:03,100
Does all the things that we take for granted, to be honest.

62
00:04:04,480 --> 00:04:08,560
Logstash is the other undervalued aspect, which is the pipeline.

63
00:04:08,680 --> 00:04:12,620
So it takes data, parses it, and spits it back out.

64
00:04:12,620 --> 00:04:17,200
And more than that, it can actually enrich it, like we'll get into with lesson number two.

65
00:04:18,060 --> 00:04:25,560
Beats is located on your actual endpoints and collects configured logs or whatever files you have and sends it to Logstash.

66
00:04:25,880 --> 00:04:30,500
So with that out of the way, very first lesson I learned.

67
00:04:31,540 --> 00:04:34,020
Isn't Elastic and Kibana the same?

68
00:04:34,020 --> 00:04:37,780
For the longest time, I didn't understand how these were different.

69
00:04:38,220 --> 00:04:43,240
And hopefully after this, you'll understand that they aren't, if you're like younger me.

70
00:04:43,240 --> 00:04:45,420
But why would I ever think this?

71
00:04:45,420 --> 00:04:46,380
Because, I mean, come on.

72
00:04:46,380 --> 00:04:48,100
Their names are two separate things.

73
00:04:48,120 --> 00:04:49,320
I just went over it.

74
00:04:49,320 --> 00:04:50,840
It should be obvious.

75
00:04:51,740 --> 00:04:58,040
However, in a lot of implementations, not all, Kibana and Elastic can be on the same page.

76
00:04:58,920 --> 00:05:00,240
The same webpage, that is.

77
00:05:00,240 --> 00:05:04,840
So in the screenshot I grabbed here from GitHub, you can actually see, you have Discover, which is Kibana.

78
00:05:04,880 --> 00:05:06,580
Kibana is featured at the top.

79
00:05:06,700 --> 00:05:09,680
But down at the bottom, you have Management and Monitoring.

80
00:05:09,680 --> 00:05:13,480
And some of that can start getting into Elastic and Logstash.

81
00:05:14,200 --> 00:05:19,060
Additionally, if you notice the error up there, it says, error request to Logstash failed.

82
00:05:19,060 --> 00:05:20,820
I'm sorry, Elastic Search.

83
00:05:22,160 --> 00:05:30,420
And that kind of introduces this confusion to beginners of, wait, if I'm using Kibana, why am I getting errors for Elastic?

84
00:05:30,740 --> 00:05:33,880
Again, as you get more familiar with this, it makes a bit more sense.

85
00:05:34,020 --> 00:05:36,600
But from a beginner, this can kind of confuse you.

86
00:05:37,680 --> 00:05:45,600
Additionally, without fully understanding the full stack, you may think that Elastic is just part of Kibana, which they're different.

87
00:05:46,520 --> 00:05:49,820
And to make it worse, when you go to Management, they both have the word index.

88
00:05:49,820 --> 00:05:53,180
Again, this should be, you know, kind of easy stuff you get used to.

89
00:05:53,280 --> 00:05:58,340
But for someone brand new to the stack, you're just going to go, Kibana, Elastic, cool, same thing.

90
00:05:59,040 --> 00:06:05,040
So what is actually happening here is that a normal analyst is probably really going to interact with Elastic itself.

91
00:06:05,040 --> 00:06:06,440
That's the goal.

92
00:06:06,600 --> 00:06:11,480
You should just be able to do everything you need through Discover, Visualize, and Dashboard, hopefully.

93
00:06:11,540 --> 00:06:16,480
Maybe a little bit of machine learning, maybe a little bit of the timeline, maybe a bit of graph.

94
00:06:16,540 --> 00:06:19,680
But overall, you should be sticking with the Kibana features itself.

95
00:06:20,140 --> 00:06:28,880
As a result, you're probably primarily interacting with Kibana, but occasionally get messages like these with Elastic, and there's more you can run into.

96
00:06:29,700 --> 00:06:34,520
Additionally, when Elastic goes down, Kibana doesn't really work.

97
00:06:34,520 --> 00:06:39,680
The data disappears and you just get ugly errors, and depending how bad it is, it's spitting errors all the time.

98
00:06:39,680 --> 00:06:46,220
And you're like, well, if they're not, you know, together, then why am I seeing those errors?

99
00:06:46,420 --> 00:06:49,160
So that causes a lot of confusion to beginners.

100
00:06:50,440 --> 00:06:56,040
So as a result, let's try to go into this a little bit more and understand better some of these differences.

101
00:06:56,600 --> 00:07:01,360
And for this, I'm just going to go into the three primary features, at least I think are important.

102
00:07:01,360 --> 00:07:06,560
The purpose of them, how you access them, and their search capabilities, because guess what?

103
00:07:06,560 --> 00:07:09,360
You can search in both of them, which introduces more confusion.

104
00:07:10,060 --> 00:07:17,080
So remember, Elastic's purpose in life is to obtain the logs, index them, and hold them in an internal database.

105
00:07:17,580 --> 00:07:22,620
Because it's already the database, you can actually have search functionality through it.

106
00:07:22,620 --> 00:07:31,740
It's just typically through cURL, or you can interact with a few other ways, like Python, for example, that we'll get into later.

107
00:07:31,900 --> 00:07:38,820
Versus Kibana, its purpose is actually just to provide you, the user, the ability to create data visualizations.

108
00:07:39,100 --> 00:07:41,980
And again, we can make that a bit more sexy with the dashboards.

109
00:07:42,000 --> 00:07:44,760
And it does provide some search capabilities.

110
00:07:44,760 --> 00:07:49,500
But if all we cared about was search, we probably wouldn't have ever introduced Kibana.

111
00:07:50,500 --> 00:07:55,860
To access Elastic, again, you can use cURL generally through GET requests.

112
00:07:55,880 --> 00:08:01,100
However, we typically interact with it with Kibana, which does all this in the background for us.

113
00:08:01,200 --> 00:08:07,060
And again, with Kibana, you just open your favorite web browser, navigate to the IP or URL, and life's good.

114
00:08:07,980 --> 00:08:14,300
And for the search capability for Elastic, again, you're going to do it through those GET requests or through a RESTful API.

115
00:08:14,300 --> 00:08:17,480
And it uses the Elastic Search Query DSL.

116
00:08:17,480 --> 00:08:23,500
And if you're used to interacting with that, you're going to have issues when you go to Kibana because it's going to keep scolding you.

117
00:08:23,500 --> 00:08:24,760
Hey, stop using that.

118
00:08:24,760 --> 00:08:29,140
Use my search capability, which is the Kibana standard query language.

119
00:08:29,220 --> 00:08:30,700
You can make it work.

120
00:08:30,700 --> 00:08:32,500
It allows you to search it directly.

121
00:08:32,500 --> 00:08:37,580
But in general, if you're new at this, just try to get familiar with Kibana standard query language.

122
00:08:37,660 --> 00:08:41,140
And again, you just access that through your handy web interface.

123
00:08:42,600 --> 00:08:47,880
So, to kind of help summarize this, Elastic is your distributed search engine.

124
00:08:47,880 --> 00:08:50,260
It takes in the logs, not even through Logstash.

125
00:08:50,260 --> 00:08:53,860
You can just feed in the logs directly in the index system.

126
00:08:54,200 --> 00:08:59,560
Again, it can receive directly through HTTP request methods or through Logstash.

127
00:08:59,900 --> 00:09:03,200
If you have another replacement for Logstash, that's fine.

128
00:09:03,200 --> 00:09:04,560
They'll accept it too.

129
00:09:05,180 --> 00:09:10,860
And it's truly the heart, the engine, that helps make searches happen so quickly for Kibana.

130
00:09:10,860 --> 00:09:16,020
Kibana does none of the heavy-duty effort to make that happen.

131
00:09:17,120 --> 00:09:22,640
So, Kibana, on the other hand, is your browser-based analytics and search dashboard for Elastic.

132
00:09:22,960 --> 00:09:26,680
And again, it's what your analyst or you are going to typically interact with.

133
00:09:26,860 --> 00:09:30,880
Again, it allows you to conduct your searches, create visualizations, and dashboards.

134
00:09:31,480 --> 00:09:34,900
After all this, you may be asking, why do I care?

135
00:09:35,360 --> 00:09:42,540
First off, incorrect terminology can, one, make you sound a little silly, two, result in you potentially being locked up behind your back.

136
00:09:43,040 --> 00:09:52,580
But mainly, the reason you should care is that understanding the differences and the interaction between the two can help you better improve your troubleshooting skills.

137
00:09:52,580 --> 00:09:57,140
That way, when you encounter those errors, you can better understand what needs to happen.

138
00:09:57,280 --> 00:09:59,800
Alternatively, you can just submit a better ticket.

139
00:10:00,100 --> 00:10:07,980
Additionally, as we'll get into soon, if you can't quite make the right search query, understanding overall the interaction with the stack is going to help you with that.

140
00:10:08,480 --> 00:10:15,280
So, even if you think, hey, I'm never going to deploy the stack, I don't care, you should care at least at a very basic level the differences.

141
00:10:16,900 --> 00:10:28,460
So, with that out of the way, second thing I still say to this day, why can't I search for, enter the field name, I see it's a field.

142
00:10:28,460 --> 00:10:32,640
In this case, host name, I'll admit this was me probably three weeks ago.

143
00:10:33,280 --> 00:10:37,500
So, again, me on a new and or existing setup all the time.

144
00:10:38,560 --> 00:10:44,240
Take a moment and briefly see if you can figure out what's the problem if you use this.

145
00:10:45,720 --> 00:10:47,600
Okay, you got your five seconds.

146
00:10:48,140 --> 00:10:49,660
A little yellow triangle.

147
00:10:49,660 --> 00:10:52,080
I like calling it the yellow triangle of doom.

148
00:10:52,180 --> 00:10:54,820
I'm sure it has a better name, but it's what I call it.

149
00:10:56,020 --> 00:10:59,480
So, what is the symptom when you're suffering from this?

150
00:10:59,480 --> 00:11:02,140
You know a field exists, but you can't reference it.

151
00:11:02,140 --> 00:11:06,200
You go to Kibana, you go and search, and you're like, take this field name and search for this value.

152
00:11:06,200 --> 00:11:08,660
And it's like, no records.

153
00:11:08,660 --> 00:11:13,900
And you're 100% positive that one, the field exists, and two, something should have that.

154
00:11:14,260 --> 00:11:23,920
So, what you do is that you find a record within Kibana, you know, you can just go discover, roll through the records, and look at them until you find that field name you care about.

155
00:11:23,920 --> 00:11:30,300
You can then search for that specific field name and its value and still have trouble.

156
00:11:30,520 --> 00:11:34,580
At that point, you'll probably notice, hey, I have that little yellow triangle.

157
00:11:35,220 --> 00:11:40,140
Roll over it, and you'll most likely see the message, no cached mapping for this field.

158
00:11:41,360 --> 00:11:53,080
And the really quick way to just fix this is go into management, go to index patterns, find your index pattern, in this case, just Logstash at the bottom of the screen, and click the little refresh button.

159
00:11:53,080 --> 00:11:54,160
That'll fix it.

160
00:11:54,160 --> 00:11:56,360
You can search it, and life is happy.

161
00:11:57,320 --> 00:12:00,420
But you should be asking yourself, why does this happen?

162
00:12:00,940 --> 00:12:06,500
It happens whenever you get either one or more new fields introduced to an index pattern.

163
00:12:07,440 --> 00:12:13,100
And remember, your index pattern is going to be the interaction between Elastic and Kibana happening here.

164
00:12:13,100 --> 00:12:20,120
So, what happens is that Kibana has a cached version of the mapping given by Elastic, and it doesn't have this field.

165
00:12:20,460 --> 00:12:23,300
So, therefore, it doesn't really know how to give you those results.

166
00:12:24,040 --> 00:12:29,760
So, refreshing the index pattern allows for the creation of this mapping, fixing it.

167
00:12:30,860 --> 00:12:33,360
So, why would this ever happen?

168
00:12:33,460 --> 00:12:35,160
You know, we're in this nice, stable organization.

169
00:12:35,160 --> 00:12:36,540
This will never happen.

170
00:12:37,940 --> 00:12:38,740
Garbage.

171
00:12:38,760 --> 00:12:45,740
You're either going to do it through log enrichment, where you might see, or you're interpreting a new log, and you want to improve it somehow.

172
00:12:46,480 --> 00:12:49,500
Alternatively, and kind of tied with that, is improved parsing.

173
00:12:49,960 --> 00:12:56,080
Let's say you have this message on the right here of the screen, where you have network connection detected, and all this stuff.

174
00:12:57,040 --> 00:13:01,040
Depending on how your setup is, this may not actually be its own fielding.

175
00:13:01,360 --> 00:13:08,300
So, you might not be able to go through and be like, hey, how do I query the image, or the user, or the protocol, or anything else you see on there?

176
00:13:08,520 --> 00:13:13,020
Maybe it's all just in that one message, which makes it kind of ugly for Kibana search purposes.

177
00:13:13,520 --> 00:13:18,180
Therefore, maybe you need to go back and improve that parsing and implement that.

178
00:13:18,180 --> 00:13:23,320
And when you do that, and you try to query, let's say, the user, it's still not going to work because you didn't refresh it.

179
00:13:24,120 --> 00:13:37,060
And also, this can happen if you have a new update in any of the aspects you used, such as if you just upgraded Elastic from an older version to a newer version that uses the Elastic Home schema.

180
00:13:37,060 --> 00:13:42,140
You're going to have a lot of new field names, and you're going to need to refresh that.

181
00:13:42,140 --> 00:13:47,980
Additionally, if you implemented any custom events that we'll talk about soon, you're also going to run into that.

182
00:13:49,080 --> 00:13:51,020
So, don't just refresh blindly.

183
00:13:51,020 --> 00:13:53,360
Think about why it happened and if you can figure it out.

184
00:13:55,400 --> 00:14:03,720
So, speaking of that, when is this suitable to actually use Logstash for log enrichment or just improving your contextual awareness?

185
00:14:04,180 --> 00:14:16,240
To me, it comes down to three primary things, either to improve your search capabilities, such as if you're doing a search on a Windows environment and you really don't want to see the service accounts, you just want to see what the users themselves are doing.

186
00:14:17,260 --> 00:14:28,100
Therefore, you could make this really complex query to remove all those IPs or host names, or you can make a complex query to include all the different hosts and their subnets they're in.

187
00:14:28,780 --> 00:14:30,820
And again, filter out those service accounts.

188
00:14:32,900 --> 00:14:38,440
Or you can just mark all service accounts as they go through Logstash, so the Windows logs.

189
00:14:38,680 --> 00:14:47,480
And if it sees that, you can just tag your service account and then you can simply tell Kibana, don't use or exclude all records with this tag.

190
00:14:47,660 --> 00:14:56,220
Makes it a lot easier, makes it easier on the analysts, makes sure you make less mistakes, and just overall improves your experience.

191
00:14:56,900 --> 00:15:00,160
Additionally, sometimes you just need to transform or fix data.

192
00:15:00,160 --> 00:15:06,960
So an example of this is often Linux will send timestamp, maybe in a weird format that you're not using.

193
00:15:06,960 --> 00:15:10,340
So your Windows and your Linux may send it in a different format.

194
00:15:11,040 --> 00:15:18,200
And as a result, that makes it hard to compare things and be able to see, did something happen at the same time or in a similar window?

195
00:15:18,520 --> 00:15:23,500
Alternatively, maybe one of your systems is set up to use UTC, but the other is using local.

196
00:15:23,620 --> 00:15:25,120
That also doesn't help.

197
00:15:25,120 --> 00:15:27,400
These are all different things you can use to fix.

198
00:15:27,920 --> 00:15:31,720
And additionally, why not improve your context?

199
00:15:31,720 --> 00:15:36,420
So rather than having you have to go like, OK, let me see if this IP is a known bad IP.

200
00:15:36,440 --> 00:15:40,160
Let me see if this URL is a bad URL or something like that.

201
00:15:40,160 --> 00:15:47,560
You can add the log session, have a query and be like, hey, is this IP that's coming in or going down part of a botnet?

202
00:15:47,760 --> 00:15:50,420
Or is someone visiting a known malware site?

203
00:15:50,460 --> 00:15:59,280
There's a lot of things you can do, and I'm not going to get into all of these per se, but these are just examples to try to get you thinking about how can you improve your environment?

204
00:16:01,300 --> 00:16:06,220
So in order to understand this, again, yes, you, the analyst, may not ever be doing this.

205
00:16:07,100 --> 00:16:10,560
But understand how this works so you can make more reasonable requests.

206
00:16:10,720 --> 00:16:14,160
And you're not doing things such as saying, I want to find a bad guy.

207
00:16:14,600 --> 00:16:15,660
You get nothing else.

208
00:16:15,660 --> 00:16:17,440
You just say, find bad guy.

209
00:16:17,720 --> 00:16:18,960
I'm sure we've all heard that a lot.

210
00:16:18,960 --> 00:16:20,000
We get irritated.

211
00:16:20,220 --> 00:16:21,520
We don't want to do that.

212
00:16:21,520 --> 00:16:23,640
We don't want to be the same people saying that.

213
00:16:23,640 --> 00:16:30,240
So as a result, if you're asking for something to be fixed or improved, have a better understanding of what is possible.

214
00:16:30,800 --> 00:16:35,000
So the overall format of a Logstash configuration files, you have input and output.

215
00:16:35,000 --> 00:16:36,820
Those are the two mandatory fields.

216
00:16:37,140 --> 00:16:39,840
Input tells Logstash, what port should I listen on?

217
00:16:39,840 --> 00:16:41,480
What protocol should I listen on?

218
00:16:41,480 --> 00:16:43,160
And how should I interpret it?

219
00:16:43,340 --> 00:16:48,260
The output says, once I'm done with it, where do I output it to and where?

220
00:16:49,080 --> 00:16:55,460
Additionally, there's a filter section that can go in between, and it's optional.

221
00:16:55,460 --> 00:16:58,780
However, all the fun things we just talked about are in there.

222
00:16:59,120 --> 00:17:09,940
And just to go a little bit more, you can have TCP, you can have UDP, you can have Beats, things like that for all of how Logstash should be listening to it.

223
00:17:09,940 --> 00:17:13,740
The codec, just briefly, is going to be how should I interpret it?

224
00:17:13,740 --> 00:17:16,000
JSON, CSV, something like that.

225
00:17:17,360 --> 00:17:18,160
Output.

226
00:17:18,600 --> 00:17:22,660
Note that here, you can obviously see the hosting, which is what we're primarily going to use.

227
00:17:22,880 --> 00:17:29,600
That also means if you see an error, like back in Lesson 1, a VIP can't resolve that, it may be a DNS problem.

228
00:17:30,020 --> 00:17:34,580
So again, just trying to tie these together and better improve our understanding of how the whole stack works.

229
00:17:35,780 --> 00:17:39,000
So with that in mind, there's a thing you can do called tags.

230
00:17:39,000 --> 00:17:42,680
I mentioned this before, like tagging user accounts, tagging the machine.

231
00:17:43,520 --> 00:17:45,260
Why would you ever want to do this?

232
00:17:45,260 --> 00:18:00,200
Again, it helps provide contextual information, because even if I don't filter for it, if I just see an IPNC, hey, this is my domain controller, or hey, this is a workstation, or hey, this belongs to the commerce or the DMZ subnet.

233
00:18:00,200 --> 00:18:05,740
This is all things that helps us quickly go like, okay, I have a better idea of what your purpose in life is.

234
00:18:06,180 --> 00:18:10,260
Alternatively, you can just say, hey, this is a Linux system, this is a Windows system.

235
00:18:10,260 --> 00:18:14,060
Again, you can probably figure that out with 5 or 10 seconds more thought.

236
00:18:14,280 --> 00:18:17,900
But this all add up and make you operate better and more efficiently.

237
00:18:18,600 --> 00:18:20,680
You can also identify the top of logs.

238
00:18:20,680 --> 00:18:28,300
So if you're wanting to parse only, or if you only want to look for a patchy like error logs to see what's going on, you can do that.

239
00:18:28,300 --> 00:18:30,040
Access logs, same thing.

240
00:18:30,620 --> 00:18:34,500
You can also, you know, obviously identify admin accounts, user accounts.

241
00:18:34,680 --> 00:18:38,120
Pretty much, if you think you can make it happen, you can do it.

242
00:18:39,300 --> 00:18:51,560
In this example you see on the slide, it's a really simple, basic one where it says, hey, if you get any logs that come from autobeat, which is something we'll talk about soon, don't worry about it.

243
00:18:51,820 --> 00:18:56,580
It's going to add a tag of Linux because autobeat only exists on Linux.

244
00:18:57,300 --> 00:19:00,000
Again, you can add a lot more complex information here.

245
00:19:00,000 --> 00:19:03,760
This is just to give you the beginnings to make you a little dangerous.

246
00:19:04,240 --> 00:19:09,280
Additionally, it's important to look at tags because Logstash will add tags when a parser fails.

247
00:19:09,720 --> 00:19:18,540
So even if you didn't set up a custom one, if something's failing along the line, it's going to let you know, hey, this part in particular errored.

248
00:19:18,540 --> 00:19:19,960
Go look at it.

249
00:19:21,000 --> 00:19:21,600
So.

250
00:19:24,000 --> 00:19:28,040
Again, some of the different things you may want to consider about with Logstash in general.

251
00:19:29,100 --> 00:19:31,780
And again, this is more about the filter section.

252
00:19:32,380 --> 00:19:35,600
Timestamps that aren't sent through Beats may not come in a format you like.

253
00:19:35,600 --> 00:19:40,220
They may come in different formats, they may come in the wrong time zone, something like that.

254
00:19:40,500 --> 00:19:45,380
Additionally, one thing I particularly like doing is I just want to split the field and make them into a new one.

255
00:19:45,560 --> 00:19:51,480
So rather than saying mycomputer.local, maybe I want to see the hostname of my computer in the domain local.

256
00:19:52,140 --> 00:19:53,500
Maybe you want to have all three.

257
00:19:53,500 --> 00:19:54,700
You can do that.

258
00:19:55,320 --> 00:19:57,160
You can remove field and tags.

259
00:19:57,160 --> 00:20:02,860
This really kind of aggravates me, but has its use, is that Beats will generate an automated tag.

260
00:20:02,900 --> 00:20:10,540
However, this can be a really lengthy tag, so you might want to rename it or you just want to remove it because you don't care that it's some complex thing.

261
00:20:11,720 --> 00:20:15,020
And again, the Translate plugin is going to be how you rename the fields.

262
00:20:15,020 --> 00:20:27,420
And more than just being like, hey, let me rename this, you know, Beats tag, it can be a lot better and actually do like, let me convert the HTTP codes into their actual meaning so you don't have to keep pulling up a little cheat sheet.

263
00:20:27,460 --> 00:20:31,820
As much as you promise you know all the error codes or all the different codes, we don't.

264
00:20:32,660 --> 00:20:37,640
So that helps you, again, save that little bit of time and be a bit more efficient.

265
00:20:39,020 --> 00:20:51,140
So the overall goal when you're filtering and improving your Logstash pipeline is to improve those logs by adding additional information that would have normally been done manually.

266
00:20:51,900 --> 00:21:02,880
Some of the more interesting plugins or plugins you might use regularly is GeoIP, which again adds just geographical info, says, hey, this IP is from China, Russia, America, whatever.

267
00:21:02,880 --> 00:21:07,080
Obviously, it's just doing it based on what the servers report.

268
00:21:07,700 --> 00:21:11,720
If you're that suspicious about it, double check it, but it can give you that quick insight.

269
00:21:12,380 --> 00:21:18,160
DNS Confer standard or reverse lookups, again, handy, saves you that little bit of time.

270
00:21:19,040 --> 00:21:25,700
Threats Classifier is one that's really interesting where it can enrich your logs and add a little bit of context based on the MITRE ATT&CK matrix.

271
00:21:26,260 --> 00:21:32,140
So again, saving you a little bit of time, obviously, especially with something like that, double check it.

272
00:21:33,100 --> 00:21:41,520
And UserAgent helps parse your UserAgent strings into fields, so you can actually search on it a bit more efficiently and not have to do maybe quite so many wildcards.

273
00:21:43,780 --> 00:21:51,080
So overall, if you, the analyst, can better understand Logstash, you can make a better reasonable request of new features and improvements.

274
00:21:51,260 --> 00:21:59,300
It allows you to better understand the errors, so again, you can submit that ticket or ask your neighbor and be like, hey, I'm getting this error and I think it means this.

275
00:22:00,460 --> 00:22:07,100
And overall, you can improve the creation of your dashboards or visualizations through asking to improve those pipelines.

276
00:22:07,440 --> 00:22:16,280
Again, I probably gave you here just enough to be dangerous and maybe really slow down your pipeline to avoid too many of those filters, but have fun.

277
00:22:18,220 --> 00:22:27,120
So with that out of the way, have you ever found yourself running some custom scripts to obtain information from your endpoints and just wish you could view it in Elastic?

278
00:22:27,120 --> 00:22:33,460
Maybe you're just running, hey, I want to get all the processes and import it in, just so I can see.

279
00:22:33,660 --> 00:22:42,400
Maybe you're just doing data analytics, any data from an incident response investigation, anything else that's not natively created in the log.

280
00:22:43,500 --> 00:22:50,720
Again, with me without coffee, maybe I just want to see a pretty visualization with it rather than just viewing it naturally.

281
00:22:51,220 --> 00:22:58,040
If this is you too, you're not alone, and we're going to go into how you actually take these custom scripts and throw it into Elastic.

282
00:22:59,260 --> 00:23:09,760
So again, me without caffeine in the morning is like, I'm tired of reading the CSV, it's 7 a.m., it's ugly, and I don't remember how to control us because I haven't had coffee.

283
00:23:11,560 --> 00:23:13,400
And I have to compare it to my dashboard.

284
00:23:13,600 --> 00:23:18,000
Why can't they be together so I can make better dashboards or just do better searches?

285
00:23:18,620 --> 00:23:20,020
Well, you can.

286
00:23:20,580 --> 00:23:24,540
First off, if you really hate yourself, you can just curl everything.

287
00:23:24,760 --> 00:23:28,500
You could go curl in that CSV and put it into Elastic.

288
00:23:29,680 --> 00:23:34,200
And I don't recommend it because at least I seem to forget curl any time over the weekend.

289
00:23:35,040 --> 00:23:36,900
You can also use Python.

290
00:23:36,900 --> 00:23:38,080
We all love Python.

291
00:23:38,680 --> 00:23:41,360
And probably a few of you are now arguing, it's all good.

292
00:23:41,780 --> 00:23:46,340
So if you want to use Python, all you have to do is install the Elasticsearch package through pip.

293
00:23:46,440 --> 00:23:50,300
You'll connect it to your Elastic, you'll do your steps, and you'll profit.

294
00:23:51,040 --> 00:23:58,220
What it actually allows you to do is you can pull data from your indices, and you can make your edits, and then you push back to it.

295
00:23:58,540 --> 00:24:04,200
It allows you to actually conduct the searches, because remember, Elastic is more than just storing the information.

296
00:24:04,280 --> 00:24:05,660
You can search with it.

297
00:24:06,140 --> 00:24:09,360
Overall, it gives you the ability to use Elastic without the bonnet.

298
00:24:09,360 --> 00:24:15,140
Again, you're not going to get the visualizations, you're not going to get the dashboards, but there's a lot you can do with it.

299
00:24:16,780 --> 00:24:18,840
So, how would you do this?

300
00:24:18,840 --> 00:24:22,480
First off, hopefully you have your data in a JSON format.

301
00:24:23,580 --> 00:24:27,760
Elastic in general prefers things to be in JSON, even Logstash.

302
00:24:27,760 --> 00:24:28,640
Why is that?

303
00:24:28,640 --> 00:24:31,760
JSON is already a structured language.

304
00:24:31,760 --> 00:24:35,120
You know, it's basically a really fancy key value type of thing.

305
00:24:35,120 --> 00:24:41,080
So, it makes it really quick to parse, versus if you have syslog coming in, which we'll talk about soon.

306
00:24:41,440 --> 00:24:47,700
That's all separated by spaces, and not all the fields are guaranteed, and there's no way to easily say it's not there.

307
00:24:48,100 --> 00:24:50,860
So, JSON in general will speed everything up.

308
00:24:51,360 --> 00:24:56,620
So, if you don't have your data already in a JSON format, you're going to have to make it into one.

309
00:24:56,760 --> 00:25:03,680
Then, you'll load it into the Python JSON object, and then you'll upload it into Elastic, simply by doing...

310
00:25:03,680 --> 00:25:16,920
calling your index, specify the index name, specify the type of the document, give it an ID number, note you want it to probably be unique if you're creating a new one, and then you put your JSON object there.

311
00:25:17,820 --> 00:25:21,380
You can do this for every single little thing you do.

312
00:25:21,820 --> 00:25:29,180
If you need to update a lot of different indices, that's going to be a lot of copy and paste, and you're probably going to make mistakes.

313
00:25:29,760 --> 00:25:33,360
So, alternatively, you can just use Elastic's bulk feature.

314
00:25:34,060 --> 00:25:39,460
Again, this isn't Elastic itself, this is the Elastic search package through Python.

315
00:25:39,460 --> 00:25:42,620
And what this does is that it allows you to do better performance.

316
00:25:42,680 --> 00:25:48,460
Just with one call, you can do multiple indexing and deletion operations, again, just in one API call.

317
00:25:48,960 --> 00:26:00,140
So, if you're interested in trying to do this, but you're like, man, I really suck at converting things into JSON, don't worry, I'm with you.

318
00:26:00,140 --> 00:26:08,200
So, what you can do is you can go to GitHub and go download the Elasticsearch loader.

319
00:26:09,040 --> 00:26:10,900
Again, link's right there on the slide.

320
00:26:11,140 --> 00:26:12,240
Fantastic one.

321
00:26:12,540 --> 00:26:23,140
What this does is that it'll take JSON, CSV, or Parquet files, and it'll go ahead, convert if needed, and push it for you just in one command.

322
00:26:24,440 --> 00:26:29,380
Additionally, you can set up some custom mappings to help make sure your data is interpreted the right way.

323
00:26:29,640 --> 00:26:31,100
Why would you want to do this?

324
00:26:31,680 --> 00:26:35,220
Do you want the number one to be an integer or a string?

325
00:26:35,320 --> 00:26:38,740
Do you want port 80 to be a number or a string?

326
00:26:39,120 --> 00:26:41,580
Do you want to be able to do less than or equal to?

327
00:26:41,800 --> 00:26:46,540
Those type of things you sometimes want to make sure are interpreted a specific way.

328
00:26:47,180 --> 00:26:50,380
Maybe with your IPs, you want it to actually be IPs and not a string.

329
00:26:50,380 --> 00:26:53,760
These are all different things you can try to do with the custom mappings.

330
00:26:54,780 --> 00:27:02,240
But what is the downside in doing all this I just talked about, whether it's CURL or Python or using the Elasticsearch loader?

331
00:27:02,720 --> 00:27:07,040
You can't really do any enrichment or tagging that we just talked about.

332
00:27:07,300 --> 00:27:15,640
And again, if you don't set up those custom mappings, which you can do in the Python method as well, it can be incorrect and you're going to have to fix that.

333
00:27:18,040 --> 00:27:22,700
So instead, if you really don't want to do that much Python, FileBeat is going to be your friend.

334
00:27:23,180 --> 00:27:30,160
What FileBeat does is that it's going to take any logs you specify and forward it either to Elastic or LogStash.

335
00:27:30,160 --> 00:27:31,160
It's up to you.

336
00:27:31,400 --> 00:27:35,220
And it supports some basic data processing and data enrichments.

337
00:27:35,320 --> 00:27:36,820
What's the benefit to that?

338
00:27:37,160 --> 00:27:42,400
Well, again, LogStash is already going to be a little bit overwhelmed with anything you're trying to tell it to do.

339
00:27:42,400 --> 00:27:49,020
So you can do some of this data enhancements like adding tags on the endpoint, have it do it there, and then send it.

340
00:27:49,020 --> 00:27:50,940
Save yourself a little bit of time.

341
00:27:51,020 --> 00:27:55,460
Remember, everything you can save improves overall performance.

342
00:27:55,460 --> 00:27:59,600
You just need to decide, are your endpoints struggling in processing power?

343
00:27:59,600 --> 00:28:00,940
Where's LogStash?

344
00:28:01,080 --> 00:28:06,520
There's a bunch of different solutions you can do out there, but we're not necessarily going to get into those nitty gritty details.

345
00:28:07,420 --> 00:28:09,740
So overall, how does FileBeat work?

346
00:28:09,740 --> 00:28:17,680
First off, you're going to use your favorite file format in the world that Elastic loves to use, or the Elastic Stack, and that's YAML.

347
00:28:19,380 --> 00:28:23,640
And YAML is great because you need to specify the spaces specifically, so good luck.

348
00:28:25,020 --> 00:28:32,340
So other than you'll crush yourself without having with that, what you do with FileBeat, at least, is again, you'll do...

349
00:28:32,340 --> 00:28:34,260
And what I have on the right is a very basic one.

350
00:28:34,260 --> 00:28:36,020
It doesn't include everything you need.

351
00:28:36,020 --> 00:28:37,680
It's just the configuration.

352
00:28:37,680 --> 00:28:41,480
We say for the inputs, I'm going to grab type log.

353
00:28:41,680 --> 00:28:46,980
The FileBeat website, depending on what version you have, will tell you what types it supports.

354
00:28:46,980 --> 00:28:49,540
And this is just going to be a Linux log type.

355
00:28:50,560 --> 00:28:58,000
And what it does is it says, hey, collect type log and collect it from, in this case, var log and anything that ends in .log.

356
00:28:58,840 --> 00:29:02,240
So FileBeat reads that and it says, hey, let's do this.

357
00:29:02,240 --> 00:29:08,060
And what it does is it starts a harvester, which is just, you know, child processes for each input type.

358
00:29:08,060 --> 00:29:10,500
So you can have type log, whatever other types.

359
00:29:10,500 --> 00:29:11,940
It's going to start one for each.

360
00:29:13,320 --> 00:29:16,380
And additionally, it's going to create one for each file.

361
00:29:16,780 --> 00:29:20,640
Don't worry, FileBeat's not going to kill your computer by creating like 100 children.

362
00:29:20,760 --> 00:29:21,560
It controls it.

363
00:29:21,560 --> 00:29:22,600
It has a mask.

364
00:29:23,480 --> 00:29:31,460
What it does is that each one goes ahead, grabs the log, opens it, reads it, sends it to an aggregator.

365
00:29:31,460 --> 00:29:41,280
And once it has everything, or when it reaches a certain threshold, it'll send it to either Logstash or Elastic, depending on how you set it up.

366
00:29:42,100 --> 00:29:46,720
So the input aspect, again, starts the harvesters and manages them.

367
00:29:46,720 --> 00:29:52,700
The harvester, all they care about is opening the file, reading it, and submitting it to the aggregator.

368
00:29:55,060 --> 00:30:01,580
So, in the previous slide, you just saw how it ended in star dot log.

369
00:30:02,160 --> 00:30:08,220
You're going to have to specify the type of the document and where it is.

370
00:30:08,360 --> 00:30:15,700
FileBeat is not recursive, so any additional directors you have, let's say under bar log, you're going to have to specify those and keep going into it.

371
00:30:16,020 --> 00:30:22,680
Additionally, with those data enrichment aspects, FileBeat by default is not going to parse those necessarily.

372
00:30:22,700 --> 00:30:31,340
Great, it's not going to consider what type of application it is and automatically parse those for you to make it say, hey, this is a string, this is an IP, all that jazz.

373
00:30:32,040 --> 00:30:34,260
Instead, what you can do is you can use modules.

374
00:30:35,180 --> 00:30:37,780
Modules, the ones I've listed here, are just the beginning.

375
00:30:37,780 --> 00:30:39,180
There's a lot more.

376
00:30:39,180 --> 00:30:46,000
It includes things like system, which are going to be your typical, like bar log ones, things you see in DMessage, things like that.

377
00:30:46,400 --> 00:30:53,240
Nginx for using Nginx, MySQL, IIS, so it's not just Linux, don't worry, and so many more.

378
00:30:53,400 --> 00:30:55,420
An example you see on the right, it's Apache.

379
00:30:55,600 --> 00:31:01,480
We can enable or disable just the access and error, specify where they are, and it works.

380
00:31:02,640 --> 00:31:04,840
So, again, this simplifies your config.

381
00:31:05,220 --> 00:31:08,660
All you do is enable it, specify where the path is, and then you're good to go.

382
00:31:09,400 --> 00:31:14,260
There's a lot more settings you can specify, and that's all going to be in Elastic Documentation.

383
00:31:16,360 --> 00:31:21,800
So, you're probably like, gosh, that's been a while.

384
00:31:22,680 --> 00:31:28,680
And you really don't care about the fact that I want to go back to my Linux origins because I'm tired of talking about Windows and that incy little bit of Linux in the slides.

385
00:31:30,020 --> 00:31:36,100
And a lot of the verbal examples I've provided have been Windows, and obviously some of this can be converted to networking devices.

386
00:31:36,540 --> 00:31:38,220
But what about Linux?

387
00:31:39,820 --> 00:31:44,360
With Linux, when we generally forward logs, we use syslog, rsyslog, or syslogng.

388
00:31:44,360 --> 00:31:46,740
Typically, those are the most popular ones.

389
00:31:47,640 --> 00:31:52,180
Some implementations such as syslogng can forward directly to Elastic.

390
00:31:52,720 --> 00:31:53,520
Fantastic.

391
00:31:54,040 --> 00:31:59,840
However, others are going to require help, whether using FileBeat or using a Python script, whatever the case may be.

392
00:32:00,700 --> 00:32:03,080
But overall, this only helps your services.

393
00:32:03,420 --> 00:32:07,600
It only helps you find things that you can configure in there and be able to read from this log file.

394
00:32:08,180 --> 00:32:13,700
And you get into a little bit of weird things when you try to talk about, I want to forward things from log in, log out.

395
00:32:13,700 --> 00:32:16,940
I want to see my sudo or sus privileges being used.

396
00:32:16,940 --> 00:32:20,000
I want to see when someone edits a very particular file.

397
00:32:20,940 --> 00:32:26,380
Things like that, it's kind of getting to the point of, can we have sysmon for Linux?

398
00:32:26,380 --> 00:32:29,100
Because let's face it, sysmon is great on Windows.

399
00:32:29,100 --> 00:32:31,460
It helps us find so many more bad things.

400
00:32:33,440 --> 00:32:37,320
Linux doesn't have a super famous way of making this happen.

401
00:32:38,620 --> 00:32:40,480
But that's what Audity is for.

402
00:32:41,120 --> 00:32:45,100
Audity is available for basically all distros out there of Linux.

403
00:32:45,740 --> 00:32:51,580
The only ones I've seen a little bit of issues with are Gentoo and maybe a little bit of Arch, but you can still make them work.

404
00:32:52,540 --> 00:32:55,900
And what it does is it allows you to log basically anything.

405
00:32:56,660 --> 00:32:59,140
Here on the slide is just some more common things you're going to do.

406
00:32:59,200 --> 00:33:03,340
It allows you to log the date, time, and type of outcome of an event.

407
00:33:03,600 --> 00:33:06,700
And don't worry, I'm going to define what an event is in just a moment.

408
00:33:06,700 --> 00:33:11,060
An association of that event and who triggered the event.

409
00:33:11,060 --> 00:33:11,800
Was it Root?

410
00:33:11,800 --> 00:33:13,680
Was it user Pinky?

411
00:33:13,680 --> 00:33:14,820
Who was it?

412
00:33:15,480 --> 00:33:19,980
Any modifications to the audit configuration to try to maybe circumvent it.

413
00:33:20,680 --> 00:33:24,620
All uses of authentication mechanisms like SSH or Kerberos.

414
00:33:24,620 --> 00:33:28,460
Because remember, Linux can't be joined to your Active Directory network.

415
00:33:29,160 --> 00:33:33,040
And any changes to trusted databases like XC Password.

416
00:33:33,040 --> 00:33:44,440
And as a little heads up, the little tweet on the slide, that will actually create or generate alerts, or events in this case, that is new process creation, just like SysMob.

417
00:33:45,680 --> 00:33:47,200
So, event.

418
00:33:47,200 --> 00:33:48,500
I said I was going to explain that.

419
00:33:48,500 --> 00:33:49,520
What is it?

420
00:33:49,720 --> 00:33:53,080
What it is, is it's the result of a rule being triggered.

421
00:33:53,080 --> 00:33:54,060
What's a rule?

422
00:33:54,460 --> 00:33:59,940
It's what you see later in the slide, but it's everything you specify in XC audit rules.

423
00:34:00,860 --> 00:34:04,380
So, there are two overall types of rules.

424
00:34:04,900 --> 00:34:07,940
Files to monitor, and monitoring system calls.

425
00:34:08,540 --> 00:34:19,780
So, generally when you want to specify a file, just as a quick, dirty intro, is that you're going to do "-w", for watch, the file to monitor, "-p", any permissions you want to trigger on.

426
00:34:19,800 --> 00:34:24,320
And this generally comes into read, write, execute, and change in attributes.

427
00:34:24,320 --> 00:34:25,180
So, for.

428
00:34:25,560 --> 00:34:28,140
And "-k", what you want the name of the rule to be.

429
00:34:28,140 --> 00:34:32,860
Make this something that makes sense, so if it gets fired off, you understand it.

430
00:34:33,480 --> 00:34:44,020
An example, in the bullet below, it says, hey, I want to watch Etsy Shadow, and anything that is written to it, or if the attribute changes, notify me with the event name, Shadow.

431
00:34:45,020 --> 00:34:49,880
For system calls, it's a little weird, and I'll be honest, every time I see it, I have to go back to the main page.

432
00:34:50,580 --> 00:34:54,960
But what it is, is that you have "-a", an action, followed by a list.

433
00:34:54,960 --> 00:35:03,520
You have "-S", the system call, "-f", which is the field, which often ends up just being like, hey, specify the architecture.

434
00:35:04,560 --> 00:35:08,160
So, is it going to be 32-bit, 64-bit, that type of deal.

435
00:35:08,580 --> 00:35:11,400
And then, "-k", the name of the rule.

436
00:35:12,620 --> 00:35:16,560
So, action must be one of two values, always or never.

437
00:35:16,820 --> 00:35:20,000
So, always alert on it, or never.

438
00:35:20,960 --> 00:35:28,000
And then, you have one of the following for the list, task, entry, exit, user, exclude.

439
00:35:28,220 --> 00:35:34,800
I find the most common one is always an exit, and what this does, is that it will always alert when that system call exits.

440
00:35:35,880 --> 00:35:39,520
So, entry would be the opposite, when that system call is initially called.

441
00:35:40,000 --> 00:35:49,480
In this case, it's going to work on 32-bit architectures, and what this means is not just 32-bit distros, but 32-bit calls for the system calls.

442
00:35:50,420 --> 00:35:54,420
And all these system calls you see here are all for changing the time.

443
00:35:54,700 --> 00:36:02,160
So, adjusting the time, setting the time of day, doing clock set time, those are all different ways you can adjust the time.

444
00:36:02,480 --> 00:36:08,180
So, the rule of calls, or the event that will be named, is time in this case.

445
00:36:09,880 --> 00:36:14,900
So, how do you get started with AuditD, because it's amazing.

446
00:36:15,220 --> 00:36:17,080
First off, you're going to install AuditD.

447
00:36:17,080 --> 00:36:18,580
It's already on your system.

448
00:36:18,580 --> 00:36:20,940
You're going to edit the config if you need.

449
00:36:21,140 --> 00:36:23,320
You're going to create all your rules by hand.

450
00:36:23,960 --> 00:36:24,620
No, you're not.

451
00:36:24,620 --> 00:36:26,900
Go to that GitHub, download it.

452
00:36:26,900 --> 00:36:36,600
It's a fantastic one that has a lot of the most common things you'll need, including things like empty password, the change of time, a lot of things that I catch bad are going to be in there.

453
00:36:37,180 --> 00:36:41,260
You're going to restart the service, and you're going to be like, I have errors.

454
00:36:41,640 --> 00:36:42,720
She lied to me.

455
00:36:42,720 --> 00:36:49,840
No, some of those things are distro-specific, or require specific users like Hrani, and you may need to comment it out.

456
00:36:50,440 --> 00:36:51,280
Do that.

457
00:36:51,280 --> 00:36:53,240
Keep fixing it until it works.

458
00:36:53,840 --> 00:37:03,000
Then you can view the actual logs, which, depending on how you set up your rules, is going to generate an event, or the proper ways to use AUsearch.

459
00:37:03,060 --> 00:37:05,160
You can look at the main page, do it all there.

460
00:37:05,600 --> 00:37:06,520
So, great.

461
00:37:07,360 --> 00:37:13,940
If you followed all these, you know, and we weren't there to talk and just actually worked through this, you'd be like, I have Audity installed.

462
00:37:14,040 --> 00:37:14,900
It's working.

463
00:37:14,900 --> 00:37:16,180
It's generating alerts.

464
00:37:17,160 --> 00:37:23,560
Gosh, it's annoying how it does it every time I use Sue, and I'm trying to be a good person and not use Root, and it's OK, because if I use Root, it's going to cause alerts anyway.

465
00:37:24,840 --> 00:37:27,400
So, hey, it's working.

466
00:37:27,720 --> 00:37:28,520
What next?

467
00:37:30,320 --> 00:37:43,520
You're going to install autobeat, because you want to, you know, you want to move those events and its logs into Kibana and Elasticsearch so you can visualize it pretty, you know, just your text files all day.

468
00:37:43,960 --> 00:37:51,160
So, you're going to point autobeats.config to autodesk.config, or you put those rules just from the GitHub into autobeat.yaml.

469
00:37:51,840 --> 00:38:03,560
And note, if you created it from scratch, or like you just extracted autobeat rather than installing it from the package manager, you need to make sure autobeat.yaml is owned by Root, otherwise it complies.

470
00:38:04,040 --> 00:38:07,780
You're going to start the service and get this exact error on the screen.

471
00:38:08,960 --> 00:38:09,660
Why?

472
00:38:10,420 --> 00:38:14,100
Well, it says, hey, failed to send it because it's already running.

473
00:38:14,520 --> 00:38:15,620
That's weird.

474
00:38:16,680 --> 00:38:17,620
I lied to you.

475
00:38:17,620 --> 00:38:19,240
You need to stop and disable Audity.

476
00:38:19,760 --> 00:38:25,540
The problem with this is that autobeats ends up actually running its own version of Audity.

477
00:38:25,760 --> 00:38:32,920
So, it partially helps avoid some of those alerts that happen of any time it reads those logs.

478
00:38:34,660 --> 00:38:37,060
Additionally, Audity has its own formatting.

479
00:38:37,560 --> 00:38:38,360
Because, again, it's going to...

480
00:38:39,080 --> 00:38:40,740
Audity doesn't use JSON.

481
00:38:40,900 --> 00:38:45,780
It's just more typical Linux formatting stuff, you know, space, tabs, all that.

482
00:38:45,780 --> 00:38:54,820
As a result, Audity is going to take longer or cause more overhead for Logstash to parse and cause you, if you have to figure Logstash, a lot of grief.

483
00:38:55,500 --> 00:39:00,280
And sometimes if Audity and autobeats are both on, it can cause a kernel panic.

484
00:39:00,280 --> 00:39:01,460
That all sucks.

485
00:39:01,760 --> 00:39:03,260
So, why did you bother?

486
00:39:03,960 --> 00:39:10,780
Do you really want to see all the times when your Audity rules fail in your Elastic?

487
00:39:11,640 --> 00:39:17,600
Probably not because it's going to confuse you the next morning or the next analyst who looks at it and goes, oh, God, it's broken.

488
00:39:17,760 --> 00:39:26,160
You want to test it locally to avoid debugging, you know, through it and having to figure out where the error is and without freaking out your fellow analysts.

489
00:39:27,600 --> 00:39:36,380
So, again, install Audity, test it, disable it and stop it, install autobeats, get it working, and life will be good.

490
00:39:37,940 --> 00:39:41,920
So, now you may be wondering, what did we just cover?

491
00:39:41,920 --> 00:39:42,780
Oh, God.

492
00:39:43,400 --> 00:39:46,000
First, Elastic and Kibana are different.

493
00:39:46,760 --> 00:39:50,800
Hopefully, you won't make the same mistake if you're guilty of saying it sometimes.

494
00:39:51,720 --> 00:39:55,180
Logstash will make your Kibana queries way better, I promise.

495
00:39:56,140 --> 00:39:59,780
Feeding in custom documents is possible and actually not that bad.

496
00:39:59,800 --> 00:40:03,800
But the first time, like anything, new experiences suck, after that's fun.

497
00:40:04,400 --> 00:40:07,040
Audity is incredibly painful to set up.

498
00:40:07,320 --> 00:40:11,080
Actually, no, tuning it is extremely painful, but it's worth it.

499
00:40:11,080 --> 00:40:14,920
Because, again, don't be blind on your Linux systems.

500
00:40:15,900 --> 00:40:22,780
And the lesson I want to briefly hint at you in this lesson or this slideshow, YAML sucks.

501
00:40:22,820 --> 00:40:29,980
Probably 90% of your errors throughout all of this will be white space issues because you are dirty and use spaces and types interchangeably.

502
00:40:30,520 --> 00:40:33,160
And if they're not set the same, life sucks.

503
00:40:35,140 --> 00:40:38,020
So, if you want to see these slides, they're actually on my GitLab.

504
00:40:39,280 --> 00:40:40,600
You can grab it from there.

505
00:40:41,300 --> 00:40:45,860
And I'll take a few minutes to answer any questions, but if not, I'll be in the chat for a bit.

506
00:40:46,140 --> 00:40:49,940
And if you don't think of it until 30 days from now, whatever, ask me on Twitter.

507
00:40:50,860 --> 00:40:51,800
That's all I have.

508
00:40:53,500 --> 00:40:55,280
Thank you very much, Dr.

509
00:40:55,300 --> 00:40:57,720
Pinky, for that wonderful presentation.

510
00:40:57,720 --> 00:41:08,460
As always, we encourage you to join our Blue Team Village Discord server and ask questions in Text Talk Track 1.

511
00:41:08,460 --> 00:41:15,900
I will try to take a quick look here and see if we have any questions for you.

512
00:41:17,600 --> 00:41:20,160
And I am not seeing anything.

513
00:41:20,160 --> 00:41:31,120
So, with that, if you do have questions, feel free to direct them to the presenter as she will be hanging around for a little bit.

514
00:41:31,120 --> 00:41:32,540
Thank you again.