1 00:00:01,690 --> 00:00:06,210 So let's finally talk about visual security event analysis. 2 00:00:11,370 --> 00:00:20,490 So what I'm going to talk about is how you analyze log files or events that you have in a visual manner. 3 00:00:22,570 --> 00:00:25,090 I don't want to talk about myself too much. 4 00:00:25,530 --> 00:00:26,750 Maybe one thing. 5 00:00:26,750 --> 00:00:43,110 I've been involved in, like, log analysis and event analysis for quite a while, and it has always interested me whether there's a way to look at the data in a way that it really, like, a lot of data, in a way that it's really apparent what is going on in that dataset. 6 00:00:43,370 --> 00:00:49,390 And if you're looking at log files, thousands of lines of logs, it's really hard to figure out at one glance what is going on. 7 00:00:49,390 --> 00:00:50,830 You have to start reading through them. 8 00:00:50,830 --> 00:00:55,890 You have to, like, use your own mind to do pattern analysis and figure out what is really going on. 9 00:00:55,890 --> 00:01:22,330 So at some point, I started working for this company that I'm working for right now called ArcSight, and I got a hand on a lot of neat tools to look at data, and I got a feature in the software that is called Event Graphs, and I really liked it, and I started looking into how can we use this feature to make it more easy for a security analyst or a person interested in a log file to figure out what is really going on in that dataset. 10 00:01:22,330 --> 00:01:26,670 So that's how kind of my passion about Event Graphs started. 11 00:01:27,530 --> 00:01:34,550 My talk here is not going to be very textual or, like, theoretical, and I'm not going to talk too much about how you generate the graphs and whatever. 12 00:01:34,610 --> 00:01:40,990 I want to show you a lot of examples on how to look at datasets and how you configure graphs to get the most out of it. 13 00:01:41,510 --> 00:01:58,230 As part of the talk, I'm going to show you a tool that I have up on SourceForge, which is called Afterglow, which is a very simple Perl script that transforms CSV files into a graph or language file, so you can go home and play with these things yourself, 14 00:01:58,230 --> 00:02:00,010 and the tool is actually also on the CD. 15 00:02:02,570 --> 00:02:06,510 I'm going a little faster in the beginning so we can catch up for the time I lost already. 16 00:02:06,510 --> 00:02:12,130 One thing, IP addresses and host names, you're going to see in all these graphs, and the descriptions are obfuscated or changed. 17 00:02:12,130 --> 00:02:17,410 The addresses are completely random and any resemblance with well-known addresses or host names are purely coincidental. 18 00:02:18,510 --> 00:02:19,390 All right. 19 00:02:19,690 --> 00:02:22,490 So let's do a little bit of an introduction here. 20 00:02:22,490 --> 00:02:23,970 Why would you want to look at these graphs? 21 00:02:23,970 --> 00:02:25,770 I guess this is a pretty good example already. 22 00:02:25,770 --> 00:02:29,330 Do you want to look at the left-hand side, or do you want to look at the right-hand side? 23 00:02:32,670 --> 00:02:36,850 Well, even if I made the font bigger on the left-hand side, it's probably still not very fun. 24 00:02:37,330 --> 00:02:38,050 All right. 25 00:02:38,670 --> 00:02:40,410 Why do you want to use event graphs? 26 00:02:40,870 --> 00:02:43,090 I guess I already motivated that quite well. 27 00:02:43,090 --> 00:02:45,090 I'd rather look at the graphs. 28 00:02:45,090 --> 00:02:47,270 They look just much nicer. 29 00:02:48,520 --> 00:03:05,390 If you actually dig into it a little more and you start looking how to configure these things, you will see pretty quickly that you can visualize different properties in those graphs, and it's very easy to see certain aspects of a log file very quickly and certain things that are important to you. 30 00:03:05,390 --> 00:03:13,690 You don't have to wade through ten thousands of lines of logs to figure out whether there is anyone sending a lot of e-mails to someone else. 31 00:03:14,590 --> 00:03:33,310 If you use this in a security operating center or in a 724 response team or something, if you have graphs, they're a great way of giving you situational awareness or real-time view on things that are happening, and you can quickly react on them. 32 00:03:33,690 --> 00:03:41,330 We have a lot of customers that are starting to use this in their SOCs, and it doesn't just look pretty on their big screens, but it's actually really useful. 33 00:03:42,010 --> 00:03:43,870 It's actually also kind of an interesting story. 34 00:03:43,870 --> 00:03:52,750 I was at one customer, and they had all these big displays, and they were empty, and I was like, wow, why do you have these nice graphs here on the analyst screens? 35 00:03:52,750 --> 00:03:54,870 Why don't you have them up there on the big screens? 36 00:03:55,110 --> 00:04:00,830 They were like, whoa, no, no, no, we tried that for a while, but then the boss came in, and he saw all these things on the screen. 37 00:04:00,830 --> 00:04:01,650 He was like, oh, what is this? 38 00:04:01,650 --> 00:04:02,050 What's that? 39 00:04:02,050 --> 00:04:02,430 What's this? 40 00:04:02,430 --> 00:04:02,850 What's that? 41 00:04:02,850 --> 00:04:03,950 So we had to explain all this. 42 00:04:03,950 --> 00:04:04,950 We took them off. 43 00:04:08,140 --> 00:04:12,800 All right, and one other thing, another story that I had with these graphs, which actually two other stories. 44 00:04:12,800 --> 00:04:18,980 One is, one customer uses these graphs to communicate with other teams, so you have the security analysts. 45 00:04:19,000 --> 00:04:27,460 They look at the log files and all that, and then they go back to the operators or the operational folks, and they're like, guys, you have to do something about this machine. 46 00:04:27,460 --> 00:04:29,200 It's talking to all the others in the network. 47 00:04:29,280 --> 00:04:37,020 And in the past, they were handing them log files to prove that it's that way, and the operators were like, whatever, we don't understand what this is. 48 00:04:37,020 --> 00:04:42,120 But now they started giving them graphs, and the operators were like, oh, my God, my machine is talking to all these? 49 00:04:42,120 --> 00:04:43,720 Okay, we're going to take care of this. 50 00:04:44,200 --> 00:05:00,380 And even worse, I had one customer that actually just told me this week that he used the graphs to analyze some data, and he found some stuff, and one of the guys got fired just because he used the graphs and found out that one of the guys was not doing his work. 51 00:05:04,090 --> 00:05:09,130 All right, so I came up with, like, two big categories of when to use these event graphs. 52 00:05:09,130 --> 00:05:18,330 One is in a real-time monitoring scenario, where you have, like, a screen app that looks real-time at certain properties of your data file, and I'm going to show you some examples on what I mean by that. 53 00:05:18,590 --> 00:05:20,950 It gives you great awareness of things that are happening. 54 00:05:20,950 --> 00:05:26,270 You can configure these things to, for example, look at transactions going to your financial server. 55 00:05:26,270 --> 00:05:29,910 So right away, if someone connects to your financial service, you will know. 56 00:05:30,010 --> 00:05:31,670 And then you can take it a level further. 57 00:05:31,670 --> 00:05:37,930 You can say, well, I know that these machines are okay to connect to my financial servers, so don't show me those, but show me everything else. 58 00:05:37,930 --> 00:05:41,270 So as soon as something pops up there, you know you've got to look into it. 59 00:05:41,610 --> 00:05:53,790 The other thing is, if you take this in a more forensic or historical way, you can, you grab a data set, you run it through some tools, and you look at the graphs on what is, on certain properties, and you figure out what's going on. 60 00:05:55,390 --> 00:05:57,310 Let me mention quickly some related work. 61 00:05:57,310 --> 00:06:02,410 There's some classics from, he did all these kind of things. 62 00:06:02,410 --> 00:06:11,450 He tried to figure out whether there are certain ways to visualize properties of data, and he took a lot of dimensions and put them all in one graph. 63 00:06:11,830 --> 00:06:19,390 What I'm going to do is, I take a lot of dimensions too, and I'm always going to stay with event graphs, so like node-to-node connections. 64 00:06:19,430 --> 00:06:27,610 I'm not going to use any of the parallel coordinates or the right-hand-to-bottom thing that I have no idea what it shows there. 65 00:06:28,310 --> 00:06:31,170 Then Ehrbacher, he also, he actually went even further. 66 00:06:31,170 --> 00:06:41,870 He tried to go even more multidimensional and map everything into these nodes, so there's like size, there's shape, there's length of these spirals going out, and so on. 67 00:06:42,810 --> 00:06:46,990 Then Greg Conti, you probably have seen his talk, I hope. 68 00:06:47,750 --> 00:06:55,910 He mainly focuses on different ways to visualize TCP dump or packet locks, and very interesting. 69 00:06:57,510 --> 00:06:59,570 Then Envision IP is something I found. 70 00:06:59,570 --> 00:07:10,530 What you see up there is a grid of, I think it's 8x8 or 64x64, and they just map IP addresses in there, so you see certain clusters of things going on. 71 00:07:10,530 --> 00:07:16,630 The nice thing about this project is, there's a lot of interactivity, so you can start playing with the data, you can zoom in and so on. 72 00:07:17,030 --> 00:07:21,910 And then finally, a few of you might know Showkey from Steve Barry. 73 00:07:22,110 --> 00:07:23,490 It's basically a scatterplot. 74 00:07:23,490 --> 00:07:28,530 You can define three different scatterplots, and then you have a 3D of that. 75 00:07:28,810 --> 00:07:40,690 So on the top right, it's not a very nice picture, but on the top right it's a three-dimensional output, so you can say one axis is your source IP, one axis is destination IP, and the other one is the destination port. 76 00:07:41,630 --> 00:07:43,010 All right, let's move on. 77 00:07:44,310 --> 00:07:45,950 How do you generate an event graph? 78 00:07:45,950 --> 00:07:46,490 Pretty easy. 79 00:07:46,490 --> 00:07:49,190 You have a log file that a device is generating. 80 00:07:49,550 --> 00:07:52,150 You take a parser. 81 00:07:52,930 --> 00:07:54,170 Why do you take a parser? 82 00:07:54,170 --> 00:08:01,970 Well, you need to know where in my log event or my log file, where's the source IP, where's the destination IP, where's the event name, where's the port? 83 00:08:01,970 --> 00:08:07,550 So you parse that, you get the syntax right, and then you visualize that in an event analyzer. 84 00:08:08,250 --> 00:08:12,330 If you have an event, let's look at this Nord event here. 85 00:08:12,870 --> 00:08:20,550 You don't have to be really able to read this, but what I want to show here is in an event graph you can have different configurations. 86 00:08:20,830 --> 00:08:26,210 The normal event graph that you will always see is like who is talking to whom, so you have source IP, destination IP. 87 00:08:26,330 --> 00:08:28,870 But you can take this a lot further. 88 00:08:28,870 --> 00:08:40,730 You can say, well, on the right top you want to have the source IP, destination IP, and destination port, so you will see someone port scanning, for example, if there's a fan with destination ports. 89 00:08:40,830 --> 00:08:45,290 So you can configure these things in pretty wild ways, and I'm going to show you some of them. 90 00:08:46,990 --> 00:08:51,210 Now let's have a sneak preview of Afterglow. 91 00:08:51,450 --> 00:08:54,010 I apologize because I don't use my laptop running it. 92 00:08:54,010 --> 00:08:55,010 I can't show you the demo. 93 00:08:55,010 --> 00:08:59,130 It's not very fun anyways. 94 00:08:59,130 --> 00:09:02,070 I will show you the output anyway, so it doesn't really matter. 95 00:09:02,130 --> 00:09:03,510 So what is Afterglow? 96 00:09:03,510 --> 00:09:07,310 Afterglow is not a security information management tool. 97 00:09:07,330 --> 00:09:16,710 It's just a script which takes CSV files, so you have to somehow from your logs generate a CSV file, comma, separated values. 98 00:09:16,830 --> 00:09:24,130 And then you take Afterglow, the tool that I'm going to show you, and you transform the CSV files into a graph language file. 99 00:09:24,750 --> 00:09:32,530 And then you take a grapher tool, and right now I'm supporting two of them, that takes this graph language and generates the graph. 100 00:09:32,530 --> 00:09:34,210 And that's not something I did. 101 00:09:34,210 --> 00:09:40,130 I just transformed the CSVs into graph files, but I'm going to show you some cool things that we implemented there. 102 00:09:41,490 --> 00:09:55,830 So how you can use this thing is you basically cap the input file, you pipe it to Afterglow, you give it a color properties file, which I'm going to introduce a little later, and then you can use the AT&T Graphverse library, .neto, I'll give you references in a second, 103 00:09:55,830 --> 00:09:57,670 and you generate a GIF file. 104 00:09:58,230 --> 00:10:06,830 The color properties file is basically just you tell it the source nodes are red, for example, the event nodes are green, and the target nodes are blue. 105 00:10:06,830 --> 00:10:09,370 And I'm going to show you how to really play with this stuff. 106 00:10:09,370 --> 00:10:15,970 We extended this quite a bit so you can have colors depending on the values in the log files. 107 00:10:17,250 --> 00:10:18,850 Big shout out to Christian. 108 00:10:18,850 --> 00:10:22,550 Thanks a lot for building the foundation of this tool. 109 00:10:23,010 --> 00:10:32,170 He built me the whole Afterglow profile, the basics and everything, and I just went in and made some small changes, but thanks a lot, Chris. 110 00:10:34,330 --> 00:10:36,390 Let's look at examples. 111 00:10:37,750 --> 00:10:42,790 I mentioned in the beginning I kind of have two use cases or two big categories of things to look at. 112 00:10:42,790 --> 00:10:47,230 One is what I call situational awareness or real-time monitoring. 113 00:10:51,120 --> 00:10:55,300 So this is one of the views that I saw at a customer. 114 00:10:55,600 --> 00:11:02,560 Basically you see here is six independent screens of different properties of your data that you want to look at. 115 00:11:02,920 --> 00:11:14,280 So here the configuration is such that on the top left the customer wanted to see just the compromise attempts, and don't ask me how to filter for just those events. 116 00:11:14,280 --> 00:11:16,080 That's a whole different story. 117 00:11:16,840 --> 00:11:30,280 But if you have a way to say these are my malware events, these are my DOS events, these are my scan events, if you can separate them by something, then you can draw all these screens and you can look at these screens and see what's going on. 118 00:11:30,500 --> 00:11:36,620 So what this customer is actually also doing, whenever an analyst starts his shift, he resets all the graphs so they're empty. 119 00:11:36,620 --> 00:11:47,340 So over time, nodes start popping up and they look at them and they're like, okay, this is fine, and they might filter them out even, or they just leave them there and say, okay, I know about this. 120 00:11:47,420 --> 00:11:52,000 And the more things that show up, the more they go along in their workflow and work with these events. 121 00:11:52,220 --> 00:12:01,080 So I have a feeling this is going to be a great tool, especially in a SOC or an analysis environment, to kick off your workflow. 122 00:12:01,080 --> 00:12:09,440 You look at these things and you take it on and you start building filters to filter out the things you don't know about, and you just see the outliers here in these screens. 123 00:12:11,300 --> 00:12:22,760 The second big group of use cases is forensic analysis or historical analysis, where you take a log file that you have and you visualize it. 124 00:12:24,520 --> 00:12:27,100 Afterglow can generate two kinds of outputs. 125 00:12:27,100 --> 00:12:34,760 One is for the AT&T graphics library for .neto, and the other one is for LGL, the large graph library. 126 00:12:35,420 --> 00:12:37,760 I will give a pointer to that in the end as well. 127 00:12:37,760 --> 00:12:40,360 This thing is generating three-dimensional graphs. 128 00:12:40,360 --> 00:12:43,860 You can actually generate VRML output and you can fly through these things. 129 00:12:44,440 --> 00:12:52,740 I didn't prepare a demo because it's kind of fun to fly through it, but the analysis value is not really high or the value you get out of it. 130 00:12:52,740 --> 00:12:56,020 It's really cool to fly through it and see that, oh, this guy sent an email. 131 00:12:56,020 --> 00:12:56,480 To whom? 132 00:12:56,480 --> 00:12:57,520 Oh, over there. 133 00:12:57,520 --> 00:12:58,120 Okay. 134 00:12:59,060 --> 00:13:03,720 It looks great, and I apologize for the colors here, but this is really how it looks. 135 00:13:04,220 --> 00:13:05,900 Anyway, let's go back to 2D. 136 00:13:07,000 --> 00:13:14,500 This is a very simple example, one of about 12 that I'm going to show, of how to analyze a log file. 137 00:13:14,600 --> 00:13:26,820 So what I had here is, I believe, a snort log or a web log or something, and I said, well, show me what is happening. 138 00:13:26,820 --> 00:13:32,800 So the event name, the destination IP are the blue ones, and then the destination ports are the white nodes. 139 00:13:32,880 --> 00:13:37,060 So what you see here, all the blue circles are my web servers. 140 00:13:37,820 --> 00:13:41,840 The white boxes are the target ports that they're being connected to. 141 00:13:41,840 --> 00:13:44,620 So you see it's 80 and 443, so that's totally legitimate. 142 00:13:44,620 --> 00:13:49,640 Only HTTP and HTTPS traffic is going to my servers, so I probably don't have to worry about that. 143 00:13:49,640 --> 00:14:01,600 If I would see a node there, like port 21, and someone connects to my web servers with FTP, maybe that's okay, but if I'm not running FTP in my environment and that shows up here, I probably want to investigate that. 144 00:14:01,720 --> 00:14:07,600 So it's a very quick way to see whether traffic is really what it's supposed to be. 145 00:14:07,600 --> 00:14:11,460 Almost same configuration. 146 00:14:11,460 --> 00:14:14,520 Here you see source IP, destination IP, destination port. 147 00:14:14,520 --> 00:14:18,680 So this would potentially show you port scans or network scans. 148 00:14:18,680 --> 00:14:24,660 Here you see a lot of machines connecting to a lot of other ones, all on the same port. 149 00:14:25,700 --> 00:14:28,860 Down here, this white node here, is the port. 150 00:14:28,860 --> 00:14:30,200 So they all connect to the same port. 151 00:14:30,200 --> 00:14:41,600 Well, it could be that a worm is probing you, but it could also be that you have a web server, or a lot of web servers, and a lot of machines are connecting to those. 152 00:14:41,600 --> 00:14:50,260 So it might be okay, but it might be a scan, and depending on what the destination port is, you might figure out what's going on in this case. 153 00:14:50,520 --> 00:15:06,760 But you can certainly tell, well, this is all similar behavior here on this fan, and then actually if you see on the bottom left here, there's some little ones, it's probably worth investigating what those are, and probably those are the outliers that you are interested in. 154 00:15:09,200 --> 00:15:11,280 Let's keep this configuration of the graph. 155 00:15:11,280 --> 00:15:14,380 This is just a little bit of a different dataset and a different time. 156 00:15:15,120 --> 00:15:16,840 It looks much prettier. 157 00:15:18,260 --> 00:15:21,980 Let's zoom in on the bottom part here. 158 00:15:22,460 --> 00:15:30,020 What this is showing you is that there are two machines, the red nodes, that are connecting to about 15 other ones, and still on the same port. 159 00:15:30,260 --> 00:15:40,020 So again, in your environment, do you really have all these other 15 or 20 machines serving this one service, and exactly two machines are connecting to them? 160 00:15:40,040 --> 00:15:49,500 So you can go and figure out what all these machines are, and hopefully they are really the ones that are serving this service, and then you can figure out whether these other two are really allowed to connect to this. 161 00:15:50,060 --> 00:15:53,460 Let's have a look at this weird thing over here. 162 00:15:55,240 --> 00:16:14,560 What you see here is, probably you can't read this, but there is a node here that says 21 FTP, so one of the properties of this graph is that the white nodes show up exactly once per target port, so everything that happens on port 21 is going to show up around that thing there. 163 00:16:14,940 --> 00:16:23,520 So again, I can say, wow, I have about six FTP servers, and about five people are connecting to them. 164 00:16:23,520 --> 00:16:28,740 So the FTP servers would be the blue nodes, and then the red ones that connect to them would be the people using it. 165 00:16:28,760 --> 00:16:31,380 So, well, is this really something I have in my environment? 166 00:16:31,380 --> 00:16:32,380 Is this allowed? 167 00:16:32,380 --> 00:16:35,920 And why are only three or five people connecting to them? 168 00:16:35,920 --> 00:16:43,400 Is that some kind of a process that, I don't know, maybe AIX that transfers log files over, something weird like that? 169 00:16:43,820 --> 00:16:45,320 Or is this something malicious? 170 00:16:45,320 --> 00:16:47,260 Is anyone having a ware server or whatever? 171 00:16:48,980 --> 00:16:52,320 If you interpret these graphs, you have to be a little careful. 172 00:16:53,060 --> 00:16:54,700 I still have the same configuration. 173 00:16:54,820 --> 00:16:58,480 Destination IPs are the blue ones, and the white ones are the destination ports. 174 00:16:58,800 --> 00:17:03,840 Well, if you look at this on the left-hand side, I would say, wow, it's a port scan. 175 00:17:04,360 --> 00:17:05,960 Well, is it really a port scan? 176 00:17:09,410 --> 00:17:12,630 I have a book for the one that gets this right. 177 00:17:12,630 --> 00:17:14,290 What is the left-hand side? 178 00:17:16,230 --> 00:17:17,830 No, not voice over IP. 179 00:17:20,860 --> 00:17:21,940 Who said that? 180 00:17:23,120 --> 00:17:23,920 Over there. 181 00:17:23,920 --> 00:17:24,840 You got the book. 182 00:17:24,840 --> 00:17:26,240 It's the source ports. 183 00:17:26,300 --> 00:17:35,700 The log file reports such that the source port is really the destination port and the other way around. 184 00:17:35,700 --> 00:17:41,640 So when you analyze these things, be careful and make sure you're really looking at what you're looking at. 185 00:17:41,640 --> 00:17:45,460 So why do I know are the source ports and the destination ports? 186 00:17:45,460 --> 00:17:47,660 Well, they're in the 50,000 range. 187 00:17:48,100 --> 00:17:50,100 They're almost contiguous. 188 00:17:50,780 --> 00:17:53,820 And I mean, who is scanning me on these high ports? 189 00:17:53,820 --> 00:17:55,580 We're not a capture the flag. 190 00:17:55,580 --> 00:17:59,520 There, they might scan you on those high ports because services are running all over. 191 00:17:59,560 --> 00:18:02,900 But here, there's really no sense why someone would scan you up there. 192 00:18:02,900 --> 00:18:05,420 There's probably nothing there listening anyway. 193 00:18:05,520 --> 00:18:07,560 So be careful when you analyze these things. 194 00:18:07,560 --> 00:18:10,020 Sometimes it's port scan, sometimes it's not. 195 00:18:10,020 --> 00:18:14,240 I'm always, my first impression is always, well, false positive. 196 00:18:14,240 --> 00:18:16,320 Let's really figure out whether that's true. 197 00:18:19,270 --> 00:18:20,930 Okay, let's get fancy here. 198 00:18:21,810 --> 00:18:23,530 This is a firewall log. 199 00:18:23,530 --> 00:18:27,970 Basically, I'll show you passes and blocks from a PF firewall. 200 00:18:29,790 --> 00:18:36,690 If I would just visualize source IPs, destination IPs, I would see big clouds of things. 201 00:18:36,710 --> 00:18:38,750 Well, I want to see a little more in this graph. 202 00:18:38,750 --> 00:18:45,330 So what I did is I have source port, rule number, that's the squares, and then destination IP. 203 00:18:45,330 --> 00:18:54,250 So you will see I'm connecting to some other machine and rule number 100 is responsible for letting me in or blocking me. 204 00:18:55,070 --> 00:19:03,530 What I did then with Afterglow is to configure my nodes such that all the external IPs are going to show up as orange. 205 00:19:03,530 --> 00:19:07,630 I'm going to show you how to configure Afterglow to get exactly this graph. 206 00:19:07,890 --> 00:19:10,570 So the external machines are orange. 207 00:19:10,650 --> 00:19:12,330 My internal machines are green. 208 00:19:12,330 --> 00:19:14,170 And you see I only have a few of them. 209 00:19:14,250 --> 00:19:16,950 And then the rules are the blue boxes. 210 00:19:17,350 --> 00:19:22,550 And then further what I said, I wanted to see whether a connection is incoming or outgoing of my network. 211 00:19:22,550 --> 00:19:31,170 So I configured the edges such that outgoing connections are blue, that's the majority here, and incoming is red. 212 00:19:31,650 --> 00:19:34,290 And I just realized it's probably exactly the other way around. 213 00:19:34,290 --> 00:19:39,470 So I hope there's more incoming than outgoing because these are web servers that are servicing some stuff. 214 00:19:39,870 --> 00:19:46,070 Anyway, so you see big clouds of things that are showing similar behavior. 215 00:19:46,070 --> 00:19:50,690 So down here there's this one rule that is responsible for letting in a whole lot of traffic. 216 00:19:51,290 --> 00:19:53,150 And probably that's fine. 217 00:19:53,390 --> 00:19:54,970 There's another one up here. 218 00:19:55,510 --> 00:20:01,910 You can, if you know the rule set, if you know your ACLs, you can go and figure out whether it's really what it's supposed to do. 219 00:20:02,130 --> 00:20:05,990 But you can also figure out some things that are kind of weird. 220 00:20:06,850 --> 00:20:09,450 You have this whole section in the middle. 221 00:20:09,450 --> 00:20:17,750 You probably want to zoom in on those and filter out all the others and just look at these things and figure out whether your rule set is really doing what it's supposed to do. 222 00:20:18,390 --> 00:20:25,390 So the next steps would be, so the first is like overview, general impression, what is going on, is there anything really sticking out. 223 00:20:25,390 --> 00:20:31,190 And then my next steps would be, I would visualize the firewall blocks of outgoing traffic. 224 00:20:31,410 --> 00:20:34,330 Why would internal machines connect outside and be blocked? 225 00:20:34,330 --> 00:20:40,090 So that's either a misconfiguration of the internal box or the firewall rule set is wrong. 226 00:20:40,390 --> 00:20:44,090 Then the second one would be , I would probably look at blocks of incoming traffic. 227 00:20:44,090 --> 00:20:46,670 I want to know who's probing me , who's poking around. 228 00:20:47,090 --> 00:21:01,090 And if you have a live system that can look at these things, you can put all these attacker machines maybe on a blacklist and if you see them again and they're successful coming in, you might actually take some action and say, whoa, now someone probed me and now he's coming in. 229 00:21:01,430 --> 00:21:02,870 And then you can take this further. 230 00:21:02,870 --> 00:21:06,270 Third step would be to visualize the passes of outgoing traffic. 231 00:21:06,270 --> 00:21:08,410 Is this really what should be leaving the network? 232 00:21:08,470 --> 00:21:13,170 Maybe you find someone FTPing stuff out that you really don't want to FTP out. 233 00:21:13,210 --> 00:21:20,010 Maybe you realize that you're running peer-to-peer clients in your internal network and they're connecting out, anything like that. 234 00:21:20,350 --> 00:21:25,810 And you can probably take a few more steps to analyze the data set, but I guess you get the idea. 235 00:21:27,090 --> 00:21:30,450 So this one is probably the wackiest graph I have. 236 00:21:30,530 --> 00:21:35,310 It's totally going away from source IPs and destination IPs. 237 00:21:35,310 --> 00:21:38,370 What I did here is I wanted to debug my firewall rule set. 238 00:21:38,370 --> 00:21:51,770 So it's still the same data set as before, but this time what I show you is the red nodes are the rule number or the ACL number, then the destination ports, and then you see the action taken, pass or block. 239 00:21:51,930 --> 00:21:57,030 So just by virtue of firewall rule configurations, you will see two distinct graphs. 240 00:21:57,110 --> 00:22:01,390 One on the left-hand side is all the passes and the right-hand side is all the blocks. 241 00:22:02,250 --> 00:22:13,330 So here you can see, if you take one of these blue nodes, for example, port 53 here, you see that this one rule is responsible for letting traffic in. 242 00:22:13,430 --> 00:22:23,270 Well, most is okay, maybe, probably, hopefully, but you have other ones where there are two nodes or three nodes connected to the same rule. 243 00:22:24,690 --> 00:22:30,210 Like over here, for example, there are three ports being blocked by this one rule. 244 00:22:30,210 --> 00:22:31,330 It might be okay. 245 00:22:31,330 --> 00:22:32,330 It might not. 246 00:22:32,330 --> 00:22:44,490 If you see something like this on the other side, that one rule passes traffic from multiple ports, maybe that's the way you configured it, but this is certainly a great way of analyzing and debugging your firewall rule set. 247 00:22:46,630 --> 00:23:03,730 This is another example where I took a TCP dump log, and what I visualized here is I take the region of where the source IP resides, so you can go on the web, there are databases that show you what the location codes are for the IP addresses. 248 00:23:03,730 --> 00:23:06,890 They're not very accurate, but in a lot of cases, they're actually pretty good. 249 00:23:07,490 --> 00:23:14,310 So I show you the source region, where it's coming from in the world, then I show you the source MAC address. 250 00:23:14,370 --> 00:23:15,970 So this is a TCP dump log. 251 00:23:15,970 --> 00:23:22,850 The source MAC address is going to be the last hub, probably my router, that let the traffic into my network. 252 00:23:22,850 --> 00:23:25,450 And then I show the destination IP. 253 00:23:25,470 --> 00:23:36,370 So what I see here, I have four boxes, the four white ones, that are my internal machines, and interesting enough, I have two entry points into my network. 254 00:23:36,370 --> 00:23:38,850 I have two source MAC addresses showing up. 255 00:23:38,850 --> 00:23:42,670 And then I have these two fans in the middle part. 256 00:23:42,890 --> 00:23:50,110 What this is showing me is I probably either have a redundant router setup, or maybe I have a load balancer. 257 00:23:50,330 --> 00:23:57,090 And what is interesting here is that all these machines up here, they're connecting through this guy. 258 00:23:57,090 --> 00:24:01,450 All these guys here are coming through both of them, and all these down here come from the other one. 259 00:24:01,450 --> 00:24:08,890 So maybe this is a regional load balancer, that everything on one side of the world takes this entry point, everything on the other side is using that entry point. 260 00:24:08,890 --> 00:24:18,070 So if that's really the case, a great way to analyze this log is this graph here, and you can actually figure out why these two in the middle are coming in for both of them. 261 00:24:19,430 --> 00:24:30,710 And this is just a zoom, where you see the one server, and here it's like, you can't even read it below the node here, but it's Sand Hill or whatever in the United States. 262 00:24:35,510 --> 00:24:41,170 Graphs like these are a lot of times indications for warms, but again, be careful. 263 00:24:41,250 --> 00:24:47,550 There's a nice fan here of source IPs using the same event going to a few destination IPs. 264 00:24:47,610 --> 00:24:58,930 Well, it could be, again, it could be a warm that is using this one event to try to get into a network, but it might again be a web server serving just legitimate files. 265 00:24:59,150 --> 00:25:07,210 What you have to do in this case, you have to figure out what is this event name here that connects all these dots, and this might give you an indication of a warm or not. 266 00:25:07,330 --> 00:25:12,070 If you're visualizing a snort log file and this talks about warm, well, duh, it's probably a warm. 267 00:25:12,210 --> 00:25:25,570 But if you visualize a TCP dump log file, like an ICMP port unreachable might be an indicator for a warm trying to probe UDP ports and you get ICMP port unreachables back. 268 00:25:25,770 --> 00:25:29,330 So you have to investigate a little bit to figure out whether it's really a warm. 269 00:25:29,330 --> 00:25:49,410 I was playing Capture the Flag actually this year and last year, and I was trying to figure out the challenge really for the analysis team on Capture the Flag is that all the traffic hitting your web server or your servers seems to be coming from the same IP address, 270 00:25:49,410 --> 00:25:59,210 and there is malicious traffic of the other teams hacking you, but there is also benign traffic from the score bots trying to figure out whether your services are still up. 271 00:25:59,290 --> 00:26:01,810 So blocking all the traffic is not working. 272 00:26:01,810 --> 00:26:12,890 So you have to figure out a certain way of distinguishing these two types of traffic, and you can't do it by the source IP, which you would normally do in your corporate environment. 273 00:26:13,350 --> 00:26:21,470 So I started visualizing this log file of TCP dump, and I just wanted to have a general feeling for what's going on. 274 00:26:21,470 --> 00:26:32,610 So what you see here, the dark blue ovals are destination ports that are below 1024, normally services that are offered to the outside world. 275 00:26:32,750 --> 00:26:44,830 Then destination ports higher than 1024 are light blue, and then you see here what I just explained, the red node, this is what all the traffic that's coming in is coming from this one IP. 276 00:26:44,830 --> 00:26:59,450 And then I configured it such that our machines, the internal targets are green, other teams' targets are brown, and then internal sources are yellow, and internal targets are the boxes. 277 00:26:59,550 --> 00:27:18,050 So what you see here, we have two servers, while there's a third one showing up down here, this was actually not a server, but I think that was one of the machines on our network that someone connected to, and that could have been one of our team members connected to someone else's machine. 278 00:27:19,270 --> 00:27:26,950 And then the services that we offered were D6, and it was all kinds of weird things. 279 00:27:27,810 --> 00:27:33,630 I don't even remember anymore what it was last year, port 9999 or something, and there was some mud running there. 280 00:27:33,770 --> 00:27:37,830 But this gives you a very quick way of looking at what is happening in this network. 281 00:27:38,850 --> 00:27:47,150 Well, I took it a little further, and I had an idea, or a few people had an idea, I said, well , maybe it's a TTL of the traffic coming in. 282 00:27:47,150 --> 00:27:50,910 Maybe you can distinguish the good traffic from the bad traffic by looking at the TTL. 283 00:27:51,030 --> 00:27:57,650 So well, what I did is I said, okay, show me the source IPs, the destination IPs, and then the TTLs of that traffic. 284 00:27:58,090 --> 00:28:11,890 So you see here, I have eight TTLs showing up, eight distinct TTLs, and I'm going to look at the everything from 35, 50, 52, 60, 62, 63, 64, 128, and 146. 285 00:28:13,470 --> 00:28:20,510 Some of them are showing up if my teammates would connect out, so I could filter them out. 286 00:28:20,510 --> 00:28:22,090 I was not interested in those. 287 00:28:22,090 --> 00:28:28,390 I'm just interested in the ones that happen of the traffic coming in from this one evil IP. 288 00:28:30,430 --> 00:28:32,150 Let's go back quickly again. 289 00:28:32,150 --> 00:28:40,810 And then I figured out, well, I still can't quite tell what is okay and what's not, so I had to reconfigure my graph and look at some other properties. 290 00:28:40,810 --> 00:28:44,630 So what I did is I said, well, I'm not really interested in the IPs. 291 00:28:44,630 --> 00:28:57,510 I know everything is coming from this one IP that I'm interested in, so it only showed me the traffic from that guy, but I'm interested in the destination ports, the TCP flags, and the TTLs. 292 00:28:57,510 --> 00:29:05,930 So I wanted to see, like, the SYNs, the SYNX, the pushes and whatever, and their TTLs, and what port they went to. 293 00:29:06,530 --> 00:29:14,130 So this yields this graph here, and in addition to that, that was not quite enough to figure out what's really going on. 294 00:29:14,130 --> 00:29:15,690 I need to have the counts. 295 00:29:15,730 --> 00:29:17,010 I'm going to lose everyone here. 296 00:29:17,010 --> 00:29:23,510 I need the counts on all these nodes, so how many times did certain flags show up? 297 00:29:23,790 --> 00:29:26,090 So let's take one example here. 298 00:29:26,090 --> 00:29:29,770 I have port 80 here. 299 00:29:29,790 --> 00:29:37,150 It showed up 2,600-and-something times, and these are all the flags that showed up with the different TTLs. 300 00:29:37,150 --> 00:29:50,810 So there were four TTLs, one, two, three, four, that were used when someone connected these ports here, and here are the TCP flags that were being used for this traffic. 301 00:29:51,270 --> 00:29:52,510 So I had a theory. 302 00:29:52,510 --> 00:29:59,530 I said, well, I'm looking at all the connection establishments, all the SYNs, and I want to see how many times they happen. 303 00:29:59,610 --> 00:30:06,450 The score about traffic is probably going to all the services, like in sequence, or probably the same amount of time. 304 00:30:06,450 --> 00:30:18,730 So I will see the same amount of SYNs going to each of the destination ports, and if the theory about the TTLs is right, they all use the same TTL. 305 00:30:19,350 --> 00:30:26,570 And it turned out that we were actually able to figure out the TTLs, and for the game last year that way, and we filtered all the other traffic. 306 00:30:26,590 --> 00:30:29,530 But it was a little late in the game, so we still didn't win. 307 00:30:33,080 --> 00:30:36,560 All right, so I have some other logs here. 308 00:30:37,420 --> 00:30:49,860 At some point I was interested in email logs, and I wrote a little send mail parser, and that's actually part of the afterglow scripts that you can have on the CD or you can get from SourceForge. 309 00:30:50,220 --> 00:30:55,010 The problem in send mail, I'll mention that in the end I guess, there are always two entries. 310 00:30:55,110 --> 00:31:07,170 It says from so-and-so, and then there's a message ID, and then the next log entry says to so-and-so, and there's a message ID to link those two events together, or just two log entries. 311 00:31:07,190 --> 00:31:15,330 So I had to write a tool to actually take them, to put them together, and then output a CSV file, and all that stuff, and the result is this one here. 312 00:31:15,330 --> 00:31:19,030 So I'm just showing who is talking to whom by email. 313 00:31:19,370 --> 00:31:30,790 And then I said, well, what's probably interesting is to see everything that's coming from my domain, the one that I'm really hosting on that server, going to people outside and what is coming into my domain. 314 00:31:30,790 --> 00:31:39,590 So I colored the whole graph that way, and if we look at a zoom up here, you see that these people here are quite popular. 315 00:31:40,330 --> 00:31:46,950 A lot of people from outside, these red nodes, are sending emails to this guy that was hosting on this server. 316 00:31:47,750 --> 00:31:49,750 Here you just see the popular people. 317 00:31:49,990 --> 00:31:50,730 It might be interesting. 318 00:31:50,730 --> 00:31:51,050 I don't know. 319 00:31:51,050 --> 00:31:52,250 Maybe you want to get to know them. 320 00:31:53,090 --> 00:31:57,270 But I was really interested in whether I'm running an open relay. 321 00:31:58,250 --> 00:32:00,530 I'm not, but I'm pretending I will. 322 00:32:01,750 --> 00:32:10,950 So what I did is I said, well, just show me everything on the outside. 323 00:32:10,950 --> 00:32:20,050 All my domains or all the, like, rafi.ch or whatever my domain is, don't grade them out and just show me the rest. 324 00:32:20,050 --> 00:32:24,130 So you see there's still some others showing up here. 325 00:32:24,210 --> 00:32:30,250 So what I did then was I used afterglow to just eliminate all those. 326 00:32:30,250 --> 00:32:31,130 I'll show you how to do that. 327 00:32:31,130 --> 00:32:32,970 You can set nodes and invisible. 328 00:32:33,330 --> 00:32:45,130 And here I realized, oh, my God, there are actually people using my server, like this guy here, to send emails to people that are not hosted on my server. 329 00:32:45,670 --> 00:32:48,910 So I'm probably running an open relay. 330 00:32:51,610 --> 00:32:57,190 Well, some other things I was interested in, well, am I running, or is anyone spamming? 331 00:32:57,210 --> 00:32:59,770 And , well, probably, hopefully I see something in here. 332 00:32:59,770 --> 00:33:06,710 Otherwise my graph would probably not be very, my configuration would not be very good because probably on every mail server you'll find spam. 333 00:33:07,270 --> 00:33:11,470 So what I did is in the email log you see the size of the messages. 334 00:33:11,470 --> 00:33:15,530 So I just visualized who is getting what sizes of messages. 335 00:33:16,010 --> 00:33:21,110 And I was interested in just the ones that are above 10,000, all the others I'm not interested in. 336 00:33:21,110 --> 00:33:26,970 So you still see that, for example, down here there are multiple recipients that get the same size messages. 337 00:33:26,970 --> 00:33:41,110 So that means that someone probably is sending emails to a few people and this email is pretty big and it's probably the same one because they have quite a high size, or the same size. 338 00:33:42,290 --> 00:33:46,990 So that would be one indication of maybe there's something spam going on. 339 00:33:46,990 --> 00:33:48,170 Probably not. 340 00:33:48,230 --> 00:33:55,010 Well, another way of looking at whether you have spam is look at the number of recipients in an email. 341 00:33:55,010 --> 00:33:57,330 You can send emails to multiple people, right? 342 00:33:57,330 --> 00:34:02,070 So the number of recipients in the log file would indicate how many people got this email. 343 00:34:02,070 --> 00:34:04,710 And what I said, well, I want to see everything that's bigger than two. 344 00:34:05,070 --> 00:34:11,690 And all the things where someone is just sending to two other people, I don't want to see. 345 00:34:11,690 --> 00:34:20,390 So I have this omit threshold thing in Afterglow, I will explain it in a second, where I can drop out nodes that only connect to one other or two others. 346 00:34:20,390 --> 00:34:23,990 So I can filter out the things that are not very interesting. 347 00:34:23,990 --> 00:34:28,290 I'm interested in the bigger, the fans or the circles showing up. 348 00:34:28,430 --> 00:34:32,770 So when I did that I realized, well, only this one comes back here. 349 00:34:32,770 --> 00:34:35,750 There are, like, ten recipients and seven and eleven. 350 00:34:36,010 --> 00:34:39,210 And it's always from this free cloud report at news.ch. 351 00:34:39,210 --> 00:34:41,250 So it was probably a newsletter. 352 00:34:41,250 --> 00:34:43,530 And maybe it was spam. 353 00:34:43,870 --> 00:34:48,130 You can take this further and further. 354 00:34:48,190 --> 00:34:51,890 You can have very big, you look at very big emails. 355 00:34:52,050 --> 00:34:56,550 Maybe someone is sending out documents from the internal network to someone outside. 356 00:34:56,850 --> 00:35:01,230 I'm going to skip over these a little quicker to show you Afterglow. 357 00:35:02,710 --> 00:35:08,590 What an interesting, another one is there's a field called delay on the email server log. 358 00:35:08,850 --> 00:35:14,370 And that shows you how long was the email queued on the mail gateway. 359 00:35:14,710 --> 00:35:19,950 And what you see here is that there was apparently some problem here with these big two clouds. 360 00:35:20,450 --> 00:35:22,450 This one here and this one down here. 361 00:35:22,450 --> 00:35:25,370 These emails stayed on the server for more than ten minutes. 362 00:35:25,910 --> 00:35:31,530 And it happened a lot of times, as you can see, to these two email addresses. 363 00:35:31,810 --> 00:35:34,110 This one here and this one down here. 364 00:35:34,110 --> 00:35:35,830 So there's some problem with those addresses. 365 00:35:35,830 --> 00:35:36,970 Maybe they're wrong. 366 00:35:37,650 --> 00:35:45,750 And you can go in and either block them or do something about them or figure out who was sending the emails out to these guys. 367 00:35:45,790 --> 00:35:55,450 And this might actually be also a way to detect spammers because they send these huge lists of emails and probably a lot of them don't exist and they will be queued on system. 368 00:35:55,450 --> 00:35:58,430 They might retry all the time, so getting a lot of delay. 369 00:35:58,470 --> 00:36:00,990 So maybe you can figure out who your spammers are. 370 00:36:01,150 --> 00:36:03,410 Okay, let's get to the gist here, to Afterglow. 371 00:36:05,090 --> 00:36:06,950 You find it on SourceForge. 372 00:36:07,370 --> 00:36:12,010 As I mentioned earlier already, I support two output formats. 373 00:36:12,590 --> 00:36:19,790 The GraphList library from AT&T, also known as .neto, and the LGL, the Large Graph Layout by Alex Adai. 374 00:36:24,510 --> 00:36:27,250 If you, so Afterglow is just a Perl script. 375 00:36:27,250 --> 00:36:31,470 You can pass it the input and read some standard in. 376 00:36:31,470 --> 00:36:33,970 You can pass it a few parameters, figure it. 377 00:36:35,270 --> 00:36:40,770 One of them is if you do a minus T on it, it expects a two-column input. 378 00:36:40,770 --> 00:36:46,110 So just from two or size and recipient or something like that. 379 00:36:46,110 --> 00:36:48,890 So minus T reads two columns. 380 00:36:49,190 --> 00:36:51,470 If you don't do the minus T, it expects three columns. 381 00:36:51,470 --> 00:36:55,890 So you can do all these like source IP destination, IP destination port things. 382 00:36:56,610 --> 00:37:00,450 If you do a minus D, it will print the count on the nodes. 383 00:37:00,450 --> 00:37:08,250 That was that weird example from Capture the Flag I showed you, that you actually see how many times did that node show up in the log file. 384 00:37:08,590 --> 00:37:12,550 Then you can pass it an edge length to kind of optimize how your graph is going to look. 385 00:37:12,550 --> 00:37:15,830 If you have a length of five, then things would be really distant from each other. 386 00:37:15,830 --> 00:37:19,170 But if you have big clouds around one of the nodes, that might be helpful. 387 00:37:20,610 --> 00:37:24,370 With minus N, you can disable the node labels. 388 00:37:25,070 --> 00:37:29,870 So sometimes you have logs, if you can't parse them really well, you have really long tokens. 389 00:37:29,870 --> 00:37:32,870 So they will really clutter your graphs. 390 00:37:32,870 --> 00:37:37,090 They just turn them off and you get kind of a feeling that there are certain things going on. 391 00:37:37,090 --> 00:37:40,030 And then you have to go back to the log file and figure out what those nodes were. 392 00:37:41,110 --> 00:37:57,030 This minus O thing, this threshold that you can set, if you have a log file of communications like someone is talking to someone else and you see a lot of people that just connect to one other machine and nothing else. 393 00:37:57,030 --> 00:37:59,890 And then another machine to something totally different. 394 00:37:59,990 --> 00:38:01,890 But then you have these big clouds. 395 00:38:01,890 --> 00:38:05,410 A lot of times, I'm not interested in these one-to-one connections. 396 00:38:05,410 --> 00:38:08,790 I really don't want to know about those, but I'm interested in the big clouds. 397 00:38:08,970 --> 00:38:13,430 So if you do a minus O one, it would kill everything that has a fan out of one. 398 00:38:14,030 --> 00:38:17,270 And this feature is not quite complete. 399 00:38:17,270 --> 00:38:22,410 It doesn't do the whole traversing of the clusters and figure out whether it's connected to other clusters. 400 00:38:22,410 --> 00:38:23,110 Play with it. 401 00:38:23,110 --> 00:38:24,090 It's kind of cool. 402 00:38:24,150 --> 00:38:27,410 There might be like one-off cases that look kind of weird. 403 00:38:27,950 --> 00:38:30,490 And then the most important thing is the minus C. 404 00:38:30,490 --> 00:38:33,070 You can pass it a color property file. 405 00:38:33,070 --> 00:38:36,570 I'm going to show this in a second how this works. 406 00:38:37,070 --> 00:38:43,550 So the property file just has color dot, for example, source equals, and then it can pass it a Perl expression. 407 00:38:43,550 --> 00:38:48,570 And the expression just has to return a color, like red, this string. 408 00:38:50,010 --> 00:39:00,950 Then there is an array called fields that contains the columns that you passed it in. 409 00:39:00,950 --> 00:39:04,350 So for every line, it will evaluate these array. 410 00:39:04,490 --> 00:39:17,690 So you can say, like, the event node is red if the first column, so that's really the second column, if fields one matches this regular expression, so it's in 192, then make it red. 411 00:39:18,570 --> 00:39:21,590 Then I have the special color called invisible. 412 00:39:21,590 --> 00:39:28,630 If you say that the target is invisible if this expression here matches, then it won't draw that node. 413 00:39:28,630 --> 00:39:32,190 So you can kick out certain things by certain expressions. 414 00:39:33,530 --> 00:39:35,350 Or you just have a simple color. 415 00:39:35,350 --> 00:39:37,630 Just say color dot edge equals blue. 416 00:39:38,210 --> 00:39:42,170 So here is the fancy example of the file graph. 417 00:39:42,170 --> 00:39:43,630 I'm just very quick. 418 00:39:43,630 --> 00:39:44,910 So this is the property file. 419 00:39:44,910 --> 00:39:58,510 You say the source is olive drab if the field zero, so if the first entry that you get is 192.141.69.4, that's one of my IP addresses. 420 00:39:58,630 --> 00:40:01,230 It's not really, so go and play with it. 421 00:40:02,350 --> 00:40:04,330 So they show up green. 422 00:40:04,950 --> 00:40:09,230 Then everything else, like first match wins. 423 00:40:09,230 --> 00:40:12,150 So if it falls through, it will just be orange-red. 424 00:40:12,150 --> 00:40:14,330 So everything outside is going to be red. 425 00:40:14,470 --> 00:40:17,990 Then you configure the nodes to be blue, the targets. 426 00:40:18,410 --> 00:40:20,090 Like, I think you get the idea here. 427 00:40:20,090 --> 00:40:36,290 You can have really complicated things on the edges, for example, for ingoing and outgoing traffic, I have to say if the first column is inside my network or the third column, which is the destination IP, is inside my network, then give it a fire break color. 428 00:40:36,850 --> 00:40:39,310 And otherwise just use the other color. 429 00:40:39,910 --> 00:40:43,870 Afterglow also has two parses I'll give you. 430 00:40:44,110 --> 00:40:47,430 And I just realized, I just put this slide in last night at like 2.30. 431 00:40:47,630 --> 00:40:49,370 I hope there are no typos. 432 00:40:50,690 --> 00:40:53,950 I realized that these two parses are actually quite useful. 433 00:40:53,950 --> 00:40:56,050 I used them at Capture the Flag yesterday. 434 00:40:56,270 --> 00:41:00,310 If you have a TCP down block file and you want to visualize that, you have one problem. 435 00:41:00,310 --> 00:41:04,370 You see requests from a high port to port 80. 436 00:41:04,490 --> 00:41:08,370 Then you see the response going from port 80 to a high port. 437 00:41:08,370 --> 00:41:13,470 If you visualize that, you will see the high port and port 80 as source ports. 438 00:41:13,550 --> 00:41:16,490 And that was what I was showing in the other graph, basically. 439 00:41:16,490 --> 00:41:19,110 And then the destination port, again, same port numbers. 440 00:41:19,110 --> 00:41:22,790 So you need to, for the request, you need to flip the ports. 441 00:41:23,790 --> 00:41:25,590 That's so that you have the connections again. 442 00:41:25,590 --> 00:41:29,310 And this, the TCP down to CSV takes care of that. 443 00:41:29,310 --> 00:41:33,850 It remembers what direction the connections are going in and automatically flips them. 444 00:41:33,850 --> 00:41:34,990 So you get nice output. 445 00:41:35,090 --> 00:41:41,210 And then the send mail parser combines the send mail log entries into one again that you need. 446 00:41:41,850 --> 00:41:43,290 That's really it. 447 00:41:43,790 --> 00:41:45,270 Any questions? 448 00:41:47,960 --> 00:41:49,540 Wow, everyone is tired. 449 00:41:49,540 --> 00:41:50,120 Yeah? 450 00:42:02,030 --> 00:42:09,710 So the question is whether I'm thinking about using, like, a database to drive or to read from or something. 451 00:42:09,710 --> 00:42:12,850 Well, AfterGrowth is really just like a Unix script, right? 452 00:42:12,850 --> 00:42:16,470 You use pipes and you pipe in data and you get data out. 453 00:42:16,630 --> 00:42:19,850 So it's pretty easy to use a SQL database. 454 00:42:19,850 --> 00:42:22,650 For example, you use the TCP down to CSV. 455 00:42:22,650 --> 00:42:33,650 You pipe that into MySQL minus E, insert into blah, blah, and you get into database and then you can have it piped into something else and you read out the stuff from the database. 456 00:42:33,650 --> 00:42:36,350 So you plug it basically into whatever you want it. 457 00:42:36,770 --> 00:42:48,370 We basically just provided this AfterGlow to take the CSV, to add the color and then everything to that graph file and generate the graph file and then you use the grapher to generate the graph. 458 00:42:48,990 --> 00:42:52,930 But please feel free to contribute anything to the SourceForest project. 459 00:42:54,010 --> 00:42:58,470 And also if you're going ahead, go home and visualize and get some cool graphs. 460 00:42:58,470 --> 00:43:02,610 I'm always interested in getting graphs, so spam me with cool pictures. 461 00:43:07,240 --> 00:43:09,000 How fast is it? 462 00:43:09,820 --> 00:43:13,000 Well, most of the time it's actually consumed by Nido. 463 00:43:13,000 --> 00:43:24,240 That's what I use to generate all these graphs here because it has to generate the layout and it really depends on how many nodes or how many log entries you're passing in. 464 00:43:25,860 --> 00:43:31,780 It's, if you have probably about, let's say we want to have a refresh rate of like 30 seconds. 465 00:43:31,780 --> 00:43:35,740 I think that's good because you need time to look at it and then it's going to change completely again. 466 00:43:35,740 --> 00:43:36,800 That's going to be one of the problems. 467 00:43:36,800 --> 00:43:38,900 You don't see nice delta updates. 468 00:43:38,900 --> 00:43:40,400 You would have to handle that somehow. 469 00:43:40,480 --> 00:43:43,600 Might be something that maybe Greg and I could work on. 470 00:43:47,160 --> 00:43:55,340 So if you're 30 seconds, it's probably safe to use up to probably 5,000 events in 30 seconds. 471 00:43:55,340 --> 00:43:56,060 That should probably work. 472 00:43:56,060 --> 00:44:00,380 I think I did something like that yesterday with the capture the flag data. 473 00:44:01,880 --> 00:44:03,300 I have two more books. 474 00:44:04,640 --> 00:44:08,340 I guess I give them to the two gentlemen that asked the questions here. 475 00:44:09,660 --> 00:44:11,020 Anyone else? 476 00:44:13,160 --> 00:44:14,820 All right, thank you very much.