start	end	text
1690	6210	So let's finally talk about visual security event analysis.
11370	17870	So what I'm going to talk about is how you analyze log files or events that you have
18770	27070	in a visual manner. I don't want to talk about myself too much. Maybe one thing. I've been
27070	33310	involved in, like, log analysis and event analysis for quite a while, and it has always
33310	38030	interested me whether there's a way to look at the data in a way that it really, like,
38210	43870	a lot of data, in a way that it's really apparent what is going on in that dataset. And if you're
43870	47850	looking at log files, thousands of lines of logs, it's really hard to figure out at one
47850	51610	glance what is going on. You have to start reading through them. You have to, like, use
51610	55890	your own mind to do pattern analysis and figure out what is really going on.
55890	61290	So at some point, I started working for this company that I'm working for right now called
61290	68910	ArcSight, and I got a hand on a lot of neat tools to look at data, and I got a feature
68910	73910	in the software that is called Event Graphs, and I really liked it, and I started looking
73910	78450	into how can we use this feature to make it more easy for a security analyst or a person
78450	82330	interested in a log file to figure out what is really going on in that dataset.
82330	88770	So that's how kind of my passion about Event Graphs started. My talk here is not going
88770	92810	to be very textual or, like, theoretical, and I'm not going to talk too much about how
92810	96850	you generate the graphs and whatever. I want to show you a lot of examples on how to look
96850	100990	at datasets and how you configure graphs to get the most out of it.
101510	107170	As part of the talk, I'm going to show you a tool that I have up on SourceForge, which
107170	113610	is called Afterglow, which is a very simple Perl script that transforms CSV files into
114070	118390	a graph or language file, so you can go home and play with these things yourself, and the
118390	120010	tool is actually also on the CD.
122570	126510	I'm going a little faster in the beginning so we can catch up for the time I lost already.
126510	130230	One thing, IP addresses and host names, you're going to see in all these graphs, and the
130230	134330	descriptions are obfuscated or changed. The addresses are completely random and any resemblance
134330	137410	with well-known addresses or host names are purely coincidental.
138510	143270	All right. So let's do a little bit of an introduction here. Why would you want to look
143270	146250	at these graphs? I guess this is a pretty good example already. Do you want to look
146250	149330	at the left-hand side, or do you want to look at the right-hand side?
152670	156250	Well, even if I made the font bigger on the left-hand side, it's probably still not very
156250	156850	fun.
157330	163090	All right. Why do you want to use event graphs? I guess I already motivated that quite well.
163090	167270	I'd rather look at the graphs. They look just much nicer.
168520	172030	If you actually dig into it a little more and you start looking how to configure these
172030	178690	things, you will see pretty quickly that you can visualize different properties in those
178690	183510	graphs, and it's very easy to see certain aspects of a log file very quickly and certain
183510	188430	things that are important to you. You don't have to wade through ten thousands of lines
188430	193690	of logs to figure out whether there is anyone sending a lot of e-mails to someone else.
194590	203990	If you use this in a security operating center or in a 724 response team or something, if
203990	210090	you have graphs, they're a great way of giving you situational awareness or real-time view
210090	213310	on things that are happening, and you can quickly react on them.
213690	217850	We have a lot of customers that are starting to use this in their SOCs, and it doesn't
217850	222370	just look pretty on their big screens, but it's actually really useful. It's actually
222370	228330	also kind of an interesting story. I was at one customer, and they had all these big displays,
228330	231950	and they were empty, and I was like, wow, why do you have these nice graphs here on
231950	235590	the analyst screens? Why don't you have them up there on the big screens? They were like,
235590	239870	whoa, no, no, no, we tried that for a while, but then the boss came in, and he saw all
239870	242570	these things on the screen. He was like, oh, what is this? What's that? What's this? What's
242570	244950	that? So we had to explain all this. We took them off.
248140	252040	All right, and one other thing, another story that I had with these graphs, which actually
252040	257440	two other stories. One is, one customer uses these graphs to communicate with other teams,
257440	260780	so you have the security analysts. They look at the log files and all that, and then they
260780	265700	go back to the operators or the operational folks, and they're like, guys, you have to
265700	270180	do something about this machine. It's talking to all the others in the network. And in the
270180	274220	past, they were handing them log files to prove that it's that way, and the operators
274220	278020	were like, whatever, we don't understand what this is. But now they started giving
278020	282120	them graphs, and the operators were like, oh, my God, my machine is talking to all these?
282120	287180	Okay, we're going to take care of this. And even worse, I had one customer that actually
287180	293520	just told me this week that he used the graphs to analyze some data, and he found some stuff,
293520	298900	and one of the guys got fired just because he used the graphs and found out that one
298900	307430	of the guys was not doing his work. All right, so I came up with, like, two big categories
307430	311950	of when to use these event graphs. One is in a real-time monitoring scenario, where
311950	316170	you have, like, a screen app that looks real-time at certain properties of your data file, and
316170	319930	I'm going to show you some examples on what I mean by that. It gives you great awareness
319930	324770	of things that are happening. You can configure these things to, for example, look at transactions
324770	329110	going to your financial server. So right away, if someone connects to your financial service,
329110	332890	you will know. And then you can take it a level further. You can say, well, I know that
332890	336950	these machines are okay to connect to my financial servers, so don't show me those, but show
336950	340470	me everything else. So as soon as something pops up there, you know you've got to look
340470	346710	into it. The other thing is, if you take this in a more forensic or historical way, you
346710	350770	can, you grab a data set, you run it through some tools, and you look at the graphs on
350770	356190	what is, on certain properties, and you figure out what's going on. Let me mention quickly
356190	362890	some related work. There's some classics from, he did all these kind of things. He tried
362890	368430	to figure out whether there are certain ways to visualize properties of data, and he took
368650	373930	a lot of dimensions and put them all in one graph. What I'm going to do is, I take a lot
373930	378390	of dimensions too, and I'm always going to stay with event graphs, so like node-to-node
378790	384710	connections. I'm not going to use any of the parallel coordinates or the right-hand-to-bottom
384710	387610	thing that I have no idea what it shows there.
388310	393890	Then Ehrbacher, he also, he actually went even further. He tried to go even more multidimensional
393890	399650	and map everything into these nodes, so there's like size, there's shape, there's length of
399650	406110	these spirals going out, and so on. Then Greg Conti, you probably have seen his talk, I
406110	414690	hope. He mainly focuses on different ways to visualize TCP dump or packet locks, and
414710	415910	very interesting.
417510	422430	Then Envision IP is something I found. What you see up there is a grid of, I think it's
423190	429710	8x8 or 64x64, and they just map IP addresses in there, so you see certain clusters of things
429710	434010	going on. The nice thing about this project is, there's a lot of interactivity, so you
434010	436630	can start playing with the data, you can zoom in and so on.
437030	442810	And then finally, a few of you might know Showkey from Steve Barry. It's basically a
442810	448530	scatterplot. You can define three different scatterplots, and then you have a 3D of that.
448810	453230	So on the top right, it's not a very nice picture, but on the top right it's a three-dimensional
453230	458850	output, so you can say one axis is your source IP, one axis is destination IP, and the other
458850	460690	one is the destination port.
461630	467370	All right, let's move on. How do you generate an event graph? Pretty easy. You have a log
467370	474250	file that a device is generating. You take a parser. Why do you take a parser? Well,
474250	479430	you need to know where in my log event or my log file, where's the source IP, where's
479430	482910	the destination IP, where's the event name, where's the port? So you parse that, you get
482910	487550	the syntax right, and then you visualize that in an event analyzer.
488250	494450	If you have an event, let's look at this Nord event here. You don't have to be really able
494450	498790	to read this, but what I want to show here is in an event graph you can have different
499750	503670	configurations. The normal event graph that you will always see is like who is talking
503670	508950	to whom, so you have source IP, destination IP. But you can take this a lot further. You
508950	515210	can say, well, on the right top you want to have the source IP, destination IP, and destination
515210	519950	port, so you will see someone port scanning, for example, if there's a fan with destination
519950	524370	ports. So you can configure these things in pretty wild ways, and I'm going to show you
524370	525290	some of them.
526990	533650	Now let's have a sneak preview of Afterglow. I apologize because I don't use my laptop
533650	539890	running it. I can't show you the demo. It's not very fun anyways. I will show you the
539890	545350	output anyway, so it doesn't really matter. So what is Afterglow? Afterglow is not a security
545350	550850	information management tool. It's just a script which takes CSV files, so you have
550850	557310	to somehow from your logs generate a CSV file, comma, separated values. And then you take
557310	562170	Afterglow, the tool that I'm going to show you, and you transform the CSV files into
562390	568110	a graph language file. And then you take a grapher tool, and right now I'm supporting
568110	573710	two of them, that takes this graph language and generates the graph. And that's not something
573710	577950	I did. I just transformed the CSVs into graph files, but I'm going to show you some cool
577950	580130	things that we implemented there.
581490	586410	So how you can use this thing is you basically cap the input file, you pipe it to Afterglow,
586410	589790	you give it a color properties file, which I'm going to introduce a little later, and
589790	595830	then you can use the AT&T Graphverse library, .neto, I'll give you references in a second,
595830	601990	and you generate a GIF file. The color properties file is basically just you tell it the source
601990	606830	nodes are red, for example, the event nodes are green, and the target nodes are blue.
606830	610350	And I'm going to show you how to really play with this stuff. We extended this quite a
610350	617690	bit so you can have colors depending on the values in the log files.
617690	623530	Big shout out to Christian. Thanks a lot for building the foundation of this tool. He built
623530	629690	me the whole Afterglow profile, the basics and everything, and I just went in and made
629690	638710	some small changes, but thanks a lot, Chris. Let's look at examples. I mentioned in the
638710	642950	beginning I kind of have two use cases or two big categories of things to look at. One
642950	653120	is what I call situational awareness or real-time monitoring. So this is one of the views that
653740	660200	I saw at a customer. Basically you see here is six independent screens of different properties
660200	666240	of your data that you want to look at. So here the configuration is such that on the
666240	672560	top left the customer wanted to see just the compromise attempts, and don't ask me how
672560	677980	to filter for just those events. That's a whole different story. But if you have a way
677980	681900	to say these are my malware events, these are my DOS events, these are my scan events,
681900	686280	if you can separate them by something, then you can draw all these screens and you can
686280	690280	look at these screens and see what's going on.
690500	694900	So what this customer is actually also doing, whenever an analyst starts his shift, he resets
694900	700940	all the graphs so they're empty. So over time, nodes start popping up and they look at them
700940	704800	and they're like, okay, this is fine, and they might filter them out even, or they just
704800	709140	leave them there and say, okay, I know about this. And the more things that show up, the
709140	713680	more they go along in their workflow and work with these events. So I have a feeling this
713680	720200	is going to be a great tool, especially in a SOC or an analysis environment, to kick
720200	723340	off your workflow. You look at these things and you take it on and you start building
723340	728120	filters to filter out the things you don't know about, and you just see the outliers
728120	729440	here in these screens.
731300	739700	The second big group of use cases is forensic analysis or historical analysis, where you
739700	747100	take a log file that you have and you visualize it. Afterglow can generate two kinds of outputs.
747100	753900	One is for the AT&T graphics library for .neto, and the other one is for LGL, the large graph
753900	759180	library. I will give a pointer to that in the end as well. This thing is generating
759180	762980	three-dimensional graphs. You can actually generate VRML output and you can fly through
762980	769080	these things. I didn't prepare a demo because it's kind of fun to fly through it, but the
769080	773660	analysis value is not really high or the value you get out of it. It's really cool
773660	778120	to fly through it and see that, oh, this guy sent an email. To whom? Oh, over there. Okay.
779060	784640	It looks great, and I apologize for the colors here, but this is really how it looks. Anyway,
784640	785900	let's go back to 2D.
787000	793180	This is a very simple example, one of about 12 that I'm going to show, of how to analyze
793480	802620	a log file. So what I had here is, I believe, a snort log or a web log or something, and
803180	809500	I said, well, show me what is happening. So the event name, the destination IP are the
809500	813640	blue ones, and then the destination ports are the white nodes. So what you see here,
813640	820480	all the blue circles are my web servers. The white boxes are the target ports that they're
820480	825280	being connected to. So you see it's 80 and 443, so that's totally legitimate. Only HTTP
825280	829640	and HTTPS traffic is going to my servers, so I probably don't have to worry about that.
829640	834260	If I would see a node there, like port 21, and someone connects to my web servers with
834260	839800	FTP, maybe that's okay, but if I'm not running FTP in my environment and that shows up here,
839800	845960	I probably want to investigate that. So it's a very quick way to see whether traffic is
845960	847600	really what it's supposed to be.
847600	854720	Almost same configuration. Here you see source IP, destination IP, destination port. So this
854720	861420	would potentially show you port scans or network scans. Here you see a lot of machines connecting
861420	868860	to a lot of other ones, all on the same port. Down here, this white node here, is the port.
868860	875720	So they all connect to the same port. Well, it could be that a worm is probing you, but
875720	880200	it could also be that you have a web server, or a lot of web servers, and a lot of machines
880200	885980	are connecting to those. So it might be okay, but it might be a scan, and depending on what
885980	890780	the destination port is, you might figure out what's going on in this case. But you
890780	895920	can certainly tell, well, this is all similar behavior here on this fan, and then actually
895920	901820	if you see on the bottom left here, there's some little ones, it's probably worth investigating
901820	906760	what those are, and probably those are the outliers that you are interested in.
909200	912860	Let's keep this configuration of the graph. This is just a little bit of a different dataset
912860	922620	and a different time. It looks much prettier. Let's zoom in on the bottom part here. What
922620	926360	this is showing you is that there are two machines, the red nodes, that are connecting
926360	932000	to about 15 other ones, and still on the same port. So again, in your environment, do you
932000	938400	really have all these other 15 or 20 machines serving this one service, and exactly two
938400	942240	machines are connecting to them? So you can go and figure out what all these machines
942240	946480	are, and hopefully they are really the ones that are serving this service, and then you
946480	949500	can figure out whether these other two are really allowed to connect to this.
950060	957580	Let's have a look at this weird thing over here. What you see here is, probably you can't
957580	965520	read this, but there is a node here that says 21 FTP, so one of the properties of this graph
965520	970560	is that the white nodes show up exactly once per target port, so everything that happens
970560	976740	on port 21 is going to show up around that thing there. So again, I can say, wow, I have
976740	984400	about six FTP servers, and about five people are connecting to them. So the FTP servers
984400	987740	would be the blue nodes, and then the red ones that connect to them would be the people
987740	992380	using it. So, well, is this really something I have in my environment? Is this allowed?
992380	998440	And why are only three or five people connecting to them? Is that some kind of a process that,
998440	1004260	I don't know, maybe AIX that transfers log files over, something weird like that? Or
1004260	1007260	is this something malicious? Is anyone having a ware server or whatever?
1008980	1014000	If you interpret these graphs, you have to be a little careful. I still have the same
1014000	1017640	configuration. Destination IPs are the blue ones, and the white ones are the destination
1017640	1023840	ports. Well, if you look at this on the left-hand side, I would say, wow, it's a port scan.
1024360	1033070	Well, is it really a port scan? I have a book for the one that gets this right. What is
1033070	1044840	the left-hand side? No, not voice over IP. Who said that? Over there. You got the book.
1044840	1054200	It's the source ports. The log file reports such that the source port is really the destination
1054200	1059300	port and the other way around. So when you analyze these things, be careful and make
1059300	1063860	sure you're really looking at what you're looking at. So why do I know are the source
1063860	1070100	ports and the destination ports? Well, they're in the 50,000 range. They're almost contiguous.
1070780	1075980	And I mean, who is scanning me on these high ports? We're not a capture the flag. There,
1075980	1080560	they might scan you on those high ports because services are running all over. But here, there's
1080560	1084520	really no sense why someone would scan you up there. There's probably nothing there listening
1084520	1089100	anyway. So be careful when you analyze these things. Sometimes it's port scan, sometimes
1089100	1094640	it's not. I'm always, my first impression is always, well, false positive. Let's really
1094640	1100930	figure out whether that's true. Okay, let's get fancy here.
1101810	1107970	This is a firewall log. Basically, I'll show you passes and blocks from a PF firewall.
1109790	1117170	If I would just visualize source IPs, destination IPs, I would see big clouds of things. Well,
1117170	1121590	I want to see a little more in this graph. So what I did is I have source port, rule
1121590	1129150	number, that's the squares, and then destination IP. So you will see I'm connecting to some
1129150	1134250	other machine and rule number 100 is responsible for letting me in or blocking me.
1135070	1142150	What I did then with Afterglow is to configure my nodes such that all the external IPs are
1142150	1146610	going to show up as orange. I'm going to show you how to configure Afterglow to get exactly
1146610	1152370	this graph. So the external machines are orange. My internal machines are green. And
1152370	1157950	you see I only have a few of them. And then the rules are the blue boxes. And then further
1157950	1162710	what I said, I wanted to see whether a connection is incoming or outgoing of my network. So
1162710	1167370	I configured the edges such that outgoing connections are blue, that's the majority
1167370	1174290	here, and incoming is red. And I just realized it's probably exactly the other way around.
1174290	1178550	So I hope there's more incoming than outgoing because these are web servers that are servicing
1178550	1186070	some stuff. Anyway, so you see big clouds of things that are showing similar behavior.
1186070	1189910	So down here there's this one rule that is responsible for letting in a whole lot of
1189910	1196270	traffic. And probably that's fine. There's another one up here. You can, if you know
1196270	1200650	the rule set, if you know your ACLs, you can go and figure out whether it's really what
1200650	1207250	it's supposed to do. But you can also figure out some things that are kind of weird. You
1207250	1211830	have this whole section in the middle. You probably want to zoom in on those and filter
1211830	1215870	out all the others and just look at these things and figure out whether your rule set
1215870	1221130	is really doing what it's supposed to do. So the next steps would be, so the first is
1221130	1224970	like overview, general impression, what is going on, is there anything really sticking
1224970	1230150	out. And then my next steps would be, I would visualize the firewall blocks of outgoing
1230150	1235730	traffic. Why would internal machines connect outside and be blocked? So that's either a
1235730	1241750	misconfiguration of the internal box or the firewall rule set is wrong. Then the second
1241750	1245330	one would be, I would probably look at blocks of incoming traffic. I want to know who's
1245330	1249910	probing me, who's poking around. And if you have a live system that can look at these
1249910	1254950	things, you can put all these attacker machines maybe on a blacklist and if you see them again
1254950	1259210	and they're successful coming in, you might actually take some action and say, whoa, now
1259210	1263010	someone probed me and now he's coming in. And then you can take this further. Third
1263010	1267530	step would be to visualize the passes of outgoing traffic. Is this really what should be leaving
1267530	1272450	the network? Maybe you find someone FTPing stuff out that you really don't want to FTP
1272450	1277630	out. Maybe you realize that you're running peer-to-peer clients in your internal network
1277630	1282390	and they're connecting out, anything like that. And you can probably take a few more
1282390	1285810	steps to analyze the data set, but I guess you get the idea.
1287090	1293470	So this one is probably the wackiest graph I have. It's totally going away from source
1293470	1298710	IPs and destination IPs. What I did here is I wanted to debug my firewall rule set. So
1298710	1303250	it's still the same data set as before, but this time what I show you is the red nodes
1303250	1309630	are the rule number or the ACL number, then the destination ports, and then you see the
1309630	1315250	action taken, pass or block. So just by virtue of firewall rule configurations, you will
1315250	1320110	see two distinct graphs. One on the left-hand side is all the passes and the right-hand
1320110	1328310	side is all the blocks. So here you can see, if you take one of these blue nodes, for example,
1328310	1333710	port 53 here, you see that this one rule is responsible for letting traffic in. Well,
1333710	1340270	most is okay, maybe, probably, hopefully, but you have other ones where there are two
1340790	1346590	nodes or three nodes connected to the same rule. Like over here, for example, there are
1346590	1352710	three ports being blocked by this one rule. It might be okay. It might not. If you see
1352710	1358870	something like this on the other side, that one rule passes traffic from multiple ports,
1358870	1362430	maybe that's the way you configured it, but this is certainly a great way of analyzing
1362430	1364490	and debugging your firewall rule set.
1366630	1373290	This is another example where I took a TCP dump log, and what I visualized here is I
1373290	1378390	take the region of where the source IP resides, so you can go on the web, there are databases
1378390	1384770	that show you what the location codes are for the IP addresses. They're not very accurate,
1384770	1389430	but in a lot of cases, they're actually pretty good. So I show you the source region, where
1389430	1395330	it's coming from in the world, then I show you the source MAC address. So this is a TCP
1395330	1400770	dump log. The source MAC address is going to be the last hub, probably my router, that
1400770	1407430	let the traffic into my network. And then I show the destination IP. So what I see here,
1407430	1413370	I have four boxes, the four white ones, that are my internal machines, and interesting
1413370	1418850	enough, I have two entry points into my network. I have two source MAC addresses showing up.
1418850	1425690	And then I have these two fans in the middle part. What this is showing me is I probably
1425690	1432070	either have a redundant router setup, or maybe I have a load balancer. And what is interesting
1432070	1437770	here is that all these machines up here, they're connecting through this guy. All these guys
1437770	1441450	here are coming through both of them, and all these down here come from the other one.
1441450	1445510	So maybe this is a regional load balancer, that everything on one side of the world takes
1445510	1449250	this entry point, everything on the other side is using that entry point. So if that's
1449250	1453890	really the case, a great way to analyze this log is this graph here, and you can actually
1453890	1460130	figure out why these two in the middle are coming in for both of them. And this is just
1460270	1466830	a zoom, where you see the one server, and here it's like, you can't even read it below
1466830	1470710	the node here, but it's Sand Hill or whatever in the United States.
1475510	1481450	Graphs like these are a lot of times indications for warms, but again, be careful. There's
1481550	1488250	a nice fan here of source IPs using the same event going to a few destination IPs. Well,
1488250	1492210	it could be, again, it could be a warm that is using this one event to try to get into
1492310	1499890	a network, but it might again be a web server serving just legitimate files. What you have
1499890	1503270	to do in this case, you have to figure out what is this event name here that connects
1503270	1508250	all these dots, and this might give you an indication of a warm or not. If you're visualizing
1508390	1513510	a snort log file and this talks about warm, well, duh, it's probably a warm. But if you
1513510	1521330	visualize a TCP dump log file, like an ICMP port unreachable might be an indicator for
1521470	1526770	a warm trying to probe UDP ports and you get ICMP port unreachables back. So you have to
1526770	1529330	investigate a little bit to figure out whether it's really a warm.
1529330	1536930	I was playing Capture the Flag actually this year and last year, and I was trying to figure
1536930	1543370	out the challenge really for the analysis team on Capture the Flag is that all the traffic
1543370	1549530	hitting your web server or your servers seems to be coming from the same IP address, and
1549530	1555710	there is malicious traffic of the other teams hacking you, but there is also benign traffic
1555710	1560050	from the score bots trying to figure out whether your services are still up. So blocking
1560050	1564490	all the traffic is not working. So you have to figure out a certain way of distinguishing
1564490	1571270	these two types of traffic, and you can't do it by the source IP, which you would normally
1571270	1572890	do in your corporate environment.
1573350	1579930	So I started visualizing this log file of TCP dump, and I just wanted to have a general
1579930	1586170	feeling for what's going on. So what you see here, the dark blue ovals are destination
1586170	1592850	ports that are below 1024, normally services that are offered to the outside world. Then
1592850	1599730	destination ports higher than 1024 are light blue, and then you see here what I just explained,
1599730	1604830	the red node, this is what all the traffic that's coming in is coming from this one IP.
1604830	1613770	And then I configured it such that our machines, the internal targets are green, other teams'
1613770	1619450	targets are brown, and then internal sources are yellow, and internal targets are the boxes.
1619550	1625510	So what you see here, we have two servers, while there's a third one showing up down
1625510	1631970	here, this was actually not a server, but I think that was one of the machines on our
1631970	1636310	network that someone connected to, and that could have been one of our team members connected
1636310	1645830	to someone else's machine. And then the services that we offered were D6, and it was all kinds
1645830	1651930	of weird things. I don't even remember anymore what it was last year, port 9999 or something,
1651930	1655930	and there was some mud running there. But this gives you a very quick way of looking
1655930	1657830	at what is happening in this network.
1658850	1664590	Well, I took it a little further, and I had an idea, or a few people had an idea, I said,
1664590	1668830	well, maybe it's a TTL of the traffic coming in. Maybe you can distinguish the good traffic
1668830	1673490	from the bad traffic by looking at the TTL. So well, what I did is I said, okay, show
1673490	1678670	me the source IPs, the destination IPs, and then the TTLs of that traffic. So you see
1678670	1685910	here, I have eight TTLs showing up, eight distinct TTLs, and I'm going to look at the
1685930	1695810	everything from 35, 50, 52, 60, 62, 63, 64, 128, and 146. Some of them are showing up
1696410	1702090	if my teammates would connect out, so I could filter them out. I was not interested in those.
1702090	1707510	I'm just interested in the ones that happen of the traffic coming in from this one evil
1707510	1712150	IP. Let's go back quickly again.
1712150	1717870	And then I figured out, well, I still can't quite tell what is okay and what's not, so
1717870	1722010	I had to reconfigure my graph and look at some other properties. So what I did is I
1722010	1725950	said, well, I'm not really interested in the IPs. I know everything is coming from this
1725950	1730610	one IP that I'm interested in, so it only showed me the traffic from that guy, but I'm
1730610	1740490	interested in the destination ports, the TCP flags, and the TTLs. So I wanted to see, like,
1740490	1745930	the SYNs, the SYNX, the pushes and whatever, and their TTLs, and what port they went to.
1746530	1753190	So this yields this graph here, and in addition to that, that was not quite enough to figure
1753190	1757010	out what's really going on. I need to have the counts. I'm going to lose everyone here.
1757010	1764650	I need the counts on all these nodes, so how many times did certain flags show up? So let's
1764650	1773810	take one example here. I have port 80 here. It showed up 2,600-and-something times, and
1773810	1779750	these are all the flags that showed up with the different TTLs. So there were four TTLs,
1779750	1786210	one, two, three, four, that were used when someone connected these ports here, and here
1786210	1793330	are the TCP flags that were being used for this traffic. So I had a theory. I said, well,
1793330	1798570	I'm looking at all the connection establishments, all the SYNs, and I want to see how many times
1798570	1804310	they happen. The score about traffic is probably going to all the services, like in sequence,
1804310	1810930	or probably the same amount of time. So I will see the same amount of SYNs going to
1810930	1816270	each of the destination ports, and if the theory about the TTLs is right, they all use
1816270	1823310	the same TTL. And it turned out that we were actually able to figure out the TTLs, and
1823330	1827110	for the game last year that way, and we filtered all the other traffic. But it was a little
1827110	1829530	late in the game, so we still didn't win.
1833080	1840100	All right, so I have some other logs here. At some point I was interested in email logs,
1840510	1846160	and I wrote a little send mail parser, and that's actually part of the afterglow scripts
1846160	1851570	that you can have on the CD or you can get from SourceForge. The problem in send mail,
1851570	1857850	I'll mention that in the end I guess, there are always two entries. It says from so-and-so,
1857850	1862870	and then there's a message ID, and then the next log entry says to so-and-so, and there's
1862970	1867830	a message ID to link those two events together, or just two log entries. So I had to write
1867950	1872750	a tool to actually take them, to put them together, and then output a CSV file, and
1873150	1875330	all that stuff, and the result is this one here.
1875330	1881410	So I'm just showing who is talking to whom by email. And then I said, well, what's probably
1881410	1885930	interesting is to see everything that's coming from my domain, the one that I'm really hosting
1885930	1891470	on that server, going to people outside and what is coming into my domain. So I colored
1891470	1897290	the whole graph that way, and if we look at a zoom up here, you see that these people
1897290	1903970	here are quite popular. A lot of people from outside, these red nodes, are sending emails
1903970	1906950	to this guy that was hosting on this server.
1907750	1911350	Here you just see the popular people. It might be interesting. I don't know. Maybe you want
1911350	1918610	to get to know them. But I was really interested in whether I'm running an open relay. I'm
1918610	1920530	not, but I'm pretending I will.
1921750	1932170	So what I did is I said, well, just show me everything on the outside. All my domains
1932170	1939630	or all the, like, rafi.ch or whatever my domain is, don't grade them out and just show me
1939630	1946610	the rest. So you see there's still some others showing up here. So what I did then was I
1947550	1951550	used afterglow to just eliminate all those. I'll show you how to do that. You can set
1951550	1958350	nodes and invisible. And here I realized, oh, my God, there are actually people using
1958350	1965130	my server, like this guy here, to send emails to people that are not hosted on my server.
1965670	1968910	So I'm probably running an open relay.
1971610	1977190	Well, some other things I was interested in, well, am I running, or is anyone spamming?
1977210	1981170	And, well, probably, hopefully I see something in here. Otherwise my graph would probably
1981170	1985490	not be very, my configuration would not be very good because probably on every mail server
1985490	1986710	you'll find spam.
1987270	1992510	So what I did is in the email log you see the size of the messages. So I just visualized
1992510	1998530	who is getting what sizes of messages. And I was interested in just the ones that are
1998530	2003090	above 10,000, all the others I'm not interested in. So you still see that, for example, down
2003090	2007950	here there are multiple recipients that get the same size messages. So that means that
2007950	2015170	someone probably is sending emails to a few people and this email is pretty big and it's
2015170	2022970	probably the same one because they have quite a high size, or the same size. So that would
2022970	2028170	be one indication of maybe there's something spam going on. Probably not.
2028230	2032790	Well, another way of looking at whether you have spam is look at the number of recipients
2034050	2038690	in an email. You can send emails to multiple people, right? So the number of recipients
2039110	2043010	in the log file would indicate how many people got this email. And what I said, well, I want
2043010	2049530	to see everything that's bigger than two. And all the things where someone is just sending
2049530	2055150	to two other people, I don't want to see. So I have this omit threshold thing in Afterglow,
2055150	2059430	I will explain it in a second, where I can drop out nodes that only connect to one other
2059430	2064450	or two others. So I can filter out the things that are not very interesting. I'm interested
2064450	2071090	in the bigger, the fans or the circles showing up. So when I did that I realized, well, only
2071090	2076350	this one comes back here. There are, like, ten recipients and seven and eleven. And it's
2076350	2081490	always from this free cloud report at news.ch. So it was probably a newsletter. And maybe
2081490	2090490	it was spam. You can take this further and further. You can have very big, you look at
2090490	2095170	very big emails. Maybe someone is sending out documents from the internal network to
2095170	2101230	someone outside. I'm going to skip over these a little quicker to show you Afterglow.
2102710	2108590	What an interesting, another one is there's a field called delay on the email server log.
2108850	2115790	And that shows you how long was the email queued on the mail gateway. And what you see
2115790	2121390	here is that there was apparently some problem here with these big two clouds. This one here
2121390	2125950	and this one down here. These emails stayed on the server for more than ten minutes. And
2126370	2132550	it happened a lot of times, as you can see, to these two email addresses. This one here
2132990	2136970	and this one down here. So there's some problem with those addresses. Maybe they're wrong.
2137650	2143850	And you can go in and either block them or do something about them or figure out who
2143850	2147850	was sending the emails out to these guys. And this might actually be also a way to detect
2147850	2153330	spammers because they send these huge lists of emails and probably a lot of them don't
2153330	2157570	exist and they will be queued on system. They might retry all the time, so getting a lot
2157570	2160990	of delay. So maybe you can figure out who your spammers are.
2161150	2168150	Okay, let's get to the gist here, to Afterglow. You find it on SourceForge. As I mentioned
2168150	2175010	earlier already, I support two output formats. The GraphList library from AT&T, also known
2175010	2179790	as .neto, and the LGL, the Large Graph Layout by Alex Adai.
2184510	2191130	If you, so Afterglow is just a Perl script. You can pass it the input and read some standard
2191130	2197710	in. You can pass it a few parameters, figure it. One of them is if you do a minus T on
2197710	2205450	it, it expects a two-column input. So just from two or size and recipient or something
2205450	2210890	like that. So minus T reads two columns. If you don't do the minus T, it expects three
2210890	2215890	columns. So you can do all these like source IP destination, IP destination port things.
2216610	2221950	If you do a minus D, it will print the count on the nodes. That was that weird example
2221950	2225790	from Capture the Flag I showed you, that you actually see how many times did that node
2225790	2231650	show up in the log file. Then you can pass it an edge length to kind of optimize how
2231650	2234790	your graph is going to look. If you have a length of five, then things would be really
2234790	2238310	distant from each other. But if you have big clouds around one of the nodes, that might
2238310	2246590	be helpful. With minus N, you can disable the node labels. So sometimes you have logs,
2246590	2251110	if you can't parse them really well, you have really long tokens. So they will really clutter
2252050	2256250	your graphs. They just turn them off and you get kind of a feeling that there are certain
2256250	2259090	things going on. And then you have to go back to the log file and figure out what those
2259090	2267650	nodes were. This minus O thing, this threshold that you can set, if you have a log file of
2269510	2274350	communications like someone is talking to someone else and you see a lot of people that
2274350	2278770	just connect to one other machine and nothing else. And then another machine to something
2278770	2283090	totally different. But then you have these big clouds. A lot of times, I'm not interested
2283090	2287690	in these one-to-one connections. I really don't want to know about those, but I'm interested
2287690	2292150	in the big clouds. So if you do a minus O one, it would kill everything that has a fan
2292150	2299410	out of one. And this feature is not quite complete. It doesn't do the whole traversing
2299410	2303110	of the clusters and figure out whether it's connected to other clusters. Play with it.
2303110	2308690	It's kind of cool. There might be like one-off cases that look kind of weird. And then the
2308690	2313430	most important thing is the minus C. You can pass it a color property file. I'm going to
2313430	2319670	show this in a second how this works. So the property file just has color dot, for
2319670	2324370	example, source equals, and then it can pass it a Perl expression. And the expression just
2324370	2333770	has to return a color, like red, this string. Then there is an array called fields that
2333770	2343410	contains the columns that you passed it in. So for every line, it will evaluate these
2343430	2350750	array. So you can say, like, the event node is red if the first column, so that's really
2350750	2356670	the second column, if fields one matches this regular expression, so it's in 192, then make
2356670	2363950	it red. Then I have the special color called invisible. If you say that the target is invisible
2363950	2370050	if this expression here matches, then it won't draw that node. So you can kick out certain
2370050	2376470	things by certain expressions. Or you just have a simple color. Just say color dot edge
2376470	2383710	equals blue. So here is the fancy example of the file graph. I'm just very quick. So
2383710	2391310	this is the property file. You say the source is olive drab if the field zero, so if the
2391310	2399710	first entry that you get is 192.141.69.4, that's one of my IP addresses. It's not really,
2399710	2409230	so go and play with it. So they show up green. Then everything else, like first match wins.
2409230	2414330	So if it falls through, it will just be orange-red. So everything outside is going to be red.
2414470	2420090	Then you configure the nodes to be blue, the targets. Like, I think you get the idea here.
2420090	2425150	You can have really complicated things on the edges, for example, for ingoing and outgoing
2425150	2431530	traffic, I have to say if the first column is inside my network or the third column,
2431530	2436290	which is the destination IP, is inside my network, then give it a fire break color.
2436850	2439310	And otherwise just use the other color.
2439910	2445630	Afterglow also has two parses I'll give you. And I just realized, I just put this slide
2445630	2452950	in last night at like 2.30. I hope there are no typos. I realized that these two parses
2452950	2456050	are actually quite useful. I used them at Capture the Flag yesterday.
2456270	2460310	If you have a TCP down block file and you want to visualize that, you have one problem.
2460310	2466510	You see requests from a high port to port 80. Then you see the response going from port
2466510	2472430	80 to a high port. If you visualize that, you will see the high port and port 80 as
2472430	2476490	source ports. And that was what I was showing in the other graph, basically.
2476490	2480570	And then the destination port, again, same port numbers. So you need to, for the request,
2480570	2486490	you need to flip the ports. That's so that you have the connections again. And this,
2486490	2491770	the TCP down to CSV takes care of that. It remembers what direction the connections are
2491770	2494990	going in and automatically flips them. So you get nice output.
2495090	2500390	And then the send mail parser combines the send mail log entries into one again that
2500390	2510120	you need. That's really it. Any questions? Wow, everyone is tired. Yeah?
2522030	2528930	So the question is whether I'm thinking about using, like, a database to drive or to read
2528930	2533970	from or something. Well, AfterGrowth is really just like a Unix script, right? You use pipes
2533970	2539850	and you pipe in data and you get data out. So it's pretty easy to use a SQL database.
2539850	2547230	For example, you use the TCP down to CSV. You pipe that into MySQL minus E, insert into
2547250	2551430	blah, blah, and you get into database and then you can have it piped into something
2551430	2555530	else and you read out the stuff from the database. So you plug it basically into whatever you
2555530	2562110	want it. We basically just provided this AfterGlow to take the CSV, to add the color and then
2562110	2566410	everything to that graph file and generate the graph file and then you use the grapher
2566410	2571890	to generate the graph. But please feel free to contribute anything to the SourceForest
2571890	2576070	project. And also if you're going ahead, go home and
2576070	2581030	visualize and get some cool graphs. I'm always interested in getting graphs, so spam me with
2581030	2593000	cool pictures. How fast is it? Well, most of the time it's actually consumed by Nido.
2593000	2598080	That's what I use to generate all these graphs here because it has to generate the layout
2598080	2604240	and it really depends on how many nodes or how many log entries you're passing in.
2605860	2611260	It's, if you have probably about, let's say we want to have a refresh rate of like 30
2611260	2614480	seconds. I think that's good because you need time to look at it and then it's going to
2614480	2618160	change completely again. That's going to be one of the problems. You don't see nice delta
2618160	2622880	updates. You would have to handle that somehow. Might be something that maybe Greg and I could
2622880	2629880	work on. So if you're 30 seconds, it's probably safe
2629880	2636300	to use up to probably 5,000 events in 30 seconds. That should probably work. I think
2636300	2640380	I did something like that yesterday with the capture the flag data.
2641880	2647680	I have two more books. I guess I give them to the two gentlemen that asked the questions
2647680	2654820	here. Anyone else? All right, thank you very much.