WEBVTT 00:14.640 --> 00:18.240 And welcome back, everybody, to DEF CON 28 Safe Mode. 00:18.360 --> 00:30.520 Continuing on with the Blue Team Village OpenSOC CTF walkthrough, we're going to be talking about Moloch today, and we have Bashar Shama here to give us a quick discussion and a walkthrough on the tool. 00:30.740 --> 00:32.960 Welcome, Bashar. 00:33.320 --> 00:34.560 Thank you. 00:34.560 --> 00:35.380 Hello, everyone. 00:35.380 --> 00:41.780 I am Bashar, and I'm going to go over how we're going to use Moloch tomorrow during the CTF. 00:41.780 --> 00:49.020 I was actually introduced to Moloch two years ago playing the same exact CTF that you're going to play tomorrow. 00:49.020 --> 00:52.620 I played it two years ago, and I really, really fell in love with Moloch. 00:52.620 --> 01:00.220 So since then, I started playing with it, tinkering with it, just use it as much as I can, and I keep using it day to day. 01:01.740 --> 01:07.520 The goal today is to really prepare you on how to use Moloch on the CTF tomorrow. 01:07.760 --> 01:11.260 So we will have time for questions at the end. 01:11.260 --> 01:17.220 Please post any questions that you have to the TechWorkshops Track 1 channel in the Discord. 01:17.620 --> 01:24.640 The moderators will monitor your questions, and we're going to ask them at the end, towards the end of the session. 01:24.640 --> 01:27.240 So keep your questions thrown away. 01:27.240 --> 01:29.880 We'll have some time at the end to go over some questions. 01:31.000 --> 01:33.960 So let's go ahead and get started. 01:34.960 --> 01:39.240 So a very brief intro on what Moloch is. 01:39.240 --> 01:41.860 It's pretty much a free open source tool. 01:42.320 --> 01:50.080 It's really a network analysis tool that you can use to analyze a large volume of packet data, or PCAPs. 01:51.040 --> 02:08.240 The simplest way to think about it is, if you've ever used Wireshark, it's like a Wireshark frontend with a huge database backend, so you can search tons and tons of PCAP data, gigs of data, hundreds of gigs of data, of PCAPs very easily, very quickly. 02:09.280 --> 02:16.240 That's a very, very, very 10,000, over 2,000 foot overview of Moloch. 02:16.480 --> 02:32.540 For our purpose here, and for the CTF tomorrow, what I have done is, actually, I have Moloch set up, and I got some, about four gigs of data as a sample from the network forensics training of March 2015. 02:32.620 --> 02:46.100 I downloaded these PCAPs, and I pretty much loaded them into Moloch to kind of show you how we can use this tool to do our investigation and answer the questions during the CTF tomorrow. 02:46.860 --> 02:57.860 So, our scenario for today is, you're part of the security team, and you receive a call saying, hey, around 1 p.m. 02:57.860 --> 03:00.540 today, or not today, 1 p.m. 03:00.540 --> 03:04.980 on March 12, 2015, our main company site has been de-fixed. 03:04.980 --> 03:08.240 Somebody has changed the way our website looks like. 03:08.300 --> 03:17.040 All we know is, we have this image of a file showing as a frog on our main site, and we don't know how this happened. 03:17.040 --> 03:18.460 Can you please help us? 03:18.680 --> 03:22.360 And all you know is, you have access to packet capture. 03:22.480 --> 03:25.300 You have packet capture, and you have access to Moloch. 03:25.300 --> 03:27.920 So, let's walk right through it. 03:28.360 --> 03:32.900 So, if you have never seen Moloch before, this is what it looks like. 03:32.900 --> 03:37.360 This is when you land or you log in. 03:37.360 --> 03:44.040 You will land pretty much in the sessions tab, and that's where you're going to probably spend most of your time tomorrow, in the sessions tab. 03:45.260 --> 03:56.320 Usually, I would like to go and just drop down here on the time, on the date, and I would do all time, all day, just to have an understanding of how big the incident is. 03:56.980 --> 04:00.600 For the purpose of tomorrow, again, it's going to be a very specific time period. 04:00.600 --> 04:08.260 But this is where I would start, and let's go ahead and deep dive into the investigation and how can we use Moloch. 04:08.260 --> 04:13.840 So, the first piece of information we know is the date of the incident. 04:14.040 --> 04:18.520 So, we know it's on 2015. 04:18.520 --> 04:27.180 So, as you can see, Moloch can give us this ability to just click through and decide on the dates that we want to investigate. 04:27.240 --> 04:28.960 I'm going to do 12. 04:28.960 --> 04:30.300 So, we have our date here. 04:30.300 --> 04:32.100 That's going to be our starting date. 04:32.320 --> 04:42.100 And then I'm going to just copy it, paste it here, and I'm going to change this to 13, which will show us here now, oh, you're looking at one day time range. 04:42.320 --> 04:44.060 So, I have my time range set up. 04:44.060 --> 04:45.540 I'm going to go search. 04:46.560 --> 04:53.440 And now I'm going to narrow it down to that 24-hour period to kind of figure out what happened during that day. 04:53.440 --> 04:59.440 Now, the other piece of information that we were given was it was our main company's website. 04:59.780 --> 05:03.280 So, how can I search for that? 05:03.280 --> 05:06.140 Probably I want to look for our host name. 05:06.140 --> 05:17.800 So, what I can do is I can type host, and then Moloch would automatically parse the different fields based on the protocols that exist in the packet. 05:17.800 --> 05:24.380 So, if it was HTTP traffic, then Moloch will say, okay, well, this is an HTTP host name. 05:24.380 --> 05:27.500 If it's an email, then it's an email host name and so on. 05:27.880 --> 05:30.960 In my case, because I want to see everything, how can I post it? 05:30.960 --> 05:32.260 I can just say host. 05:32.640 --> 05:38.480 And to specify what I'm looking for in Moloch, I would just say equal equal. 05:39.400 --> 05:47.060 So, equal equal means show me everything that matches exactly our host name. 05:47.680 --> 05:49.940 On ad.ase. 05:50.220 --> 05:51.100 Okay? 05:51.260 --> 05:52.660 That's the first bit of it. 05:52.660 --> 05:53.980 And I can run that. 05:54.200 --> 06:01.720 I also know, since it's our website, I want to say, well, it's a website, it's going to only run on two ports, for example. 06:01.720 --> 06:11.580 So, to kind of add more queries into Moloch, what you need to do is, you just do two amp signs as an end. 06:11.740 --> 06:15.540 And I'm going to open parentheses and add the ports that we need. 06:15.540 --> 06:20.600 So, port equals 80, just like we did before. 06:22.120 --> 06:30.200 And port equals 443, which we know they both are HTTP ports. 06:30.200 --> 06:39.320 Now, before we go any further, you know, under ports I can say, well, I can specify I want it to be a destination port, I can have it be a source port, or so on. 06:39.540 --> 06:48.320 For, just to keep it general, and we can see everything here, I'm just going to say, okay, these ports, this host name, and let's see what we find. 06:48.560 --> 06:51.000 And I messed up something, of course. 06:51.180 --> 06:55.140 Oh, I'm saying end here, and that should be or. 06:55.140 --> 06:58.300 So, it's either port 80 or port 443. 06:59.440 --> 07:06.920 So, in that case, I just do two pipelines, and run that. 07:07.940 --> 07:09.460 Not pipelines, but pipes. 07:09.460 --> 07:14.720 And now I'm saying, okay, show me everything for this host on port 80 and port 443. 07:14.720 --> 07:17.140 And now I can see the traffic. 07:18.300 --> 07:22.140 So, let's go and dive deeper. 07:22.960 --> 07:31.840 On this traffic, I can see all kind of requests, and let's just open something very randomly. 07:31.940 --> 07:36.500 Let's open up this request, and I see it's a get request. 07:36.500 --> 07:39.380 And before we go down, let's spend some time here. 07:39.380 --> 07:44.260 So, idcmalloc will parse all the fields in the packet. 07:44.260 --> 07:50.280 So, I can click on any of these, and I can say, okay, well, what protocol did it come from? 07:50.280 --> 07:52.220 Which IP, which ports? 07:53.260 --> 08:00.360 If it's HTTP packet, then it will also parse the method, the status code, all of these things. 08:00.680 --> 08:02.580 It will also parse the user agent. 08:02.580 --> 08:11.980 Let's say I'm interested in knowing all the user agents that happened in that specific day that visited our website. 08:12.040 --> 08:18.640 I can easily click on user agents, and I can say, okay, export unique user agents with counts. 08:18.640 --> 08:29.800 Once I click that, I will see a new page showing me the unique user agents, along with how many times we've seen this in that specific day, and the other one. 08:29.800 --> 08:34.780 So, we only see two different user agents, nothing abnormal, nothing suspicious. 08:34.780 --> 08:37.580 So, nothing to worry about here. 08:38.300 --> 08:45.360 If we scroll a bit down, we can actually see the actual requests, the raw requests. 08:45.360 --> 08:50.600 So, I can see which hosts they're requesting, where the URI they're going after, and so on. 08:51.320 --> 09:01.440 Because I know we had an incident, and I know the website has been defaced, the attacker must have sent some kind of data to our website. 09:01.440 --> 09:04.540 So, most likely they will not be doing a GET request. 09:04.540 --> 09:07.280 When we see GET, it's just pulling data from our website. 09:07.280 --> 09:10.040 They're going to be posting or sending some kind of data. 09:10.040 --> 09:12.660 So, this excludes this. 09:12.860 --> 09:18.160 So, I'm going to click GET, and we don't want GET anymore, so I'm just going to say net GET. 09:18.420 --> 09:20.940 That automatically adds that to our query. 09:22.580 --> 09:24.820 And let's hit search and see what we find. 09:24.820 --> 09:28.280 So, now we went down to only 12 entries. 09:28.280 --> 09:34.500 Okay, that's much easier to kind of go through and investigate. 09:35.080 --> 09:47.460 Again, we're not sure what happened, but one thing I can sort for is by time, to kind of understand the timeline of these different requests and the events. 09:48.220 --> 09:51.000 And this just opens on the random. 09:53.680 --> 09:55.800 So, again, same kind of thing. 09:56.040 --> 09:59.660 So, now it's a POST request. 10:02.920 --> 10:05.400 And it's just a request in index.php. 10:10.470 --> 10:12.650 And I see a test on sleep. 10:12.650 --> 10:14.850 I'm not sure what all this stuff really means. 10:14.850 --> 10:17.510 And I also see this jumper stuff. 10:17.510 --> 10:18.770 What is this? 10:19.070 --> 10:25.510 So, if we look at the header, which I've already displayed for you, it turns out it's actually encoded with gzip. 10:25.510 --> 10:33.170 Gzip is just a method of compression that web browsers use to compress the data to transfer the least amount of traffic. 10:33.370 --> 10:41.750 So, the nice thing with Molek, what we can do is I can just click uncompress, and now it automatically will decode this packet for us. 10:41.750 --> 10:47.190 And now I can easily read it and see, oh, okay, this is what loaded, this is what the page is showing. 10:47.570 --> 10:52.110 And from what we see here, nothing of interest yet. 10:52.110 --> 10:54.130 Okay, that was useful. 10:54.130 --> 10:56.270 But let's keep going. 10:56.270 --> 10:57.730 Let's find out what happened. 10:58.270 --> 11:02.330 So, another request, and see this. 11:02.330 --> 11:05.410 But this is like, this looks like an IP address. 11:05.410 --> 11:06.490 But what is all this? 11:06.490 --> 11:07.470 I don't know. 11:07.930 --> 11:13.810 Well, what we can do is Molek has Cybershift built in. 11:13.950 --> 11:25.510 Well, Cybershift is a separate open source project, that you can actually just go and do Cybershift, and you can load it outside of... 11:26.510 --> 11:28.330 Cybershift, there we go. 11:31.600 --> 11:35.540 And you can just load it outside of Molek, and you can do your decoding. 11:35.540 --> 11:36.380 But let's get back here. 11:36.380 --> 11:43.860 The beauty of this is when we do it on Molek, it will automatically take that packet data and put it in for us. 11:43.880 --> 11:45.640 So, what is Cybershift? 11:45.640 --> 11:55.360 It's pretty much a tool, a web GUI, that you can use to do different decoding of different languages, decoding methods, and so on. 11:55.360 --> 12:00.600 So, in this case, it's taking the hex code and just decoding it. 12:00.800 --> 12:05.560 And while I'm looking at this, I can see these percentage signs. 12:05.960 --> 12:09.400 Percentage sign means it's a URL kind of thing. 12:09.400 --> 12:12.860 So, I can easily just drag the URL decode. 12:12.860 --> 12:16.700 And now I can see the decoded message here. 12:16.920 --> 12:25.000 Now I can see, oh, it's trying to ping this IP address, which is the same IP that's trying to visit. 12:25.000 --> 12:27.200 Okay, this is interesting. 12:27.200 --> 12:35.120 I'm not sure if they were actually able to ping, but it looks like this IP is trying to do something here. 12:37.380 --> 12:40.540 Let's keep going, see what else can we find. 12:42.860 --> 12:46.620 Next packet, same thing. 12:46.620 --> 12:50.060 Tests, okay, nothing here. 12:50.480 --> 12:54.180 Next packet, so on. 12:55.640 --> 12:57.780 Oh, there's an NC. 12:57.840 --> 12:58.980 Okay, what is this? 12:58.980 --> 12:59.460 Hold on a second. 12:59.460 --> 13:02.220 Let's pull it up again in CyberChef. 13:02.220 --> 13:05.860 Do the same method, decode the URL. 13:05.940 --> 13:10.960 And now we see an NC IP address and port. 13:10.960 --> 13:19.040 So, NC stands for Netcat, which is the utility attackers can use to have a server connect back to them and get a show in that box. 13:19.040 --> 13:24.920 So, what this is saying is, okay, connect back to my IP over this port. 13:25.600 --> 13:34.520 So, what this means, if this actually succeeded, that means our server connected back on this IP address. 13:34.800 --> 13:37.820 Well, let's see if this actually happens. 13:37.820 --> 13:46.600 I'm going to take that port, I'm going to clean all this up, and then I'm going to add the IP address. 13:47.780 --> 13:49.500 It's not going to be the source. 13:52.140 --> 13:57.400 And port equals the port that we're looking for. 13:57.480 --> 13:58.520 Let's search for it. 13:58.520 --> 13:59.700 And we have traffic. 13:59.700 --> 14:00.420 Uh-oh. 14:00.880 --> 14:02.420 This is not good. 14:02.420 --> 14:08.800 Looks like our box, which is the source here, connected back to the server over this port. 14:09.040 --> 14:14.240 We can see how many packets and the amount of data that was sent back and forth. 14:14.600 --> 14:24.960 So, immediately, I would say, okay, well, this is the first connection that has the highest number of packets, that might have something interesting in it, so let me open it up. 14:26.540 --> 14:33.280 Scroll down, and we can actually see the whole conversation back and forth now. 14:33.280 --> 14:47.320 Well, it looks like the attacker ran a command, which is id, which is equal to whoami on Windows, to figure out who the attacker is running as on this box, and they're running as apache. 14:47.580 --> 14:55.220 Then they try to figure out which folder they're in, and they try to access file systems. 14:55.220 --> 14:56.820 Okay, this is not good. 14:57.140 --> 15:07.740 And then they did this cat-index.php, which looks like it's our website, so they're looking into what's inside our main file and so on. 15:07.900 --> 15:09.840 Okay, what is all this? 15:09.840 --> 15:22.220 Oh, and then I see another netcat command here, and saving the connection as a cm0.php file, and our box, locally. 15:22.500 --> 15:26.820 So I'm interested in knowing what this is, but let's keep going. 15:26.820 --> 15:29.020 Let's see what else they do in this box. 15:30.180 --> 15:32.760 And if we come here, they actually did it for us. 15:32.760 --> 15:35.140 They did a cat on this file that they created. 15:36.420 --> 15:40.220 Okay, when they did that, we can see, oh, it's a PHP backdoor. 15:40.480 --> 15:44.420 This is not good, so they used a PHP backdoor on our site. 15:45.060 --> 15:48.800 So how am I going to now figure out what did they do? 15:49.060 --> 15:57.820 I know because of how backdoors work, they have to visit this PHP file explicitly to kind of load the command they want to run. 15:57.820 --> 16:03.240 So what I can say to Molek is, well, show me all the URIs. 16:03.520 --> 16:06.060 So when we say a URI is... 16:07.880 --> 16:14.300 pretty much everything after the domain will be a URI, so anything from here on, that's a URI. 16:14.300 --> 16:18.480 So I want to say, show me everything that has this in it. 16:18.600 --> 16:33.320 And since I can't just do, like, all, what I do in Molek is I just use wildcards at the beginning, so I don't care what's before this, and the end, meaning I don't care what's after it, just show me anything that has this string in the URI. 16:33.580 --> 16:44.820 And when I search it, now, here, we can see all these URLs or URIs with all these different links in them. 16:44.820 --> 16:46.400 What are they trying to do? 16:46.600 --> 16:51.240 Well, how can I know, like, how can I get all this in one nice place? 16:51.240 --> 16:57.240 I can just click on info, and I want to say export unique URI with two counts. 16:58.380 --> 17:06.560 Now, Molek will take this for us and tell us, okay, well, this command was run five times, this command was run three times, and so on. 17:06.560 --> 17:16.900 So now I kind of know which commands they ran, because I can see it's command equal cat, command equal cat, or else, or whatever they're trying to do. 17:17.180 --> 17:20.280 And I kind of see here there's a JPEG file as well. 17:20.280 --> 17:29.460 Okay, so this is how they probably got that JPEG file or the image of the fraud into our website. 17:29.820 --> 17:35.920 So let's find out where did they, like, how did they do this? 17:37.240 --> 17:43.000 Let's go back here, and let's do... 17:43.000 --> 17:47.700 Instead of this, we are looking for that specific JPEG file. 17:48.220 --> 17:53.020 So I'm going to do wildcard and show me anything that has JPEG in it. 17:53.960 --> 17:55.180 Load it up. 17:56.060 --> 17:58.180 And it's going to show me a lot of stuff. 17:59.620 --> 18:06.640 I'm going to say probably it's going to be the biggest file, so I'm going to sort by the size of the data. 18:08.060 --> 18:10.740 And this is the biggest file that we have. 18:14.970 --> 18:16.350 So it's loading. 18:17.010 --> 18:19.950 And here I can see, yeah, it's an image file. 18:23.290 --> 18:25.510 But I'm not sure what it looks like. 18:25.510 --> 18:33.310 So what I can do in Moloch is I can just click on show images and files, and it will actually render it for us right here in the browser. 18:34.390 --> 18:45.030 Now, let's say, you know, I'm not comfortable with doing this or I want to dig in deeper to it or I want to understand what actually happens using Wireshark because I've never used Moloch before. 18:45.030 --> 18:53.090 At any session, at any point, you can always just click on download PCAP, and it will kind of, like, save the PCAPs for you. 18:53.470 --> 19:00.710 So you can open up in Wireshark and, you know, do your analysis manually if you'd like to. 19:04.650 --> 19:12.270 Pretty much that's how they get the file, and that's how they deface our website using a backdoor. 19:12.270 --> 19:17.810 If we have time, I think we have another... I think that's pretty much it. 19:17.810 --> 19:25.050 Okay, so with that being said, I think that's the highlight of Moloch and how to use it. 19:25.050 --> 19:27.290 I want to keep some time for questions. 19:28.590 --> 19:30.950 Please let me know any questions you've got. 19:31.170 --> 19:32.750 I'm also going to be in Discord. 19:32.750 --> 19:38.450 I'll be on Twitter if you need anything else, but I'm going to be here waiting for any questions that you guys have. 19:39.610 --> 19:40.610 Cool. 19:41.290 --> 19:42.470 Thank you, Bashar. 19:42.630 --> 19:54.370 I went ahead and put a note in the text window under the workshop track one with a link over to Recon Episec's OpenSOC Moloch Discord channel. 19:54.830 --> 19:56.250 So definitely check that out. 19:56.250 --> 20:05.370 Obviously, hit Bashar up on Twitter and Discord, but we're trying to help everybody kind of connect with the right people here so you can get the help you need. 20:06.870 --> 20:07.590 Perfect. 20:08.830 --> 20:09.890 Thanks a lot, Bashar. 20:09.890 --> 20:10.990 I appreciate it. 20:11.130 --> 20:11.810 Thank you. 20:11.810 --> 20:12.170 Thank you. 20:12.170 --> 20:12.870 I appreciate your time. 20:12.870 --> 20:13.570 Thank you. 20:13.590 --> 20:14.210 Take care.