00:00:00.125-->00:00:03.958 >> So uh thank you everyone for being here. We’re really excited to be at DefCon. This is my 00:00:03.958-->00:00:10.292 second time. Thomas is his first. Uh and it’s it’s amazing to be around people who care 00:00:10.292-->00:00:16.250 about security and sharing information. So our talk today is about malicious CDNs. And 00:00:16.250-->00:00:20.917 we’re gonna cover one particular one. I mean, there aren’t many, but Zbot has been uh and 00:00:20.917-->00:00:26.375 interesting fast flex proxy network in the past few years. We- we’re gonna show how we have 00:00:26.375-->00:00:32.542 been studying it uh using like SSL scans and combining like a few interesting heuristics that 00:00:32.542-->00:00:37.542 uh use graph theory and uh some statistics—basic. >> Hi so my name is Thomas Mathew and uh I’m 00:00:39.917-->00:00:46.083 a researcher at uh Cisco Umbrella which is formerly OpenDNS and my main focus is on 00:00:46.083-->00:00:52.250 data science and machine learning. >> And I’m Dhia. I’m the head of security research at 00:00:52.250-->00:00:57.250 um Cisco Umbrella and my interests are in graph theory and security uh overall. So what 00:00:59.292-->00:01:05.250 are we talking about today? Uh just a quick overview or a few words about CDNs. Most of us 00:01:05.250-->00:01:10.125 know about them. These are like uh a very interesting and powerful technology that enables 00:01:10.125-->00:01:15.125 uh people who have content especially popular sites to deliver the content. Uh in an 00:01:15.125-->00:01:20.375 efficient way so then people around the world can get the content with low lea- latency 00:01:20.375-->00:01:25.375 based on the edge nodes that are closer to these uh users. So for the sake this talk uh certain 00:01:28.792-->00:01:33.250 features or requirements of a CDN infrastructure are of particular interest of us. So 00:01:33.250-->00:01:40.000 specifically that the uh content of a customer will be delivered uh with low latency. The 00:01:40.000-->00:01:45.458 customer’s website will also be protected against DDoS attacks. The customers also try to hide 00:01:45.458-->00:01:51.750 their origin IP behind the CDN infrastructure. And then if uh the site is uh communicating via 00:01:51.750-->00:01:56.750 HTPS then you can you can also have a SSL search deployed on the edge nodes. Uh so then you 00:01:59.083-->00:02:05.500 can you can guarantee that end to end uh secure com. Now most of us know about the legit ones, 00:02:05.500-->00:02:11.083 so Akamai, Cloudflare, Google Cloud, Amazon Cloudfront. Cloudfare tends to be abused a 00:02:11.083-->00:02:17.167 few times by some kind of bad content. But they are the legit side and we work with them to 00:02:17.167-->00:02:22.333 kind of mitigate some of these uh threats. But then, there are like some pure criminal content 00:02:22.333-->00:02:26.583 delivery networks here. They’re more on the reverse proxy, fast-flex network side. And Zbot 00:02:26.583-->00:02:31.292 is one of them. In fact we’ve been studying this network uh for the past few years. We’ve 00:02:31.292-->00:02:37.458 had the talk BlackHat 2014, 2016 in BotConf and I invite you to go check like the details on how 00:02:37.458-->00:02:42.458 to detect it. And uh some of the other uh features of the infrastructure. So this is kind 00:02:44.792-->00:02:50.042 of a overview of how uh this network operates, so you have in fact around thirty to forty 00:02:50.042-->00:02:56.167 thousand uh uh compromised machines. Mainly routers and access points in Ukraine and 00:02:56.167-->00:03:01.542 Russia and they are maintained and harvested by uh the actors. So it’s not necessarily the guy 00:03:01.542-->00:03:07.042 who is selling the service in the underground. The same as the ones who are basically 00:03:07.042-->00:03:11.917 compromising the machines. Usually you can buy installs for thousands of machines and then 00:03:11.917-->00:03:16.083 you can go and provision them to your customers. So, as we know like there’s this big 00:03:16.083-->00:03:21.583 segmentation of uh expertise in the underground. And now what you do is the, this actual offer 00:03:21.583-->00:03:26.792 in the underground in terms of uh fast flex, so then as a customer let say you are willing 00:03:26.792-->00:03:33.083 to deliver um you know like malware, ransomware sites uh fishing especially uh and we 00:03:33.083-->00:03:37.792 also saw a lot of discarding and cyber crime forums, cyber crime forums who will switch between 00:03:37.792-->00:03:42.500 these infrastructures to protect the contents so it’s not taken down. So it’s not revealed to uh 00:03:42.500-->00:03:47.500 security researchers et cetera. And in that case what they do is they hide behind the Zbot CDN or 00:03:47.500-->00:03:54.083 fast flex proxy network and when they buy the service for a couple hundred dollars, or I 00:03:54.083-->00:03:58.292 would say in that range then you will get uh you will be provisioned ef- I would say 00:03:58.292-->00:04:03.750 between forty to fifty. Sometimes it goes up but it’s in that range, um number of IPs or 00:04:03.750-->00:04:09.583 bots that in which you’ll be having your SSL uh installed and that way you will guarantee that 00:04:09.583-->00:04:14.458 the end to end communication from the victims talking to ransomware c2s, crimeware 00:04:14.458-->00:04:20.958 consumers or researchers like us will be talking HTPS to the uh end to end. So this is kind of 00:04:20.958-->00:04:24.708 an interesting infrastructure, in fact its been around for years and uh, it’s worth 00:04:24.708-->00:04:30.042 investigating from the SSL perspective like we said because all of the bots will have SSL uh 00:04:30.042-->00:04:34.583 certs install- uh installed on them. And when you have like scans you can figure out a lot 00:04:34.583-->00:04:41.292 of interesting patterns. Quickly. I mentioned crimeware forums, dump shops, malware, 00:04:41.292-->00:04:47.583 these are just like screen- screenshots, you can find uh anywhere on the web. And I guess 00:04:47.583-->00:04:53.375 quickly as the cert as we know, it will have the section called common name or or a field, which 00:04:53.375-->00:04:59.208 has to match the domain you’re trying to protect with HTPS. Uh obviously I will not get to get 00:04:59.208-->00:05:04.083 into details where you can have like wildcards and sub-domains and stuff like that. And then 00:05:04.083-->00:05:08.250 the main point is that like we said if you want to protect your site end to end, then you have 00:05:08.250-->00:05:13.375 to have your cert off the let’s say crimeware forums form deployed on the bots you got 00:05:13.375-->00:05:18.375 provisioned so then they can, uh deliver your content with uh SSL encryption. >> So uh, the main 00:05:20.625-->00:05:26.000 objective in today’s is to provide researchers with a series of statistical tools that 00:05:26.000-->00:05:31.000 can allow them to analyze uh large sets of SSL data. And uh all of the data that we’re going 00:05:33.625-->00:05:40.292 to be discussing in today’s talk is actually available uh at the following URL and it’s collected 00:05:40.292-->00:05:45.292 by rapid7 and University of Michigan. So this is a high level uh overview of kind of how 00:05:47.708-->00:05:53.292 the the talk’s gonna gonna go. We’re gonna discuss what exactly is contained in the SSL sonar 00:05:53.292-->00:05:59.083 data and then once we do that, we can then model that data using a bipartite graph which 00:05:59.083-->00:06:05.417 slits the uh data into uh common names and then ASNs. Once we have that bipartite graph we can 00:06:05.417-->00:06:10.417 then start co- collecting kind of what we call global information through a series of 00:06:10.417-->00:06:15.750 histograms that like calculate the relative frequencies of the popularity of both ASNs and 00:06:15.750-->00:06:21.750 domains. Uh these histograms then allow us to create a micro local features at at a per 00:06:21.750-->00:06:26.958 domain basis. And then once we have those tiny histograms we can ce- we can use uh a 00:06:26.958-->00:06:31.250 bucketing scheme to convert them into a vector and then this vector can then be measured 00:06:31.250-->00:06:36.208 against other domains in the same kind of neighborhood of popularity. In the graph. And 00:06:36.208-->00:06:41.583 then ultimately we can then use uh just like a very simple anomaly detection mechanism to 00:06:41.583-->00:06:46.208 see whether a domain in a particular neighborhood is unlike it’s other neighbors. And 00:06:46.208-->00:06:51.417 that’s how we can identify whether a domain is potentially a Zbot host. So with SSL we 00:06:51.417-->00:06:58.333 don’t really need to dig into the de- uh the details too much we just want to know that SSL is 00:06:58.333-->00:07:03.250 a secure socket layer, it’s used for encrypting traffic over HTTP and there’s just been an 00:07:03.250-->00:07:09.417 increase in websites uh employing uh SSL. And the type of SSL data that we’re working 00:07:09.417-->00:07:16.042 with is the x509 certificate and the x509 certificate contains uh information regarding the issuer 00:07:16.042-->00:07:21.708 as well as the subject. And for today’s talk we are more interested in the subject. That 00:07:21.708-->00:07:27.000 is, the person who the SSL certificate belongs to. And in particular, when we were looking 00:07:27.000-->00:07:31.500 at the subject information, we’re interested in a field that Dhia mentioned, it’s called the 00:07:31.500-->00:07:36.458 common name. Now a common name can be any alpha numeric string. But in particular we’re 00:07:36.458-->00:07:41.875 interested inc common names that are uh legitimate domain names. And that’s because we want to 00:07:41.875-->00:07:47.875 see what are the domain names that are associated with a particular x509 certificate. So 00:07:47.875-->00:07:54.292 this is just uh an example of how an x509 certificate looks if you uh decode the uh base 00:07:54.292-->00:08:01.000 sixty-four and one of the useful things about uh SSL data is that it allows us to map out not only 00:08:01.000-->00:08:07.583 residential IPs verse commercial IP space, but also allows us to uh understand where a network 00:08:07.583-->00:08:14.458 can be uh spread out over a a series of IPs. So let’s say we have uh a set of x509 00:08:14.458-->00:08:18.208 certificates and their corresponding IPs. If you look at the common name information, 00:08:18.208-->00:08:24.250 we can see how a particular uh common name is spread out over a series of IPs and then we can 00:08:24.250-->00:08:29.250 make a guess that that common name entity is somehow involved in some way with the uh hosting 00:08:31.708-->00:08:36.708 or the co-location at that IP. And so with sonar data, uh its its about a twice or four times 00:08:38.833-->00:08:43.792 monthly scan. Over the entire IPv4 space. But in actuality because certain network 00:08:43.792-->00:08:49.667 operators don’t allows the University of Michigan to uh scan their IP range, we don’t 00:08:49.667-->00:08:56.292 get information from certain ranges. And just a basic scan on port 443. And then uh the most 00:08:56.292-->00:09:01.292 important part is that we have an x5- x509 certificate as well as information regarding the IP 00:09:03.458-->00:09:10.083 that it was found on. And this is kind of the flow chart that we used uh in order to kind of 00:09:10.083-->00:09:15.250 get our data prepped before we performed the analysis. We take uh a raw monthly scan of sonar 00:09:15.250-->00:09:22.167 data. We can extract out the common names and then we map uh an IP to an ASN and then we come 00:09:22.167-->00:09:28.958 up with this kind of quadruple which is the SSL SHA, the IP, the common name and then the ASN 00:09:28.958-->00:09:33.958 that the IP belongs to. And what’s really go- great about the sonar study is uh because 00:09:33.958-->00:09:39.250 this is a produced on a monthly basis, we can kind of see how hosting patterns emerge and 00:09:39.250-->00:09:44.250 change over a five month period. >> So real quick here what you can see is the range of a I 00:09:47.708-->00:09:53.000 would say number of SHAs we collect every month. So we can see that between 250K and a 00:09:53.000-->00:09:58.625 million unique SHAs, here in a five month period and then from those obviously the data also 00:09:58.625-->00:10:04.625 has the uh the raw certs so you can decode it and then extract the common name. So the point 00:10:04.625-->00:10:09.625 here is that we have a quite a big enough uh I would say a data set that allows us to uh try to 00:10:11.750-->00:10:17.500 find some interesting patterns and find these anomalies and uh I would say threats in general. 00:10:17.500-->00:10:21.500 And the other things is that its difficult to manually inspect these domains. That’s why you 00:10:21.500-->00:10:27.500 are looking we looking for a large scale abstraction model that can help us uh do these 00:10:27.500-->00:10:32.958 kind of analytics. And in the sense, graphs as you know are very useful to do a lot of 00:10:32.958-->00:10:38.333 things so um most of you know about bipartite graphs you have like two sets are disjoint. And 00:10:38.333-->00:10:42.958 in this case we took like a simpler presentation where you have a common name set, 00:10:42.958-->00:10:48.542 connected to the ASNs which means the common name is hosted on an IP, but then that IP 00:10:48.542-->00:10:54.167 belongs to an ASN. And we found that by uh lumping all of the IPs within and the ASN node, 00:10:54.167-->00:10:59.167 that’s more useful for our investigation than keeping the CN to IP bipartite graph. And 00:11:01.667-->00:11:06.333 that’s basically what you end up having. Uh this has been useful to do the type of analysis that 00:11:06.333-->00:11:11.417 we’re gonna describe uh in a second. So I guess the first takeaway is that the bipartite 00:11:11.417-->00:11:16.417 graph is a useful representation of uh this problem that helped us uh solve the uh uh issues. >> 00:11:21.583-->00:11:25.708 So classically there are multiple methods for uh analyzing a graph. And kind of 00:11:25.708-->00:11:30.042 figuring out uh or identifying various sub structures. One is uh you can use a graph 00:11:30.042-->00:11:34.375 factorization technique or you can identify the various connected components within the 00:11:34.375-->00:11:39.292 graph and study each of those connected components. Or kind of calculate the minimum spanning 00:11:39.292-->00:11:43.792 tree. Uh for this talk we’re actually not going to use any of those three methods, we’re 00:11:43.792-->00:11:48.250 instead gonna be looking at a- another set of statistics. But in general our goal is to 00:11:48.250-->00:11:53.708 identify uh possibly anomalous sub-structures within the graph. So a sub-structure within the 00:11:53.708-->00:12:00.458 graph can be thought of as a certain set of domains and ASNs that have some sort of odd shape 00:12:00.458-->00:12:06.875 and I know that sounds slightly vague right now, but as the talk goes forward you’ll start to see 00:12:06.875-->00:12:13.292 what we mean by a sub-structure uh within the graph. Uh so when we are analyzing the graph, we 00:12:13.292-->00:12:19.708 need to come up first with uh a baseline metric with what we consider a normal. And so what 00:12:19.708-->00:12:26.125 we- we wanted to do is create a a metric that is based on the topological features of the 00:12:26.125-->00:12:30.875 graph. That means we’re looking at somehow the relationship between uh domains and their 00:12:30.875-->00:12:36.542 mapping to ASNs and vice versa. Now uh a really easy way to do that is to kind of just look at 00:12:36.542-->00:12:40.875 the popularity of each common name. And so the popularity of each common name becomes as 00:12:40.875-->00:12:47.750 defined as the essentially the degree count at each domain vertex and so we then calculate 00:12:47.750-->00:12:53.375 the frequencies of how a particular domain name is distributed across a set of 00:12:53.375-->00:13:00.292 ASNs. And then for each ASN we model that ASN as having a particular type. And now the 00:13:00.292-->00:13:06.417 type of the ASN is referred to its uh popularity and the popularity of the ASN is how 00:13:06.417-->00:13:12.042 many unique common names uh appear on it. So there’s like this type of weird mirror 00:13:12.042-->00:13:16.292 relationship between the two uh popularity scores that we create. And so Dhia now will 00:13:16.292-->00:13:21.750 kind of show with a simple example of how this works. >> So lets break it down here in in a 00:13:21.750-->00:13:28.167 like a very simple example. So we see our common name uh I would say red set linking to the 00:13:28.167-->00:13:33.292 other set of the bipartite graph which is the ASNs in blue. And uh the nice analogy we’re gonna 00:13:33.292-->00:13:39.458 use in this talk is common names take them as people individuals and ASNs are basically cities or 00:13:39.458-->00:13:44.333 states and you can see that let’s say a person like John at the top, he lived in lets say 00:13:44.333-->00:13:49.417 one, two, three, four, four cities which are four ASNs or he lived in four different states. 00:13:49.417-->00:13:55.708 And what we try to do here is to study the the uh the ASN part. So in, in a sense you looking at 00:13:55.708-->00:14:01.750 the uh behavior of cities in terms of how many people they hosted. So what we have here is 00:14:01.750-->00:14:07.750 that the ASN at the top, the blue one, has three incident edges which means it has a 00:14:07.750-->00:14:12.708 degree of three and it had three common names hosted on it and you look at the other ac- uh 00:14:12.708-->00:14:19.458 follower uh following like ASNs, the second one has an incident uh one incident um edge. Which 00:14:19.458-->00:14:23.125 means it has a degree of one. Anyhow, so what you end up having is like the three bullets 00:14:23.125-->00:14:27.708 at the bottom where you have two occurrences of an ASN with degree one, three occurrences of 00:14:27.708-->00:14:32.708 an ASN with degree two and two occurrences of a degree of three. And the simplified 00:14:32.708-->00:14:37.833 histogram at the bottom you can see ASN on the x. Number of occurrences of that event on the 00:14:37.833-->00:14:43.750 y. And that’s how you can scale that to a bigger data set. >> So when we apply then this 00:14:43.750-->00:14:49.000 technique to the uh entire January data set that we’re on twenty-two thousand ASNs uh the 00:14:49.000-->00:14:55.542 histogram on the right eh is a sampling, a stratified sampling from a a set now of 5k. And what 00:14:55.542-->00:15:01.125 we notice is that there’s a definite, kind of long tailed to this unto this distribution so 00:15:01.125-->00:15:07.667 the majority of the ASNs that are kind of lumped in the zero to five range is you can see 00:15:07.667-->00:15:13.583 what’s being circled but then theirs a couple ASNs way out to the right that host more than 00:15:13.583-->00:15:18.583 fifty thousand unique uh domain names. And- >> Yeah, like the quick takeaway is that uh as 00:15:22.208-->00:15:27.917 Thomas said, the majority of the ASNs are basically hosting between one and a hundred 00:15:27.917-->00:15:32.375 domains. In a sense, the majority of cities in the U.S. for example, are hosting between 00:15:32.375-->00:15:39.083 one and a hundred people. Just to use the analogy. >> So here’s some kind of numbers uh saw raw 00:15:39.083-->00:15:43.958 numbers to kind of back up the that statistic so if you look at the number of ANSs hosting just 00:15:43.958-->00:15:48.625 one uh unique common name it’s around seven thousand six hundred and the number of ASNs 00:15:48.625-->00:15:53.167 hosting two common names its uh a little under four thousand. And then if you look at the 00:15:53.167-->00:15:58.417 number of ASNs who’s hosting uh under twenty unique common names it’s nineteen thousand. So 00:15:58.417-->00:16:04.750 that’s more than ninety-seven percent of all uh ASNs kind of fall within that uh twenty 00:16:04.750-->00:16:11.708 unique uh common band. And of course ASNs hosting more than one hundred are uh only less 00:16:11.708-->00:16:17.208 than a thousand. >> So lets take the mirror set now the common names which are let’s say the 00:16:17.208-->00:16:21.792 people or individuals so in a sense you’re trying to figure out some uh some behavior or 00:16:21.792-->00:16:27.208 like some uh better understanding. So in here we can see that the first red dot has 00:16:27.208-->00:16:33.375 one two three four outgoing edges so it has a degree of four. The next one has five um 00:16:33.375-->00:16:37.917 outgoing edges so a degree of five and you end up having that list of uh I would say events, 00:16:37.917-->00:16:42.583 so two times we have a common name with degree one. Et cetera, et cetera. And you end up 00:16:42.583-->00:16:48.208 constructing that histogram at the bottom where you have like the common name degrees on the x 00:16:48.208-->00:16:53.208 and the number of times that even happened in terms of uh degree value. >> So what happens 00:16:57.083-->00:17:03.583 when we apply then this metric to the global data set, well out of around uh eight hundred fifty 00:17:03.583-->00:17:08.833 thousand domains we kind of did a a sampling kind of represented on the histogram. And we can see 00:17:08.833-->00:17:14.292 again that there’s again uh a very close clustering towards the one to two hundred uh range. 00:17:14.292-->00:17:19.167 And we can kind of see this if we zoom in uh that’s the bottom histogram where we can see the 00:17:19.167-->00:17:25.375 majority of uh common names mapped to say one to three uh unique ASNs and then there’s a 00:17:25.375-->00:17:31.583 very short or sharp drop off and you get ticks at essentially every other uh kind of count in 00:17:31.583-->00:17:37.417 between one to one hundred, a hundred and forty. And so let’s now just take a really quick 00:17:37.417-->00:17:43.500 quick look at the outliers. So uh you can see that one of the outliers is dlink and the other 00:17:43.500-->00:17:47.458 one is uh google video. Both domains that people are pretty familiar with and they’re 00:17:47.458-->00:17:53.375 definitely not malicious. >> Yeah, the the quick takeaway I would say is the google video as 00:17:53.375-->00:18:00.083 we see is found on as a common name is found on two thousand different ASNs. That’s basically 00:18:00.083-->00:18:05.542 the common name you’ll find uh on the search that are used for uh YouTube uh content delivery. 00:18:05.542-->00:18:12.125 Obviously, google has deployed a lot of uh I would say, caching mechanisms on g-nodes around the 00:18:12.125-->00:18:16.958 world so that’s why you see this big diversity of ASNs for google video. So google video is not 00:18:16.958-->00:18:23.750 fast-flux, it’s just a a core uh CDN, uh I would say um common name you find on the search that 00:18:23.750-->00:18:31.000 are serving the content. >> So uh the I guess the one other point I was gonna mention is 00:18:31.000-->00:18:37.375 that there is an exponential kind of drop off when it comes to how uh domain counts are 00:18:37.375-->00:18:43.333 distributed uh across ASNs. So you can see the the jump from sinology to example and then 00:18:43.333-->00:18:49.667 example to dlink and then dlink to Apple iTunes is very rapid. Which uh kind of just shows how 00:18:49.667-->00:18:55.292 quickly everything starts to converge towards uh use extremely tw- towards common 00:18:55.292-->00:19:02.042 names that mapped to very few ASNs. And so why are we kind of talking about all of this and 00:19:02.042-->00:19:05.625 kind of doing all these histograms. Well, the goal of this talk is to find kind of 00:19:05.625-->00:19:11.750 sub-structures uh within in the graph. And so as we know that as you move toward the right of uh 00:19:11.750-->00:19:16.750 this graph and we start to see how an ASN, I’m sorry, a common name maps to more and more ASNs, 00:19:16.750-->00:19:22.708 we gain more information about that uh common name. So for example when we know that an ASN 00:19:22.708-->00:19:28.250 mapped, a common name mapped, to say a thousand ASNs, it’s very easy to understand possibly what 00:19:28.250-->00:19:33.625 that uh common names behavioral role is. But as you move towards say an ASN, I’m sorry a common 00:19:33.625-->00:19:39.583 name that maps to only one uh one ASN it’s very difficult to understand what’s going on. So 00:19:39.583-->00:19:47.625 uh what we can then see is that around ninety-seven percent of all domains uh just map to a 00:19:47.625-->00:19:53.917 single ASN. And the problem here then is that in that range of one to ten mappings. There’s 00:19:53.917-->00:19:59.125 just not enough information to kind of make any type of inference. >> Quick thought here 00:19:59.125-->00:20:05.500 is that uh in general if you’re trying to do data analysis, uh data is useful, but then if uh 00:20:05.500-->00:20:10.917 the data is too sparse, there’s no chance to find anything interesting and if the data is 00:20:10.917-->00:20:16.292 let’s say google video and dlink, they are like so popular that you are not expecting to 00:20:16.292-->00:20:21.458 find anything useful. That’s why here, we kind of focus like Thomas said on a very smaller 00:20:21.458-->00:20:27.708 range that we believe has the core or the instance, the instance of some of the 00:20:27.708-->00:20:31.667 interesting patterns we are trying to track. And in general, we could have taken like a 00:20:31.667-->00:20:35.208 clustering approach, cause this is kind of, kind of an unsupervised method on this big 00:20:35.208-->00:20:39.792 data set, but then we found that using like some simple statistical uh techniques like 00:20:39.792-->00:20:45.625 histograms is very good to find like the uh to get like build this understanding, step by step 00:20:45.625-->00:20:50.333 of the data you’re looking at. So you’re trying to isolate your focus on specific regions that 00:20:50.333-->00:20:59.750 then you can go and like peal them off with other techniques. >> So uh with the following kind 00:20:59.750-->00:21:04.167 of information that we just kind of discussed, it’s very easy to come up with uh a simply 00:21:04.167-->00:21:08.042 heuristic to filter out nine-nine percent of all the domains. We just remove, or we 00:21:08.042-->00:21:13.625 just don’t look at domains that map to fewer or more than uh 10 different ASNs and to kind of 00:21:13.625-->00:21:18.875 expand again, it’s it’s kind of like when you’re analyzing a document. If a document only has 00:21:18.875-->00:21:23.875 a single word or a couple words, its very difficult to know whether that document was just 00:21:23.875-->00:21:29.958 created by some random process. Somebody just kind of typing it out a word and then spitting out 00:21:29.958-->00:21:34.042 that document, but as the document increases in size, it’s potentially a lot easier to find 00:21:34.042-->00:21:39.083 some sort of topic within the document. So the goal here is to kind of find the topic of a a 00:21:39.083-->00:21:45.542 domain. We want to see kind of how the domain deployed within this larger ASNs uh structure 00:21:45.542-->00:21:50.250 and in particular we’re trying to understand whether that topic can be considered malicious. So 00:21:50.250-->00:21:56.917 we’ve talked a lot about kind of the macro level of the graph, but we haven’t talked that much 00:21:56.917-->00:22:00.875 about the micro level. And so the micro level is understanding how a particular domain is 00:22:00.875-->00:22:10.500 mapped to a set of ASNs. So the two histograms right here uh kind of give uh frequency counts 00:22:10.500-->00:22:16.833 at at domain uh level. So the x axis denotes the type of ASNs and just as a refresher the type 00:22:16.833-->00:22:23.167 of ASN um means how many other domains are are mapped to that ASN. And then the y axis 00:22:23.167-->00:22:28.417 represents the frequency of that type of ASN. For the particular domain. So if you look at the 00:22:28.417-->00:22:34.708 top uh histogram for naranyamarket, we can see that it contains one ASN that is 00:22:34.708-->00:22:41.375 extremely popular so this d- uh ASN hosts more than uh 25k other unique uh domains. And then it 00:22:41.375-->00:22:49.667 has a concentration of ASNs that are hosting at least a thousand other unique uh domains. And 00:22:49.667-->00:22:56.375 that’s kind of where its, general, kind of mass density is concentrated. But if you look at 00:22:56.375-->00:23:00.792 meenyousecu and its ASNs frequencies, we notice that the ASN that contains the most uh 00:23:00.792-->00:23:05.958 oh- hosts the most other domains is only a little under two thousand. And then the majority 00:23:05.958-->00:23:13.042 of its ASNs that its found on are only hosting one or five other unique domain names. So 00:23:13.042-->00:23:19.792 there’s clearly a difference in how these two uh domains are distributed. >> To bring back 00:23:19.792-->00:23:24.292 the analogy of earlier so uh I guess that people living in cities or states like 00:23:24.292-->00:23:31.500 naranyamarket think about this that as let’s say John and John happens to have lived on uh one 00:23:31.500-->00:23:36.042 single city that had twenty-five thousand people in it, but most of the time he lived in the 00:23:36.042-->00:23:40.667 cities that had between zero and lets say one and a thousand people. So you can see kind of 00:23:40.667-->00:23:45.583 the how this guy is migrating between different cities. Similarly for meenyousecu, it 00:23:45.583-->00:23:54.000 happened to have lived only on one city or ASN that had also re- or re- like hosted fifteen 00:23:54.000-->00:24:00.125 hundred plus other common names or people, but most of the time, the meenyousecu has kind of 00:24:00.125-->00:24:06.375 rotated around a cities or ASNs that were in that smaller range at the bottom. So these are like 00:24:06.375-->00:24:10.792 very lowly populated ASNs. And this as we will see is very interesting because it will tell 00:24:10.792-->00:24:15.458 you what’s this common name used for depending on uh how- where is it being res- where is it 00:24:15.458-->00:24:23.000 residing and how is it moving around on the ASN ecosystem. >> So I understand that pictures 00:24:23.000-->00:24:27.625 can sometimes be a little bit confusing, especially at the resolutional scales that I’ve 00:24:27.625-->00:24:32.500 had them up, so hopefully this kind of numeric information can kind of further highlight what 00:24:32.500-->00:24:38.292 we were discussing. We can see that for meenyousecu, uh uh the ASNs that it’s hosted on are all 00:24:38.292-->00:24:43.500 ASNs that just host one two or three other uh unique domain names. While for uh 00:24:43.500-->00:24:49.167 naranyamarket, there’s not a single ASN that hosts uh less than a thousand other unique 00:24:49.167-->00:24:55.708 domain names. So the kind of, this general intuition it’s only natural to ask, how can we come 00:24:55.708-->00:25:01.833 up with the mechanism to determine how far a part or how different uh naranyamarket and 00:25:01.833-->00:25:11.458 uh meenyousecu are? Real quick in fact this one the meenyousecu is part of Zbot so having that 00:25:11.458-->00:25:16.583 interesting patter will be uh highlighted later and we are able to find it with an 00:25:16.583-->00:25:23.417 unsupervised method. >> So uh as I mentioned, you you can directly compare uh these two 00:25:23.417-->00:25:29.292 histograms because they’re on completely different scales. Uh, meenyousecu is only having 00:25:29.292-->00:25:35.042 counts under a thousand essentially and uh naranyamarket is having counts over a 00:25:35.042-->00:25:42.083 thousand. So we need some sort of representation of the entire spectrum of possible uh domain 00:25:42.083-->00:25:49.625 ASNs counts uh to kind of be created. And so this object will be unique per domain and then we 00:25:49.625-->00:25:55.583 can use some kind of vector as a mechanism to determine similarity and so in order to 00:25:55.583-->00:26:00.417 create this vector we need to have a a bucketing scheme, which maps certain regions of counts 00:26:00.417-->00:26:07.375 to a particular dimension within that vector. And so in this case we’re interested in domains that 00:26:07.375-->00:26:12.125 might be mapped to a variety of different ASNs, but the ASNs that they’re mapped to are 00:26:12.125-->00:26:17.917 actually quite unpopular. So what we’re looking for, what we’re most interested in are 00:26:17.917-->00:26:23.417 then uh ASNs that are at a very low frequency and as a result we then create a bucketing scheme 00:26:23.417-->00:26:29.750 that is incredibly sensitive to low frequencies. Uh the best to think about this is perhaps in a 00:26:29.750-->00:26:35.167 picture. Let’s say you’re interested in a certain color uh then you divide a filter that 00:26:35.167-->00:26:40.500 essentially blocks out other colors and then kind of just focuses either just on the gray 00:26:40.500-->00:26:47.958 or the purples and same way you can think of the distribution of uh domain uh ASN distribution as 00:26:47.958-->00:26:52.958 kind of belonging to colors. So like low frequencies are more like blues, high frequencies are 00:26:52.958-->00:26:59.542 more like reds and we care more about blues. So as a result when we bucket uh the histogram, per 00:26:59.542-->00:27:04.417 domain we bucket into nine different bans and each ban refers to a kind of an index of 00:27:04.417-->00:27:11.042 popularity. So you have a band thats uh counting the number of ASNs that map to one five, five 00:27:11.042-->00:27:16.042 and ten, ten and twenty, but then as we go larger, we increase the size of each 00:27:16.042-->00:27:22.125 bucket. So that means that for example all the uh numbers that range between one thousand and 00:27:22.125-->00:27:27.750 four thousand will all be mapped to the same bucket in the vector. And so again this allows 00:27:27.750-->00:27:33.042 us to have a much better resolution of understanding how a a domain is mapped to lower 00:27:33.042-->00:27:38.333 frequency uh ASNs. And I- if this has become a little bit comp- I kind of messed up the 00:27:38.333-->00:27:45.333 explanation, this slide kind of gives a really nice uh pictorial representation. >> Yeah, so if 00:27:45.333-->00:27:52.375 we look here, we can see meenyousecu and it has that long uh I would say um, uh table or 00:27:52.375-->00:27:59.000 sorry array. So the array as we can see it has all of these one one one’s and based on the 00:27:59.000-->00:28:04.583 bucketing that Thomas described, one five, we’re basically counting how many numbers uh are 00:28:04.583-->00:28:09.250 in the range of one five so you can see one one one, all the way to five, so that it it gives you 00:28:09.250-->00:28:13.208 like fifteen occurrences of those numbers. And you cannot keep going, so you have the five 00:28:13.208-->00:28:17.875 to ten. You have basically five numbers occurring in the range and that’s how you build your 00:28:17.875-->00:28:22.292 vector for both your top domain and the bottom domain. And you can see that that way you can 00:28:22.292-->00:28:27.042 have these two vectors, that will allow you to compare these two uh uh domains. Uh in the 00:28:27.042-->00:28:39.167 same scale. >> Yeah, I guess just a, the takeaway three here is that the majority of domains 00:28:39.167-->00:28:46.167 we saw earlier are mapping or hosted or living in between one to two hundred ASNs and ask 00:28:46.167-->00:28:50.833 Thomas said we had to devise a bucketing that is sensitive to low popularity ASNs. In other 00:28:50.833-->00:28:56.917 words you have a configurable variable resolution uh depending on the lower uh I would say bins 00:28:56.917-->00:29:02.542 of interest. Now the next, the next step here is that we want to go back and focus on the 00:29:02.542-->00:29:07.958 common name and how many ASNs each domain maps to. And that will be like the next step that 00:29:07.958-->00:29:13.458 will lead us to uh explain to you how we find these outliers and hence these Zbot domains 00:29:13.458-->00:29:20.208 within this very big data set. >> K, so we now are going to go back to the original list of uh 00:29:20.208-->00:29:27.625 domains that were found in the x509 certificate. And so as we know we can kind of filter out 00:29:27.625-->00:29:32.792 the make mega outliers, the dlinks the googles because we kind of already have a very good 00:29:32.792-->00:29:37.792 idea of what they are and now what we want to do is come up with a mechanism to kind of 00:29:37.792-->00:29:45.208 create neighborhoods of domains. And in this case a neighborhood of a domain is uh other domains 00:29:45.208-->00:29:52.417 that share a very close uh count in how many other ASNs they’re mapped to. So for example uh on 00:29:52.417-->00:29:58.875 the picture you can see that let’s say we were looking at the list of domains that map to a 00:29:58.875-->00:30:03.792 hundred and sixty to a hundred and fifty other ASNs. Well in that kind of neighborhood you 00:30:03.792-->00:30:11.458 would have like iTunes at apple dot com, uh asos-media, download dot mcafee dot com and so all of 00:30:11.458-->00:30:16.625 these domains we say belong to the same neighborhood. Because they all kind of map to the same 00:30:16.625-->00:30:23.458 out of ASNs. So once we have a neighborhood uh we also have a histogram, a histogram vector 00:30:23.458-->00:30:28.083 that we created for each domain. And now this is where we can just apply uh a really simple 00:30:28.083-->00:30:35.583 pairwise Euclidean distance between any of these two domains using the domain’s histogram. >> 00:30:35.583-->00:30:42.458 Real quick, like a quick analogy again is think about the bands as like income. So you have like 00:30:42.458-->00:30:47.667 people and they are in the band of income like one fifty, one sixty k, and you have like these 00:30:47.667-->00:30:52.417 cities with neighborhoods and you’re trying to find people who are within a, how close are they 00:30:52.417-->00:30:57.042 to each other if they’re making within that range of income. And you’ll see later with Thomas 00:30:57.042-->00:31:01.375 that some of them will have some interesting outliers and they they are maybe anomalous, 00:31:01.375-->00:31:07.500 they’re making this much money, but maybe it’s there’s something fishy about them. >> So this is 00:31:07.500-->00:31:12.375 a a hypo- uh it’s like a hypothetical distance matrix for a band or a neighborhood that 00:31:12.375-->00:31:17.500 contains like three domains. Domain one, domain two, domain three. So uh each cell you can 00:31:17.500-->00:31:24.875 think of the value there is calculated by uh calculating the distance between any two uh 00:31:24.875-->00:31:31.000 domains. So let’s just look at the red column. So uh the distance between domain one and 00:31:31.000-->00:31:36.750 itself is naturally going to be zero right? Then the uh the distance between uh domain two 00:31:36.750-->00:31:41.708 and domain one is also gonna be some value and then the distance between domain three and domain 00:31:41.708-->00:31:45.833 one is also going to be some value. And so what this means is that if I look at the red 00:31:45.833-->00:31:51.875 column, I can then see the distance between d1 and ev- and every other domain in it’s band. 00:31:51.875-->00:31:59.708 And uh this naturally kind of means that if you want to find a domains that are very different 00:31:59.708-->00:32:06.292 from its neighbors, we just calculate the uh- Euclidean norm of each column of this of this 00:32:06.292-->00:32:12.833 matrix. Uh and that then allows us to kind of figure out uh how different it is from its 00:32:12.833-->00:32:17.042 neighbor. And of course the larger the value, that’s gonna mean that its more different 00:32:17.042-->00:32:22.542 than its neighbors. So over the January, Feb- uh data set as a trial, we kind of ran this 00:32:22.542-->00:32:27.375 awhile ago. And we had one really interesting case in the neighborhood of a hundred to a 00:32:27.375-->00:32:35.125 hundred and ten. And so as you can see, the average in this ban or in this neighborhood is 00:32:35.125-->00:32:40.375 around a hundred and twenty-eight or so. But there’s one very clear outlier, which 00:32:40.375-->00:32:45.083 has an overall distance from its neighbors from around five hundred and sixty-seven and so 00:32:45.083-->00:32:49.917 in the histogram you can see how the averages are all kind of bunched up in this really tall 00:32:49.917-->00:32:56.292 spike and then you have these two outliers way out at four hundred and around five hundred. 00:32:56.292-->00:33:01.375 And just kind leads it kind of the its its easy to kind of calculate the standard deviation 00:33:01.375-->00:33:05.417 then of these two and then notice that they’re go that they’re definitely two standard 00:33:05.417-->00:33:10.542 deviations away. And so what was great was when we ran this we found this domain called 00:33:10.542-->00:33:16.250 tangerine dash secure dot com. Which uh through some kind of more further manual probing uh 00:33:16.250-->00:33:22.958 we were able to actually identify as a Zbot domain. But Zbot lives in other ranges as 00:33:22.958-->00:33:28.917 well and so again, in the neighborhood of thirty to forty different ASNs, we found a 00:33:28.917-->00:33:34.708 couple of other outliers. So in this case, uh as the amount of kind of information decreases 00:33:34.708-->00:33:39.750 right? ‘Cause they’re going from say a hundred, which is a lot more information that say like, 00:33:39.750-->00:33:44.292 thirty, the the spectrum of possible distances increases. So it becomes a little bit nosier. 00:33:44.292-->00:33:48.667 But at the same time if you look at the tail of the histogram. There’s still ins- interesting 00:33:48.667-->00:33:53.000 domains. So the majority of distances are kind of all, kind of all nicely lumped together, 00:33:53.000-->00:33:58.167 but then if you cross the two hundred barrier, uh the two hundred distance mark, you can 00:33:58.167-->00:34:03.125 see they co- they’re actually five domains. And uh, out of these five domains, uh three 00:34:03.125-->00:34:09.833 actually turned out to be uh malicious. So meenyousecu, securedataassl and secure 00:34:09.833-->00:34:17.292 tangerine access dot com. And uh the the kind of the further validation was done actually 00:34:17.292-->00:34:22.000 through some more passive DNS down the road and then some more active probing. Uh and what’s 00:34:22.000-->00:34:26.375 really good about this method is we were able to take a set of around eight thousand, eight 00:34:26.375-->00:34:31.000 hundred thousand domains and then reduce it to a far more kind of manageable set of around 00:34:31.000-->00:34:35.500 eight domains that we can inspect manually. Uh by hand or give on to an analyst. And also 00:34:35.500-->00:34:41.792 gain IP information. >> Look about the previous the analogy you could think about, but also 00:34:41.792-->00:34:46.583 you have all of these Zbot domains are trying to kind of hide within the large how would 00:34:46.583-->00:34:51.708 say SSL ASN IP space ecosystem and they’re part of the same gang, however, you can see that 00:34:51.708-->00:34:57.958 some of them have lived in low to medium to high income that that’s the analogy and then with 00:34:57.958-->00:35:05.167 this method you were able to find them uh with all of that pipeline of of macro micro 00:35:05.167-->00:35:10.500 distance measurement uh calculation that we described to you. So I guess the, a few final 00:35:10.500-->00:35:16.583 thoughts here is that when we found these this last list that was reduced from eight thousand 00:35:16.583-->00:35:22.500 k to I would say you know less than ten, uh we had to use some extra signals to verify the true 00:35:22.500-->00:35:28.833 positives and also we doubt the false positives and for that we used some simple ones like how 00:35:28.833-->00:35:34.917 many uh SHAs does the common name map to? And also the a ratio between the IPs that the 00:35:34.917-->00:35:43.125 SHA was found on uh to the uh ASNs where those IPs are living. And you can see that that ratio 00:35:43.125-->00:35:49.333 ha- a ha- was very revealing to find all of these Zbot domains, like meenyousecu, the secure 00:35:49.333-->00:35:55.792 data SSL and then tangerine. They happen to have IP count over ASN count between one and 00:35:55.792-->00:36:00.125 two. In other words, actually that was confirmed uh, that confirms the uh the business 00:36:00.125-->00:36:05.917 model of the actor behind Zbot, where when he sells you between forty and fifty IPs, he will 00:36:05.917-->00:36:12.167 never give you uh IPs that belong to uh the same ASN. Usually you will get like on IP 00:36:12.167-->00:36:16.583 per ASN and he tries to diversify uh the the offering that he gives to his uh 00:36:16.583-->00:36:22.792 customers. Um so yeah, this has been like very useful to find this actionable intelligence so 00:36:22.792-->00:36:27.208 we can like block these domains or or further investigate them. Obviously we have like some 00:36:27.208-->00:36:31.750 other systems to catch these uh Zbot domains, but the whole exercise that we tried to share 00:36:31.750-->00:36:36.083 with you is that you can start with like very large data set and kind of try to peal it off 00:36:36.083-->00:36:40.667 by building this understanding from a macro micro and then as the intuitions kind of 00:36:40.667-->00:36:45.792 strengthen, you end up having like these interesting ways to uh find like uh reduce the set 00:36:45.792-->00:36:52.833 and have it um managed have it uh, to the scale where its manageable uh by hand. Or by uh 00:36:52.833-->00:37:00.125 eyeballing. So I guess uh just a quick comparison here. You have secure tangerine which is Zbot. 00:37:00.125-->00:37:05.000 Blue apron which is a legit uh domain and they happen to live in the same neighborhood, like 00:37:05.000-->00:37:10.875 they’re making between thirty and forty k uh money wise. Uh the idea is that th- they live 00:37:10.875-->00:37:16.500 in ASNs that have uh actually they they are mapped they live in a neighborhood where the most 00:37:16.500-->00:37:22.333 people, most common names there are mapping to thirty to forty ASNs. And for tangerine, you can 00:37:22.333-->00:37:26.375 see that all of the ASNs we mention here are all like Ukranian ISPs. It’s basically 00:37:26.375-->00:37:32.250 residential uh you know IPs that are either pop bought, routers or access points uh that have 00:37:32.250-->00:37:38.292 been uh leveraged for this infrastructure. Where as blue apron is hosted on, even though 00:37:38.292-->00:37:42.375 its in the same band, it’s hosted on like here legit Akamais and Orange and Amazon. 00:37:42.375-->00:37:50.250 So I guess, the takeaways I’ll let Thomas go over those. >> Um so I guess one of the big 00:37:50.250-->00:37:55.167 takeaways we took is you can use kind of the global structure of the ASN uh domain name graph. To 00:37:55.167-->00:38:01.750 help us inform decisions at the local level. And then there’s also a really kind of easy 00:38:01.750-->00:38:05.958 statistical tools that you can use to whittle down a pretty large data set into something 00:38:05.958-->00:38:11.125 that’s far more uh manageable. And what was kind of great about this is that we start off in 00:38:11.125-->00:38:16.292 January and then we kind of ran this approach uh every month on the subsequent uh kind of sonar 00:38:16.292-->00:38:24.083 SSL feeds and started monitoring the the IP space for the domains that we found and Dhia will kind 00:38:24.083-->00:38:29.292 of show some more uh examples in in the later months, like in April and June. Of what else we 00:38:29.292-->00:38:34.958 found. >> So one more thought actually about the main takeaways. So you might think ok 00:38:34.958-->00:38:40.625 you spent all this time to just catch like eight or three things, uh fair enough, but the 00:38:40.625-->00:38:45.042 exercise is useful because you could take this generic method and apply it on any other data 00:38:45.042-->00:38:54.500 set so most of research is often is not about the problem itself that you’re trying to solve, its 00:38:54.500-->00:38:56.833 about the whole methodology and the mental exercise you go through with your team and 00:38:56.833-->00:39:00.000 you’re peers, so that’s why we felt like this might be useful for you know as a general 00:39:00.000-->00:39:03.625 thought process so we could apply it on other data sets where you can basically 00:39:03.625-->00:39:12.458 represent the the data as a bipartite graph x to y. Now some bonus slides about the Zbot 00:39:12.458-->00:39:17.083 infrastructure itself. By studying SSL we were seeing like some interesting um patterns 00:39:17.083-->00:39:22.625 actually. The slide got messed up there unfortunately. But the two ta- timeworn shows the 00:39:22.625-->00:39:28.417 malware C2 domain that has a different way to operate with SSL than a domain at the bottom 00:39:28.417-->00:39:34.708 which is private zone dot w s which is a known crimeware forum. So the top you can see 00:39:34.708-->00:39:40.792 that uh orospu.cc was created on April of this year. The first DNS query we saw on our traffic 00:39:40.792-->00:39:44.583 were four days later, then it was hosted two days later on another bulletproof 00:39:44.583-->00:39:49.333 infrastructure we call Alex, that we actually covered on Thursday at Black Hat. And then 00:39:49.333-->00:39:56.708 on April twenty-third, uh there was a cert created that was then deployed on the Zbot fast flex 00:39:56.708-->00:40:00.167 and the domain started being hosted on Zbot. So you can see here, as you want as you buy the 00:40:00.167-->00:40:06.167 service, you’re immediately provisioned with a SSL either you buy it yourself and you 00:40:06.167-->00:40:10.917 provide it to the actor and he will push it on the machines on the nodes. Or he will do it for 00:40:10.917-->00:40:15.667 you. Similarly for private zone dot w s, which is a known crimeware forum for years, the 00:40:15.667-->00:40:21.583 domain was created ike four years ago, like 2014 I would say, for a long time its been 00:40:21.583-->00:40:27.375 hiding behind the cloudFlare, but actually it had an origin IP um that was unknown lets say. 00:40:27.375-->00:40:33.292 Unless you have other ways, like SSL, passive DNS probing. But then over that period of 00:40:33.292-->00:40:39.333 CloudFlare protection, they were using a variety of SSLs. Uh provisioned by CloudFlare. Then 00:40:39.333-->00:40:45.250 on May the seventh 2017, they created a dedicated SSL cert and then on May the seventh, the 00:40:45.250-->00:40:50.792 same day they started being hosted on the Alex fast flex infrastructure. Uh and then on 00:40:50.792-->00:40:56.917 June twenty-seventh now they became, uh hosted on the Zbot with the same old SSL and 00:40:56.917-->00:41:02.125 finally on July nineteenth they created a second SSL and pushed it on the uh edge nodes that are 00:41:02.125-->00:41:07.500 bought by the customer which are around forty to fifty machines in Ukraine and Russia. So what 00:41:07.500-->00:41:12.625 they’re trying to show here is that it’s kind of interesting to see how the actor uh sets up his 00:41:12.625-->00:41:17.625 backend infrastructure and how he maintains all of these uh different like domain creations, 00:41:17.625-->00:41:25.000 SSL creation, hosting change of SSLs, et cetera. I guess the final slide here is like the 00:41:25.000-->00:41:31.625 same one as we showed earlier. Just like maybe to a kind of do a a bring it all together. So 00:41:31.625-->00:41:36.208 again its like uh an infrastructure that is uh provided for customers to hide 00:41:36.208-->00:41:41.792 their content behind the scenes. You can have one SSL cert per domain or you can have um, often 00:41:41.792-->00:41:48.708 times act- actually we saw even if the bot is still holding the common name of a known uh 00:41:48.708-->00:41:54.875 crimeware forum, it doesn’t prevent uh any other domain to be uh uh I would say hosted or 00:41:54.875-->00:41:59.167 delivered through that IP. But then that case it will not be using the SSL encryption. 00:41:59.167-->00:42:03.542 Because the the domain of the new site will not match the common name of let’s say private 00:42:03.542-->00:42:11.333 zone. So yeah. That was it. Thanks again or your attention and hopefully [applause] 00:42:11.333-->00:42:25.917 Questions? >> Yeah? >> What tools are you using to process the data and generate the graph? 00:42:25.917-->00:42:31.125 >> Uh we just used uh Python and CBORD. Uh, yeah, so. You actually uh what’s kind of funny 00:42:31.125-->00:42:36.792 is you don’t really need any fancy machine learning stuff, you just kind of just gotta use 00:42:36.792-->00:42:41.208 some basic stats and you find interesting stuff in the data. >> So yeah, like another thing 00:42:41.208-->00:42:49.625 the the the whole uh scans were pushed into hbase that way we can kind of do the the search at 00:42:49.625-->00:42:55.167 scale. But yeah, like Thomas said, it’s mainly like uh >> [inaudible off mic comment] >> 00:42:55.167-->00:43:00.292 Yeah, and like some some good, the I would say judgement and discussions with the team. >> 00:43:00.292-->00:43:18.458 [inaudible off mic question] >> Oh uh, he asked whether we found anything interesting about the 00:43:18.458-->00:43:24.750 certificate authorities uh for the SSL cert, for like the the domains that were hosting uh 00:43:24.750-->00:43:30.667 Zbot. >> Well, unfortunately your typical uh abused ones like the Commodore and Lets Encrypt 00:43:30.667-->00:43:35.417 and I don’t want a, you know, name names, but those are the ones you see a lot being used, 00:43:35.417-->00:43:41.792 ‘cause either they offer either like free certs or they have a lot of resellers. Usually you 00:43:41.792-->00:43:46.333 find a lot of these uh I would say suspicious or bullet proof hosting providers who will offer 00:43:46.333-->00:43:53.208 you hosting plus SSL uh certs. So it became like a very um, like a very common commodity to 00:43:53.208-->00:44:12.083 get like a cert with the hosting uh space. >> [inaudible off mic question] >> You’re saying did 00:44:12.083-->00:44:19.000 we use the same method to find some other bot nets? Good question, so at the moment Zbot 00:44:19.000-->00:44:24.333 happened, we we are tracking other uh I would say bullet proof hosting infrastructures 00:44:24.333-->00:44:28.583 that are uh I would say distributed, but this one happens to be the only one that 00:44:28.583-->00:44:35.417 uses a CDN like uh structure. So the others they will have certs, but they are deployed on only 00:44:35.417-->00:44:42.125 one or two domains as we saw sorry, uh IPs, so as we saw earlier, is the do- information 00:44:42.125-->00:44:47.500 is to sparse, then there isn’t much to find with this method. Then you’ll you’re better off 00:44:47.500-->00:44:51.958 like using some other techniques that are much simpler. And no need for like complicating your 00:44:51.958-->00:44:58.708 life basically. But, >> [inaudible off mic comment] >> Ok, sorry you mean DDoS 00:44:58.708-->00:45:04.708 commander control. We haven’t. So that that would be a good uh discussion. We can talk. Yeah, 00:45:04.708-->00:45:15.083 thank you. >> [inaudible off mic question] >> Well, Akamai were not a problem. Like we just 00:45:15.083-->00:45:20.750 said, they were good and they were hosting good stuff. >> Yeah, it was a bunch of like 00:45:20.750-->00:45:26.333 residential people who actually were like unsuspectingly hosting a lot of a lot of these guys. 00:45:26.333-->00:45:31.875 Akamai for the most part is actually uh really uh have a clean network. >> Yeah, like a 00:45:31.875-->00:45:36.750 the reminder is that uh, most of these bot IPs are on Ukraine and Russia. So they’re mostly 00:45:36.750-->00:45:41.917 residential ISPs that are being I would say abused. So, Akamai was an example that we showed it 00:45:41.917-->00:00:00.000 was a legit, clean uh uh I would say infrastructure. Thank you all. [applause]