[00:00.000 --> 00:04.180]  Hey, everyone. My name is Jason, and thanks for joining us this morning.
[00:04.380 --> 00:09.420]  Super big thanks to the Red Team Village for having me and all the other speakers on
[00:09.420 --> 00:14.160]  and the staff for facilitating everything that's going on with the conference for DEF CON
[00:14.160 --> 00:17.340]  for hosting the Village. A big place in my heart for DEF CON.
[00:17.340 --> 00:22.320]  So today I'm going to talk a little bit about my methodology for recon.
[00:22.420 --> 00:26.700]  I've been doing this talk called the Bug Hunters Methodology for five years now,
[00:26.700 --> 00:30.900]  and I basically update it every year, and I talk about new tools and techniques
[00:30.900 --> 00:34.880]  in different spaces of the Bug Bounty scene and Red Teaming.
[00:34.880 --> 00:36.780]  So today we're going to go over the recon stuff.
[00:37.820 --> 00:41.640]  Now, the Bug Hunters Methodology, TBHM, is a presentation, like I said,
[00:41.640 --> 00:44.800]  I've been running for a long time, and basically it's too big for any one conference slot,
[00:44.800 --> 00:48.060]  so I split it into two sections. One is recon, which we're going over today,
[00:48.060 --> 00:53.340]  and the other is application analysis, which is on-site hacking and stuff like that.
[00:53.340 --> 00:57.540]  So today we're going to do recon, and I'm working on this year's version
[00:57.540 --> 01:00.800]  of application analysis, haven't finished it yet. I'm a giant slacker, sorry.
[01:01.340 --> 01:03.240]  So yeah, today we're going to go over recon.
[01:04.480 --> 01:10.140]  A little bit about me. I'm a husband, father, hacker, gamer, and I stream sometimes.
[01:10.140 --> 01:12.540]  These are my socials, so if you want to reach out and ask a question
[01:12.540 --> 01:14.840]  after the talk or something like that, just hit me up.
[01:15.540 --> 01:20.200]  I was a bug hunter, and then I worked at Bug Crowd for many years,
[01:20.200 --> 01:23.600]  and now I'm the head of security at Ubisoft,
[01:23.600 --> 01:28.520]  and so I lead the security team there. Awesome team, great people.
[01:28.740 --> 01:32.120]  And I'm also a gamer at heart, so I play some video games.
[01:32.120 --> 01:36.380]  These are my kids, Arcadia and Avalon in the top picture.
[01:36.380 --> 01:40.440]  I took them to DEF CON for the first year. Last year I did DEF CON Kids or Roots.
[01:40.440 --> 01:43.760]  It was amazing, they had a great time. And then that's my son Arlen,
[01:43.760 --> 01:46.640]  him and I being pirates in the bottom right-hand corner.
[01:46.640 --> 01:49.640]  It's a little bit about me. I've been doing pen testing for about...
[01:51.720 --> 01:53.440]  a long time, I'm old.
[01:55.460 --> 01:59.640]  I have a lot of related experience to this talk.
[01:59.780 --> 02:03.860]  The first thing I want to talk about is today we're going to go over
[02:04.520 --> 02:08.540]  recon methodology, and we do recon in red teaming engagements,
[02:08.540 --> 02:11.420]  and we do recon in bug bounties primarily. And you can do it
[02:11.420 --> 02:14.920]  in some wide-scope pen tests as well. And one of the first things
[02:14.920 --> 02:18.540]  I talk about is when you're doing an assessment, some sort of assessment,
[02:18.540 --> 02:21.180]  whether it's one of those three, you have to have a way
[02:21.180 --> 02:24.140]  that jives with you to record your work.
[02:24.140 --> 02:27.180]  Otherwise, you're just throwing darts and forgetting what you're doing.
[02:27.180 --> 02:33.460]  And this is counterproductive to basically keeping organized.
[02:33.460 --> 02:37.560]  So the first thing that I talk about here is how I keep notes.
[02:37.620 --> 02:41.460]  And I use a tool called X-Mind, which is a mind-mapping tool.
[02:41.460 --> 02:44.860]  And basically, I create nodes, and I create methodologies
[02:44.860 --> 02:48.000]  as checklists inside these nodes, and I track all my domains
[02:48.000 --> 02:50.420]  and which ones I've worked on by color codes.
[02:50.560 --> 02:53.500]  So this is something I do. I've seen other people do
[02:53.780 --> 02:57.520]  a lot of project tracking for assessments in just Notepad or Vim,
[02:57.520 --> 03:00.620]  or there's some specialized pen test tools. If you're really fancy
[03:00.620 --> 03:02.820]  and you have a subscription, you can probably do it in something fancy
[03:02.820 --> 03:06.260]  like Dratus or something like that. So there's a lot of tools,
[03:06.260 --> 03:10.100]  but it's just really important to track your work when you're doing recon.
[03:10.100 --> 03:13.540]  Recon is the art of finding as many assets as possible
[03:13.540 --> 03:19.220]  related to a target, and it can get pretty data-heavy and dense
[03:19.220 --> 03:22.680]  as you do it. So having a way that works for you to track your data
[03:22.680 --> 03:25.720]  is the first thing that's going to guide you to success.
[03:25.900 --> 03:28.240]  So this is an example of how I track my data
[03:28.240 --> 03:31.880]  when I'm on a target. So Tesla, Tesla Motors,
[03:31.880 --> 03:36.080]  has a pretty open-scope bounty on Bug Crowd.
[03:36.220 --> 03:39.500]  And this is how I would start off my project with Tesla.
[03:39.500 --> 03:42.380]  So in the middle, I have a node in this mind-mapping program
[03:42.380 --> 03:46.080]  I use called xMind. And then on the left-hand side,
[03:46.080 --> 03:49.920]  I have their autonomous system numbers, their ASNs.
[03:49.920 --> 03:54.140]  I have any acquisitions they've made. I have some other notes,
[03:54.140 --> 03:56.980]  LinkedIn discovery, and a link to their reverse-who-is information.
[03:57.380 --> 03:59.220]  And on the right-hand side, I'm starting to build out
[04:00.020 --> 04:02.720]  their root domains, or their seeds. Roots or seeds
[04:02.720 --> 04:07.580]  are terms for things like something.com,
[04:07.580 --> 04:10.260]  or teslamotors.com, or tesla.com, or SolarCity.
[04:11.020 --> 04:13.760]  And so I've started to build out their seeds
[04:13.760 --> 04:16.260]  on the right-hand side. So you can see Tesla Motors,
[04:16.260 --> 04:19.600]  Tesla, SolarCity, etc. And as I do this,
[04:19.600 --> 04:21.800]  I'm going to start collecting a lot of data. As I do recon
[04:21.800 --> 04:25.420]  on Tesla, I'm going to collect a lot of data. So that turns into
[04:25.420 --> 04:28.880]  this, which on the left-hand side, you can see
[04:28.880 --> 04:31.920]  for tesla.com, that seed,
[04:31.920 --> 04:35.360]  I've enumerated all their subdomains and all of the live links.
[04:35.360 --> 04:38.440]  So these are all the live links for tesla.com,
[04:38.440 --> 04:42.060]  for their subdomains. So things like pages.tesla.com,
[04:42.060 --> 04:43.980]  shop.tesla.com, etc.
[04:44.520 --> 04:47.240]  And then you can drill down into any one of these.
[04:47.240 --> 04:50.580]  And so if I drill into www.tesla.com, then I have
[04:50.580 --> 04:53.140]  my methodology notes for that single site, which
[04:54.040 --> 04:56.580]  inside of that node, I keep
[04:56.580 --> 04:59.240]  all of the questions I ask myself when I'm basically
[04:59.240 --> 05:02.920]  looking to hack a website. Like, does the site have multiple user roles?
[05:02.920 --> 05:05.840]  How does it reference users? How does it handle special characters?
[05:06.060 --> 05:09.260]  What are dynamic parameters? Does it have an API component?
[05:09.260 --> 05:11.780]  What kind of errors am I seeing? Does it have
[05:11.780 --> 05:14.800]  file uploads? Have I done JavaScript analysis
[05:14.800 --> 05:17.580]  for paths? Have I done content discovery?
[05:17.940 --> 05:20.680]  All these questions that you're normally going to
[05:20.680 --> 05:24.280]  keep in your methodology, I apply
[05:24.280 --> 05:26.860]  these all to each one of these subnodes. And then I
[05:26.860 --> 05:30.280]  color code my work based on where I am.
[05:30.280 --> 05:31.980]  And this is one of the most important things, I think,
[05:31.980 --> 05:36.000]  is that everything that is not filled in hasn't been
[05:36.000 --> 05:38.420]  worked on yet, so most of this hasn't been worked on yet.
[05:38.420 --> 05:41.020]  But everything in orange means I'm currently working on something,
[05:41.020 --> 05:44.700]  and everything in green, which is there's nothing green on here, I've already done.
[05:45.400 --> 05:47.880]  And so just the way I can do this,
[05:47.880 --> 05:50.100]  and then I can add checkmarks and other stuff in XMind
[05:50.100 --> 05:53.280]  to these mind maps helps me track where I am, so I can easily
[05:53.280 --> 05:55.680]  put down a project and pick it back up if I want.
[05:56.980 --> 05:59.900]  Okay, so today's mission, we're going to talk about
[05:59.900 --> 06:02.700]  wide recon. This is the art of finding as
[06:02.700 --> 06:05.480]  many assets related to a target as possible.
[06:05.480 --> 06:08.460]  So you're a red teamer, or you're a bug bounty hunter, and you have
[06:09.040 --> 06:11.260]  a scope of X company.
[06:11.460 --> 06:14.080]  And one of the things you need to do
[06:14.560 --> 06:17.420]  is identify all the websites that they own.
[06:17.420 --> 06:20.560]  Because the more websites you identify, or the more
[06:20.560 --> 06:23.460]  infrastructure you identify, the better your chances
[06:23.460 --> 06:26.740]  of getting in or finding a bounty, right? And this is a
[06:26.740 --> 06:29.920]  core component in both of those skill sets.
[06:29.920 --> 06:32.740]  And I break down what we call recon into
[06:32.740 --> 06:36.120]  a couple of domains. So finding first the in-scope
[06:36.120 --> 06:40.360]  domains via the program or your project
[06:40.360 --> 06:42.840]  brief, you're finding acquisitions
[06:42.840 --> 06:45.160]  for the company or the target,
[06:45.160 --> 06:47.960]  doing ASN enumeration, doing reverse whois,
[06:47.960 --> 06:51.140]  doing a whole bunch of subdomain enumeration, and then doing port analysis.
[06:51.140 --> 06:53.840]  And then we're going to go into some related topics
[06:54.940 --> 06:56.780]  like some vulnerability scanning
[06:56.780 --> 06:59.080]  and some automation
[07:00.420 --> 07:03.260]  information related to recon, because you'll probably do it
[07:03.260 --> 07:05.040]  in this phase of your workflow.
[07:06.220 --> 07:08.940]  Okay, so you've decided you're going to do an assessment
[07:08.940 --> 07:12.040]  or you've been handed a red team assessment or you've been handed a bug bounty
[07:12.040 --> 07:14.980]  that's wide scope, right? That's where we are right now. So the first thing
[07:14.980 --> 07:18.080]  we need to do is parse the program page.
[07:18.080 --> 07:21.260]  So I'm going to use a bounty here, a couple bounties to illustrate
[07:21.260 --> 07:23.920]  where you would start. So here is
[07:23.920 --> 07:26.180]  the brief page for the aforementioned Tesla
[07:27.040 --> 07:29.780]  for BugCraft. So in their brief, they have
[07:29.780 --> 07:32.980]  star.tesla.com, star.tesla.cn, Tesla Motors
[07:32.980 --> 07:35.320]  and Tesla Services. And then they also have this
[07:35.860 --> 07:39.040]  catch-all sentence here that says any host verified
[07:39.040 --> 07:41.560]  or owned by Tesla Motors that you can find
[07:41.900 --> 07:44.940]  is also included in scope. So this is what we consider a wide scope
[07:44.940 --> 07:48.060]  bug bounty. There's lots of stuff that you can do with this.
[07:48.260 --> 07:51.180]  So your first four seed domains are already listed here.
[07:51.180 --> 07:54.140]  Tesla.services, TeslaMotors.com, TeslaCN, and Tesla.com.
[07:54.140 --> 07:56.800]  So we're going to focus on these four and add them to our list.
[07:57.000 --> 07:59.640]  And then anything else we can find that is Tesla's is also fair game.
[07:59.640 --> 08:03.080]  So that's great. If you look at a platform like HackerOne,
[08:03.620 --> 08:06.060]  they also have a pretty wide scope bounty
[08:06.060 --> 08:09.080]  and probably the most prolific bounty available on the internet
[08:09.080 --> 08:11.840]  which is Verizon Media. And this is like the granddaddy,
[08:11.840 --> 08:14.560]  the biggest scope program I think
[08:14.560 --> 08:17.720]  I know of in the bug bounty scene. So Verizon Media
[08:17.720 --> 08:20.920]  has gobbled up so many brands and they have
[08:20.920 --> 08:23.700]  so much infrastructure related to these brands and sites
[08:23.700 --> 08:26.780]  and subdomains. It's giant. So a lot of hackers
[08:26.780 --> 08:29.780]  make their whole career hacking on
[08:29.780 --> 08:32.840]  Verizon Media, which is awesome. And they're a great team
[08:32.840 --> 08:35.680]  and they support the bug bounty community. They also have this
[08:35.680 --> 08:38.520]  catch-all kind of phrase in their scope that says if you've found
[08:38.680 --> 08:41.400]  a vulnerability that affects an asset belonging to them but it's not
[08:41.400 --> 08:44.600]  in scope, report it to the program. And so it's a
[08:44.600 --> 08:47.460]  really advanced team to work with. You couldn't even fit
[08:47.460 --> 08:50.440]  all of the domains that you have for this program on one page, which I
[08:50.440 --> 08:54.100]  didn't even try. So you will parse your seed domains
[08:54.100 --> 08:56.580]  and your in-scope domains from the program page
[08:56.580 --> 08:59.660]  to start off. And this will give you a starting place to
[08:59.660 --> 09:03.280]  work from. The next thing that I do
[09:03.280 --> 09:05.440]  when I'm scoping out a company is I want to find
[09:05.440 --> 09:09.080]  all their acquisitions. And there's a couple reasons I want to do this.
[09:09.140 --> 09:11.660]  I want to basically see
[09:11.660 --> 09:14.300]  if any of the infrastructure for those acquired companies has
[09:14.300 --> 09:17.360]  not been taken off the line or any of the subdomains
[09:17.360 --> 09:20.360]  or websites or integrations that were too hard to port
[09:20.360 --> 09:22.960]  over to new infrastructure are still online.
[09:23.260 --> 09:26.420]  This gives me more and more infrastructure and sites to
[09:26.420 --> 09:29.520]  hack. So to find the acquisitions, I use
[09:29.620 --> 09:31.840]  a tool called Crunchbase. And Crunchbase is a business
[09:32.880 --> 09:35.860]  intelligence portal. Basically, you add any business name
[09:35.860 --> 09:38.900]  to this search box on Crunchbase and you'll be able to
[09:38.900 --> 09:41.940]  look at that company and find out what
[09:41.940 --> 09:44.680]  are their employees? Are there investor rounds?
[09:44.680 --> 09:47.800]  What acquisitions have they had? What are their socials, etc.?
[09:47.800 --> 09:50.960]  So here I've used the example of Twitch.com. We're streaming
[09:50.960 --> 09:53.240]  on Twitch today, which is awesome. And Twitch
[09:53.840 --> 09:56.840]  as an organization here has an entry. And if I click
[09:56.840 --> 09:59.620]  on that, I get a lot of information about Twitch.
[09:59.620 --> 10:02.820]  And I can see that under their acquisitions tab,
[10:02.820 --> 10:05.200]  they have acquired four companies
[10:05.200 --> 10:08.640]  in the last eight years, Revlo,
[10:08.640 --> 10:11.420]  Clipmind, Curse, and Goodgame. And then also,
[10:11.420 --> 10:14.700]  they were acquired themselves on the left-hand side by Amazon.
[10:14.840 --> 10:16.620]  So this gives me a good idea of
[10:17.820 --> 10:20.640]  what they used to be, where I can also
[10:20.640 --> 10:23.480]  look for some infrastructure if it's a recent acquisition,
[10:23.480 --> 10:26.480]  like Revlo, for instance. If I click on Revlo
[10:26.480 --> 10:29.460]  in this page, it'll take me to Revlo's page, which will give me their main domain,
[10:29.460 --> 10:32.080]  which is Revlo.com. So I might put that in scope of
[10:32.660 --> 10:35.300]  a wide scope assessment or a Red Team assessment.
[10:35.680 --> 10:38.480]  Acquisitions are important to keep track of. Now, the one thing that you
[10:38.480 --> 10:41.360]  want to make sure here is that on some of those brief pages that we
[10:41.360 --> 10:44.260]  looked at in the last couple pages, some of these are
[10:44.260 --> 10:47.860]  explicitly out of scope. So they'll say in the brief,
[10:47.860 --> 10:50.460]  especially for Tesla, SolarCity is not in
[10:50.460 --> 10:53.320]  scope of their wide scope bounty.
[10:53.320 --> 10:56.060]  So even though we found that as an acquisition of Twitch
[10:56.060 --> 10:58.840]  when I looked on this page, when I've done it many times,
[10:58.840 --> 11:02.240]  it's explicitly out of scope. So remember to refer to your project page
[11:02.240 --> 11:05.280]  or understand what is okay and what is not okay to hack
[11:05.280 --> 11:06.680]  or do recon on.
[11:07.960 --> 11:11.260]  So after I find the acquisitions, which has given me
[11:11.260 --> 11:13.880]  seed domains like Revlo.com and new
[11:13.880 --> 11:17.220]  seed domains other than Tesla Motors and Tesla and stuff
[11:17.220 --> 11:20.320]  like that, or any of these top-level seed domains,
[11:20.320 --> 11:22.960]  what I want to do is I want to find this company's autonomous
[11:22.960 --> 11:26.020]  system numbers. So every company that gets
[11:26.020 --> 11:29.180]  large enough ends up having a collection of networks
[11:29.180 --> 11:32.420]  basically applied an AS number to them.
[11:32.420 --> 11:35.160]  And an AS number, autonomous system number, is just a collection
[11:35.160 --> 11:38.080]  of your known IP ranges for your networks, for your
[11:38.080 --> 11:41.200]  infrastructure. And so you can find someone's
[11:41.200 --> 11:44.000]  autonomous system number in many ways. There's
[11:44.000 --> 11:46.840]  many search engines to do this kind of
[11:46.840 --> 11:49.600]  lookup, but I use one called Hurricane Electric
[11:49.600 --> 11:52.580]  or bgp.he.net. I use this
[11:52.580 --> 11:55.620]  when I'm doing manual searching because it has a freeform
[11:55.620 --> 11:58.220]  text box at the top and I can just search the keyword
[11:58.220 --> 12:01.680]  Twitch. And here you can see I found Twitch's
[12:01.680 --> 12:04.840]  two autonomous system numbers, AS46489
[12:05.860 --> 12:08.300]  and AS397153.
[12:08.300 --> 12:10.600]  And then their associated IP ranges
[12:10.600 --> 12:13.840]  and IPv4 ranges and IPv6 ranges.
[12:13.840 --> 12:16.980]  So this is a good representation of the infrastructure
[12:16.980 --> 12:19.800]  or the IP space that they own. And I can
[12:19.800 --> 12:22.720]  assume that it belongs to... I can guarantee
[12:22.720 --> 12:25.560]  that it belongs to them. I don't have to worry about
[12:26.000 --> 12:28.840]  scope really here because I know that all this is
[12:28.840 --> 12:31.240]  owned by them. So if they have that catch-all phrase in their
[12:31.240 --> 12:34.640]  bounty project or you've got the go-ahead in your red team engagement
[12:34.640 --> 12:37.700]  to open scope, then you know this is your target.
[12:37.880 --> 12:40.500]  One thing that this doesn't represent is their cloud assets.
[12:40.500 --> 12:43.020]  So this is all owned IP space.
[12:43.200 --> 12:46.280]  It doesn't represent things like AWS and Azure and GCP
[12:46.280 --> 12:49.320]  ranges. So you won't find stuff that they host
[12:49.320 --> 12:51.620]  in those environments inside of these ranges.
[12:52.180 --> 12:55.140]  You just want to be cognizant of that. So here I've searched
[12:55.140 --> 12:58.160]  Twitch. I got their ASNs. With every
[12:58.840 --> 13:01.720]  tool or method here in my methodology,
[13:02.160 --> 13:04.660]  I like to give you a manual way to do it
[13:04.660 --> 13:07.500]  because context is important and getting used to the idea
[13:07.500 --> 13:10.340]  is important. And I like to give you an automated way to do it in case you're going to script
[13:10.340 --> 13:13.380]  up your own recon framework, which is all the fad these days
[13:13.380 --> 13:16.300]  and I have one of my own and everything. So there are
[13:16.300 --> 13:19.640]  two tools you can use to parse ASNs
[13:19.640 --> 13:22.480]  and IP ranges from companies that
[13:22.480 --> 13:25.500]  exist on the command line if you're going to script this stuff up yourself.
[13:25.720 --> 13:28.280]  So one is called Metabigor.
[13:28.640 --> 13:31.120]  I'm still not sure if I'm saying that name of the tool right. By
[13:31.620 --> 13:34.500]  JesseJJJ. And this one utilizes
[13:34.500 --> 13:37.260]  the site that we just saw, bgp.he.net
[13:37.260 --> 13:40.320]  and another site called ASN Lookup. And it will
[13:40.320 --> 13:43.280]  grab the information from those websites, scrape it off and give you
[13:43.280 --> 13:46.540]  your IP ranges for an organization. The other one
[13:46.540 --> 13:49.740]  is called ASN Lookup by Yasin. Same idea.
[13:49.740 --> 13:52.560]  It uses a different data source called the MaxMind database.
[13:52.560 --> 13:55.220]  It'll pull off their IPv4 ranges from an ASN
[13:55.220 --> 13:58.640]  and you can give these both tools
[13:58.860 --> 14:01.640]  a term instead of an AS number or anything like that.
[14:01.640 --> 14:04.400]  So they will pull it off based on that keyword
[14:04.400 --> 14:07.220]  that we're looking for. In this case, Tesla or something like that.
[14:07.240 --> 14:10.440]  Now, one thing to know about these tools is they are
[14:10.440 --> 14:13.640]  looking off your keyword here, which we put on the command line,
[14:13.640 --> 14:16.480]  Tesla. And there are multiple companies with
[14:16.480 --> 14:19.360]  Tesla in the name. There's like Tesla Research Lab or something
[14:19.360 --> 14:21.480]  like that in Sweden or something like that.
[14:22.140 --> 14:25.600]  Just be sure that when you get these ranges and you start seeing
[14:25.600 --> 14:28.620]  these sites, you can identify, oh no, I might have gotten
[14:28.620 --> 14:31.020]  one that's actually not Tesla Motors.
[14:32.140 --> 14:34.860]  Both these tools, I verified, pull up the right
[14:34.860 --> 14:37.700]  data for Tesla. But just be cognizant
[14:37.700 --> 14:40.700]  of, since you're using a keyword here to search, that you don't
[14:40.700 --> 14:43.660]  get other IP ranges that are not your targets.
[14:43.660 --> 14:45.900]  You'll have to verify somewhat manually there.
[14:47.140 --> 14:49.640]  Okay, so we've gotten some IP space from the ASNs.
[14:49.640 --> 14:52.140]  We've gotten some seed domains. We want to continue gathering
[14:52.140 --> 14:54.260]  as many seed domains as possible.
[14:55.020 --> 14:57.380]  We have the AS number, and for here
[14:57.380 --> 14:59.120]  we're going to use Twitch as an example again.
[14:59.480 --> 15:03.300]  Their autonomous system number is 46489.
[15:03.660 --> 15:06.220]  And here what we want to do is we're going to
[15:06.220 --> 15:09.100]  get our first introduction to Amass.
[15:09.100 --> 15:12.440]  And Amass is a framework for
[15:12.440 --> 15:14.920]  domain intelligence, I would call it, I guess.
[15:14.920 --> 15:18.080]  And it is written by Jeff Foley. There is a workshop you should absolutely
[15:18.080 --> 15:21.140]  sign up for. He's going to do a full workshop on Amass
[15:21.140 --> 15:24.400]  and how to use it. Jeff is as associated
[15:24.400 --> 15:27.160]  to the Red Team Village, so super cool. And here we're going
[15:27.160 --> 15:30.080]  to use Amass and feed it our autonomous
[15:30.080 --> 15:33.200]  system number, 46489. And what it's going to do
[15:33.200 --> 15:36.080]  is it's going to go out to all of the IP ranges that
[15:36.080 --> 15:38.720]  are represented by that autonomous system number.
[15:38.720 --> 15:42.240]  And it's going to scan the certificates for all HTTPS
[15:42.240 --> 15:45.160]  sites. And then it's going to parse those certificates and tell us,
[15:45.160 --> 15:48.240]  hey, here are the seed domains I found
[15:48.240 --> 15:51.220]  that are owned by them. And here you can see that we found some
[15:51.220 --> 15:54.200]  seed domains that we didn't know about before. Justin.tv, which is the
[15:54.200 --> 15:58.420]  company Twitch used to be. TTVNW.net,
[15:58.420 --> 16:00.220]  which is, I think, associated to
[16:00.220 --> 16:03.580]  Twitch's streaming protocol. Twitch.tv,
[16:03.580 --> 16:06.400]  obviously, which is their main site. TwitchCon, which is their conference.
[16:06.400 --> 16:08.460]  And SocialCam, which I have no idea what is.
[16:08.960 --> 16:12.420]  And all the links for these tools and the methods
[16:12.420 --> 16:15.260]  are posted in the slides, so you can click
[16:15.260 --> 16:18.220]  through when the slides get released and go grab the tools
[16:18.220 --> 16:20.860]  and play with them on your own. So here we've already expanded
[16:23.740 --> 16:24.540]  our assessment
[16:25.040 --> 16:27.220]  a lot, right? We have a whole bunch of IP space,
[16:27.220 --> 16:30.440]  we have a whole bunch of top-level domain seed domains,
[16:30.440 --> 16:33.180]  and we have the seed domains already given to us by our
[16:33.180 --> 16:35.880]  brief page. Okay, so the next method is
[16:35.880 --> 16:39.060]  reverse whois. And what reverse whois does is it will take the
[16:39.060 --> 16:42.120]  whois entry of... you can search your domain
[16:42.120 --> 16:45.080]  in Twitch.tv here. And you can do this
[16:45.080 --> 16:47.960]  with a lot of sites. I use huoxi.com because
[16:47.960 --> 16:50.940]  it's pretty cheap and you can get a free API
[16:50.940 --> 16:53.480]  key for so many uses. And
[16:53.900 --> 16:57.040]  you basically supply it your domain and it will give
[16:57.040 --> 16:59.820]  you the whois data and then you can
[16:59.820 --> 17:02.760]  use that whois data to correlate who else has
[17:02.760 --> 17:05.860]  registered stuff. So an example here is we gave this site
[17:05.860 --> 17:08.940]  huoxi.com, twitch.tv. And then it said
[17:08.940 --> 17:11.940]  that over the course of the years, there have been several
[17:11.940 --> 17:14.780]  companies' names related to the whois data and also
[17:14.780 --> 17:18.020]  registered emails. And so here you
[17:18.020 --> 17:21.560]  can see that the company, as of March 26,
[17:21.560 --> 17:23.940]  2015, was justin.tv
[17:23.940 --> 17:26.800]  and the email in the domain was domainmaster
[17:26.800 --> 17:30.140]  at justin.tv. And then later in 2015, it changed
[17:30.140 --> 17:33.120]  to twitchinteractive. And if you click the button right next
[17:33.120 --> 17:35.840]  to those entries, you can see there's 20 other domains still
[17:35.840 --> 17:38.860]  registered to justin.tv, which could be interesting to us. And there's
[17:39.700 --> 17:41.380]  575 domains registered
[17:41.380 --> 17:44.800]  under twitchinteractive. And so we can start pulling back more
[17:44.800 --> 17:47.460]  data, more subdomains, more root domains from
[17:48.480 --> 17:50.800]  these links when we click on them. Now,
[17:50.800 --> 17:53.580]  one thing to know here is that whois data is
[17:53.580 --> 17:56.560]  registered data. And so a lot of these companies
[17:56.560 --> 17:59.680]  will park domains to combat
[17:59.680 --> 18:02.680]  phishing and to hold stuff for
[18:02.680 --> 18:05.600]  marketing campaigns and stuff like that. So they might not be live sites.
[18:05.600 --> 18:07.920]  They might just be parsed. So this is a medium fidelity
[18:08.800 --> 18:11.500]  type of technique here, but I have found some
[18:11.500 --> 18:14.560]  really good stuff using reverse whois and the registrant
[18:14.560 --> 18:17.520]  data of these sites. There is
[18:17.620 --> 18:20.400]  a tool to do this type of analysis. It's called domlink,
[18:20.400 --> 18:23.420]  written by Vincent Yu via security on Twitter. And
[18:23.420 --> 18:26.840]  it uses that exact site, hoaxy.com.
[18:27.100 --> 18:29.480]  And you can basically give it a domain
[18:29.480 --> 18:32.300]  and it will find every other associated domain by both
[18:32.820 --> 18:35.640]  registrant email and organization name or company
[18:35.640 --> 18:38.600]  name. And it will do it on the command line and return it to you
[18:38.600 --> 18:41.520]  in a script. And it's recursive. So a really cool tool
[18:41.520 --> 18:44.480]  here. You can use this to automate some of that reverse whois
[18:45.060 --> 18:47.400]  lookup. It's called domlink. The tool is called domlink.
[18:48.880 --> 18:50.540]  So the next method that we have now
[18:50.540 --> 18:53.320]  that we've maybe gotten some more seeds and subdomains from
[18:53.320 --> 18:56.320]  reverse whois and we're building out our list of things that we can
[18:56.320 --> 18:59.140]  hack is we want to look at the ad and analytics
[18:59.140 --> 19:02.180]  relationships of our target site. We want to see what
[19:02.180 --> 19:05.180]  other sites are using the same ad and analytics codes
[19:05.180 --> 19:07.400]  as our main site. And this will give us
[19:08.480 --> 19:11.120]  a good understanding of maybe what are their most
[19:11.120 --> 19:14.100]  popular domains and maybe what are some of their less popular domains
[19:14.100 --> 19:17.220]  that are still using the codes and we didn't know about before.
[19:17.340 --> 19:20.260]  So every page usually embeds a
[19:20.260 --> 19:23.080]  Google Analytics code or a New Relic code
[19:23.080 --> 19:26.020]  or some kind of ad or analytics code
[19:26.020 --> 19:29.040]  in the main page. So here if we use a site called
[19:29.040 --> 19:32.140]  builtwith.com, we can search twitch.tv
[19:32.140 --> 19:35.080]  and twitch.tv will give us a list under the
[19:35.080 --> 19:38.120]  relationship profile tab of all of the keys that are on
[19:38.120 --> 19:41.300]  twitch.tv. And then you can
[19:41.300 --> 19:44.160]  click on those and see what other sites are using the same
[19:44.160 --> 19:47.220]  ad and analytics codes and maybe we didn't know
[19:47.220 --> 19:50.040]  about those before. So here we can... it's hard to see on the bottom
[19:50.040 --> 19:53.280]  but we have a couple entries that we didn't know about before.
[19:53.420 --> 19:57.280]  manvsgame.tv, twitc.tv.
[19:57.360 --> 19:59.060]  We knew about twitchcon.com
[19:59.060 --> 20:01.480]  from the previous method but we're starting to build up more
[20:02.040 --> 20:05.000]  domains that we can see that are related to
[20:05.000 --> 20:08.020]  our target. So ad and
[20:08.020 --> 20:11.620]  analytics relationship profiles is a pretty powerful technique.
[20:11.660 --> 20:14.240]  I have used it for success a couple
[20:14.240 --> 20:17.220]  times. You can also do this directly from Firefox or Chrome.
[20:17.220 --> 20:20.260]  They have an extension for this site called built with and
[20:20.260 --> 20:23.140]  you can just install it. You make a free account
[20:23.140 --> 20:25.920]  and then you visit your site in the browser, click the
[20:25.920 --> 20:29.080]  Chrome extension button and it will give you the relationship information
[20:29.080 --> 20:31.500]  right in your browser and you can start clicking around there.
[20:31.500 --> 20:34.600]  So pretty easy to institute while you're
[20:34.600 --> 20:36.680]  doing your recon in a browser.
[20:38.760 --> 20:41.180]  So you can also use a tool to do this
[20:41.180 --> 20:44.080]  in the command line. So Malik, after
[20:44.080 --> 20:46.940]  my first iteration of this talk, quickly scripted up an
[20:46.940 --> 20:50.460]  awesome tool to do this. It's called getrelationship.py
[20:50.460 --> 20:53.400]  and you just pass it your target here.
[20:53.400 --> 20:56.140]  Here he's targeted Uber.com which also has a bounty
[20:56.140 --> 20:59.080]  program and passed it to this Python script and it will pull out on the command
[20:59.080 --> 21:01.860]  line all of the related domains. And you'll have to
[21:01.860 --> 21:05.220]  manually kind of verify these but the ones I would look at first
[21:05.220 --> 21:07.720]  are the ones that actually have Uber in the
[21:08.320 --> 21:09.660]  domain somewhere like
[21:10.920 --> 21:14.000]  daft.uber or driveuber.co.nz
[21:14.000 --> 21:16.400]  or things like that. So you'll have to
[21:16.400 --> 21:19.300]  sift through this but you can do it on the command line now too.
[21:21.870 --> 21:23.410]  Then you can just use some general
[21:23.410 --> 21:25.730]  Google hacking or Google-fu to try to find
[21:26.330 --> 21:29.250]  associated sites to your target. You can use
[21:29.250 --> 21:32.090]  the copyright text, the terms of service, and the
[21:32.090 --> 21:35.070]  privacy policy and you can just copy those from the bottom of any of the pages
[21:35.070 --> 21:38.070]  and here you can see Twitch interactive ink would be something
[21:38.070 --> 21:41.230]  I'd Google for. And then you can see any other page that is
[21:41.230 --> 21:44.030]  hosting that text at the bottom of their page
[21:44.030 --> 21:46.410]  which could be something that you didn't know about before
[21:46.990 --> 21:50.090]  in your recon so far. So you can just do
[21:50.090 --> 21:53.190]  this manually and you can use
[21:53.190 --> 21:56.130]  other search operators like in URL Twitch or you can
[21:56.130 --> 21:59.170]  just look for these policies. One tip here is
[21:59.170 --> 22:01.430]  that you don't just want to target a certain
[22:02.010 --> 22:05.090]  year. Like here I have 2019 but you also want to check 2018
[22:05.090 --> 22:07.650]  and 2017. In fact, the more stuff you find
[22:07.650 --> 22:11.030]  that goes back farther in age, the more
[22:11.030 --> 22:14.010]  likely it is to be vulnerable and less likely to have a lot of security
[22:14.010 --> 22:17.190]  assessment associated to it. So probably more vulnerable for your
[22:17.190 --> 22:20.190]  Red Team engagement or your bug bounty.
[22:22.170 --> 22:23.070]  Alright, the next
[22:23.070 --> 22:25.750]  method that we're going to use to find some seed domains or
[22:25.750 --> 22:28.930]  related infrastructure is Shodan. I have to admit I'm not a Shodan
[22:28.930 --> 22:32.130]  ninja but I know a lot of Red Teamers are and I actually love Shodan
[22:32.130 --> 22:34.930]  and it's just a failing of mine. I need to get better at it. But there's
[22:35.050 --> 22:38.170]  a ton of search operators we could use here on our domain twitch.tv
[22:38.170 --> 22:41.430]  and what Shodan is is it's the site that is a
[22:41.430 --> 22:44.190]  site that hosts information from an infrastructure
[22:44.190 --> 22:47.330]  based spider. And a spider goes to every site on the
[22:47.330 --> 22:50.170]  internet and basically captures
[22:50.170 --> 22:53.550]  its HTTP responses, its
[22:53.550 --> 22:56.190]  technology stack, its certificate data
[22:58.030 --> 22:58.970]  and cross links it
[22:58.970 --> 23:01.810]  so you can click on any of these pieces of information and find out
[23:01.810 --> 23:04.990]  other things it's seen related and what organizations it's
[23:04.990 --> 23:08.030]  related to and what common tech that they're
[23:08.030 --> 23:10.690]  using, what web servers, what JavaScript
[23:10.690 --> 23:14.330]  framers, you could do anything. So Shodan is a really powerful tool
[23:15.010 --> 23:16.910]  and you can query our
[23:16.910 --> 23:18.590]  domain here and try to find
[23:19.670 --> 23:22.950]  related infrastructure. For instance, in one of the entries when I just
[23:22.950 --> 23:26.330]  didn't even use a great operator here, I just searched my domain twitch.tv
[23:26.330 --> 23:29.370]  I ended up seeing in the SSL certificate data
[23:30.230 --> 23:32.010]  of the search there was
[23:32.010 --> 23:35.430]  twitch.amazon.eu and now I need
[23:35.430 --> 23:38.390]  to ask myself a question like is that in scope, right? Like that's not
[23:39.590 --> 23:41.370]  verbatim a Twitch domain but in a
[23:41.370 --> 23:44.610]  red team assessment if that's related to Twitch infrastructure, well
[23:45.250 --> 23:47.350]  maybe it's a target that I can go after.
[23:47.350 --> 23:50.630]  So Shodan can give you a lot of good information.
[23:50.630 --> 23:53.370]  Okay, so we found seed domains, stuff to start
[23:53.370 --> 23:56.250]  with. Now what we're going to do is we're going to try to find subdomains
[23:56.250 --> 23:59.290]  for those seed domains. So these are things
[23:59.290 --> 24:02.230]  like www is a subdomain
[24:02.230 --> 24:05.010]  but also admin.tesla.com or
[24:06.150 --> 24:08.390]  forums.tesla.com or something like that
[24:08.390 --> 24:11.210]  or forums.twitch.tv or whatever. We're going to try to find
[24:11.210 --> 24:14.450]  subdomains. And so this idea of starting
[24:14.450 --> 24:17.170]  with seeds and then moving to subs really
[24:17.170 --> 24:20.170]  is going to multiply. The more seeds you find
[24:20.170 --> 24:23.010]  the more subdomain enumeration you can do. The more subdomains
[24:23.010 --> 24:26.050]  you find is related to how many
[24:26.050 --> 24:29.290]  sites you find. The more sites you find, the more successful you'll be
[24:29.290 --> 24:31.470]  inside of your red team engagement or bug bounty
[24:32.130 --> 24:34.990]  program. So for subdomain enumeration
[24:34.990 --> 24:37.650]  I use three-ish
[24:38.090 --> 24:40.690]  different methods to get subdomain information. One is
[24:40.690 --> 24:44.250]  LinkedIn JavaScript discovery. Another one is
[24:44.250 --> 24:46.970]  subdomain scraping techniques and then subdomain
[24:46.970 --> 24:49.930]  brute force. And then there's some auxiliary stuff I use as well.
[24:49.930 --> 24:52.910]  So we're going to run into those right now. And this is the bulk of what a lot of tools
[24:52.910 --> 24:56.090]  are doing right now. And so
[24:56.090 --> 24:58.970]  you'll see a lot of favorites in these slides. So the first area
[24:58.970 --> 25:01.410]  we're going to use to find subdomain data is
[25:01.970 --> 25:04.730]  or the first technique is linked discovery and JavaScript
[25:04.730 --> 25:08.190]  discovery. So linked discovery, what is it?
[25:08.190 --> 25:10.810]  It's basically using a web spider to
[25:10.810 --> 25:13.030]  visit a site like twitch.tv
[25:13.650 --> 25:15.630]  land on it and spider all the
[25:15.630 --> 25:19.050]  HTML links. Pretty simple. And any of those that
[25:19.050 --> 25:21.870]  come back with a subdomain
[25:22.390 --> 25:25.350]  we just add to our list of in-scope targets.
[25:25.350 --> 25:28.230]  And so this is pretty self-explanatory. It happens
[25:28.230 --> 25:31.230]  naturally when you're using tools like burp suite and you're spidering
[25:31.370 --> 25:33.830]  a site. But I'm going to walk you through how I do it.
[25:33.830 --> 25:37.050]  I'm going to use burp 1.7 because I like the UI better but it
[25:37.050 --> 25:40.030]  works as well with the crawler or the scanner
[25:40.030 --> 25:43.070]  in burp 2.0. So use whatever
[25:43.070 --> 25:46.070]  you want. And then I'm going to give you some other methods to do it. So the first thing
[25:46.070 --> 25:49.190]  we do is we load up burp suite which is on the right
[25:49.190 --> 25:52.070]  here. It's a side-by-side to our site. And
[25:52.070 --> 25:54.990]  we just visit our site through burp suite the proxy.
[25:55.350 --> 25:58.150]  And burp suite the proxy will capture all this
[25:58.150 --> 26:01.210]  data. We've only visited one page. If you visit
[26:01.210 --> 26:04.330]  twitch.tv ever, you see that it has
[26:04.330 --> 26:07.290]  cross-linked and it does request a lot
[26:07.290 --> 26:10.110]  of stuff because it's a streaming media site and so there's a lot
[26:10.110 --> 26:12.390]  of stuff going in the background when you go to twitch.tv.
[26:13.110 --> 26:16.150]  So you can see all that data on the right-hand side. This is everything that's
[26:16.150 --> 26:18.910]  either seen been linked on this page
[26:18.910 --> 26:22.110]  or it's actually been requested. Actually been requested is the
[26:22.110 --> 26:24.510]  stuff in black, seen is the stuff that's in gray.
[26:24.950 --> 26:26.630]  And so what you do is you visit your page
[26:27.770 --> 26:30.770]  and then you have to set up a rule, some type of rule.
[26:30.770 --> 26:33.910]  And you can do this by going into the target tab and
[26:33.910 --> 26:36.650]  the scope tab and then you click this box that says
[26:36.650 --> 26:39.730]  use advanced scope control. And what I do is I enter
[26:39.730 --> 26:42.670]  in just a keyword here in advanced scope control
[26:42.670 --> 26:45.230]  which I just say twitch, right? So I want to see
[26:45.810 --> 26:48.710]  in burp anything that has twitch in the URL
[26:48.710 --> 26:51.690]  at all, no matter where it is, what has that word twitch. And you could also
[26:51.690 --> 26:54.210]  add some more here because we've seen in our previous analysis
[26:55.610 --> 26:57.410]  TWT is also part of their
[26:57.410 --> 27:00.630]  domain naming nomenclature, right?
[27:00.630 --> 27:02.970]  And so you can add a couple of these scope rules
[27:03.450 --> 27:06.290]  and then you go back to your sitemap and you
[27:06.290 --> 27:09.330]  click on the ribbon bar up at top or the filter
[27:09.330 --> 27:12.250]  bar and then you say show only
[27:12.250 --> 27:15.090]  in scope items. So now this will just show
[27:15.090 --> 27:18.270]  in our sitemap all the twitch.tv links, right? So we already
[27:18.270 --> 27:22.030]  have a good list of subdomains here, right?
[27:22.050 --> 27:24.390]  Sentinel.twitchservice.net
[27:24.390 --> 27:27.490]  API.twitch.tv. So we have a lot
[27:27.490 --> 27:30.390]  of stuff with twitch in the name here. We know that most of this is probably
[27:30.390 --> 27:32.950]  related to twitch. This gives us exponential
[27:33.990 --> 27:36.130]  areas to hack, right? And this is pretty good.
[27:36.130 --> 27:39.210]  But if we select all these and then we spider them
[27:39.690 --> 27:42.390]  using burp spider, it will then go to all of
[27:42.390 --> 27:45.610]  them and find their links and find subdomains
[27:45.610 --> 27:48.470]  references in their pages. And we can do this recursively
[27:48.470 --> 27:51.470]  until there's nothing left to find. So here I'm
[27:51.470 --> 27:54.250]  going to select all of these and then I'm going to spider
[27:54.250 --> 27:57.130]  them. And now you can see, since I had them selected,
[27:57.130 --> 28:00.170]  everything in orange on the right-hand side was stuff I knew about before I
[28:00.170 --> 28:03.010]  spidered. And then after I spidered with burp,
[28:03.010 --> 28:05.770]  everything in white is new stuff. And you can see I found
[28:07.010 --> 28:09.190]  a combination of a lot of stuff here. I found
[28:09.190 --> 28:12.310]  subdomains for twitch.tv and I've also found some new
[28:12.310 --> 28:15.150]  root and seed domains like twitchapp.net
[28:15.150 --> 28:17.310]  and twitchservice.net and
[28:17.310 --> 28:21.250]  x-twitch.tv. So this is a
[28:21.250 --> 28:23.850]  hybrid technique. It can find you new seed domains or roots
[28:23.850 --> 28:27.030]  and subdomains. And so now I have all of this domain
[28:27.030 --> 28:30.190]  data. I can now select all of these and spider them again
[28:30.190 --> 28:33.130]  and spider those pages for links. And eventually you can do
[28:33.130 --> 28:35.350]  this until you get kind of spider fatigue.
[28:36.230 --> 28:38.970]  And eventually you'll have a large sitemap buildup
[28:38.970 --> 28:42.070]  of targets, which is awesome. Now, how
[28:42.070 --> 28:44.990]  do you get this data out of Burp Suite? There's
[28:44.990 --> 28:47.790]  not a great way to do this. Pro has
[28:47.990 --> 28:50.730]  a thing called Engagement Tools.
[28:50.730 --> 28:53.610]  And in Engagement Tools, you can do this thing called
[28:53.610 --> 28:56.650]  Analyze Target. And in Analyze Target, you can create
[28:56.650 --> 28:59.830]  an HTML report of all the targets
[28:59.830 --> 29:02.890]  in your site tree that are selected. So that's what I do.
[29:02.890 --> 29:05.690]  I select everything in the site tree. I go Engagement
[29:05.690 --> 29:08.710]  Tools, Analyze Target, Generate HTML Report. And then
[29:08.710 --> 29:11.730]  in the HTML report, I have a parsable list of
[29:11.730 --> 29:14.550]  the domains that it's seen at the beginning of that HTML report. And then I
[29:14.550 --> 29:17.510]  take that and I put it in a text file for later analysis.
[29:17.510 --> 29:19.590]  And then I dump it into my mind map as well.
[29:20.930 --> 29:23.290]  Okay. So the whole
[29:23.810 --> 29:26.250]  linked discovery thing counts on Burp
[29:26.250 --> 29:28.790]  Spider, right? A lot of times I hadn't seen
[29:28.790 --> 29:32.210]  before the last couple of years, great options to do this
[29:32.210 --> 29:35.210]  in the command line because it would require
[29:35.370 --> 29:38.610]  a lot of custom coding, bash scripting
[29:38.610 --> 29:41.330]  or Python to create my own spider and do that same
[29:41.330 --> 29:44.250]  process. Well, now there does exist some tools that are
[29:44.250 --> 29:47.370]  command line spiders with bug hunters and red teamers
[29:47.370 --> 29:50.110]  in mind. There's two. There's one called Ghost Spider written by Jesse
[29:50.110 --> 29:53.530]  JJJ. And it also designates
[29:53.530 --> 29:56.810]  the types of things it's parsing when it's visiting a URL.
[29:56.810 --> 29:58.670]  So it'll give you...
[30:25.120 --> 30:28.140]  Okay. Hopefully you guys can hear me. Headphone problems.
[30:28.140 --> 30:29.300]  Going to plug it in.
[30:29.780 --> 30:33.940]  Okay. Great, great, great. I was getting beeping in my ears.
[30:33.940 --> 30:36.200]  So maybe it's just running batteries. My bad. Okay.
[30:36.200 --> 30:39.420]  So you have two spiders here that you can use in the command line.
[30:39.420 --> 30:42.440]  Ghost Spider and Hackcrawler by HackLuke.
[30:42.440 --> 30:45.660]  And both these are awesome. They both have some functionality
[30:46.500 --> 30:48.860]  that will parse out the types of things
[30:48.860 --> 30:51.460]  you're getting, like JavaScript files,
[30:52.020 --> 30:54.200]  subdomains, URL endpoints.
[30:54.560 --> 30:57.940]  Some of them will give you, I think, parameter names and stuff like that.
[30:57.940 --> 31:00.800]  So you could institute the whole process
[31:00.800 --> 31:03.120]  of link discovery by scripting up
[31:03.800 --> 31:06.040]  Ghost Spider or Hackcrawler if you wanted.
[31:06.040 --> 31:09.700]  I still use Burp, but these are invaluable tools
[31:09.700 --> 31:12.540]  to have at your disposal, just like a crawler that can do
[31:12.540 --> 31:15.320]  some analysis. So keep them bookmarked
[31:15.320 --> 31:16.960]  for when you might need them.
[31:18.420 --> 31:21.440]  Okay. So the next place we're going to get subdomain information
[31:22.300 --> 31:23.540]  is by analyzing
[31:24.300 --> 31:27.480]  some JavaScript. And one of the tools I like here,
[31:27.480 --> 31:30.020]  just because it has this kind of little added benefit that I think
[31:30.020 --> 31:33.080]  that I haven't seen many places before, but maybe some other tools
[31:33.080 --> 31:35.780]  are starting to implement it now, is a tool called
[31:35.780 --> 31:39.440]  Subdomainizer by Neeraj Edwards. And what it'll do
[31:39.440 --> 31:42.300]  is it'll take a JavaScript file, you have to point it to a JavaScript
[31:42.300 --> 31:45.440]  file, and it will parse out all the cloud services,
[31:45.440 --> 31:48.240]  the subdomains, and it'll do this little extra
[31:48.240 --> 31:51.300]  thing where it uses the Shannon entropy formula
[31:51.300 --> 31:54.260]  or algorithm to identify things that
[31:54.260 --> 31:57.260]  look like API keys hard-coded in JavaScript,
[31:57.260 --> 32:00.120]  which is already kind of a vulnerability if you find a private API
[32:00.120 --> 32:03.180]  key or credential hard-coded in JavaScript,
[32:03.180 --> 32:06.240]  which I know sounds crazy to a lot of people. Like, you would just find
[32:06.380 --> 32:09.180]  a hard-coded private API key, but
[32:09.180 --> 32:12.440]  this happens all the time. Like, all the time it happens.
[32:12.860 --> 32:15.080]  And so this is
[32:15.080 --> 32:18.180]  what I like to call a forward-thinking type of thing, is using
[32:18.180 --> 32:21.040]  an algorithm like this to identify keys. Sometimes
[32:21.040 --> 32:23.980]  it's a little noisy, but a lot of times it finds you
[32:23.980 --> 32:27.060]  good stuff. So I like this tool to point at all the
[32:27.060 --> 32:29.460]  JavaScript files I found already on
[32:30.020 --> 32:33.180]  some of these sites. If you're just looking for subdomain
[32:33.740 --> 32:36.020]  information, there's another tool called Subscraper by Celian
[32:36.020 --> 32:38.820]  Collins, which has recursion built into it, which
[32:38.820 --> 32:42.240]  can do this method as well, but it doesn't do the API key part.
[32:42.740 --> 32:45.040]  So point this at JavaScript files you find
[32:45.040 --> 32:47.440]  on your site, and you can get back a whole bunch of
[32:49.860 --> 32:51.480]  subdomains and cloud
[32:52.200 --> 32:55.280]  services that the site might use or the target might use.
[32:56.540 --> 32:57.340]  All right. So that's
[32:57.340 --> 33:00.460]  link discovery and JavaScript analysis for
[33:00.460 --> 33:03.540]  subdomains. Now we're going to get into the big meats of
[33:03.540 --> 33:06.320]  what a lot of people do is subdomain scraping.
[33:06.420 --> 33:09.920]  Now the idea of subdomain scraping is
[33:09.920 --> 33:12.820]  going out to these websites on the
[33:12.820 --> 33:16.120]  internet that have search boxes, basically.
[33:17.080 --> 33:19.740]  There's all these projects on the internet, search
[33:19.740 --> 33:22.360]  engines, security websites, certificate
[33:23.060 --> 33:26.120]  projects and stuff like that. They all do different stuff.
[33:26.120 --> 33:28.820]  So Robtex gives you infrastructure
[33:28.820 --> 33:31.520]  information. The Wayback Machine houses
[33:31.520 --> 33:34.380]  information about domains and their responses
[33:34.900 --> 33:37.700]  years past. Everyone's probably used the Wayback
[33:37.700 --> 33:40.740]  Machine. The certificate sources down
[33:40.740 --> 33:43.620]  here, some of them are certificate projects to provide certificate
[33:43.620 --> 33:46.400]  transparency. Search engines, obviously you know what search
[33:46.400 --> 33:49.440]  engines are. And then there's a whole bunch of security sites
[33:49.440 --> 33:53.060]  that do different things, like give you a
[33:53.060 --> 33:56.260]  rating on how malicious a URL or a site might be.
[33:56.260 --> 33:58.820]  The common thing that these all have is either have an API or a
[33:58.820 --> 34:01.660]  search box where you can put in a domain. And they will search
[34:01.660 --> 34:04.700]  that domain and they will tell you anything they've seen
[34:04.700 --> 34:07.700]  related to that domain. And if you put in a domain like
[34:07.700 --> 34:10.420]  Tesla.com, the information that comes back is
[34:10.420 --> 34:13.420]  parsable and you could possibly find out that they know about
[34:13.420 --> 34:16.620]  some subdomains of Tesla.com that we don't know about.
[34:16.620 --> 34:19.360]  So this is the process of subdomain scraping.
[34:19.360 --> 34:22.380]  We're going to go to all of these sources, all of these sources and ask
[34:22.380 --> 34:25.400]  them, hey, what do you know about Tesla.com or
[34:25.400 --> 34:28.780]  Twitch.tv? Do you know of any subdomains that maybe I don't know of?
[34:28.800 --> 34:30.880]  Okay, cool. Let's do that. Now,
[34:31.440 --> 34:33.920]  there are many more sources that are on this page, right?
[34:33.920 --> 34:37.220]  These are only a subset and new sources
[34:37.220 --> 34:39.980]  to parse are coming out as fast as new websites are coming out.
[34:39.980 --> 34:43.080]  So the tools have to keep up adding new
[34:43.660 --> 34:46.300]  novel sources to find subdomain data or URL
[34:46.300 --> 34:48.800]  data as they
[34:49.360 --> 34:52.180]  mature. Now, this is the example of using
[34:52.360 --> 34:55.420]  a search engine like Google to do it, right? So here, what we're doing
[34:55.420 --> 34:57.840]  is we're searching with the search operators
[34:57.840 --> 35:01.160]  site.twitch.tv and we're saying, I already know about
[35:01.820 --> 35:03.580]  www.twitch.tv, so we're saying
[35:03.580 --> 35:06.640]  minus www.twitch.tv. And we already know about
[35:06.640 --> 35:09.700]  watch.twitch.tv, so minus watch.twitch.tv
[35:09.700 --> 35:12.760]  and minus dev.twitch.tv because we already know about that. So now,
[35:12.760 --> 35:15.740]  Google's only showing us things that are not those
[35:15.740 --> 35:18.460]  and so we can do this process, keep on
[35:18.460 --> 35:21.620]  minusing out domains until Google doesn't know anything
[35:21.620 --> 35:24.260]  more about subdomains related to twitch.tv.
[35:24.520 --> 35:27.580]  And that way, we can basically get a full
[35:27.580 --> 35:30.600]  inventory of what Google knows and what Google knows about subdomains
[35:30.600 --> 35:33.580]  for twitch.tv. So this is an example of doing it manually
[35:33.580 --> 35:36.540]  with Google. But luckily, you don't have to do this yourself. There's
[35:36.540 --> 35:39.720]  tools out there that will do this type of analysis, this scraping
[35:39.720 --> 35:42.620]  for you. The first
[35:42.620 --> 35:45.900]  one is Amass by Jeff Foley and the Amass team.
[35:45.900 --> 35:48.760]  There's a whole team behind this. And there's two tools I use
[35:48.760 --> 35:51.380]  here, Amass and Subfinder, but we're going to look at Amass first.
[35:51.580 --> 35:54.360]  Amass has probably the most
[35:54.360 --> 35:57.320]  sources for any subdomain
[35:57.320 --> 36:00.980]  scraping tool in existence in its enum section.
[36:00.980 --> 36:03.220]  Amass is a framework for domain information,
[36:03.220 --> 36:06.360]  but the one that actually pulls this method's
[36:06.360 --> 36:09.280]  results, subdomain scraping, is called Amass Enum.
[36:09.940 --> 36:12.680]  And here on the right-hand side, you can see we gave it twitch.tv
[36:12.680 --> 36:15.500]  and it went out to a whole bunch of sources it's had
[36:15.500 --> 36:17.880]  in its databases and it reached out to those web pages
[36:18.400 --> 36:21.180]  using curl or whatever. I don't know if they
[36:21.180 --> 36:23.620]  use headless chromium. I don't know exactly what they're using.
[36:23.620 --> 36:26.960]  And then they parse those pages for subdomains for twitch.tv
[36:26.960 --> 36:30.400]  and they give us a list of all the subdomains for twitch.tv.
[36:30.960 --> 36:32.940]  And if we do this on all of our
[36:32.940 --> 36:35.860]  seed domains that we've gathered, we have now started to exponentially
[36:36.520 --> 36:39.500]  build out the amount of sites that we have to attack.
[36:39.500 --> 36:42.480]  So this tool is invaluable.
[36:42.600 --> 36:44.800]  Amass is becoming the go-to
[36:44.800 --> 36:48.420]  tool for all subdomain enumeration,
[36:48.420 --> 36:51.220]  especially this thing, subdomain
[36:51.220 --> 36:54.380]  scraping. The other tool I used here is
[36:54.380 --> 36:57.540]  oh, also, this is still Amass. What Amass does at the end
[36:57.540 --> 37:00.400]  of a run, not only does it give you all the subdomains,
[37:00.400 --> 37:03.200]  but it also gives you this great table, which I feel like is underutilized
[37:03.360 --> 37:06.160]  a little bit. And this table tells you, okay, I discovered
[37:08.300 --> 37:09.180]  439 subdomains
[37:09.180 --> 37:12.200]  and here is where they were inside
[37:12.200 --> 37:15.320]  of these ASNs. And you can see that most
[37:15.320 --> 37:18.440]  of twitches were in
[37:18.440 --> 37:21.520]  Amazon's ranges, which makes sense because they were acquired by Amazon,
[37:21.520 --> 37:23.780]  but you can see some of them were in other
[37:24.800 --> 37:27.380]  ASNs, right? And I've had instances where,
[37:27.380 --> 37:30.400]  not on Twitch, but on other projects where
[37:30.400 --> 37:33.220]  there was a whole ASN related to the company
[37:33.220 --> 37:36.140]  I didn't know about that I found out because
[37:36.140 --> 37:39.300]  Amass built this table. They're like, hey, 70
[37:39.300 --> 37:42.420]  of these subdomains we discovered appeared in this ASN
[37:42.420 --> 37:45.380]  or these IP ranges. And then I look at that and I'm like,
[37:45.380 --> 37:48.720]  oh, I wasn't initially looking at those ranges.
[37:48.720 --> 37:51.260]  Weird. Let's go back to the beginning of my workflow
[37:51.260 --> 37:54.060]  and start enumerating those ranges. So this table is
[37:54.560 --> 37:57.620]  super cool. It also gives you information on their third
[37:57.620 --> 38:00.020]  party kind of tools they're using.
[38:00.700 --> 38:02.160]  Here you can see that
[38:03.220 --> 38:05.980]  Twitch is using Bitly and SendGrid
[38:05.980 --> 38:09.520]  and some other stuff. So it also gives you information there in Fastly.
[38:11.040 --> 38:12.340]  Okay, the other tool I use for
[38:12.340 --> 38:15.280]  subdomain scraping is Subfinder.
[38:15.280 --> 38:18.020]  Originally written by Iceman and Michael Skelton, I think
[38:18.020 --> 38:21.300]  now navigated to the ProjectDiscovery.io team, which is
[38:21.540 --> 38:24.400]  a group of bug hunters releasing some stellar tools.
[38:24.760 --> 38:27.240]  And they also have multiple sources,
[38:27.240 --> 38:30.760]  extensible output, also a really great tool.
[38:30.760 --> 38:33.840]  Both of these tools are really good.
[38:34.960 --> 38:36.320]  Each one of them have different
[38:36.320 --> 38:39.460]  sources. So what I end up doing is in my automation, I run both of them
[38:39.460 --> 38:42.240]  and then I cat the output together
[38:42.240 --> 38:45.640]  and sort it and unique it from both the tools.
[38:45.640 --> 38:48.540]  So they both have a couple of different sources
[38:48.540 --> 38:51.880]  and a lot of the same sources. So that's what I do for my stuff
[38:51.880 --> 38:53.360]  is I just use both of them.
[38:55.460 --> 38:57.700]  Okay, so this one is somewhat new
[38:57.700 --> 39:00.680]  and it has been integrated into
[39:01.220 --> 39:03.540]  Amass a little bit.
[39:04.400 --> 39:04.700]  But
[39:08.060 --> 39:10.580]  I've had inconsistency with run
[39:10.580 --> 39:13.800]  between this standalone tool, GitHub
[39:13.800 --> 39:16.520]  subdomains.py and the output of Amass.
[39:16.520 --> 39:19.520]  So I still use this independently inside of my automation
[39:19.520 --> 39:21.640]  when I'm looking for subdomains.
[39:21.640 --> 39:24.700]  So this tool is called GitHub subdomains.py
[39:24.700 --> 39:28.180]  and it's still scraping and what this is doing is it's going out to GitHub
[39:28.180 --> 39:30.840]  as a source. And basically you're providing GitHub
[39:30.840 --> 39:33.440]  and if you've ever been on GitHub, they have the search box up top
[39:33.440 --> 39:36.060]  and you're saying search for twitch.tv
[39:36.060 --> 39:38.840]  and then anything that comes back with twitch.tv
[39:39.180 --> 39:42.660]  as a piece of source code, it will parse up the subdomains
[39:42.660 --> 39:44.700]  from that piece of source code.
[39:44.840 --> 39:48.600]  Now this was written by Gwendolyn Kuik
[39:48.600 --> 39:51.340]  and I still don't know if I'm saying Gwendolyn's name
[39:51.340 --> 39:53.420]  right and that's okay.
[39:53.540 --> 39:57.560]  And Gwendolyn wrote a whole suite of tools for GitHub
[39:57.560 --> 40:00.540]  enumeration. How to find secret keys in
[40:00.540 --> 40:03.680]  GitHub related to an organization. How to pull email
[40:03.680 --> 40:06.500]  addresses out. He has a wonderful blog
[40:06.500 --> 40:09.680]  and it's linked here in this slide with a whole bunch of GitHub
[40:09.680 --> 40:13.340]  tools. Now GitHub subdomains is just looking for subdomains.
[40:13.920 --> 40:15.800]  The thing about using this tool
[40:15.800 --> 40:18.120]  is that the GitHub API, not the tool
[40:19.080 --> 40:21.520]  is just kind of instable and
[40:21.520 --> 40:24.540]  doesn't give you like somewhat returns rate limited
[40:24.540 --> 40:26.980]  results sometimes. So what I do inside of my
[40:26.980 --> 40:30.280]  automation and this has given me subdomains I haven't
[40:30.280 --> 40:33.060]  found anywhere else using this method is I run GitHub
[40:33.060 --> 40:34.820]  search or GitHub subdomains.py
[40:36.220 --> 40:39.360]  like five times and I sleep in between each run
[40:39.360 --> 40:42.220]  so that I give it a little while for the rate limiting to die down
[40:42.780 --> 40:45.420]  and then I give it a big sleep at the end before I run
[40:45.420 --> 40:47.760]  another one and then that seems
[40:47.760 --> 40:51.100]  and then I cat all those results together and unique them
[40:51.100 --> 40:53.160]  and that seems to give me more consistency
[40:53.800 --> 40:56.360]  when parsing the API. And that has nothing to do with the tool that Gwendo
[40:56.360 --> 40:59.360]  wrote. It's all about what GitHub does with their
[40:59.360 --> 41:02.240]  search API. So this is an awesome tool to
[41:02.240 --> 41:05.360]  find lesser known subdomains and I found some
[41:05.360 --> 41:07.940]  great stuff using this tool.
[41:09.160 --> 41:11.160]  Okay, the next one is
[41:11.620 --> 41:14.460]  show sub go by incogbite
[41:14.460 --> 41:17.100]  and this one will parse
[41:17.580 --> 41:20.280]  using a number of search operators with your
[41:20.280 --> 41:23.340]  API key. It will parse showdown.
[41:23.460 --> 41:27.020]  So I include this in my script as well.
[41:27.320 --> 41:29.360]  Again, this is one of those tools that some of
[41:29.360 --> 41:31.720]  the subdomain frameworks like Amass
[41:32.280 --> 41:35.380]  and Subfinder might do for you.
[41:35.400 --> 41:38.380]  I find using the standalone tool for
[41:38.380 --> 41:41.320]  some reason just works better for me. So I don't know exactly why that is
[41:41.320 --> 41:44.460]  but I've run them side by side and gotten different outputs so I still use this
[41:45.120 --> 41:47.380]  verbatim. And it's also a fast
[41:47.380 --> 41:50.400]  running script. It's not like I'm waiting for it to complete for 5-10
[41:50.400 --> 41:53.380]  minutes or something like that. It usually runs in a minute or two. So it doesn't
[41:53.380 --> 41:56.480]  skin off my nose if I have to wait an extra couple of minutes to
[41:56.480 --> 41:59.140]  ensure that I get coverage out of showdown. So
[41:59.140 --> 42:02.000]  show sub go will take a domain and your
[42:02.000 --> 42:05.580]  showdown API key and parse showdown using some search operators
[42:05.580 --> 42:08.400]  and give you back all the subdomains related to here
[42:08.400 --> 42:09.640]  twitch.tv.
[42:11.880 --> 42:14.500]  Okay, so the last... well, not the last, but one of the
[42:14.500 --> 42:17.140]  other methods to look for subdomain enumeration
[42:17.140 --> 42:20.440]  or subdomain scraping is the cloud ranges, right? So we talked about this
[42:20.560 --> 42:23.180]  a little bit earlier is that we have all this IP space
[42:23.180 --> 42:26.400]  and now we've started to identify subdomains and you'll notice that some
[42:26.400 --> 42:28.600]  of those subdomains are resolving to
[42:29.320 --> 42:32.180]  infrastructure or sites in the cloud. Now
[42:32.180 --> 42:35.200]  there's this idea that not a lot of people have been doing is just
[42:35.200 --> 42:38.140]  going out recently or not recently but
[42:38.140 --> 42:41.360]  people have been doing it for a while but it hasn't been much public is just
[42:41.360 --> 42:44.560]  scanning the entire ranges for AWS, GCP
[42:44.560 --> 42:47.240]  and Azure for SSL sites, right? Anything that
[42:47.240 --> 42:50.680]  responds to... anything that responds
[42:50.680 --> 42:53.380]  to, you know, or responds
[42:53.380 --> 42:56.320]  to a connection on 443. And you basically
[42:56.320 --> 42:59.420]  scan the cloud ranges and you scan
[42:59.420 --> 43:02.600]  it by IP address and then when it responds
[43:02.600 --> 43:05.580]  it'll give you its SSL certificate and then you parse
[43:05.580 --> 43:08.460]  the organization name or the domain name out of the
[43:08.460 --> 43:11.440]  certificate data. And then you look at that data and you say
[43:11.440 --> 43:14.440]  does it match my target twitch.tv or does it have
[43:14.440 --> 43:17.500]  the keyword twitch in it? And you're like, cool, I found some stuff
[43:17.500 --> 43:20.460]  that these people have put in the cloud that wasn't part of their
[43:20.460 --> 43:23.540]  ASN that I probably didn't know about before. Now
[43:23.540 --> 43:26.540]  that's a tremendous amount of scanning, right? Those
[43:26.540 --> 43:29.460]  ranges are huge. And you can do it
[43:29.460 --> 43:32.500]  yourself with some tools like mass scan and you
[43:32.500 --> 43:35.480]  can script it up yourself to scan but
[43:39.380 --> 43:41.940]  it's going to cost you a little bit of money
[43:41.940 --> 43:45.080]  on your VPS that you're using, etc. There's a
[43:45.080 --> 43:48.220]  service by Sam Erb and a DEF CON talk he did
[43:48.220 --> 43:51.080]  two years ago where he created a service called
[43:51.080 --> 43:53.860]  bufferover.run. It's an API and you can give it a domain
[43:53.860 --> 43:56.960]  and it will go out and he does this scanning
[43:56.960 --> 44:00.320]  every once in a while. And so you basically
[44:00.320 --> 44:02.860]  take that data, you parse it, and then you get
[44:02.860 --> 44:05.400]  a list of subdomains that were in
[44:05.400 --> 44:08.540]  the cloud ranges. Now, I think he runs his
[44:08.540 --> 44:11.420]  scans every two weeks and updates the service so
[44:11.420 --> 44:14.740]  it's not exactly live data. And
[44:14.740 --> 44:17.620]  this is one of the sources included in a mass.
[44:17.700 --> 44:20.100]  Again, I just wanted to outline the
[44:20.100 --> 44:23.400]  single-shot tool here. You probably could feel safe using a mass
[44:23.400 --> 44:25.780]  to get most of this data back. But
[44:26.400 --> 44:29.300]  yeah, you can use the service as well. So
[44:29.300 --> 44:32.460]  this method also finds some really good
[44:32.460 --> 44:35.320]  stuff. A lot of people are putting shadow IT infrastructure
[44:35.320 --> 44:38.640]  and registering websites on a credit card or
[44:39.300 --> 44:41.620]  just not paying attention or thinking that anybody
[44:41.620 --> 44:44.640]  will find their cloud infrastructure because they've never published
[44:45.200 --> 44:47.880]  those domains anywhere other than internally.
[44:48.260 --> 44:50.620]  And so you can find a lot of things for your
[44:50.620 --> 44:54.300]  target organization that are just sitting in the cloud, pretty much unsecured.
[44:54.300 --> 44:56.700]  In fact, it's been a big part of my research
[44:56.700 --> 44:59.640]  lately is scanning the cloud ranges and
[44:59.640 --> 45:02.600]  just finding wickedly undersecured
[45:02.600 --> 45:06.200]  stuff because people just don't think you'll ever find it. So this is a good method.
[45:07.320 --> 45:08.640]  All right. So we've scraped
[45:08.800 --> 45:11.560]  a lot of stuff to find subdomains. Now we want to brute force
[45:11.560 --> 45:14.560]  for subdomains. This is just a tried
[45:14.560 --> 45:17.460]  and true method that I'm sure every red team or pen tester has
[45:17.460 --> 45:20.100]  done before, usually with a tool like Fierce or
[45:20.100 --> 45:23.120]  one of the other tools like that in the past 10 years.
[45:23.420 --> 45:26.200]  You just try to give a word out of a dictionary
[45:26.780 --> 45:29.720]  and add it before your company.com
[45:29.720 --> 45:32.880]  and see if it resolves. It's a pretty simple
[45:32.880 --> 45:35.840]  method. Now, iteration in this field has
[45:35.840 --> 45:38.380]  come along in the last four years where
[45:39.940 --> 45:41.800]  a lot of the tools that we were using to do this
[45:41.800 --> 45:44.760]  were great, but they were using one DNS resolver,
[45:44.760 --> 45:47.780]  one DNS server to resolve. And
[45:47.780 --> 45:50.900]  it took a long time. I remember running Fierce as a pen tester
[45:51.720 --> 45:53.440]  10 years ago, and it just
[45:53.440 --> 45:56.480]  took so long to finish a large dictionary to do
[45:56.480 --> 45:59.720]  subdomain brute forcing. And in the last
[45:59.720 --> 46:02.540]  four years,
[46:02.540 --> 46:05.420]  the idea of using multiple
[46:05.420 --> 46:08.240]  resolvers to speed up the process was pioneered by
[46:08.240 --> 46:11.440]  MassDNS. And MassDNS was the king for a little
[46:11.440 --> 46:14.720]  while. Now Amass also includes
[46:14.720 --> 46:17.680]  the idea of using multiple resolvers. So Amass
[46:17.680 --> 46:20.000]  uses eight DNS resolvers
[46:22.360 --> 46:24.780]  to parse DNS data when it brute
[46:24.780 --> 46:27.820]  forces by default. And you can add even more
[46:27.820 --> 46:30.980]  by using some flags. So
[46:30.980 --> 46:34.280]  here we're just running Amass over, you know,
[46:34.280 --> 46:36.900]  Twitch again. And we're doing brute forcing this time
[46:36.900 --> 46:40.060]  with the dash brute
[46:40.060 --> 46:43.100]  option. And then we're adding the source so we can
[46:43.100 --> 46:46.080]  see where some of these came from with the scraping
[46:46.080 --> 46:49.320]  part. And then you can specify the number of resolvers
[46:49.320 --> 46:52.260]  with dash RF. So RF means resolver
[46:52.260 --> 46:55.720]  file, I think. And if you basically
[46:55.720 --> 46:58.680]  give it a list of DNS resolvers or DNS servers
[46:58.680 --> 47:01.740]  that you trust, it will use more than one to do
[47:01.740 --> 47:04.500]  your brute forcing. So this speeds up your brute
[47:04.500 --> 47:07.440]  forcing significantly. I've also heard that
[47:07.440 --> 47:10.200]  another alternative to using Amass for this is
[47:10.740 --> 47:13.340]  ASDNS brute. I haven't used it yet, but I've heard it's
[47:13.340 --> 47:16.060]  also wicked fast and has multiple resolvers. So if you wanted an
[47:16.060 --> 47:19.920]  alternative. Shuffle DNS by the project
[47:19.920 --> 47:22.660]  discovery team also does this. I think it's a
[47:22.660 --> 47:25.780]  wrapper around mass DNS. It is a
[47:25.780 --> 47:28.940]  wrapper around mass DNS. And if you prefer to
[47:28.940 --> 47:31.800]  break out that type of brute force from Amass
[47:31.800 --> 47:34.760]  for whatever reason, whether it's stability or
[47:34.760 --> 47:37.560]  mass, you know, is taking too long for you.
[47:37.560 --> 47:40.480]  I haven't benchmarked the tools side by side, but a
[47:40.480 --> 47:42.860]  lot of people like Shuffle DNS as well to do
[47:42.860 --> 47:46.700]  subdomain brute forcing. So a
[47:46.700 --> 47:50.000]  subdomain brute force tool is only as good as the
[47:50.000 --> 47:52.900]  dictionary or word list you give it, right? Because it's just
[47:53.040 --> 47:55.200]  trying to resolve a whole bunch of words in front of your
[47:55.200 --> 47:58.920]  target, right? Twitch.tv. And so
[47:58.920 --> 48:01.680]  in this mind, there's two kind of ways that you can
[48:01.680 --> 48:04.380]  approach what word list you feed to these tools. One
[48:04.380 --> 48:07.760]  is a tailored word list where you can
[48:07.760 --> 48:09.900]  build one based off of words that appear,
[48:09.900 --> 48:13.440]  brand names that appear, you know, you can build
[48:13.440 --> 48:15.900]  like a contextual based word list to your
[48:16.960 --> 48:20.000]  target, which Tom Nom Nom, who's a prolific
[48:20.000 --> 48:23.220]  bug hunter, awesome human, have a lot of respect
[48:23.220 --> 48:25.380]  for this guy and the tools he makes and the
[48:25.380 --> 48:28.320]  contributions he makes to the community. He did a talk
[48:28.320 --> 48:31.380]  at NahomCon, which is a conference a little while
[48:31.380 --> 48:33.960]  ago now, a couple months ago, where he did a whole talk
[48:33.960 --> 48:36.960]  on word list and how to generate contextual word list for your target.
[48:36.960 --> 48:40.020]  So I suggest going to watch that if you want to make some tailored word list. It's called
[48:40.020 --> 48:42.780]  Who, What, When, Where word list. And then
[48:42.780 --> 48:45.880]  you can also use a massive word list. And so over
[48:45.880 --> 48:48.880]  the years, we've had many, many tools that do
[48:48.880 --> 48:51.600]  DNS brute forcing. I went out and I took the
[48:51.600 --> 48:54.480]  word list for all of them and put them into
[48:55.560 --> 48:57.980]  one, and it's called all.txt. It's
[48:57.980 --> 49:00.720]  linked in this presentation. People use it. It's on my GitHub.
[49:01.100 --> 49:03.940]  And it basically sort and uniques all of the DNS
[49:03.940 --> 49:06.800]  names that you've seen forever. Now it has a lot of
[49:06.800 --> 49:09.840]  crap in it. It's true. It's a lot of lines. It's like a million
[49:09.840 --> 49:12.440]  or two million. I can't remember. It's a ton. With
[49:12.940 --> 49:15.700]  using the multiple resolvers, it actually doesn't
[49:15.700 --> 49:18.740]  take too long to do the brute forcing. I
[49:18.740 --> 49:21.880]  don't necessarily care if I'm sending crap to
[49:22.340 --> 49:23.600]  a DNS resolver like
[49:26.140 --> 49:27.960]  0001.twitch.tv. Obviously
[49:27.960 --> 49:30.380]  that's usually not going to resolve, but
[49:30.380 --> 49:33.600]  the list has some gems in it that I
[49:33.600 --> 49:35.520]  just can't get past, you know, like
[49:36.920 --> 49:40.080]  its efficacy. So I use all.txt
[49:40.080 --> 49:43.020]  when I'm doing subdomain brute forcing, and I think it's pretty good.
[49:44.460 --> 49:45.480]  There are some
[49:45.480 --> 49:48.060]  newer school pieces of research to pull out
[49:48.640 --> 49:51.200]  subdomain enumeration, right? That file
[49:51.200 --> 49:54.640]  that I made parsed out all of the kind of
[49:54.640 --> 49:57.520]  subdomain brute forcing tools that have existed for the last, you know,
[49:57.520 --> 50:00.320]  10 years or something like that and put them into one file.
[50:00.320 --> 50:02.920]  But there's some new research by the team at AssetNote,
[50:02.920 --> 50:06.460]  which is an attack surface mapping company.
[50:06.500 --> 50:09.040]  And they did some cool research using
[50:09.040 --> 50:12.680]  Google BigQuery to generate and discover
[50:12.680 --> 50:14.980]  subdomains that were used on, like, the Alexa
[50:14.980 --> 50:18.320]  top 10 or Alexa top 1,000 or 10,000
[50:18.320 --> 50:20.780]  or something like that. Or Reddit. Or anytime
[50:20.780 --> 50:24.480]  someone referenced a URL on Reddit, what was the subdomain in that link?
[50:24.480 --> 50:27.540]  Or Stack Overflow. Anytime someone
[50:27.540 --> 50:30.420]  referenced a URL on Stack Overflow, they parsed these sites
[50:30.420 --> 50:32.300]  with BigQuery, and then they made these
[50:33.760 --> 50:36.600]  subdomain lists that you can use. So they called this project
[50:36.600 --> 50:39.280]  the Common Speak project. I've integrated Common Speak 1
[50:39.280 --> 50:42.260]  into the list, all.txt,
[50:42.260 --> 50:45.240]  but Common Speak 2 came out a couple years ago. It's not in
[50:45.240 --> 50:48.600]  all.txt. It's got some other sites that they decided
[50:48.600 --> 50:51.400]  to target. I recommend checking it out for kind of newer school
[50:51.400 --> 50:54.760]  research on what subdomain names look like.
[50:55.340 --> 50:57.780]  Then there's this idea of alteration scanning.
[50:57.780 --> 51:00.360]  This is a type of brute forcing. So you have
[51:00.360 --> 51:03.320]  dev.company.com, but you could also have dev1,
[51:03.320 --> 51:06.160]  dev2, or dev-1, or dev.1.
[51:06.600 --> 51:09.580]  company.com. And so this technique was
[51:09.580 --> 51:12.560]  pioneered by Shubbs and Nathy when they wrote a tool called AltDNS.
[51:12.560 --> 51:15.320]  It's now been this permutation
[51:15.320 --> 51:18.480]  or alteration scanning, whatever you want to call it, has been built
[51:18.480 --> 51:21.480]  into a mass. So you can use a mass
[51:21.480 --> 51:24.280]  and it will try to find these naming conventions
[51:24.280 --> 51:27.500]  for you. And it will sometimes
[51:27.500 --> 51:30.460]  give you gold because people name stuff predictably
[51:30.460 --> 51:32.200]  in their subdomains.
[51:33.640 --> 51:36.140]  Some of the things I've done with permutation scanning
[51:36.140 --> 51:39.680]  is bypassing web application firewalls.
[51:39.680 --> 51:42.540]  So where I've had SQL injection on a main
[51:42.540 --> 51:45.680]  target and getting blocked by like a web application
[51:45.680 --> 51:48.400]  firewall, I've managed to find a permutation
[51:48.400 --> 51:51.420]  like dubdub2 and then managed to bypass the
[51:51.420 --> 51:54.820]  firewall because it wasn't applied to dubdub2.
[51:55.020 --> 51:57.420]  I've also managed to bypass things like
[51:57.420 --> 52:00.480]  Akamai by finding their origin via predictably
[52:00.480 --> 52:03.200]  named alterations like origin-sub or
[52:03.200 --> 52:06.520]  origin.sub domain to bypass filtering to go to the source.
[52:06.520 --> 52:09.380]  So these are some things you can use that origin scanning to
[52:09.380 --> 52:12.420]  do, as well as just find new surface to
[52:12.420 --> 52:13.140]  attack.
[52:14.300 --> 52:17.340]  All right, so now we're going to go into some other stuff that's related to
[52:17.340 --> 52:20.640]  Widescope Reon. We have seed domains, we've got a lot of seed
[52:20.640 --> 52:23.520]  domains, a lot of subdomains right now, and we should have a pretty good
[52:23.520 --> 52:27.300]  map of the infrastructure that belongs to this company.
[52:28.040 --> 52:30.080]  All right, so one that I didn't talk about,
[52:30.080 --> 52:33.140]  and this is new to this specific talk,
[52:33.140 --> 52:35.920]  that's why I called it 4.002,
[52:35.920 --> 52:38.760]  is favicon analysis. So there is this idea
[52:38.760 --> 52:41.680]  that every page in the tab at the top of your browser
[52:41.680 --> 52:44.860]  has a favicon, right? You see that little Tesla in the bottom left-hand corner
[52:44.860 --> 52:47.540]  of my Chrome tab? That's a favicon.
[52:47.960 --> 52:50.700]  Now, favicons are little
[52:50.700 --> 52:53.580]  images, and what you can do is you can take a hash
[52:53.580 --> 52:56.880]  of that favicon, and then you can search for the hash
[52:56.880 --> 52:59.720]  of the favicon on Shodan. And Shodan
[52:59.720 --> 53:02.900]  will then show you every other site that has that favicon,
[53:02.900 --> 53:05.740]  which will find you some gems, because people tend to reuse
[53:05.740 --> 53:08.960]  favicons on a lot of their sites of domains you might not have seen
[53:08.960 --> 53:12.200]  before. There are some newer tools to do this.
[53:12.200 --> 53:14.620]  I used to do this a lot in my recon testing.
[53:14.640 --> 53:17.880]  I then took it out for a little while, and I recently
[53:17.880 --> 53:20.800]  put it back in. So favicon analysis is kind of
[53:20.960 --> 53:23.760]  a fringe technique, but it's pretty
[53:23.760 --> 53:25.880]  cool. So there's a new tool called favfreakout by
[53:27.100 --> 53:29.440]  Devansh Batham. They're at Twitter at
[53:29.440 --> 53:32.820]  Asmodeus, and basically he does a couple
[53:32.820 --> 53:35.060]  things here. He parses from Shodan.
[53:35.240 --> 53:38.980]  He'll also look for
[53:38.980 --> 53:41.780]  hashes in different places, but he also uses
[53:41.780 --> 53:44.720]  the hash. He also has hashes for
[53:44.720 --> 53:48.480]  common infrastructure, like a scanner would,
[53:48.480 --> 53:51.020]  to look for different types of infrastructure. So on the bottom
[53:51.020 --> 53:53.140]  right-hand side, you can see that he has some
[53:54.160 --> 53:57.080]  fingerprints on different types of things, like Spring Boot pages
[53:57.080 --> 54:00.040]  or Big IP pages or Slack instances. They all have common
[54:00.040 --> 54:02.920]  favicons. And so when you scan a list of
[54:02.920 --> 54:06.180]  URLs or a set of domains on
[54:06.180 --> 54:08.920]  port 80 or 443 and you retrieve their favicon and their
[54:08.920 --> 54:11.900]  hash matches one of those hashes, you know, oh, shoot, I've stumbled
[54:11.900 --> 54:15.020]  upon a Slack instance for this organization. And you
[54:15.020 --> 54:17.980]  may not have found that doing anything else.
[54:17.980 --> 54:21.080]  You can also do the same thing if you scan the cloud
[54:21.080 --> 54:24.080]  and you correlate one of the previous techniques with this one.
[54:24.080 --> 54:27.000]  Let me find things that have my domain and the certificate data and have
[54:27.000 --> 54:30.120]  these favicon hashes associated to them. So this is
[54:30.580 --> 54:33.300]  a fringe kind of analysis technique to find
[54:33.300 --> 54:36.260]  even more kind of esoteric
[54:37.700 --> 54:39.020]  related sites.
[54:40.780 --> 54:42.080]  Okay. So then we're
[54:42.080 --> 54:45.020]  going to go back to a tried and true method
[54:45.020 --> 54:48.080]  is port scanning. We have a lot of domains now. We have a lot of C domains
[54:48.080 --> 54:50.820]  and subdomains and sites that we can work with.
[54:50.960 --> 54:54.040]  We want to port scan them because they may have services that are
[54:54.040 --> 54:56.960]  not 80 or 443 on these pieces of
[54:56.960 --> 55:00.440]  infrastructure. And so the, you know,
[55:00.440 --> 55:03.140]  the tried and true hacker education
[55:03.140 --> 55:05.900]  will tell you to use nmap here. But I use
[55:05.900 --> 55:09.040]  mass scan because I believe it's faster. It has a
[55:09.860 --> 55:12.160]  rewritten TCP IP stack, true multi
[55:12.660 --> 55:15.520]  threading. It's written in C, you know, directly
[55:15.520 --> 55:18.340]  calls, you know, a lot of stuff. So mass scan, in my
[55:18.340 --> 55:20.820]  experience, has been faster than using nmap, even with flags
[55:20.820 --> 55:23.920]  like min parallelism, which there was a huge debate on Twitter
[55:23.920 --> 55:26.720]  the other day of which is faster, min parallelism or using
[55:26.720 --> 55:29.060]  mass scan and people were like, whatever. When I run
[55:30.540 --> 55:32.820]  a scanning tool, a port scanning tool across,
[55:32.820 --> 55:35.640]  you know, 400,000 hosts,
[55:35.640 --> 55:37.960]  I have always found mass scan and it's
[55:38.920 --> 55:41.720]  syntax, it's advantages
[55:41.720 --> 55:44.800]  to beat out nmap. So that's just my personal
[55:44.800 --> 55:47.600]  experience. If you really like nmap to do this, you totally can.
[55:47.600 --> 55:50.460]  And this is strictly for finding ports. It's not to do
[55:50.460 --> 55:53.260]  service scanning, right? Nmap obviously wins in those areas. If you're
[55:53.260 --> 55:56.400]  going to do banner analysis, service scanning, script
[55:56.400 --> 55:59.360]  scanning, like that all pans down. Mass scan
[55:59.360 --> 56:02.440]  only does one thing, finds open ports. That's it. So what I
[56:02.440 --> 56:05.360]  do, also, if you want to learn how to use mass scan,
[56:05.360 --> 56:08.120]  Daniel Miesler, one of my best friends in the whole world,
[56:08.120 --> 56:11.260]  he wrote a study guide on mass scan and all of its syntax.
[56:11.260 --> 56:14.100]  It's one of the best ones I've seen, even better than man page. So go check
[56:14.100 --> 56:17.220]  that out on how to use mass scan and how to set the flags
[56:17.220 --> 56:19.860]  correctly and how to scan. So
[56:19.860 --> 56:23.120]  what I do is I'll take mass scan and then the problem
[56:23.120 --> 56:26.200]  about mass scan is it only scans IP
[56:26.200 --> 56:29.160]  addresses. It won't scan a domain name. So
[56:29.160 --> 56:31.960]  you can use a tool called dnmasscan, which will convert
[56:31.960 --> 56:35.120]  your domains that we have, right? We have subdomains and
[56:35.120 --> 56:38.200]  seed domains and it'll convert them into IP addresses and then scan them with mass
[56:38.200 --> 56:41.460]  scan and tell you all the open ports on each IP address.
[56:41.460 --> 56:43.860]  So what I do is I take mass scan
[56:44.760 --> 56:47.400]  and I scan that over all of my subdomains
[56:47.400 --> 56:50.480]  and then I feed that output, because I know it's
[56:50.480 --> 56:53.500]  open, to nmap to do service scanning. And
[56:53.500 --> 56:56.500]  service scanning will start to give me more information
[56:56.500 --> 56:59.560]  on these services that are open on those boxes. And then what
[56:59.560 --> 57:02.460]  I do is I do a quick default credential spray
[57:02.460 --> 57:05.380]  across everything that has certain services open,
[57:05.380 --> 57:09.160]  which is FTP, SMTP, SSH, Telnet,
[57:09.160 --> 57:11.880]  any type of SQL database, basic
[57:11.880 --> 57:14.880]  authorization and some other stuff. And so this is a tool
[57:14.880 --> 57:17.940]  that I use to do that. It's called Brute Spray. Brute Spray will
[57:17.940 --> 57:21.000]  take the output of your nmap scan, so you feed mass
[57:21.000 --> 57:23.800]  scan to nmap, nmap does the full service
[57:23.800 --> 57:26.920]  scan and outputs an XML file, and then you feed that
[57:26.920 --> 57:29.680]  XML file to Brute Spray to do a quick
[57:29.680 --> 57:32.920]  credential, default credential brute force against all
[57:32.920 --> 57:35.720]  of the services that allow you to do that. And so I've found
[57:35.720 --> 57:38.460]  some good wins doing this, logging
[57:38.460 --> 57:41.460]  straight into SQL databases and SSH
[57:41.460 --> 57:44.480]  and just some horrible stuff that people leave
[57:44.480 --> 57:46.700]  unsecured on the Internet, so on services.
[57:47.800 --> 57:50.860]  All right. So while I'm doing
[57:50.860 --> 57:53.660]  all of this stuff, right, we're in kind of the other category right now. So while
[57:53.660 --> 57:56.500]  I'm doing all of this analysis, and I have most of this automated
[57:56.500 --> 57:59.620]  in a giant ugly shell script that I use, and we'll talk about
[57:59.620 --> 58:02.780]  recon frameworks in a second. But while I'm doing all this,
[58:02.780 --> 58:05.640]  it takes a little while for these tools to run. The conglomeration
[58:05.640 --> 58:07.980]  of tools and techniques we've already talked about takes
[58:08.740 --> 58:12.000]  anywhere between 5 and 15 minutes,
[58:12.000 --> 58:15.080]  maybe a little bit more if I have a lot of subdomains
[58:15.080 --> 58:17.980]  or it's a big project. It'll take to run all of these
[58:17.980 --> 58:21.000]  tools and give me output. So while I'm doing this,
[58:21.520 --> 58:23.700]  I do a technique called GitHub dorking
[58:23.700 --> 58:27.360]  and this has found me many, many great things.
[58:28.240 --> 58:29.860]  And what you do is
[58:29.860 --> 58:32.800]  you basically just go to GitHub and type in your domain as a search operator.
[58:32.800 --> 58:35.780]  So twitch.tv. And you go to GitHub and you type in
[58:35.780 --> 58:39.020]  twitch.tv and then you just start browsing source code
[58:39.020 --> 58:42.220]  that has referenced twitch.tv or tesla.com
[58:42.220 --> 58:44.140]  or whatever, teslamotors.com.
[58:44.460 --> 58:47.680]  And you can find all kinds of sensitive data that former
[58:47.680 --> 58:49.780]  employees or current employees
[58:51.380 --> 58:54.140]  have accidentally put on GitHub.
[58:54.140 --> 58:56.740]  And so I have built a script
[58:56.740 --> 58:59.800]  just to build these search queries for me. These dorks, I call them.
[58:59.800 --> 59:02.300]  And it's right here in a gist. You can grab it.
[59:02.440 --> 59:05.880]  And it looks for things like my domain twitch.tv that I'm looking at
[59:05.880 --> 59:08.120]  right now and password or
[59:10.220 --> 59:12.120]  npmrcauth or dockerconfig
[59:12.120 --> 59:14.240]  or pemprivate for certificates or
[59:15.600 --> 59:17.620]  s3config or htpassword or
[59:17.620 --> 59:21.720]  credentials or bashrcprofiles or sshconfigs.
[59:21.760 --> 59:23.600]  And so it searches these key terms
[59:23.600 --> 59:26.620]  along with the domain. And if it comes up
[59:26.620 --> 59:27.960]  that someone has accidentally
[59:29.400 --> 59:32.720]  put this on GitHub, it's usually automatically a finding and it'll help me
[59:32.720 --> 59:35.640]  get into other systems. Let me give you an example here of something that happened in the real
[59:35.640 --> 59:38.920]  world. I found an admin page for a
[59:38.920 --> 59:41.620]  site. I couldn't do anything with it.
[59:41.620 --> 59:44.660]  I fuzzed the form. Great,
[59:44.660 --> 59:48.280]  didn't work, etc., etc. Then I found
[59:48.280 --> 59:50.480]  some dude who had basically
[59:50.480 --> 59:53.540]  posted a password not for that specific site, but it was
[59:53.540 --> 59:56.460]  related to my domain on GitHub for some other system.
[59:56.640 --> 59:59.680]  That system was internal, so I couldn't access it.
[59:59.680 --> 01:00:02.660]  But then I tried to use that password and username on this admin
[01:00:02.660 --> 01:00:05.440]  portal I had found that I had no luck before. Bam, got in,
[01:00:05.440 --> 01:00:08.380]  stole credit cards, millions of records of data.
[01:00:08.380 --> 01:00:11.320]  It was game over at that point. So this can
[01:00:11.320 --> 01:00:14.260]  help. There's a whole awesome talk here
[01:00:14.260 --> 01:00:16.960]  called GitHub Recon and Sensitive Data Exposure on
[01:00:16.960 --> 01:00:20.120]  Bugcrowd University by the gentleman. It's probably the best
[01:00:20.120 --> 01:00:23.140]  primer on doing this type of dorking to find sensitive
[01:00:23.140 --> 01:00:26.400]  stuff on GitHub that I've ever seen. I highly recommend
[01:00:26.400 --> 01:00:29.380]  you check that out. And then Gwendolyn also wrote a
[01:00:29.380 --> 01:00:32.280]  GitHub search tool that allows you to do this in the command line and
[01:00:32.280 --> 01:00:41.480]  not in the browser, which is...
[01:00:41.480 --> 01:00:44.600]  Okay, so now you have a bunch
[01:00:45.160 --> 01:00:47.520]  of subdomains, and I think I'm running close to time, so I'm going to try
[01:00:47.520 --> 01:00:50.300]  to hurry it up here. You have a bunch
[01:00:50.300 --> 01:00:53.360]  of subdomains here, and now we want to prioritize
[01:00:53.360 --> 01:00:56.440]  which ones we test, right? So what you can do is feed
[01:00:56.440 --> 01:00:59.720]  all of your subdomains to a tool that does screenshotting.
[01:00:59.940 --> 01:01:02.300]  And there are several tools that do
[01:01:02.300 --> 01:01:05.460]  this. Currently, I use Eyewitness. I was using Aquatone.
[01:01:05.460 --> 01:01:08.480]  I go back and forth, but there's four tools here. Aquatone,
[01:01:08.480 --> 01:01:10.760]  HTTP Screenshot, Eyewitness, and WitnessMe.
[01:01:11.060 --> 01:01:14.140]  All are tools that help you take
[01:01:14.140 --> 01:01:16.740]  screenshots, and then some do, like, additional analysis
[01:01:17.880 --> 01:01:20.120]  on your domains that you feed it, but normally
[01:01:20.120 --> 01:01:23.000]  I just use them for screenshotting. It doesn't matter which
[01:01:23.000 --> 01:01:26.000]  one you use. Try them out. See which one you like. I like Eyewitness.
[01:01:26.100 --> 01:01:28.660]  I feed it a list of domains. It will
[01:01:28.660 --> 01:01:32.140]  take those domains, visit them with the headless browser,
[01:01:32.140 --> 01:01:34.940]  take a screenshot of the page, and then I will just look at that
[01:01:34.940 --> 01:01:37.820]  folder and see, okay, which one
[01:01:37.820 --> 01:01:41.040]  of these looks like I want to prioritize first?
[01:01:41.240 --> 01:01:44.020]  Does it look like it redirects to the main site? Well, obviously,
[01:01:44.020 --> 01:01:46.000]  I'm not looking for the main site right now.
[01:01:46.720 --> 01:01:49.740]  It's some kind of back-end admin portal. Okay, I want to prioritize
[01:01:49.740 --> 01:01:52.740]  that. So screenshots can help you prioritize your work.
[01:01:54.900 --> 01:01:55.700]  When you're
[01:01:55.700 --> 01:01:58.980]  given a large list of subdomains like you have now, you can start
[01:01:58.980 --> 01:02:01.960]  to look for some vulnerabilities. One is subdomain takeover.
[01:02:02.180 --> 01:02:04.560]  There's a repo called CanITakeOverXYZ by
[01:02:04.560 --> 01:02:07.720]  edoverflow, and it gives you a list of all of these
[01:02:07.720 --> 01:02:10.960]  services and their fingerprints that
[01:02:10.960 --> 01:02:14.200]  might indicate that you can take over that subdomain. I'm not going to go
[01:02:14.200 --> 01:02:17.040]  crazy into subdomain takeover because it's short on time,
[01:02:17.040 --> 01:02:19.980]  obviously. But you can use a tool to check for
[01:02:19.980 --> 01:02:23.080]  subdomain takeovers. The best one right now is Nuclei
[01:02:23.640 --> 01:02:26.180]  and SubOver, I think, are my two favorites
[01:02:26.180 --> 01:02:29.100]  right now. SubOver was an independent tool,
[01:02:29.100 --> 01:02:31.800]  has been since ported over into the Nuclei framework that
[01:02:31.800 --> 01:02:35.220]  Project Discovery is making. But go check out Nuclei.
[01:02:35.220 --> 01:02:38.120]  It has the most
[01:02:38.120 --> 01:02:41.160]  subdomain takeover checks that I've
[01:02:41.160 --> 01:02:44.080]  seen for any tool. And I would
[01:02:44.080 --> 01:02:47.040]  check that out. And you can run it just across a list of large
[01:02:47.040 --> 01:02:49.880]  domains, and it'll tell you, hey, possible subdomain takeover
[01:02:49.880 --> 01:02:52.960]  at this address. So this is another wide scope tool that I
[01:02:52.960 --> 01:02:55.860]  use in the end part of my recon when I have all these subdomains.
[01:02:55.860 --> 01:02:58.780]  And you can find some volumes just straight off of
[01:02:58.780 --> 01:03:01.960]  using this. Okay, so
[01:03:01.960 --> 01:03:05.000]  we're going to blow through automation real quick. Okay, so when you have
[01:03:05.000 --> 01:03:08.020]  some of the tools and you have a long methodology
[01:03:08.020 --> 01:03:10.900]  like I have, you end up automating it, right? I write a horrible
[01:03:10.900 --> 01:03:13.620]  bash script to do my stuff. It
[01:03:13.620 --> 01:03:16.660]  works for me, but it could be better.
[01:03:16.800 --> 01:03:19.880]  Some of the tools that I use don't do certain things. They're not
[01:03:19.880 --> 01:03:22.860]  threaded. They don't take certain types of
[01:03:22.860 --> 01:03:25.040]  inputs, like list inputs or
[01:03:25.920 --> 01:03:28.860]  glob notation like Nmap does or range notation like
[01:03:28.860 --> 01:03:31.820]  Nmap does. And sometimes I need to feed a tool with another
[01:03:31.820 --> 01:03:34.540]  tool. And so I can't do that.
[01:03:35.040 --> 01:03:37.780]  So Michael Skelton, also known as Kadingo, wrote a tool
[01:03:37.780 --> 01:03:40.920]  called Interlace, which basically wraps around other tools and lets
[01:03:40.920 --> 01:03:43.640]  them do those things. It will thread them. It'll allow you to take different
[01:03:43.640 --> 01:03:46.880]  sources of input. It'll allow you to distribute tools.
[01:03:46.880 --> 01:03:49.920]  Here is an example of a blog written by Hackloop,
[01:03:49.920 --> 01:03:52.740]  which talks about basically threading Nikto, which
[01:03:52.740 --> 01:03:56.120]  Nikto doesn't support inherent threading.
[01:03:56.120 --> 01:03:58.640]  So you can use Interlace
[01:03:58.640 --> 01:04:01.800]  to basically glue together a lot of stuff
[01:04:01.800 --> 01:04:04.480]  that your other tools don't do. Any tool written by
[01:04:04.480 --> 01:04:06.840]  TomNomNom is awesome in automation, right?
[01:04:07.840 --> 01:04:09.940]  HttpProbe, WaybackURLs, Meg.
[01:04:10.560 --> 01:04:14.140]  TomNomNom has several talks out there talking about his tools and how they work.
[01:04:14.260 --> 01:04:16.380]  I use HttpProbe a lot for the glue
[01:04:16.380 --> 01:04:19.600]  between me finding subdomains and you feed
[01:04:19.600 --> 01:04:22.640]  it a lot of subdomains that you have found after your initial analysis.
[01:04:22.640 --> 01:04:25.260]  And then it tells you which ones have actual live listening
[01:04:25.980 --> 01:04:29.180]  web servers associated to them. So that's HttpProbe.
[01:04:29.580 --> 01:04:32.360]  WaybackURLs will find you URLs associated to any...
[01:04:32.980 --> 01:04:34.820]  old URLs associated to any
[01:04:34.820 --> 01:04:38.200]  site that you're currently looking at. Meg is like a directory brute
[01:04:38.200 --> 01:04:41.260]  forcer, but for many hosts. And I use this to find some stuff
[01:04:41.260 --> 01:04:44.360]  too. So all of TomNomNom's tools are just
[01:04:44.360 --> 01:04:47.300]  amazing. Alright, last part. I promise.
[01:04:48.060 --> 01:04:50.960]  Omar, are we okay to finish this last little bit?
[01:04:52.280 --> 01:04:53.160]  Absolutely.
[01:04:53.880 --> 01:04:56.940]  Awesome. Okay, so maybe
[01:04:56.940 --> 01:05:00.120]  recon is not your thing. I definitely run into hackers who are like, this is
[01:05:00.120 --> 01:05:03.000]  the most boring part of assessment to me.
[01:05:03.000 --> 01:05:05.960]  And then I run into other people who are like, yeah, recon
[01:05:05.960 --> 01:05:09.240]  is awesome. It's like my favorite part of the assessment. So it could
[01:05:09.240 --> 01:05:11.780]  be that recon is not really your thing. Finding all these sites
[01:05:11.780 --> 01:05:14.760]  is not your thing. Hacking the sites is more your thing.
[01:05:14.820 --> 01:05:17.960]  And that's cool. There are a lot of new tools out here
[01:05:17.960 --> 01:05:21.100]  these days that basically automate all this
[01:05:21.100 --> 01:05:23.340]  for you. And they're called recon frameworks.
[01:05:23.820 --> 01:05:26.800]  So if you've ever looked at a video game before,
[01:05:26.800 --> 01:05:29.760]  like Diablo, every once in a while,
[01:05:29.760 --> 01:05:33.060]  content creators will make these tiers of builds for Diablo
[01:05:33.060 --> 01:05:36.140]  or something like that. So I've put recon frameworks into a couple
[01:05:36.140 --> 01:05:39.160]  of tiers. C tier, B tier, A tier, and S tier.
[01:05:39.160 --> 01:05:41.960]  C tier are recon frameworks that are built around
[01:05:41.960 --> 01:05:44.460]  scripting up other tools in Bash or Python.
[01:05:44.820 --> 01:05:47.680]  They're step-based. They don't really have a workflow. They only have a few
[01:05:47.680 --> 01:05:50.740]  techniques and they're not really super extensible. But they work
[01:05:50.740 --> 01:05:53.560]  really well. Let me put out a big disclaimer here.
[01:05:53.560 --> 01:05:57.080]  My tools that I use personally on all bug hunts
[01:05:57.080 --> 01:05:59.820]  are C tier tools. And they work
[01:05:59.820 --> 01:06:02.540]  for what I want and they do. And they're great.
[01:06:02.540 --> 01:06:05.440]  So there's nothing bad to say about a C tier tool.
[01:06:05.440 --> 01:06:08.560]  A B tier tool, in my mind, and this is all
[01:06:08.560 --> 01:06:11.800]  very rough, right? These classifications are not
[01:06:11.800 --> 01:06:14.960]  in any way super serious.
[01:06:14.960 --> 01:06:17.420]  But a B tier tool, in my mind, has
[01:06:17.960 --> 01:06:21.120]  some of its own modules it's doing, some of its own sources,
[01:06:21.500 --> 01:06:23.960]  maybe has a GUI, maybe has some
[01:06:23.960 --> 01:06:26.880]  workflow in it where they're bringing data in from the end of the
[01:06:26.880 --> 01:06:30.000]  recon process back to the beginning if they find new stuff.
[01:06:30.000 --> 01:06:32.600]  It has more techniques than a C tier.
[01:06:32.760 --> 01:06:36.020]  But it still runs that point in time and it's still working on flat files
[01:06:36.020 --> 01:06:39.220]  to track the data for recon.
[01:06:39.280 --> 01:06:41.980]  A tier is maybe
[01:06:41.980 --> 01:06:43.660]  writing all of their own modules.
[01:06:43.760 --> 01:06:46.640]  These tools start to have some GUIs, some of them
[01:06:46.640 --> 01:06:49.880]  they run iteratively. So like
[01:06:49.880 --> 01:06:53.180]  Cron or something like that, they'll run at a schedule.
[01:06:53.180 --> 01:06:56.180]  And they start to manage things via database so you can compare
[01:06:56.180 --> 01:06:58.940]  scans and recon scans over time.
[01:06:59.100 --> 01:07:02.980]  And then S tier are kind of the highest level of what's out right now.
[01:07:02.980 --> 01:07:05.300]  They write a lot of their own modules. They have a GUI.
[01:07:05.300 --> 01:07:08.180]  They run iteratively. They manage all their data via database.
[01:07:08.180 --> 01:07:11.480]  They scale across multiple boxes to make the scanning faster.
[01:07:11.480 --> 01:07:14.100]  They send alerts back to the user via email and text
[01:07:14.100 --> 01:07:17.360]  and Slack and whatever when they find
[01:07:17.360 --> 01:07:20.320]  new things. They have some novel techniques that not a lot of other people
[01:07:20.320 --> 01:07:23.200]  are doing. So these are how I classify some of the frameworks. I'm going to show
[01:07:23.200 --> 01:07:26.180]  some of them to you now and you can pick one that works for you if you're not
[01:07:26.180 --> 01:07:28.900]  into this recon stuff. Warning.
[01:07:29.000 --> 01:07:31.920]  I had to put this in here. I don't want to hurt anyone's feelings.
[01:07:31.920 --> 01:07:35.120]  I am scaling some tools and giving them grades. It's not
[01:07:35.120 --> 01:07:37.940]  serious. It's my rough experience. My gut feel.
[01:07:38.380 --> 01:07:41.100]  All these tools are wonderful and I respect the authors of them
[01:07:41.100 --> 01:07:43.560]  so darn much of just putting code out there.
[01:07:43.560 --> 01:07:47.340]  I can't say enough about the people that open source their code
[01:07:47.340 --> 01:07:49.940]  and even C tier. Like I said, my tool is a C tier
[01:07:49.940 --> 01:07:53.000]  tool and it works for me. So this is just subjective
[01:07:53.000 --> 01:07:55.920]  and based off my own experience. Okay. So
[01:07:55.920 --> 01:07:59.060]  some C tier tools that you could look into that are wrapping around
[01:07:59.380 --> 01:08:02.140]  a lot of other existing tools.
[01:08:02.140 --> 01:08:05.260]  One of the ones called ultimate recon here
[01:08:05.260 --> 01:08:08.220]  I like is listed here. And the reason I like
[01:08:08.220 --> 01:08:10.800]  ultimate recon is because it's using the new nuclei
[01:08:10.800 --> 01:08:13.340]  scanner that's out by project discovery and it
[01:08:14.360 --> 01:08:17.080]  implements finding subdomains using a couple
[01:08:17.080 --> 01:08:19.980]  different tools. It will then take all of those. It will
[01:08:19.980 --> 01:08:23.140]  port scan them and then it will run nuclei templates on
[01:08:23.140 --> 01:08:26.660]  them. So it's pretty good. I like it.
[01:08:26.660 --> 01:08:28.540]  All of these different C tier tools
[01:08:29.020 --> 01:08:31.840]  they're all choosing their own tools to wrap around
[01:08:31.840 --> 01:08:34.540]  that they like. So there's no best one really. But
[01:08:35.340 --> 01:08:37.680]  I tend to like ultimate recon in this
[01:08:37.680 --> 01:08:41.280]  one. There's also other good ones in here too.
[01:08:41.280 --> 01:08:43.900]  So check these out. B tier
[01:08:43.900 --> 01:08:47.440]  frameworks. This is lazy recon by Captain Milo.
[01:08:47.440 --> 01:08:49.940]  And here you can see his workflow. Very
[01:08:49.940 --> 01:08:52.800]  similar to what we talked about in this presentation. Amass and sub finder
[01:08:52.800 --> 01:08:56.040]  combine those two. You get a final subdomain count.
[01:08:56.040 --> 01:08:58.680]  Then you start to do some vulnerability analysis with
[01:08:58.680 --> 01:09:01.760]  subjack and core scanner to find some vulnerabilities related
[01:09:01.760 --> 01:09:05.020]  around subdomain takeover and cores vulnerabilities.
[01:09:05.020 --> 01:09:08.080]  Then you start brute forcing. Then
[01:09:08.080 --> 01:09:10.420]  you start port scanning. Then you do some
[01:09:10.420 --> 01:09:12.860]  screenshots. And then you have a final
[01:09:13.820 --> 01:09:16.780]  output. So a lot of stuff we talked about today is
[01:09:16.780 --> 01:09:19.680]  implemented inside of this workflow. So lazy recon is what I like to
[01:09:19.680 --> 01:09:22.700]  point out inside of the B tier here.
[01:09:24.180 --> 01:09:26.080]  A tier, there's a couple.
[01:09:26.200 --> 01:09:28.300]  They all teeter on being S tier, right?
[01:09:28.700 --> 01:09:31.560]  I wonder if I should even really have
[01:09:31.560 --> 01:09:34.880]  like an S tier and an A tier. Usually
[01:09:34.880 --> 01:09:37.860]  the thing that's separating some of these is GUI from the other ones.
[01:09:37.860 --> 01:09:40.540]  So the one I have referenced here on this slide is called find domain.
[01:09:40.540 --> 01:09:43.620]  And find domain does pretty much everything. It does subdomain
[01:09:43.620 --> 01:09:46.540]  discovery and scraping and brute force and all of that
[01:09:47.220 --> 01:09:49.780]  via amass and sub finder and asset finder
[01:09:49.780 --> 01:09:52.920]  and everything. And then it stores it in a database.
[01:09:52.920 --> 01:09:55.800]  It does iterative scanning. It'll text you. If it finds new
[01:09:55.800 --> 01:09:58.900]  domains from its last scan, it'll email you, which you see in the bottom left
[01:09:58.900 --> 01:10:01.900]  hand corner, which is great. The only thing it lacks
[01:10:01.900 --> 01:10:04.760]  is a GUI. It's a command line based tool. So
[01:10:04.760 --> 01:10:07.900]  it kind of teeters on doing a lot of the stuff we want to do.
[01:10:08.040 --> 01:10:10.980]  And in the enterprise version, it does port scanning
[01:10:10.980 --> 01:10:13.980]  and screenshots. And that
[01:10:13.980 --> 01:10:17.040]  version is a for pay version, but it's affordable to bounty
[01:10:17.040 --> 01:10:20.160]  hunters. Some of those are really good subscriptions to pay for.
[01:10:20.160 --> 01:10:21.880]  So find domain, pretty cool.
[01:10:22.960 --> 01:10:25.760]  Here's our S tier framework. Some of them you'll never get to use
[01:10:25.760 --> 01:10:28.800]  because they're commercial products, but I include them here to give people
[01:10:28.800 --> 01:10:32.040]  ideas as to what they should be working towards in their frameworks.
[01:10:32.220 --> 01:10:35.080]  Intrigue.io is an asset discovery tool
[01:10:35.080 --> 01:10:37.540]  written by Jonathan Cran, who's a friend of mine.
[01:10:38.800 --> 01:10:41.280]  He builds Intrigue and he has a
[01:10:41.280 --> 01:10:44.080]  GUI for it and it's a SaaS platform and businesses buy it
[01:10:44.080 --> 01:10:47.160]  to map their attack surface. But it's doing the same techniques that we've talked
[01:10:47.160 --> 01:10:49.920]  about today. The cool thing about Intrigue
[01:10:49.920 --> 01:10:52.940]  is even if you don't pay for the subscription of
[01:10:53.720 --> 01:10:56.180]  the SaaS version that he's selling to businesses,
[01:10:56.180 --> 01:10:59.180]  he open sources almost all of the code for Intrigue.io
[01:10:59.180 --> 01:11:02.180]  so you can run your own hosted version or at least
[01:11:02.180 --> 01:11:05.760]  see what he's doing and his kind of analysis. So Intrigue is pretty cool.
[01:11:05.840 --> 01:11:08.140]  Asset note is another one that's
[01:11:08.540 --> 01:11:11.400]  a B2B type play for recon.
[01:11:11.400 --> 01:11:14.260]  It's a team of former bug hunters who made this awesome platform.
[01:11:14.260 --> 01:11:16.620]  This is probably the gold standard of
[01:11:18.600 --> 01:11:20.420]  asset management or attack surface
[01:11:20.420 --> 01:11:23.160]  management tools. You can see that it's pretty.
[01:11:23.160 --> 01:11:26.920]  It breaks things up by assets that need attention.
[01:11:26.960 --> 01:11:29.340]  It's got a lot of graphing. It does
[01:11:29.640 --> 01:11:32.180]  a lot of custom vulnerability checking.
[01:11:32.420 --> 01:11:35.340]  So asset note is probably, it's S tier, but
[01:11:35.340 --> 01:11:38.440]  it's a high price tag so you're probably not going to use it as a bug hunter
[01:11:38.440 --> 01:11:41.220]  or a red teamer. But it is kind of the gold standard that a lot of the tool
[01:11:41.220 --> 01:11:43.780]  makers should probably work towards.
[01:11:45.000 --> 01:11:47.440]  Spiderfoot is another one that does
[01:11:47.560 --> 01:11:50.900]  a lot of OSINT type domain correlation information.
[01:11:50.900 --> 01:11:52.560]  It does some of the techniques we've talked about
[01:11:53.340 --> 01:11:56.340]  in this presentation. They're adding more all the time. I know the author
[01:11:56.340 --> 01:11:59.540]  is really strictly interested
[01:11:59.540 --> 01:12:02.580]  in adding more bug bounty focused features
[01:12:02.580 --> 01:12:05.540]  in it. And it's got a great graphing library and monitoring
[01:12:05.540 --> 01:12:08.160]  and everything. So Spiderfoot's pretty cool.
[01:12:08.320 --> 01:12:11.580]  The unreleased project discovery, we've talked about project discovery a couple times
[01:12:11.580 --> 01:12:14.560]  in the presentation. That team, their unreleased framework looks
[01:12:14.560 --> 01:12:17.580]  to be a killer. Looks to be awesome. I'm really excited about it
[01:12:17.580 --> 01:12:20.660]  when it comes out. So yeah, you can see here
[01:12:20.840 --> 01:12:23.580]  a dashboard of a project, all your discovered domains
[01:12:23.580 --> 01:12:26.540]  and seeds, what the scan
[01:12:26.540 --> 01:12:30.100]  process is. And then if you dig into any of those, you get screenshots,
[01:12:30.100 --> 01:12:32.680]  port scans, technology identification of the
[01:12:32.680 --> 01:12:36.000]  site, its activity, its changes over time.
[01:12:36.000 --> 01:12:38.640]  So the discovery framework when it comes out will
[01:12:38.640 --> 01:12:41.840]  be pretty cool. Jails is a vulnerability
[01:12:41.840 --> 01:12:44.680]  scanner written by Jesse, JJJ and team.
[01:12:44.760 --> 01:12:47.680]  It is also really good. It is a
[01:12:48.240 --> 01:12:51.120]  GUI flow that wraps around some command line stuff.
[01:12:51.120 --> 01:12:53.940]  It does vulnerability scanning, which, you know,
[01:12:53.940 --> 01:12:57.840]  if you're working with Nessus or, you know, not a lot of red teams are
[01:12:57.840 --> 01:12:59.620]  scanning with Nessus. But if you want to look for some
[01:12:59.620 --> 01:13:02.520]  very pointed stuff, you can use Jails and look at their
[01:13:02.520 --> 01:13:06.060]  kind of CVE checks that they use.
[01:13:06.060 --> 01:13:08.400]  And you can see signatures there for
[01:13:08.400 --> 01:13:11.760]  the Zoho management page and fuel CMS RCE.
[01:13:11.760 --> 01:13:14.600]  So they're looking for some cool volume Jails.
[01:13:14.600 --> 01:13:17.600]  I really like it. Osmedias is
[01:13:17.600 --> 01:13:20.560]  awesome as well. It's very similar to some of the previous
[01:13:20.560 --> 01:13:23.380]  pages we've seen. It'll take all of your
[01:13:23.380 --> 01:13:27.200]  domains. It will do... it'll take all your subdomains.
[01:13:27.200 --> 01:13:29.940]  It will resolve them to IP. It will do port scanning on them.
[01:13:29.940 --> 01:13:31.760]  It'll tell you the technologies.
[01:13:32.720 --> 01:13:35.580]  It will do screenshots. It'll do all kinds of stuff.
[01:13:35.580 --> 01:13:39.320]  So Osmedias is also pretty easy to stand up and really good.
[01:13:39.360 --> 01:13:41.960]  Huntersuite.io, also same kind of idea.
[01:13:41.960 --> 01:13:45.080]  You can see technology parsing, services,
[01:13:46.080 --> 01:13:47.880]  domain discovery, etc.
[01:13:49.100 --> 01:13:51.220]  Bounty.offensive.ai,
[01:13:51.220 --> 01:13:54.180]  same type of deal, right? Includes some vulnerability scanning,
[01:13:54.180 --> 01:13:57.960]  technology fingerprinting, subdomain finding,
[01:13:57.960 --> 01:13:59.800]  and then attack graphing.
[01:14:00.060 --> 01:14:03.640]  Reengine is the one I used last week or a week before.
[01:14:03.640 --> 01:14:06.200]  Reengine is pretty sick.
[01:14:06.260 --> 01:14:08.800]  You can have projects in it that are separate.
[01:14:08.800 --> 01:14:12.060]  It'll do the subdomain scanning. It basically takes the methodology
[01:14:12.060 --> 01:14:15.220]  that I outlined in this presentation and turns it into
[01:14:15.400 --> 01:14:18.040]  a tool, a hosted tool.
[01:14:18.400 --> 01:14:21.160]  And they're making some improvements every
[01:14:21.160 --> 01:14:24.420]  week. So lots of potential on
[01:14:24.420 --> 01:14:28.020]  Reengine, which I highly
[01:14:28.020 --> 01:14:29.560]  suggest checking out.
[01:14:30.240 --> 01:14:33.280]  And then Scout PDP on the SecApps team. They've been working
[01:14:33.280 --> 01:14:35.720]  on tools for well over a decade.
[01:14:35.820 --> 01:14:39.160]  And they have built out Scout, which is a paid offering,
[01:14:39.160 --> 01:14:42.300]  but it's affordable enough for bounty hunters. Same idea, right?
[01:14:42.400 --> 01:14:45.140]  Projects, subdomain enumeration, related
[01:14:45.140 --> 01:14:48.300]  domains, reverse DNS. It supports screenshotting.
[01:14:48.300 --> 01:14:49.120]  It's awesome.
[01:14:50.540 --> 01:14:51.960]  Awesome tool.
[01:14:52.280 --> 01:14:56.000]  And then lastly, Nuclei.
[01:14:56.000 --> 01:14:59.080]  I talked about it very briefly, but anyone who's not using
[01:14:59.080 --> 01:15:02.420]  Nuclei in their bounty scanning right now is at a disadvantage.
[01:15:02.440 --> 01:15:05.060]  They're coming out with some epic templates for CVE
[01:15:05.840 --> 01:15:06.800]  identification.
[01:15:08.080 --> 01:15:11.660]  And you can basically build your own via a YAML file.
[01:15:11.660 --> 01:15:14.080]  And it's a scanner that'll go out and scan all your domains for
[01:15:14.680 --> 01:15:17.920]  vulnerabilities, subdomain takeovers, all kinds of stuff.
[01:15:17.920 --> 01:15:21.400]  So I really have been using Nuclei to great success lately.
[01:15:22.320 --> 01:15:23.280]  And that's it.
[01:15:23.280 --> 01:15:25.700]  That's all I got. So that's the Bug Hunters methodology.
[01:15:25.960 --> 01:15:29.680]  Thanks for giving me the time today. I really appreciate it. Sorry I went over a little bit.
[01:15:29.680 --> 01:15:32.440]  No worries. Thank you so much for the presentation
[01:15:32.440 --> 01:15:34.120]  and amazing support.
[01:15:34.780 --> 01:15:37.840]  Once again, from the bottom of my heart, thank you for supporting
[01:15:37.840 --> 01:15:40.120]  DEF CON and the DEF CON Red Team Village.
[01:15:40.120 --> 01:15:43.480]  And a reminder for everyone here,
[01:15:43.480 --> 01:15:46.360]  please take a look at all the talks and activities that we have
[01:15:46.360 --> 01:15:49.460]  in our website. We have the CTF, of course,
[01:15:49.460 --> 01:15:52.400]  the Cyber Raid contest as well.
[01:15:52.680 --> 01:15:55.440]  Tons and tons of activities throughout this weekend.
[01:15:55.700 --> 01:15:58.040]  The link should be in the description at
[01:15:58.040 --> 01:15:59.980]  the bottom of your stream. And all that stuff.