Jason Haddix and this is domain discovery expanding your scope
like a boss. So we did some strike through here because it's
not just domain discovery. I added a little bit of web
discovery here. This talk is primarily focused around my
discovery methodology. I by no means think I have the best
discovery methodology but I think it gets me a lot of sites
to hunt and bug bounties and pen tests when I'm going after a
certain entity or business or something like that. So that's
my twitter if you wanna yell at me twitter. I work for bug crowd
who does bug bounties so we manage enterprise bug bounties.
On the global leader board of 600 thousand register testers on
59th currently on bug crowd. And I play a lot of video games.
That's me and my son. And we're both wearing matching dinosaur
shirts if you can see that. Okay so this is my methodology in
a nut shell. If you guys might have seen this a couple weeks
ago. I did release a.
version of this but I've added more since I did that version.
So my version of a methodology to do discovery for hacking
websites, this isn't an OSINT talk where I'm going to go into
hacking people and individual information. This is a
methodology to find more web assets for me to go after in a
bug bounty or pen test. Usually when you find undiscovered
assets, web assets, they are less secured than the main
domain. So in bug bounty world that means that they will be
less secured and I can find better, more critical bounties
on these sites. So we start with identifying our targets, their
main TLDs. We move on to domain scraping for those discovered
TLDs. We go into domain brute forcing. Then we go into
permutation scanning and port scanning. We do some visual
identification. We do some auxiliary stuff. And then we
go into the sites that we have now discovered and we start
doing platform identification, content discovery and parameter
discovery. So this is kind of how it works. All right. So the
first thing you do when you start trying to find or
discover, you know, assets or IPs or anything associated to a
domain, and it's hard to see this, but you actually have to
look up their ASN, right? The autonomous system number for a
company, right? So this is their kind of registered IP space.
They're all assigned to a specific domain. They're all
assigned an ASN number. Here I've looked up Tesla. So Tesla has a
couple different blocks here. So the site to use this, the site to
do this that I use to do this is Hurricane Electric. It's one of
the only free sites that will let you pull down the large set
of ASN data. Some of them will let you pull it down, but only
will limit you at like 500 requests or 500 lines of the
output. So here I can see that Tesla Motors has a registered
net block of 209.133.79.0. So that's where we're going to
go slash 24. So this will be one way I start gathering or
basically searching for their IP space so I can start looking for
assets. This is just one out of all. Usually when I start,
though, I'm starting from a bug bounty. So I already have a
couple targets to start with, usually their main domain. So for
Tesla, we're using them as an example because they have an open
bug bounty and it's pretty permissive. They say anything
that we own is in scope of the bug bounty, including the cars
and the web assets and stuff like that. So I would already probably
have tesla.com or teslamotors.com in scope and then I
would look up their ASN. So then you want to do reverse who
is. So manually you can use these two sites. There's also
some tools that automate this. So one of the frameworks I'm
going to use a lot, I'm going to rely a lot on in some of the
next slides is called Recon NG and they have a module for
reverse who is. But if you want to do it manually, these are the
two, again, the two sites that you want to use because they
don't cap you at the amount of stuff that they pull back. So if
you want to do it manually, again, all these sites that do
this online kind of reverse DNS information or give you
information about domains and stuff like that, they want to
charge you at some point for some of this info. And as a pen
tester, I'm not going to pay, you know, $70 every lookup I
need to do. So reverse DNS or reverse who is, excuse me, will
find some more scope targets for this type of stuff. The last one
in this area is acquisitions. So this is actually the end all be
all site for the end all be all site for the end all be all site
for acquisitions. Crunch base marks all this stuff when a
company acquires a new company. They mark it all in their
acquisitions table. So you can just go up to the URL at crunch
base. It's crunch base.com, organization, then your
organization name. And you can see here I have a list or a
history of everything that Tesla has acquired in the last ten
years. So depending on if your pen test is really open scope or
your bug bounty is really open scope, these domains are now top
level domains that I have to put back into the beginning of my
discovery. So Grumman Engineering, Solar City, Riviera
Tool. I'm sure if I was doing this, I would also want to check
out SpaceX just because I know that they're related to Tesla.
So you also want to add these top level domains to your
testing. This should be automated. Like, some of these
sites should be automated, but these sites really want you to
pay for this data. So they're all protected by different bot
protection tools. So this one's protected by Distill, which is
actually pretty good. Using a web browser, you can actually get a
web scraper. Doesn't work like all the normal web scraping
libraries that you would because otherwise I would automate this
as part of, like, a tool or something like that. Okay. So
that is gathering from, like, OSINT sites. What I want to do
now in this section of the methodology is move on to
finding subdomains. So I found a whole bunch of top level
domains. I found IP addresses that they've used. Now what I
want to do is find probably subdomains. And there's a
couple different ways to do this, but they involve scraping.
Scraping more of these open source sites. So what do we do in
this case? Well, basically, when you're doing subdomain scraping
and you're trying to find more subdomains of a top level
target, a TLD, you want to scrape search engines, right? So
the idea here is you would go to Google.com and search site
colon Tesla.com. And you would get a whole bunch of results that
come back. And that says Tesla.com, right? And then also a
search result that comes back is admin.Tesla.com, forum.Tesla.com.
And iteratively, you would get a whole bunch of results that come
back. And iteratively, you make more Google searches, removing
those ones that you found from the list until you have no more
results and you have a full cache of everything Google's ever seen
for Tesla.com, all the subdomains. So that's manually
is kind of a hard process. So you use tooling to do this. The
other place that you can look for subdomain data is inside of
certificate transparency projects. So cert.sh, Google
Certificate Transparency Project, and their OpenSSL subject
alternative name space.
Now, these are kind of individual talks. You can go
into, like, scraping subdomains out of certificates, or you can
go into search engine scraping. I think there's already been a
couple talks on this. So we're just going to go into the
automation of these and what I use to make this quick. So
there's two tools for doing all this subdomain scraping and
finding targets. One is Recon NG, which I have created a
script around called Enum All that wraps around it. It's a
Python wrapper. And there's another one called Sublister.
And they're both really good. They both have a lot of
resources. They both do the same thing. These are the sources
they pull from. It's kind of hard to see. But on the right,
Sublister does or in the middle, they both do Google. They both
scrape Google, like I just described. They do the same
thing on Bing. They look in the cert.sh project, which is the
SSL certificates directory. They look on the Threat Crowd API and
the Netcraft API. Now, individually, they also do some
different sources. Sublister handles Baidu, Ask, DNS Dumpster,
which is the scans.io project, VirusTotal, and PT Archive. In
some way or another, these sites all have a website that's
aggregating domain information somehow. And these tools will
scrape that out of there for your target. So they will find
Tesla subdomains for me. Recon NG individually will do SSL
Tools API, Hacker Target API, and Shodan, which is a pretty
popular one. And then they have a whole bunch of optional modules
that you can bake into Recon NG. So disparate tools, but the two
best tools. So if I want to scrape all of these, I really
want to have to use both.
So this is a tool called Sublister. And so this is the one
I showed you, Shublister. And so this is a run on the right. It's
hard to see. But basically, I've set it to go after Tesla.com. It
runs. It searches Baidu, Yahoo, Google Bing, Netcraft, DNS
Dumpster, VirusTotal, Threat Cloud, SSL certificates, and
passes DNS.com. And then it just gives me an output of all of the
domains. And you can kind of see them. You can see Tesla.com,
Auth.Tesla.com, Autodiscover, Dev.Tesla.com, Fortis.com. And
forums.Tesla.com, et cetera. So I've significantly expanded my
scope now from just Tesla.com to go after.
So the other tool was enumall that I wrote. And I don't really
feel like running them all independently. I want some form
of automation around this. So this is a project called Brute
Subs, which basically can take any Recon tool written in any
language if you make a Docker file for it and spin it up and
run it and give you the output from such tools.
So Brute Subs specifically takes my tool, enumall, and
sublister, and a couple other tools we're talking about, and
we'll run them. There's some configuration required. You have
to set up the Docker image with some of the modules that aren't
included by default, make a custom environment file. But once
you do that, all you do is Docker up, and it will run all of
this domain scraping for you and just give you an output text
file from three or four different tools that we're going
to talk about in the next couple seconds. So this is the
run on the left. You can see it does a little bit of everything.
It does the scraping. One of the tools is actually a brute
forcer. We're not going to use it because we're going to use
something else for brute forcing. All right. Some other
subdomain finding tools that are scraping data, the one that I
find really cool here is Cloudflare. So Cloudflare,
basically, when you log into the Cloudflare site and you go to add
Cloudflare to a domain, you know, potentially that you own,
you type in here, Disney.com, and it's going to give you a
link to Disney.com, right? And it will tell you if that's
already used, basically, if Cloudflare is already running on
that domain. So this isn't my screenshot, by the way. This is
a tool author screenshot. So by iteratively putting in names into
that search field on Cloudflare once you're logged in, you can
verify or not verify that they have a subdomain in existence
with Cloudflare. Cloudflare runs on 15% of the Internet right
now. So you can get a massive amount of, basically, data that
basically OSINT information from this kind of API or web portal
that they have. So here, it's run against Disney. You can see
that it's returned some success querying the DNS archives of
Cloudflare. There's another one that's individually test
census.io, and that's another project for aggregation of web
data. So these are bespoke tools that don't really fit into
anything, but I really like this Cloudflare one. Okay. So we
scraped a whole bunch of stuff off the Internet, searched a lot
of search engines, a whole bunch of open source sites. Now we
have to move on to basically guessing at our target, right? So
I might have some search engine results, but now I want to do
this idea of brute forcing the subdomain. So the classic
example of this is that you have tesla.com, and then you try to
resolve admin.tesla.com. And if it resolves and you go somewhere,
then that site exists. If it doesn't resolve, it doesn't
exist. And you iterate over this in a tool. And then you try to
solve it. Now, over the years of pen testing, we've done
multiple tools that have come out for this. Fierce was probably
the first and best one. New school ones are like SubBrute,
Black Sheep Wall, DNS Parallel Brute Forcer. Now, the problem
with these is that they take a long time with the big list that
you're trying to brute force, right? And different projects
have come out with different lists as well, and some of those
lists are really long. Like this one is a million lines long. And
to do that normally, or at least in my experience, took a lot of
time and effort to do that. So I'm going to go ahead and show you
this. So I'm going to go ahead and show you this. So I'm going to
go ahead and show you this. So I'm going to go ahead and show you
a few more. Now we have past results. You can see that it
took days to weeks to run through a brute force of that
kind of stuff. Now, this is actually some research that I
did. I basically benchmarked all these tools. So you can see
that the two best ones here are two tools called GoBuster and
Mast DNS. GoBuster is written in Go, obviously. It does
subdomain brute forcing. With this list, this one million line
subdomain brute forcing list it completed that whole thing in 21
minutes. Mast DNS finished it in a minute, 24 seconds. Now, the
reason Mast DNS is subdomain brute forcing is, well, m σ is U,
fast is because it's written in C and instead of using just your
DNS infrastructure that you're connected to, it has a list of
DNS servers that it cycles through in parallel to resolve
the sites. It gives you more false positives but it runs
very, very quickly. So these are the two tools that I integrate
into my methodology to do DNS brute forcing or subdomain brute
forcing. Yeah? Yeah. So I'm going to go back to the
No. No. Well, SubBrute obviously
aired out. But DNS Parallel Prober and Black Sheep Wall
returned 61 and 43 respectively but they were all included in
the data sets of GoBuster and MassDNS. Yeah. Okay. So that
file that I said was a million lines that you're doing for
subdomain brute forcing, that is a file that I'm going to
make. It's a file that is made out of basically every file that
I could find that has ever done this type of work, subdomain
brute forcing. So the fierce list is in here, DNS scan, the deep
magic top 500 prefixes for subdomains. There was some
research earlier this year by a guy named BitQuark who did
scraping on the web of the top million most popular subdomains.
Remember, we're brute forcing just names to see if they
resolve. So we want to have a good list to brute force from. So
this is everything that's ever existed. A lot of the stuff that
I could find. So it's all catted into one file for you to use at
that gist. You can just grab it. And the idea here is because
MassDNS is so fast that why not just use a huge list? It
doesn't matter, right? It gives us better coverage. So MassDNS
can complete this in a minute 30. So the yeah, there's even
some extra stuff in here that were not subdomain lists. So
like the raft list. And these were the subdomain lists that
were URL brute forcing. So using words to find URL paths. And I
found that to be sometimes useful in fuzzing subdomains as
well or brooding subdomains. So those are incorporated into this
list as well. Daniel's project, robots disallowed, I ported to
a DNS structure to be in here as well. Okay. So I've done a ton
of brute forcing and I've found some stuff that resolves. I've
scraped a whole bunch of stuff off the internet and I've found a
whole bunch of stuff that might exist because it was off search
engines or in all these.
Open source caches. Now what else do I do? Well, a quick part
of the methodology is trying to find permutations on the stuff
that I've found already. So the idea is that, you know, I may
have brute force for admin.Tesla.com, but a lot of
people use subdomain nomenclature that's like ACS.admin.Tesla.com,
right? So what AltDNS does is it takes keywords that it's seen
before, permutations, and throws them into your results of
other tools. So you take the results of other tools. You
go ahead and run it against the domain and it will add these
permutations all over and then find additional subdomains
against stuff that you've already found because they use
these naming, these weird naming structures in a lot of
places. The other one, this is kind of interesting. I've only
used this once in success, but I really like the idea behind this
is it's called SDBF. It's called Smart DNS Brute Forcer. The
idea here is to use Markov chains and some statistics and an
Ngram model to basically generate these permutations of
domains. I actually don't know what all that means. I've just
used this tool and it's kind of cool. So, yeah, so the white
paper, this was an academic research project. He's published
the code for this. I'd say the variance between this and AltDNS
is actually very low, maybe like 10% more, I found, with this
than AltDNS, but I've only run it a couple times on a couple
of projects. But I like the idea and it has fancy names. Yes,
with 100% coverage so far, yeah. Okay. So, okay, so I've done
acquisitions. I've done their ASN. I've done subdomain
scraping from all kinds of public resources. I've done
permutation scanning to find bespoke, like, weirdly named
subdomains. Now I need to move on to doing some sort of actual
port scanning, right? So I have a whole bunch of targets, right?
There's really no other solution better than mass scan. It's just
what you do, right? Nmap will take forever with a large ASN,
not to mention the amount of time it would take to add on all
those domains I found. So you just can't use it. It's going to
take forever. So for large targets ASN, like 65,000 hosts
live or something like that, the run from Nmap is kind of
infinite and you fall asleep. For mass scan, you can do a
scan like this in 11 minutes. And so the problem with mass
scan is that by default it doesn't have a default ports
list. It's kind of meant for projects where you scan the
whole Internet for one port. So you'd have to specify in the
command line all the ports you want. These are all the ports
that are the default Nmap ports here. So you can use this
command line syntax and just plug it in and go. Or you can
use a config file and put all these ports in your config file
for mass scan and it will go out and search. It will go out and
port scan all this stuff. So it's written in C. It's
distributed really fast. And you don't have to worry about
any of the functions. So that's why it can do what Nmap can't.
It doesn't have any of the functions that Nmap has for
server versioning, for Nmap scripting. It doesn't have any
of that. It's just a port scanner to tell me what's up and
it's really fast. Only TCP. The output of the tool? I think
it's just a list. Yeah, a list in the command line, yeah.
Oh, actually, no. It supports OG, which is the Nmap syntax for
XML, comma separated value, and something else, too. Right? List
as well. Greppable. Greppable. There we go. Greppable. Yeah.
Cool. This tool will melt your boxes. So make sure to do it on
DigitalOcean and not your home network. It will DOS your
router. Okay. So I have port scanned a lot of stuff. I have
scraped a lot of stuff. I've done a lot of stuff. I've done a
lot of stuff. I have a lot of sources that I'm building from.
But in the first part of the methodology, we were scraping
public sources, and those things might not be on the Internet
anymore. Like, they might have been taken off. They might
actually have a registration, but they're redirecting to the
home page of the site I'm after. Right? So a lot of the, when I
did this run, when I did Tesla, a lot of the sites redirected
back to just teslamotors.com, because that's their main
domain. Right? They had registered something, either so
that some, you know, nerd wouldn't hijack a domain that
sounds like them, or they had maybe planned to use a site
eventually but never did or something like that. So there's a
lot of registration and redirects and stuff like that. So
you need to do some visual identification. But I'm not
going to go in my browser and open up, like, 400 tabs of sites.
That's inefficient. So what you do is you use a tool called
eyewitness. There's a couple tools that do this, but I like
eyewitness the best. What they'll do is they'll take your
list of output. And since I've been scraping from search
engines, sometimes I don't get the protocol that I'm supposed
to visit for that site. So it might be HTTPS, it might be
HTTP, depending on the source where I'm scraping from. Now,
what eyewitness will do is it'll try to visit a subdomain
or a domain in both HTTP and HTS, and it'll take a
screenshot, and it'll dump it in a folder. And then it will sort
them by the length of that screenshot. They'll also pull
header information and content type and sort them into folders
for that as well now, too. So what I do is I run this on the
output of everything I've already done, and I start
visually inspecting.
And so you can see some thumbnails there for kind of what the
screenshots look like. But eventually, what you'll see in a
large engagement like this against an OpenScope target is
like an employee login, and that's where kind of I would
want to start, or a partner login, or maybe an old
marketing page that has, you know, long since been forgotten,
or a page that looks like it's coded in a language that you
could probably kick and it would fall over. So that's kind of
where you would want to start.
And this is a run of eyewitness on the left. The added benefit
of eyewitness is it has a library to take screenshots for a
couple other protocols, too, not just HTTP and HPS. It'll also
take screenshots of RDP and VNC. So if these sites have those
ports open, it will take a screenshot of the RDP login. So
I've actually scored pretty good money on sites that have left
RDP exposed to the Internet with admins still logged in, so I had
their usernames, and all I need to do is brute force their
passwords to get into their RDP instance.
Male Speaker 2 in audience RDP and VNC.
Male Speaker 2 in audience RDP and VNC.
Male Speaker 2 in audience RDP and VNC.
Male Speaker 2 in audience RDP and VNC.
Male Speaker 2 in audience RDP and VNC.
Male Speaker 2 in audience RDP and VNC.
Not in my methodology. Usually I'm going after
web targets, but the mass scan output will give you all those
ports. So if I want to, I can go back later
and look at SSH, FTP, whatever protocol I want. I would probably
look at public-facing databases. Like, nobody should have that,
right? You want to brute force those. I don't know. Other
interesting ports, so. Yeah.
Cool. Some auxiliary stuff for enumeration here is if people have DNSSEC enabled, you can use a couple of DNSSEC tools in the LDNS utils package to walk the relationship of DNSSEC registered domains.
And this has to do with a presentation that was at a conference that I held called Level Up at BugCrowd.
We invited a whole bunch of our bug hunters to come and talk about their techniques.
He did a whole talk on this idea of LDNS walking and NSEC walking and then even NSEC 3 walking, which are all technologies related to DNSSEC.
And so the presentation there taught me a lot about these couple methods.
There was another one for basically using GitHub to find secret keys, internal credentials, API endpoints, and domain patterns.
Inside of GitHub, so just searching your target inside of GitHub really is all it is.
You go to the search box, you say tesla.com, and now any partner that's ever integrated with them that has open source code, you get their domains.
They might have left API keys and stuff in there like that.
So there's also just like the simple fact that people also don't actually just visit the site and see where it takes you.
So this isn't passive, but this is active.
So one method is to use just one method.
It's burp suite and scope filters to find additional domain coverage.
So I'm going to show that in a second.
But basically you load up your main target through an interception proxy burp suite for web testers.
You visit it, and then the JavaScript will start firing.
You walk it a little bit, and you'll start to, you know, your site tree on the left will start to build up.
And then you can just nuke your scope down to a keyword of like tesla.com and start looking at additional sites that they were linking to.
And then you can also do a ton of Google dorking for things.
Like people's ads keys for Google.
So if a company registers an ad key for Google, they're going to use it globally across a lot of domains.
So if I want to find all of their sites, I might want to Google for their ads key.
Also, companies across all of their sites use the same privacy policy in terms of service.
So if I can find that, I can Google for that string and find domains using that.
And then looking for people's S3 buckets and AWS buckets.
That's in a presentation.
This is a presentation by Ben Sekhdaipour, I think is how you say his last name.
He was an intern of mine at Bug Crowd.
He works at HackerOne now.
And he did a presentation on that, which is linked on the right as well.
Those are methods that take kind of a while and could have their own talk.
So I don't go into all of them right here.
So this is just like a simple idea of what you do with Burp.
So I'm going to kind of try to do a live demo.
I'm going to fire up Burp Suite.
So this is using Burp to kind of find all of the domains you can that are linked on one site.
So I'll start at Burp.
Any of you in here use Burp?
Yeah?
It just sits between your traffic and the server and it shows you everything that goes by.
It has a whole bunch of helper tools to manipulate that web traffic.
They have a scanner in it.
They do.
They have a web scanner in it.
But today we're not going to focus on any of that.
Okay.
So I've opened up Burp Suite.
And I have a Chrome tab here that's directed to pass through Burp Suite.
So if I refresh tesla.com, you'll see my proxy tab lights up.
I'll let this through.
So all the traffic is now flowing through Burp Suite to the Internet.
You can see on the left-hand side a whole bunch of stuff has how many domains have traveled through.
Now, I want to make sure that I completely spider Tesla so I can find every related domain.
So here's tesla.com down here.
I can right-click here and spider this host.
It'll say, would you like to modify the scope to include these items?
And you want to answer yes here.
And I'll say yes here.
All right.
So this is going to spider tesla.com with the web spider.
You can see it's transferring bytes.
It's making requests.
It's got some forms queued.
I'm going to stop it here because this could take a long time.
And now I've got a whole bunch of stuff in this site tree on the side, right?
And this will expand out pretty large if I continue that spider to its finish.
Not all these are Tesla sites, right?
These are all links that were on their sites.
But an easy way to just find more scope for your target is just to go to your scope tab here.
And you can add a new rule here in your scope and just add a keyword under your host here.
As Tesla.
And then click here on the ribbon and say show only in scope sites.
Now these are all Tesla sites that were linked from their main domain.
Now what I'd want to do is select all of these.
Spider all of these recursively.
Find out what they're linking to.
And use the same filter to try to find more domains.
So recursively I could do this and probably find a ton of their stuff that they're linking to on their sites.
Um.
Any questions there?
Cool.
All right.
That was the only one I didn't have an animated GIF for.
So.
All right.
All right.
So I've identified a ton of sites to hack now on a large scope program or pen test.
Um.
Now.
I want to do some actual identification of what software they're running to actually do the hacking.
Right?
I found a ton of scope.
So there are some tools that already exist to tell you what technology stacks that these sites are using.
Um.
Wapalyzer and built with are two browser extensions that you can use that will automatically just based on the headers that come back from the site.
Based on strings in the source code of the page.
Um.
The HTML source code.
Um.
They just know it's built with WordPress.
Or they know it's built with whatever.
Whatever framework.
Right?
They can even go as deep as telling you sometimes what the databases are that are running the project.
So.
Wapalyzer and built with.
They're just little buttons that sit inside of your browser.
And you can click them and get full stack information.
The reason I want to know the full stack information is because it helps me know what techniques I'm going to use to try to hack that server.
Um.
There's also another thing called retire.js.
Which is a scanner to find outdated server side JavaScript libraries.
Um.
Which is super cool.
You can use that.
Um.
I use it a lot.
Um.
And then this.
This is a new tool called.
Um.
Vulners.
Or it's not a new site.
But it's a new burp extension that I like.
So.
Uh.
Vulners.com is a repository of basically CVEs.
Um.
And instead of the actual CVE advisory.
They give you exploitation data on how to exploit old versions of software.
Um.
Like.
You know.
Old hacker sites used to do.
Um.
So they give you all the write ups of the original person who exploited it.
It's not just like.
Hey.
There might.
There is a buffer overflow in this area of the application.
This is the CVE number.
Um.
So what Burp Vulner Scanner will do.
Is it will find.
Um.
Basically pages.
You load it into Burp.
You browse a site.
Uh.
And it will tell you what technologies the site is using.
And what CVEs.
Based on the version number it's returning.
It might be vulnerable to.
And it has complete links inside of Burp.
In the bottom right hand corner there.
That you can go to and click.
And.
Uh.
And find exploitation info.
So.
Um.
It's a pretty sweet Burp Suite tool that.
Uh.
I like a lot right now.
It's plugin.
Yes.
Burp.
Burp Suite plugin.
Yeah.
So.
Is it from Vulner's or.
No.
Vulner's just released it.
Um.
I think.
Two weeks ago.
Yeah.
It's not in the BAP store yet.
At least it's not as far as I'm aware.
So it's on the GitHub on Vulner's.
Vulner's dot com.
Burp Vulner's Scanner.
That's where.
That's where it is.
So this is just loading the extension and running it on something.
Eventually in the bottom right hand corner you'll see the.
Um.
You'll see the output.
Right.
So.
So.
So.
Let's just.
Go into a site.
Selecting a site.
Yeah.
It sees Vulner's detected.
And there's all the CVEs that might be associated to that site because of its versions on the
bottom right hand corner.
Cool.
All right.
So I now have a bunch of targets.
I've started to identify their platforms.
I have auxiliary ports to test.
I really have a lot to test at this point on a large scope pen tester bug bounty.
Now what you have to do in any like web test.
If it's a web technologies.
You have to discover all the content of the website.
Not just the web content.
by spidering stuff that exists
but trying to find stuff that doesn't exist.
So this is a thing called content discovery
or directory brute forcing on websites.
You'll learn about this if you take any web hacking class.
It's basically brute forcing paths on your URL.
So let's say I'm hacking tesla.com,
and there's a hidden page
that is basically the admin's page, and it's slash admin.
But that's never linked anywhere on the site.
Only employees know about that path to go there.
Well, the way you find this stuff is you brute force
words after that path, after the main tesla.com path.
So here, this, just like the old tools in DNS,
used to take a ton of time to do.
Tools like Potador
or any other directory brute forcing
or content discovery tool,
even the content discovery tool in Burp,
not super fast.
Go Buster is kind of the new school tool to do this.
It's written in Go. It's multi-threaded.
It really does a good job for this.
So you can burn through a large list of brute forcing,
this kind of stuff, with Go Buster in,
I can tell you what it's completed in.
Running, sorry?
It is, yes.
It has some of the same functionalities
as the cooler tools where you can,
so it took 40 seconds to run on a 500,000 line
directory brute force list,
which is really fast.
If you don't know, that's super fast.
Which, like, cracker for BNC or BNP,
do you recommend?
For brute forcing the passwords?
Yeah, right.
I think Medusa was the one I used last, I think.
Which one?
Medusa.
Medusa.
Yeah, Medusa.
Anyway, okay, so you're brute forcing URLs, right?
So you have a fast tool, Go Buster,
but you also need a great list
to find out what common paths there are.
So there's three lists,
that are really kind of the industry standard now.
SecList is a project that Daniel and I run together,
and it has lists of paths that usually exist
that are sensitive.
Dan did a separate project called Robots Disallowed.
He went out to the whole internet, right, Dan?
And basically found every robots.txt file that existed,
which is the stuff that admins don't want you to spider,
and then put that into a list so you can then spider it.
Yeah.
So it's like the whole internet status.
So it's super cool.
You run that through Go Buster.
It's really fast.
You'll find good stuff there.
Then there's these Digger Word Lists.
These Digger Word Lists are two projects put out by,
I forget what consultancy it was.
It used to be Stack & Lou,
which is now Bishop Fox, I think.
Yeah, yeah.
So they're cool guys.
They had these Digger tools,
and what they did is they went to
the source code repositories, GitHub,
and there's another one.
I don't know what the other one was.
What?
Might have been Bitbucket.
I'm not 100% sure.
Anyway, they went there,
and they spidered all the code from those projects,
and then parsed out the paths,
any paths that they saw for directories,
and then made lists based off of that
in reoccurring instances.
So you can use any three of those lists,
but use them with Go Buster.
You can also do this inside of Burp,
but it's just not as fast.
But if you want to keep it all in Burp,
you can do it in Burp.
All right, so now I have paths.
I have sites.
I've got a ton of stuff to work with.
Is there anything else I need to do?
There can be.
I don't do this a lot,
because this is like an extra step,
but it is worth mentioning.
I think it's a cool idea.
So the idea here is that I might have found
a whole bunch of scripts,
and maybe I found an admin script
or resource on the page,
but this resource doesn't tell me how to use it.
I'm not an admin.
I don't know what parameters to send
or data to pass into this script to do anything.
So the idea here is you can actually
brute force that as well.
So this is a tool called Parameth.
You pass it through here,
and you pass it a resource name
or a script name here on the bottom.
It found a simple test.php script,
but it has no idea what parameters
to pass this to execute anything.
Or it could find a path or something like that.
So what it does is it brute forces
parameter names for that.
There's another project that Burp actually put out
called Backslash Powered Scanner,
and they did a similar research to Dan's,
but instead of crawling all of the robots.tech,
files on the internet,
and putting that into a list,
they basically,
I don't know where they got the data,
but they pulled out the top 2,500
used parameter names on the internet.
So you can load that into Parameth
and try to find parameters that will execute.
You know when you're successful
because this tool will do
like a reject response discovery
and a comparison between sending it nothing
and then sending it this brute force parameter.
So it'll tell you,
hey, this differed.
You should go check this out
because I think I actually executed something here.
Yeah.
No, that is not.
It's a different thing.
Yeah.
No, this is a command line tool.
Yeah.
You could do it in Burp.
You would just have to do it in Intruder.
Yeah.
I mean, command line tools are usually faster,
but if you do it in Burp,
you keep it inside of your Burp workflow,
which is nice.
Yeah.
Okay.
So parameter brute forcing is a thing.
All right.
So there's a lot of stuff that we've done in here.
You know, like how do you keep track of it?
How do you automate it?
Like really there's not great solutions right now.
The problem here is I see a lot of people
making these frameworks to do all this for you.
And that's really great.
Like J. Crane at my work has one called Intrigue,
and it is the slickest looking OSINT tool.
There's one called DataSploit, which is badass,
which these guys actually make, which is cool.
But my problem is that the tools and the sites
that I'm scraping and that are written right now,
they change a lot.
Like sometimes those sites go offline.
Sometimes they start limiting your API access.
The tool and the technology may be distributed
next time it's written.
I mean, a lot of these tools are writing
their own tools to do this.
And so it's never as fast or as complete
as I need it to be.
So I always end up having to go back
to manually using these tools to get the data I want.
So there are two projects,
one called Hodor and one called Kubot by Onshuman Baris.
And basically the idea here for Hodor
is that you set everything up as a Docker container
with a standard Docker config file.
So if you have a new tool and it's the new hotness,
you load it into your Docker config file,
and then you run one command in Hodor,
and it distributes your command across all of the machines.
Let's say I just say Docker up slash something,
and then I say Tesla.com.
It'll stand up new Docker instances for everything,
run all my tools simultaneously,
and then spit back the output to my host machine.
And so that's where I think kind of the future is of everything,
because then if I find a new tool I like,
it doesn't matter what language it's written.
I can just make a Docker container for it
and start it and bring back the output.
Then all I have to do with that output
is parse it a little bit on the command line,
and it's usable.
So I think Hodor and Kubot
are what I'm going to use in the future.
There's also, in the bug bounty world,
there's this idea of in order to hack these things,
I have to be the first to know
about them.
When someone stands up a new subdomain,
I need to be the first to know about it
so I can hack it and get a good bug.
Asset note is a framework
that will do some of this for you,
and you can write custom modules for,
and it will basically text you
when it finds a new domain.
So I have a couple of bug bounty hunters
who use this in great success.
They get an alert on their phone
that says Tesla stood up a new domain.
You need to go home right now,
stop whatever you're doing,
and go hack it before the rest of the 500,
1,000 bounty hunters are going to go after it.
And so really, it makes a difference.
It does when you're trying to do bug bounty full time
for a living.
Oh, that's it.
Any questions?
Yes?
Where can we find your talk?
I haven't published this version of the slides.
I don't know.
Are you guys publishing somehow the slides at all?
Yeah, I'll give them to them,
and they'll probably be on the Recon Village site.
So, yeah.
You have a question?
Have you used AWS Azure or Cloudflare APIs
to do some of your recon?
Yeah, so there's also the idea
of looking for that kind of stuff.
I haven't found any tooling that I've settled on yet.
That's why I didn't include any of it.
There's probably four or five tools
that will scour the internet
for misconfigured Git directories
and also misconfigured AWS S3 buckets.
And so I haven't really found one I liked yet,
so I didn't add it in the presentation,
but that is a method that I do use.
Yes?
Yes?
Do you have any suggestions for UBs
on how to do UB space safely?
I guess, you don't want to go home.
Maybe you need that.
I don't do anything from home anymore,
so I did a bounty a couple weeks ago,
and I went up against Akamai's WAF,
and if you get blocked by Akamai's WAF,
it blocks you from PayPal, eBay, all your banks.
I tried to register for a hotel.
It wouldn't let me to that site.
My wife was really angry with me.
So, yeah.
So I do everything from DigitalOcean VPSs
pretty much now.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
They haven't bothered me yet.
Yeah.
Yeah.
What?
Nothing yet.
Yeah.
Yeah.
Oh.
So, we have extra recon badges,
which are super sweet.
How am I going to do this?
Somebody name off one of the tools I use in my methodology.
All right.
Who said BurpSuite first over here?
It was you, yes.
Okay.
One of these.
All right.
What was one of these subdomain scraping things
that I talked about?
I think he said it first.
Subluster.
Yes.
Cool.
All right.
Sorry.
I only had two.
Oh, you dropped it.
You need the battery.
You need the battery.
It's right there.
I also have a couple printed copies of proof of concept
or get the F out.
So, if anybody wants one of those,
they can just come up and grab one.
So, I'll just put them right here.
Yeah.
Cool.
Well, really,
thanks for coming.
I really appreciate it.
Thank you.
Thank you.
Thank you.
Thank you.