start	end	text
0	4180	Hey, everyone. My name is Jason, and thanks for joining us this morning.
4380	9420	Super big thanks to the Red Team Village for having me and all the other speakers on
9420	14160	and the staff for facilitating everything that's going on with the conference for DEF CON
14160	17340	for hosting the Village. A big place in my heart for DEF CON.
17340	22320	So today I'm going to talk a little bit about my methodology for recon.
22420	26700	I've been doing this talk called the Bug Hunters Methodology for five years now,
26700	30900	and I basically update it every year, and I talk about new tools and techniques
30900	34880	in different spaces of the Bug Bounty scene and Red Teaming.
34880	36780	So today we're going to go over the recon stuff.
37820	41640	Now, the Bug Hunters Methodology, TBHM, is a presentation, like I said,
41640	44800	I've been running for a long time, and basically it's too big for any one conference slot,
44800	48060	so I split it into two sections. One is recon, which we're going over today,
48060	53340	and the other is application analysis, which is on-site hacking and stuff like that.
53340	57540	So today we're going to do recon, and I'm working on this year's version
57540	60800	of application analysis, haven't finished it yet. I'm a giant slacker, sorry.
61340	63240	So yeah, today we're going to go over recon.
64480	70140	A little bit about me. I'm a husband, father, hacker, gamer, and I stream sometimes.
70140	72540	These are my socials, so if you want to reach out and ask a question
72540	74840	after the talk or something like that, just hit me up.
75540	80200	I was a bug hunter, and then I worked at Bug Crowd for many years,
80200	83600	and now I'm the head of security at Ubisoft,
83600	88520	and so I lead the security team there. Awesome team, great people.
88740	92120	And I'm also a gamer at heart, so I play some video games.
92120	96380	These are my kids, Arcadia and Avalon in the top picture.
96380	100440	I took them to DEF CON for the first year. Last year I did DEF CON Kids or Roots.
100440	103760	It was amazing, they had a great time. And then that's my son Arlen,
103760	106640	him and I being pirates in the bottom right-hand corner.
106640	109640	It's a little bit about me. I've been doing pen testing for about...
111720	113440	a long time, I'm old.
115460	119640	I have a lot of related experience to this talk.
119780	123860	The first thing I want to talk about is today we're going to go over
124520	128540	recon methodology, and we do recon in red teaming engagements,
128540	131420	and we do recon in bug bounties primarily. And you can do it
131420	134920	in some wide-scope pen tests as well. And one of the first things
134920	138540	I talk about is when you're doing an assessment, some sort of assessment,
138540	141180	whether it's one of those three, you have to have a way
141180	144140	that jives with you to record your work.
144140	147180	Otherwise, you're just throwing darts and forgetting what you're doing.
147180	153460	And this is counterproductive to basically keeping organized.
153460	157560	So the first thing that I talk about here is how I keep notes.
157620	161460	And I use a tool called X-Mind, which is a mind-mapping tool.
161460	164860	And basically, I create nodes, and I create methodologies
164860	168000	as checklists inside these nodes, and I track all my domains
168000	170420	and which ones I've worked on by color codes.
170560	173500	So this is something I do. I've seen other people do
173780	177520	a lot of project tracking for assessments in just Notepad or Vim,
177520	180620	or there's some specialized pen test tools. If you're really fancy
180620	182820	and you have a subscription, you can probably do it in something fancy
182820	186260	like Dratus or something like that. So there's a lot of tools,
186260	190100	but it's just really important to track your work when you're doing recon.
190100	193540	Recon is the art of finding as many assets as possible
193540	199220	related to a target, and it can get pretty data-heavy and dense
199220	202680	as you do it. So having a way that works for you to track your data
202680	205720	is the first thing that's going to guide you to success.
205900	208240	So this is an example of how I track my data
208240	211880	when I'm on a target. So Tesla, Tesla Motors,
211880	216080	has a pretty open-scope bounty on Bug Crowd.
216220	219500	And this is how I would start off my project with Tesla.
219500	222380	So in the middle, I have a node in this mind-mapping program
222380	226080	I use called xMind. And then on the left-hand side,
226080	229920	I have their autonomous system numbers, their ASNs.
229920	234140	I have any acquisitions they've made. I have some other notes,
234140	236980	LinkedIn discovery, and a link to their reverse-who-is information.
237380	239220	And on the right-hand side, I'm starting to build out
240020	242720	their root domains, or their seeds. Roots or seeds
242720	247580	are terms for things like something.com,
247580	250260	or teslamotors.com, or tesla.com, or SolarCity.
251020	253760	And so I've started to build out their seeds
253760	256260	on the right-hand side. So you can see Tesla Motors,
256260	259600	Tesla, SolarCity, etc. And as I do this,
259600	261800	I'm going to start collecting a lot of data. As I do recon
261800	265420	on Tesla, I'm going to collect a lot of data. So that turns into
265420	268880	this, which on the left-hand side, you can see
268880	271920	for tesla.com, that seed,
271920	275360	I've enumerated all their subdomains and all of the live links.
275360	278440	So these are all the live links for tesla.com,
278440	282060	for their subdomains. So things like pages.tesla.com,
282060	283980	shop.tesla.com, etc.
284520	287240	And then you can drill down into any one of these.
287240	290580	And so if I drill into www.tesla.com, then I have
290580	293140	my methodology notes for that single site, which
294040	296580	inside of that node, I keep
296580	299240	all of the questions I ask myself when I'm basically
299240	302920	looking to hack a website. Like, does the site have multiple user roles?
302920	305840	How does it reference users? How does it handle special characters?
306060	309260	What are dynamic parameters? Does it have an API component?
309260	311780	What kind of errors am I seeing? Does it have
311780	314800	file uploads? Have I done JavaScript analysis
314800	317580	for paths? Have I done content discovery?
317940	320680	All these questions that you're normally going to
320680	324280	keep in your methodology, I apply
324280	326860	these all to each one of these subnodes. And then I
326860	330280	color code my work based on where I am.
330280	331980	And this is one of the most important things, I think,
331980	336000	is that everything that is not filled in hasn't been
336000	338420	worked on yet, so most of this hasn't been worked on yet.
338420	341020	But everything in orange means I'm currently working on something,
341020	344700	and everything in green, which is there's nothing green on here, I've already done.
345400	347880	And so just the way I can do this,
347880	350100	and then I can add checkmarks and other stuff in XMind
350100	353280	to these mind maps helps me track where I am, so I can easily
353280	355680	put down a project and pick it back up if I want.
356980	359900	Okay, so today's mission, we're going to talk about
359900	362700	wide recon. This is the art of finding as
362700	365480	many assets related to a target as possible.
365480	368460	So you're a red teamer, or you're a bug bounty hunter, and you have
369040	371260	a scope of X company.
371460	374080	And one of the things you need to do
374560	377420	is identify all the websites that they own.
377420	380560	Because the more websites you identify, or the more
380560	383460	infrastructure you identify, the better your chances
383460	386740	of getting in or finding a bounty, right? And this is a
386740	389920	core component in both of those skill sets.
389920	392740	And I break down what we call recon into
392740	396120	a couple of domains. So finding first the in-scope
396120	400360	domains via the program or your project
400360	402840	brief, you're finding acquisitions
402840	405160	for the company or the target,
405160	407960	doing ASN enumeration, doing reverse whois,
407960	411140	doing a whole bunch of subdomain enumeration, and then doing port analysis.
411140	413840	And then we're going to go into some related topics
414940	416780	like some vulnerability scanning
416780	419080	and some automation
420420	423260	information related to recon, because you'll probably do it
423260	425040	in this phase of your workflow.
426220	428940	Okay, so you've decided you're going to do an assessment
428940	432040	or you've been handed a red team assessment or you've been handed a bug bounty
432040	434980	that's wide scope, right? That's where we are right now. So the first thing
434980	438080	we need to do is parse the program page.
438080	441260	So I'm going to use a bounty here, a couple bounties to illustrate
441260	443920	where you would start. So here is
443920	446180	the brief page for the aforementioned Tesla
447040	449780	for BugCraft. So in their brief, they have
449780	452980	star.tesla.com, star.tesla.cn, Tesla Motors
452980	455320	and Tesla Services. And then they also have this
455860	459040	catch-all sentence here that says any host verified
459040	461560	or owned by Tesla Motors that you can find
461900	464940	is also included in scope. So this is what we consider a wide scope
464940	468060	bug bounty. There's lots of stuff that you can do with this.
468260	471180	So your first four seed domains are already listed here.
471180	474140	Tesla.services, TeslaMotors.com, TeslaCN, and Tesla.com.
474140	476800	So we're going to focus on these four and add them to our list.
477000	479640	And then anything else we can find that is Tesla's is also fair game.
479640	483080	So that's great. If you look at a platform like HackerOne,
483620	486060	they also have a pretty wide scope bounty
486060	489080	and probably the most prolific bounty available on the internet
489080	491840	which is Verizon Media. And this is like the granddaddy,
491840	494560	the biggest scope program I think
494560	497720	I know of in the bug bounty scene. So Verizon Media
497720	500920	has gobbled up so many brands and they have
500920	503700	so much infrastructure related to these brands and sites
503700	506780	and subdomains. It's giant. So a lot of hackers
506780	509780	make their whole career hacking on
509780	512840	Verizon Media, which is awesome. And they're a great team
512840	515680	and they support the bug bounty community. They also have this
515680	518520	catch-all kind of phrase in their scope that says if you've found
518680	521400	a vulnerability that affects an asset belonging to them but it's not
521400	524600	in scope, report it to the program. And so it's a
524600	527460	really advanced team to work with. You couldn't even fit
527460	530440	all of the domains that you have for this program on one page, which I
530440	534100	didn't even try. So you will parse your seed domains
534100	536580	and your in-scope domains from the program page
536580	539660	to start off. And this will give you a starting place to
539660	543280	work from. The next thing that I do
543280	545440	when I'm scoping out a company is I want to find
545440	549080	all their acquisitions. And there's a couple reasons I want to do this.
549140	551660	I want to basically see
551660	554300	if any of the infrastructure for those acquired companies has
554300	557360	not been taken off the line or any of the subdomains
557360	560360	or websites or integrations that were too hard to port
560360	562960	over to new infrastructure are still online.
563260	566420	This gives me more and more infrastructure and sites to
566420	569520	hack. So to find the acquisitions, I use
569620	571840	a tool called Crunchbase. And Crunchbase is a business
572880	575860	intelligence portal. Basically, you add any business name
575860	578900	to this search box on Crunchbase and you'll be able to
578900	581940	look at that company and find out what
581940	584680	are their employees? Are there investor rounds?
584680	587800	What acquisitions have they had? What are their socials, etc.?
587800	590960	So here I've used the example of Twitch.com. We're streaming
590960	593240	on Twitch today, which is awesome. And Twitch
593840	596840	as an organization here has an entry. And if I click
596840	599620	on that, I get a lot of information about Twitch.
599620	602820	And I can see that under their acquisitions tab,
602820	605200	they have acquired four companies
605200	608640	in the last eight years, Revlo,
608640	611420	Clipmind, Curse, and Goodgame. And then also,
611420	614700	they were acquired themselves on the left-hand side by Amazon.
614840	616620	So this gives me a good idea of
617820	620640	what they used to be, where I can also
620640	623480	look for some infrastructure if it's a recent acquisition,
623480	626480	like Revlo, for instance. If I click on Revlo
626480	629460	in this page, it'll take me to Revlo's page, which will give me their main domain,
629460	632080	which is Revlo.com. So I might put that in scope of
632660	635300	a wide scope assessment or a Red Team assessment.
635680	638480	Acquisitions are important to keep track of. Now, the one thing that you
638480	641360	want to make sure here is that on some of those brief pages that we
641360	644260	looked at in the last couple pages, some of these are
644260	647860	explicitly out of scope. So they'll say in the brief,
647860	650460	especially for Tesla, SolarCity is not in
650460	653320	scope of their wide scope bounty.
653320	656060	So even though we found that as an acquisition of Twitch
656060	658840	when I looked on this page, when I've done it many times,
658840	662240	it's explicitly out of scope. So remember to refer to your project page
662240	665280	or understand what is okay and what is not okay to hack
665280	666680	or do recon on.
667960	671260	So after I find the acquisitions, which has given me
671260	673880	seed domains like Revlo.com and new
673880	677220	seed domains other than Tesla Motors and Tesla and stuff
677220	680320	like that, or any of these top-level seed domains,
680320	682960	what I want to do is I want to find this company's autonomous
682960	686020	system numbers. So every company that gets
686020	689180	large enough ends up having a collection of networks
689180	692420	basically applied an AS number to them.
692420	695160	And an AS number, autonomous system number, is just a collection
695160	698080	of your known IP ranges for your networks, for your
698080	701200	infrastructure. And so you can find someone's
701200	704000	autonomous system number in many ways. There's
704000	706840	many search engines to do this kind of
706840	709600	lookup, but I use one called Hurricane Electric
709600	712580	or bgp.he.net. I use this
712580	715620	when I'm doing manual searching because it has a freeform
715620	718220	text box at the top and I can just search the keyword
718220	721680	Twitch. And here you can see I found Twitch's
721680	724840	two autonomous system numbers, AS46489
725860	728300	and AS397153.
728300	730600	And then their associated IP ranges
730600	733840	and IPv4 ranges and IPv6 ranges.
733840	736980	So this is a good representation of the infrastructure
736980	739800	or the IP space that they own. And I can
739800	742720	assume that it belongs to... I can guarantee
742720	745560	that it belongs to them. I don't have to worry about
746000	748840	scope really here because I know that all this is
748840	751240	owned by them. So if they have that catch-all phrase in their
751240	754640	bounty project or you've got the go-ahead in your red team engagement
754640	757700	to open scope, then you know this is your target.
757880	760500	One thing that this doesn't represent is their cloud assets.
760500	763020	So this is all owned IP space.
763200	766280	It doesn't represent things like AWS and Azure and GCP
766280	769320	ranges. So you won't find stuff that they host
769320	771620	in those environments inside of these ranges.
772180	775140	You just want to be cognizant of that. So here I've searched
775140	778160	Twitch. I got their ASNs. With every
778840	781720	tool or method here in my methodology,
782160	784660	I like to give you a manual way to do it
784660	787500	because context is important and getting used to the idea
787500	790340	is important. And I like to give you an automated way to do it in case you're going to script
790340	793380	up your own recon framework, which is all the fad these days
793380	796300	and I have one of my own and everything. So there are
796300	799640	two tools you can use to parse ASNs
799640	802480	and IP ranges from companies that
802480	805500	exist on the command line if you're going to script this stuff up yourself.
805720	808280	So one is called Metabigor.
808640	811120	I'm still not sure if I'm saying that name of the tool right. By
811620	814500	JesseJJJ. And this one utilizes
814500	817260	the site that we just saw, bgp.he.net
817260	820320	and another site called ASN Lookup. And it will
820320	823280	grab the information from those websites, scrape it off and give you
823280	826540	your IP ranges for an organization. The other one
826540	829740	is called ASN Lookup by Yasin. Same idea.
829740	832560	It uses a different data source called the MaxMind database.
832560	835220	It'll pull off their IPv4 ranges from an ASN
835220	838640	and you can give these both tools
838860	841640	a term instead of an AS number or anything like that.
841640	844400	So they will pull it off based on that keyword
844400	847220	that we're looking for. In this case, Tesla or something like that.
847240	850440	Now, one thing to know about these tools is they are
850440	853640	looking off your keyword here, which we put on the command line,
853640	856480	Tesla. And there are multiple companies with
856480	859360	Tesla in the name. There's like Tesla Research Lab or something
859360	861480	like that in Sweden or something like that.
862140	865600	Just be sure that when you get these ranges and you start seeing
865600	868620	these sites, you can identify, oh no, I might have gotten
868620	871020	one that's actually not Tesla Motors.
872140	874860	Both these tools, I verified, pull up the right
874860	877700	data for Tesla. But just be cognizant
877700	880700	of, since you're using a keyword here to search, that you don't
880700	883660	get other IP ranges that are not your targets.
883660	885900	You'll have to verify somewhat manually there.
887140	889640	Okay, so we've gotten some IP space from the ASNs.
889640	892140	We've gotten some seed domains. We want to continue gathering
892140	894260	as many seed domains as possible.
895020	897380	We have the AS number, and for here
897380	899120	we're going to use Twitch as an example again.
899480	903300	Their autonomous system number is 46489.
903660	906220	And here what we want to do is we're going to
906220	909100	get our first introduction to Amass.
909100	912440	And Amass is a framework for
912440	914920	domain intelligence, I would call it, I guess.
914920	918080	And it is written by Jeff Foley. There is a workshop you should absolutely
918080	921140	sign up for. He's going to do a full workshop on Amass
921140	924400	and how to use it. Jeff is as associated
924400	927160	to the Red Team Village, so super cool. And here we're going
927160	930080	to use Amass and feed it our autonomous
930080	933200	system number, 46489. And what it's going to do
933200	936080	is it's going to go out to all of the IP ranges that
936080	938720	are represented by that autonomous system number.
938720	942240	And it's going to scan the certificates for all HTTPS
942240	945160	sites. And then it's going to parse those certificates and tell us,
945160	948240	hey, here are the seed domains I found
948240	951220	that are owned by them. And here you can see that we found some
951220	954200	seed domains that we didn't know about before. Justin.tv, which is the
954200	958420	company Twitch used to be. TTVNW.net,
958420	960220	which is, I think, associated to
960220	963580	Twitch's streaming protocol. Twitch.tv,
963580	966400	obviously, which is their main site. TwitchCon, which is their conference.
966400	968460	And SocialCam, which I have no idea what is.
968960	972420	And all the links for these tools and the methods
972420	975260	are posted in the slides, so you can click
975260	978220	through when the slides get released and go grab the tools
978220	980860	and play with them on your own. So here we've already expanded
983740	984540	our assessment
985040	987220	a lot, right? We have a whole bunch of IP space,
987220	990440	we have a whole bunch of top-level domain seed domains,
990440	993180	and we have the seed domains already given to us by our
993180	995880	brief page. Okay, so the next method is
995880	999060	reverse whois. And what reverse whois does is it will take the
999060	1002120	whois entry of... you can search your domain
1002120	1005080	in Twitch.tv here. And you can do this
1005080	1007960	with a lot of sites. I use huoxi.com because
1007960	1010940	it's pretty cheap and you can get a free API
1010940	1013480	key for so many uses. And
1013900	1017040	you basically supply it your domain and it will give
1017040	1019820	you the whois data and then you can
1019820	1022760	use that whois data to correlate who else has
1022760	1025860	registered stuff. So an example here is we gave this site
1025860	1028940	huoxi.com, twitch.tv. And then it said
1028940	1031940	that over the course of the years, there have been several
1031940	1034780	companies' names related to the whois data and also
1034780	1038020	registered emails. And so here you
1038020	1041560	can see that the company, as of March 26,
1041560	1043940	2015, was justin.tv
1043940	1046800	and the email in the domain was domainmaster
1046800	1050140	at justin.tv. And then later in 2015, it changed
1050140	1053120	to twitchinteractive. And if you click the button right next
1053120	1055840	to those entries, you can see there's 20 other domains still
1055840	1058860	registered to justin.tv, which could be interesting to us. And there's
1059700	1061380	575 domains registered
1061380	1064800	under twitchinteractive. And so we can start pulling back more
1064800	1067460	data, more subdomains, more root domains from
1068480	1070800	these links when we click on them. Now,
1070800	1073580	one thing to know here is that whois data is
1073580	1076560	registered data. And so a lot of these companies
1076560	1079680	will park domains to combat
1079680	1082680	phishing and to hold stuff for
1082680	1085600	marketing campaigns and stuff like that. So they might not be live sites.
1085600	1087920	They might just be parsed. So this is a medium fidelity
1088800	1091500	type of technique here, but I have found some
1091500	1094560	really good stuff using reverse whois and the registrant
1094560	1097520	data of these sites. There is
1097620	1100400	a tool to do this type of analysis. It's called domlink,
1100400	1103420	written by Vincent Yu via security on Twitter. And
1103420	1106840	it uses that exact site, hoaxy.com.
1107100	1109480	And you can basically give it a domain
1109480	1112300	and it will find every other associated domain by both
1112820	1115640	registrant email and organization name or company
1115640	1118600	name. And it will do it on the command line and return it to you
1118600	1121520	in a script. And it's recursive. So a really cool tool
1121520	1124480	here. You can use this to automate some of that reverse whois
1125060	1127400	lookup. It's called domlink. The tool is called domlink.
1128880	1130540	So the next method that we have now
1130540	1133320	that we've maybe gotten some more seeds and subdomains from
1133320	1136320	reverse whois and we're building out our list of things that we can
1136320	1139140	hack is we want to look at the ad and analytics
1139140	1142180	relationships of our target site. We want to see what
1142180	1145180	other sites are using the same ad and analytics codes
1145180	1147400	as our main site. And this will give us
1148480	1151120	a good understanding of maybe what are their most
1151120	1154100	popular domains and maybe what are some of their less popular domains
1154100	1157220	that are still using the codes and we didn't know about before.
1157340	1160260	So every page usually embeds a
1160260	1163080	Google Analytics code or a New Relic code
1163080	1166020	or some kind of ad or analytics code
1166020	1169040	in the main page. So here if we use a site called
1169040	1172140	builtwith.com, we can search twitch.tv
1172140	1175080	and twitch.tv will give us a list under the
1175080	1178120	relationship profile tab of all of the keys that are on
1178120	1181300	twitch.tv. And then you can
1181300	1184160	click on those and see what other sites are using the same
1184160	1187220	ad and analytics codes and maybe we didn't know
1187220	1190040	about those before. So here we can... it's hard to see on the bottom
1190040	1193280	but we have a couple entries that we didn't know about before.
1193420	1197280	manvsgame.tv, twitc.tv.
1197360	1199060	We knew about twitchcon.com
1199060	1201480	from the previous method but we're starting to build up more
1202040	1205000	domains that we can see that are related to
1205000	1208020	our target. So ad and
1208020	1211620	analytics relationship profiles is a pretty powerful technique.
1211660	1214240	I have used it for success a couple
1214240	1217220	times. You can also do this directly from Firefox or Chrome.
1217220	1220260	They have an extension for this site called built with and
1220260	1223140	you can just install it. You make a free account
1223140	1225920	and then you visit your site in the browser, click the
1225920	1229080	Chrome extension button and it will give you the relationship information
1229080	1231500	right in your browser and you can start clicking around there.
1231500	1234600	So pretty easy to institute while you're
1234600	1236680	doing your recon in a browser.
1238760	1241180	So you can also use a tool to do this
1241180	1244080	in the command line. So Malik, after
1244080	1246940	my first iteration of this talk, quickly scripted up an
1246940	1250460	awesome tool to do this. It's called getrelationship.py
1250460	1253400	and you just pass it your target here.
1253400	1256140	Here he's targeted Uber.com which also has a bounty
1256140	1259080	program and passed it to this Python script and it will pull out on the command
1259080	1261860	line all of the related domains. And you'll have to
1261860	1265220	manually kind of verify these but the ones I would look at first
1265220	1267720	are the ones that actually have Uber in the
1268320	1269660	domain somewhere like
1270920	1274000	daft.uber or driveuber.co.nz
1274000	1276400	or things like that. So you'll have to
1276400	1279300	sift through this but you can do it on the command line now too.
1281870	1283410	Then you can just use some general
1283410	1285730	Google hacking or Google-fu to try to find
1286330	1289250	associated sites to your target. You can use
1289250	1292090	the copyright text, the terms of service, and the
1292090	1295070	privacy policy and you can just copy those from the bottom of any of the pages
1295070	1298070	and here you can see Twitch interactive ink would be something
1298070	1301230	I'd Google for. And then you can see any other page that is
1301230	1304030	hosting that text at the bottom of their page
1304030	1306410	which could be something that you didn't know about before
1306990	1310090	in your recon so far. So you can just do
1310090	1313190	this manually and you can use
1313190	1316130	other search operators like in URL Twitch or you can
1316130	1319170	just look for these policies. One tip here is
1319170	1321430	that you don't just want to target a certain
1322010	1325090	year. Like here I have 2019 but you also want to check 2018
1325090	1327650	and 2017. In fact, the more stuff you find
1327650	1331030	that goes back farther in age, the more
1331030	1334010	likely it is to be vulnerable and less likely to have a lot of security
1334010	1337190	assessment associated to it. So probably more vulnerable for your
1337190	1340190	Red Team engagement or your bug bounty.
1342170	1343070	Alright, the next
1343070	1345750	method that we're going to use to find some seed domains or
1345750	1348930	related infrastructure is Shodan. I have to admit I'm not a Shodan
1348930	1352130	ninja but I know a lot of Red Teamers are and I actually love Shodan
1352130	1354930	and it's just a failing of mine. I need to get better at it. But there's
1355050	1358170	a ton of search operators we could use here on our domain twitch.tv
1358170	1361430	and what Shodan is is it's the site that is a
1361430	1364190	site that hosts information from an infrastructure
1364190	1367330	based spider. And a spider goes to every site on the
1367330	1370170	internet and basically captures
1370170	1373550	its HTTP responses, its
1373550	1376190	technology stack, its certificate data
1378030	1378970	and cross links it
1378970	1381810	so you can click on any of these pieces of information and find out
1381810	1384990	other things it's seen related and what organizations it's
1384990	1388030	related to and what common tech that they're
1388030	1390690	using, what web servers, what JavaScript
1390690	1394330	framers, you could do anything. So Shodan is a really powerful tool
1395010	1396910	and you can query our
1396910	1398590	domain here and try to find
1399670	1402950	related infrastructure. For instance, in one of the entries when I just
1402950	1406330	didn't even use a great operator here, I just searched my domain twitch.tv
1406330	1409370	I ended up seeing in the SSL certificate data
1410230	1412010	of the search there was
1412010	1415430	twitch.amazon.eu and now I need
1415430	1418390	to ask myself a question like is that in scope, right? Like that's not
1419590	1421370	verbatim a Twitch domain but in a
1421370	1424610	red team assessment if that's related to Twitch infrastructure, well
1425250	1427350	maybe it's a target that I can go after.
1427350	1430630	So Shodan can give you a lot of good information.
1430630	1433370	Okay, so we found seed domains, stuff to start
1433370	1436250	with. Now what we're going to do is we're going to try to find subdomains
1436250	1439290	for those seed domains. So these are things
1439290	1442230	like www is a subdomain
1442230	1445010	but also admin.tesla.com or
1446150	1448390	forums.tesla.com or something like that
1448390	1451210	or forums.twitch.tv or whatever. We're going to try to find
1451210	1454450	subdomains. And so this idea of starting
1454450	1457170	with seeds and then moving to subs really
1457170	1460170	is going to multiply. The more seeds you find
1460170	1463010	the more subdomain enumeration you can do. The more subdomains
1463010	1466050	you find is related to how many
1466050	1469290	sites you find. The more sites you find, the more successful you'll be
1469290	1471470	inside of your red team engagement or bug bounty
1472130	1474990	program. So for subdomain enumeration
1474990	1477650	I use three-ish
1478090	1480690	different methods to get subdomain information. One is
1480690	1484250	LinkedIn JavaScript discovery. Another one is
1484250	1486970	subdomain scraping techniques and then subdomain
1486970	1489930	brute force. And then there's some auxiliary stuff I use as well.
1489930	1492910	So we're going to run into those right now. And this is the bulk of what a lot of tools
1492910	1496090	are doing right now. And so
1496090	1498970	you'll see a lot of favorites in these slides. So the first area
1498970	1501410	we're going to use to find subdomain data is
1501970	1504730	or the first technique is linked discovery and JavaScript
1504730	1508190	discovery. So linked discovery, what is it?
1508190	1510810	It's basically using a web spider to
1510810	1513030	visit a site like twitch.tv
1513650	1515630	land on it and spider all the
1515630	1519050	HTML links. Pretty simple. And any of those that
1519050	1521870	come back with a subdomain
1522390	1525350	we just add to our list of in-scope targets.
1525350	1528230	And so this is pretty self-explanatory. It happens
1528230	1531230	naturally when you're using tools like burp suite and you're spidering
1531370	1533830	a site. But I'm going to walk you through how I do it.
1533830	1537050	I'm going to use burp 1.7 because I like the UI better but it
1537050	1540030	works as well with the crawler or the scanner
1540030	1543070	in burp 2.0. So use whatever
1543070	1546070	you want. And then I'm going to give you some other methods to do it. So the first thing
1546070	1549190	we do is we load up burp suite which is on the right
1549190	1552070	here. It's a side-by-side to our site. And
1552070	1554990	we just visit our site through burp suite the proxy.
1555350	1558150	And burp suite the proxy will capture all this
1558150	1561210	data. We've only visited one page. If you visit
1561210	1564330	twitch.tv ever, you see that it has
1564330	1567290	cross-linked and it does request a lot
1567290	1570110	of stuff because it's a streaming media site and so there's a lot
1570110	1572390	of stuff going in the background when you go to twitch.tv.
1573110	1576150	So you can see all that data on the right-hand side. This is everything that's
1576150	1578910	either seen been linked on this page
1578910	1582110	or it's actually been requested. Actually been requested is the
1582110	1584510	stuff in black, seen is the stuff that's in gray.
1584950	1586630	And so what you do is you visit your page
1587770	1590770	and then you have to set up a rule, some type of rule.
1590770	1593910	And you can do this by going into the target tab and
1593910	1596650	the scope tab and then you click this box that says
1596650	1599730	use advanced scope control. And what I do is I enter
1599730	1602670	in just a keyword here in advanced scope control
1602670	1605230	which I just say twitch, right? So I want to see
1605810	1608710	in burp anything that has twitch in the URL
1608710	1611690	at all, no matter where it is, what has that word twitch. And you could also
1611690	1614210	add some more here because we've seen in our previous analysis
1615610	1617410	TWT is also part of their
1617410	1620630	domain naming nomenclature, right?
1620630	1622970	And so you can add a couple of these scope rules
1623450	1626290	and then you go back to your sitemap and you
1626290	1629330	click on the ribbon bar up at top or the filter
1629330	1632250	bar and then you say show only
1632250	1635090	in scope items. So now this will just show
1635090	1638270	in our sitemap all the twitch.tv links, right? So we already
1638270	1642030	have a good list of subdomains here, right?
1642050	1644390	Sentinel.twitchservice.net
1644390	1647490	API.twitch.tv. So we have a lot
1647490	1650390	of stuff with twitch in the name here. We know that most of this is probably
1650390	1652950	related to twitch. This gives us exponential
1653990	1656130	areas to hack, right? And this is pretty good.
1656130	1659210	But if we select all these and then we spider them
1659690	1662390	using burp spider, it will then go to all of
1662390	1665610	them and find their links and find subdomains
1665610	1668470	references in their pages. And we can do this recursively
1668470	1671470	until there's nothing left to find. So here I'm
1671470	1674250	going to select all of these and then I'm going to spider
1674250	1677130	them. And now you can see, since I had them selected,
1677130	1680170	everything in orange on the right-hand side was stuff I knew about before I
1680170	1683010	spidered. And then after I spidered with burp,
1683010	1685770	everything in white is new stuff. And you can see I found
1687010	1689190	a combination of a lot of stuff here. I found
1689190	1692310	subdomains for twitch.tv and I've also found some new
1692310	1695150	root and seed domains like twitchapp.net
1695150	1697310	and twitchservice.net and
1697310	1701250	x-twitch.tv. So this is a
1701250	1703850	hybrid technique. It can find you new seed domains or roots
1703850	1707030	and subdomains. And so now I have all of this domain
1707030	1710190	data. I can now select all of these and spider them again
1710190	1713130	and spider those pages for links. And eventually you can do
1713130	1715350	this until you get kind of spider fatigue.
1716230	1718970	And eventually you'll have a large sitemap buildup
1718970	1722070	of targets, which is awesome. Now, how
1722070	1724990	do you get this data out of Burp Suite? There's
1724990	1727790	not a great way to do this. Pro has
1727990	1730730	a thing called Engagement Tools.
1730730	1733610	And in Engagement Tools, you can do this thing called
1733610	1736650	Analyze Target. And in Analyze Target, you can create
1736650	1739830	an HTML report of all the targets
1739830	1742890	in your site tree that are selected. So that's what I do.
1742890	1745690	I select everything in the site tree. I go Engagement
1745690	1748710	Tools, Analyze Target, Generate HTML Report. And then
1748710	1751730	in the HTML report, I have a parsable list of
1751730	1754550	the domains that it's seen at the beginning of that HTML report. And then I
1754550	1757510	take that and I put it in a text file for later analysis.
1757510	1759590	And then I dump it into my mind map as well.
1760930	1763290	Okay. So the whole
1763810	1766250	linked discovery thing counts on Burp
1766250	1768790	Spider, right? A lot of times I hadn't seen
1768790	1772210	before the last couple of years, great options to do this
1772210	1775210	in the command line because it would require
1775370	1778610	a lot of custom coding, bash scripting
1778610	1781330	or Python to create my own spider and do that same
1781330	1784250	process. Well, now there does exist some tools that are
1784250	1787370	command line spiders with bug hunters and red teamers
1787370	1790110	in mind. There's two. There's one called Ghost Spider written by Jesse
1790110	1793530	JJJ. And it also designates
1793530	1796810	the types of things it's parsing when it's visiting a URL.
1796810	1798670	So it'll give you...
1825120	1828140	Okay. Hopefully you guys can hear me. Headphone problems.
1828140	1829300	Going to plug it in.
1829780	1833940	Okay. Great, great, great. I was getting beeping in my ears.
1833940	1836200	So maybe it's just running batteries. My bad. Okay.
1836200	1839420	So you have two spiders here that you can use in the command line.
1839420	1842440	Ghost Spider and Hackcrawler by HackLuke.
1842440	1845660	And both these are awesome. They both have some functionality
1846500	1848860	that will parse out the types of things
1848860	1851460	you're getting, like JavaScript files,
1852020	1854200	subdomains, URL endpoints.
1854560	1857940	Some of them will give you, I think, parameter names and stuff like that.
1857940	1860800	So you could institute the whole process
1860800	1863120	of link discovery by scripting up
1863800	1866040	Ghost Spider or Hackcrawler if you wanted.
1866040	1869700	I still use Burp, but these are invaluable tools
1869700	1872540	to have at your disposal, just like a crawler that can do
1872540	1875320	some analysis. So keep them bookmarked
1875320	1876960	for when you might need them.
1878420	1881440	Okay. So the next place we're going to get subdomain information
1882300	1883540	is by analyzing
1884300	1887480	some JavaScript. And one of the tools I like here,
1887480	1890020	just because it has this kind of little added benefit that I think
1890020	1893080	that I haven't seen many places before, but maybe some other tools
1893080	1895780	are starting to implement it now, is a tool called
1895780	1899440	Subdomainizer by Neeraj Edwards. And what it'll do
1899440	1902300	is it'll take a JavaScript file, you have to point it to a JavaScript
1902300	1905440	file, and it will parse out all the cloud services,
1905440	1908240	the subdomains, and it'll do this little extra
1908240	1911300	thing where it uses the Shannon entropy formula
1911300	1914260	or algorithm to identify things that
1914260	1917260	look like API keys hard-coded in JavaScript,
1917260	1920120	which is already kind of a vulnerability if you find a private API
1920120	1923180	key or credential hard-coded in JavaScript,
1923180	1926240	which I know sounds crazy to a lot of people. Like, you would just find
1926380	1929180	a hard-coded private API key, but
1929180	1932440	this happens all the time. Like, all the time it happens.
1932860	1935080	And so this is
1935080	1938180	what I like to call a forward-thinking type of thing, is using
1938180	1941040	an algorithm like this to identify keys. Sometimes
1941040	1943980	it's a little noisy, but a lot of times it finds you
1943980	1947060	good stuff. So I like this tool to point at all the
1947060	1949460	JavaScript files I found already on
1950020	1953180	some of these sites. If you're just looking for subdomain
1953740	1956020	information, there's another tool called Subscraper by Celian
1956020	1958820	Collins, which has recursion built into it, which
1958820	1962240	can do this method as well, but it doesn't do the API key part.
1962740	1965040	So point this at JavaScript files you find
1965040	1967440	on your site, and you can get back a whole bunch of
1969860	1971480	subdomains and cloud
1972200	1975280	services that the site might use or the target might use.
1976540	1977340	All right. So that's
1977340	1980460	link discovery and JavaScript analysis for
1980460	1983540	subdomains. Now we're going to get into the big meats of
1983540	1986320	what a lot of people do is subdomain scraping.
1986420	1989920	Now the idea of subdomain scraping is
1989920	1992820	going out to these websites on the
1992820	1996120	internet that have search boxes, basically.
1997080	1999740	There's all these projects on the internet, search
1999740	2002360	engines, security websites, certificate
2003060	2006120	projects and stuff like that. They all do different stuff.
2006120	2008820	So Robtex gives you infrastructure
2008820	2011520	information. The Wayback Machine houses
2011520	2014380	information about domains and their responses
2014900	2017700	years past. Everyone's probably used the Wayback
2017700	2020740	Machine. The certificate sources down
2020740	2023620	here, some of them are certificate projects to provide certificate
2023620	2026400	transparency. Search engines, obviously you know what search
2026400	2029440	engines are. And then there's a whole bunch of security sites
2029440	2033060	that do different things, like give you a
2033060	2036260	rating on how malicious a URL or a site might be.
2036260	2038820	The common thing that these all have is either have an API or a
2038820	2041660	search box where you can put in a domain. And they will search
2041660	2044700	that domain and they will tell you anything they've seen
2044700	2047700	related to that domain. And if you put in a domain like
2047700	2050420	Tesla.com, the information that comes back is
2050420	2053420	parsable and you could possibly find out that they know about
2053420	2056620	some subdomains of Tesla.com that we don't know about.
2056620	2059360	So this is the process of subdomain scraping.
2059360	2062380	We're going to go to all of these sources, all of these sources and ask
2062380	2065400	them, hey, what do you know about Tesla.com or
2065400	2068780	Twitch.tv? Do you know of any subdomains that maybe I don't know of?
2068800	2070880	Okay, cool. Let's do that. Now,
2071440	2073920	there are many more sources that are on this page, right?
2073920	2077220	These are only a subset and new sources
2077220	2079980	to parse are coming out as fast as new websites are coming out.
2079980	2083080	So the tools have to keep up adding new
2083660	2086300	novel sources to find subdomain data or URL
2086300	2088800	data as they
2089360	2092180	mature. Now, this is the example of using
2092360	2095420	a search engine like Google to do it, right? So here, what we're doing
2095420	2097840	is we're searching with the search operators
2097840	2101160	site.twitch.tv and we're saying, I already know about
2101820	2103580	www.twitch.tv, so we're saying
2103580	2106640	minus www.twitch.tv. And we already know about
2106640	2109700	watch.twitch.tv, so minus watch.twitch.tv
2109700	2112760	and minus dev.twitch.tv because we already know about that. So now,
2112760	2115740	Google's only showing us things that are not those
2115740	2118460	and so we can do this process, keep on
2118460	2121620	minusing out domains until Google doesn't know anything
2121620	2124260	more about subdomains related to twitch.tv.
2124520	2127580	And that way, we can basically get a full
2127580	2130600	inventory of what Google knows and what Google knows about subdomains
2130600	2133580	for twitch.tv. So this is an example of doing it manually
2133580	2136540	with Google. But luckily, you don't have to do this yourself. There's
2136540	2139720	tools out there that will do this type of analysis, this scraping
2139720	2142620	for you. The first
2142620	2145900	one is Amass by Jeff Foley and the Amass team.
2145900	2148760	There's a whole team behind this. And there's two tools I use
2148760	2151380	here, Amass and Subfinder, but we're going to look at Amass first.
2151580	2154360	Amass has probably the most
2154360	2157320	sources for any subdomain
2157320	2160980	scraping tool in existence in its enum section.
2160980	2163220	Amass is a framework for domain information,
2163220	2166360	but the one that actually pulls this method's
2166360	2169280	results, subdomain scraping, is called Amass Enum.
2169940	2172680	And here on the right-hand side, you can see we gave it twitch.tv
2172680	2175500	and it went out to a whole bunch of sources it's had
2175500	2177880	in its databases and it reached out to those web pages
2178400	2181180	using curl or whatever. I don't know if they
2181180	2183620	use headless chromium. I don't know exactly what they're using.
2183620	2186960	And then they parse those pages for subdomains for twitch.tv
2186960	2190400	and they give us a list of all the subdomains for twitch.tv.
2190960	2192940	And if we do this on all of our
2192940	2195860	seed domains that we've gathered, we have now started to exponentially
2196520	2199500	build out the amount of sites that we have to attack.
2199500	2202480	So this tool is invaluable.
2202600	2204800	Amass is becoming the go-to
2204800	2208420	tool for all subdomain enumeration,
2208420	2211220	especially this thing, subdomain
2211220	2214380	scraping. The other tool I used here is
2214380	2217540	oh, also, this is still Amass. What Amass does at the end
2217540	2220400	of a run, not only does it give you all the subdomains,
2220400	2223200	but it also gives you this great table, which I feel like is underutilized
2223360	2226160	a little bit. And this table tells you, okay, I discovered
2228300	2229180	439 subdomains
2229180	2232200	and here is where they were inside
2232200	2235320	of these ASNs. And you can see that most
2235320	2238440	of twitches were in
2238440	2241520	Amazon's ranges, which makes sense because they were acquired by Amazon,
2241520	2243780	but you can see some of them were in other
2244800	2247380	ASNs, right? And I've had instances where,
2247380	2250400	not on Twitch, but on other projects where
2250400	2253220	there was a whole ASN related to the company
2253220	2256140	I didn't know about that I found out because
2256140	2259300	Amass built this table. They're like, hey, 70
2259300	2262420	of these subdomains we discovered appeared in this ASN
2262420	2265380	or these IP ranges. And then I look at that and I'm like,
2265380	2268720	oh, I wasn't initially looking at those ranges.
2268720	2271260	Weird. Let's go back to the beginning of my workflow
2271260	2274060	and start enumerating those ranges. So this table is
2274560	2277620	super cool. It also gives you information on their third
2277620	2280020	party kind of tools they're using.
2280700	2282160	Here you can see that
2283220	2285980	Twitch is using Bitly and SendGrid
2285980	2289520	and some other stuff. So it also gives you information there in Fastly.
2291040	2292340	Okay, the other tool I use for
2292340	2295280	subdomain scraping is Subfinder.
2295280	2298020	Originally written by Iceman and Michael Skelton, I think
2298020	2301300	now navigated to the ProjectDiscovery.io team, which is
2301540	2304400	a group of bug hunters releasing some stellar tools.
2304760	2307240	And they also have multiple sources,
2307240	2310760	extensible output, also a really great tool.
2310760	2313840	Both of these tools are really good.
2314960	2316320	Each one of them have different
2316320	2319460	sources. So what I end up doing is in my automation, I run both of them
2319460	2322240	and then I cat the output together
2322240	2325640	and sort it and unique it from both the tools.
2325640	2328540	So they both have a couple of different sources
2328540	2331880	and a lot of the same sources. So that's what I do for my stuff
2331880	2333360	is I just use both of them.
2335460	2337700	Okay, so this one is somewhat new
2337700	2340680	and it has been integrated into
2341220	2343540	Amass a little bit.
2344400	2344700	But
2348060	2350580	I've had inconsistency with run
2350580	2353800	between this standalone tool, GitHub
2353800	2356520	subdomains.py and the output of Amass.
2356520	2359520	So I still use this independently inside of my automation
2359520	2361640	when I'm looking for subdomains.
2361640	2364700	So this tool is called GitHub subdomains.py
2364700	2368180	and it's still scraping and what this is doing is it's going out to GitHub
2368180	2370840	as a source. And basically you're providing GitHub
2370840	2373440	and if you've ever been on GitHub, they have the search box up top
2373440	2376060	and you're saying search for twitch.tv
2376060	2378840	and then anything that comes back with twitch.tv
2379180	2382660	as a piece of source code, it will parse up the subdomains
2382660	2384700	from that piece of source code.
2384840	2388600	Now this was written by Gwendolyn Kuik
2388600	2391340	and I still don't know if I'm saying Gwendolyn's name
2391340	2393420	right and that's okay.
2393540	2397560	And Gwendolyn wrote a whole suite of tools for GitHub
2397560	2400540	enumeration. How to find secret keys in
2400540	2403680	GitHub related to an organization. How to pull email
2403680	2406500	addresses out. He has a wonderful blog
2406500	2409680	and it's linked here in this slide with a whole bunch of GitHub
2409680	2413340	tools. Now GitHub subdomains is just looking for subdomains.
2413920	2415800	The thing about using this tool
2415800	2418120	is that the GitHub API, not the tool
2419080	2421520	is just kind of instable and
2421520	2424540	doesn't give you like somewhat returns rate limited
2424540	2426980	results sometimes. So what I do inside of my
2426980	2430280	automation and this has given me subdomains I haven't
2430280	2433060	found anywhere else using this method is I run GitHub
2433060	2434820	search or GitHub subdomains.py
2436220	2439360	like five times and I sleep in between each run
2439360	2442220	so that I give it a little while for the rate limiting to die down
2442780	2445420	and then I give it a big sleep at the end before I run
2445420	2447760	another one and then that seems
2447760	2451100	and then I cat all those results together and unique them
2451100	2453160	and that seems to give me more consistency
2453800	2456360	when parsing the API. And that has nothing to do with the tool that Gwendo
2456360	2459360	wrote. It's all about what GitHub does with their
2459360	2462240	search API. So this is an awesome tool to
2462240	2465360	find lesser known subdomains and I found some
2465360	2467940	great stuff using this tool.
2469160	2471160	Okay, the next one is
2471620	2474460	show sub go by incogbite
2474460	2477100	and this one will parse
2477580	2480280	using a number of search operators with your
2480280	2483340	API key. It will parse showdown.
2483460	2487020	So I include this in my script as well.
2487320	2489360	Again, this is one of those tools that some of
2489360	2491720	the subdomain frameworks like Amass
2492280	2495380	and Subfinder might do for you.
2495400	2498380	I find using the standalone tool for
2498380	2501320	some reason just works better for me. So I don't know exactly why that is
2501320	2504460	but I've run them side by side and gotten different outputs so I still use this
2505120	2507380	verbatim. And it's also a fast
2507380	2510400	running script. It's not like I'm waiting for it to complete for 5-10
2510400	2513380	minutes or something like that. It usually runs in a minute or two. So it doesn't
2513380	2516480	skin off my nose if I have to wait an extra couple of minutes to
2516480	2519140	ensure that I get coverage out of showdown. So
2519140	2522000	show sub go will take a domain and your
2522000	2525580	showdown API key and parse showdown using some search operators
2525580	2528400	and give you back all the subdomains related to here
2528400	2529640	twitch.tv.
2531880	2534500	Okay, so the last... well, not the last, but one of the
2534500	2537140	other methods to look for subdomain enumeration
2537140	2540440	or subdomain scraping is the cloud ranges, right? So we talked about this
2540560	2543180	a little bit earlier is that we have all this IP space
2543180	2546400	and now we've started to identify subdomains and you'll notice that some
2546400	2548600	of those subdomains are resolving to
2549320	2552180	infrastructure or sites in the cloud. Now
2552180	2555200	there's this idea that not a lot of people have been doing is just
2555200	2558140	going out recently or not recently but
2558140	2561360	people have been doing it for a while but it hasn't been much public is just
2561360	2564560	scanning the entire ranges for AWS, GCP
2564560	2567240	and Azure for SSL sites, right? Anything that
2567240	2570680	responds to... anything that responds
2570680	2573380	to, you know, or responds
2573380	2576320	to a connection on 443. And you basically
2576320	2579420	scan the cloud ranges and you scan
2579420	2582600	it by IP address and then when it responds
2582600	2585580	it'll give you its SSL certificate and then you parse
2585580	2588460	the organization name or the domain name out of the
2588460	2591440	certificate data. And then you look at that data and you say
2591440	2594440	does it match my target twitch.tv or does it have
2594440	2597500	the keyword twitch in it? And you're like, cool, I found some stuff
2597500	2600460	that these people have put in the cloud that wasn't part of their
2600460	2603540	ASN that I probably didn't know about before. Now
2603540	2606540	that's a tremendous amount of scanning, right? Those
2606540	2609460	ranges are huge. And you can do it
2609460	2612500	yourself with some tools like mass scan and you
2612500	2615480	can script it up yourself to scan but
2619380	2621940	it's going to cost you a little bit of money
2621940	2625080	on your VPS that you're using, etc. There's a
2625080	2628220	service by Sam Erb and a DEF CON talk he did
2628220	2631080	two years ago where he created a service called
2631080	2633860	bufferover.run. It's an API and you can give it a domain
2633860	2636960	and it will go out and he does this scanning
2636960	2640320	every once in a while. And so you basically
2640320	2642860	take that data, you parse it, and then you get
2642860	2645400	a list of subdomains that were in
2645400	2648540	the cloud ranges. Now, I think he runs his
2648540	2651420	scans every two weeks and updates the service so
2651420	2654740	it's not exactly live data. And
2654740	2657620	this is one of the sources included in a mass.
2657700	2660100	Again, I just wanted to outline the
2660100	2663400	single-shot tool here. You probably could feel safe using a mass
2663400	2665780	to get most of this data back. But
2666400	2669300	yeah, you can use the service as well. So
2669300	2672460	this method also finds some really good
2672460	2675320	stuff. A lot of people are putting shadow IT infrastructure
2675320	2678640	and registering websites on a credit card or
2679300	2681620	just not paying attention or thinking that anybody
2681620	2684640	will find their cloud infrastructure because they've never published
2685200	2687880	those domains anywhere other than internally.
2688260	2690620	And so you can find a lot of things for your
2690620	2694300	target organization that are just sitting in the cloud, pretty much unsecured.
2694300	2696700	In fact, it's been a big part of my research
2696700	2699640	lately is scanning the cloud ranges and
2699640	2702600	just finding wickedly undersecured
2702600	2706200	stuff because people just don't think you'll ever find it. So this is a good method.
2707320	2708640	All right. So we've scraped
2708800	2711560	a lot of stuff to find subdomains. Now we want to brute force
2711560	2714560	for subdomains. This is just a tried
2714560	2717460	and true method that I'm sure every red team or pen tester has
2717460	2720100	done before, usually with a tool like Fierce or
2720100	2723120	one of the other tools like that in the past 10 years.
2723420	2726200	You just try to give a word out of a dictionary
2726780	2729720	and add it before your company.com
2729720	2732880	and see if it resolves. It's a pretty simple
2732880	2735840	method. Now, iteration in this field has
2735840	2738380	come along in the last four years where
2739940	2741800	a lot of the tools that we were using to do this
2741800	2744760	were great, but they were using one DNS resolver,
2744760	2747780	one DNS server to resolve. And
2747780	2750900	it took a long time. I remember running Fierce as a pen tester
2751720	2753440	10 years ago, and it just
2753440	2756480	took so long to finish a large dictionary to do
2756480	2759720	subdomain brute forcing. And in the last
2759720	2762540	four years,
2762540	2765420	the idea of using multiple
2765420	2768240	resolvers to speed up the process was pioneered by
2768240	2771440	MassDNS. And MassDNS was the king for a little
2771440	2774720	while. Now Amass also includes
2774720	2777680	the idea of using multiple resolvers. So Amass
2777680	2780000	uses eight DNS resolvers
2782360	2784780	to parse DNS data when it brute
2784780	2787820	forces by default. And you can add even more
2787820	2790980	by using some flags. So
2790980	2794280	here we're just running Amass over, you know,
2794280	2796900	Twitch again. And we're doing brute forcing this time
2796900	2800060	with the dash brute
2800060	2803100	option. And then we're adding the source so we can
2803100	2806080	see where some of these came from with the scraping
2806080	2809320	part. And then you can specify the number of resolvers
2809320	2812260	with dash RF. So RF means resolver
2812260	2815720	file, I think. And if you basically
2815720	2818680	give it a list of DNS resolvers or DNS servers
2818680	2821740	that you trust, it will use more than one to do
2821740	2824500	your brute forcing. So this speeds up your brute
2824500	2827440	forcing significantly. I've also heard that
2827440	2830200	another alternative to using Amass for this is
2830740	2833340	ASDNS brute. I haven't used it yet, but I've heard it's
2833340	2836060	also wicked fast and has multiple resolvers. So if you wanted an
2836060	2839920	alternative. Shuffle DNS by the project
2839920	2842660	discovery team also does this. I think it's a
2842660	2845780	wrapper around mass DNS. It is a
2845780	2848940	wrapper around mass DNS. And if you prefer to
2848940	2851800	break out that type of brute force from Amass
2851800	2854760	for whatever reason, whether it's stability or
2854760	2857560	mass, you know, is taking too long for you.
2857560	2860480	I haven't benchmarked the tools side by side, but a
2860480	2862860	lot of people like Shuffle DNS as well to do
2862860	2866700	subdomain brute forcing. So a
2866700	2870000	subdomain brute force tool is only as good as the
2870000	2872900	dictionary or word list you give it, right? Because it's just
2873040	2875200	trying to resolve a whole bunch of words in front of your
2875200	2878920	target, right? Twitch.tv. And so
2878920	2881680	in this mind, there's two kind of ways that you can
2881680	2884380	approach what word list you feed to these tools. One
2884380	2887760	is a tailored word list where you can
2887760	2889900	build one based off of words that appear,
2889900	2893440	brand names that appear, you know, you can build
2893440	2895900	like a contextual based word list to your
2896960	2900000	target, which Tom Nom Nom, who's a prolific
2900000	2903220	bug hunter, awesome human, have a lot of respect
2903220	2905380	for this guy and the tools he makes and the
2905380	2908320	contributions he makes to the community. He did a talk
2908320	2911380	at NahomCon, which is a conference a little while
2911380	2913960	ago now, a couple months ago, where he did a whole talk
2913960	2916960	on word list and how to generate contextual word list for your target.
2916960	2920020	So I suggest going to watch that if you want to make some tailored word list. It's called
2920020	2922780	Who, What, When, Where word list. And then
2922780	2925880	you can also use a massive word list. And so over
2925880	2928880	the years, we've had many, many tools that do
2928880	2931600	DNS brute forcing. I went out and I took the
2931600	2934480	word list for all of them and put them into
2935560	2937980	one, and it's called all.txt. It's
2937980	2940720	linked in this presentation. People use it. It's on my GitHub.
2941100	2943940	And it basically sort and uniques all of the DNS
2943940	2946800	names that you've seen forever. Now it has a lot of
2946800	2949840	crap in it. It's true. It's a lot of lines. It's like a million
2949840	2952440	or two million. I can't remember. It's a ton. With
2952940	2955700	using the multiple resolvers, it actually doesn't
2955700	2958740	take too long to do the brute forcing. I
2958740	2961880	don't necessarily care if I'm sending crap to
2962340	2963600	a DNS resolver like
2966140	2967960	0001.twitch.tv. Obviously
2967960	2970380	that's usually not going to resolve, but
2970380	2973600	the list has some gems in it that I
2973600	2975520	just can't get past, you know, like
2976920	2980080	its efficacy. So I use all.txt
2980080	2983020	when I'm doing subdomain brute forcing, and I think it's pretty good.
2984460	2985480	There are some
2985480	2988060	newer school pieces of research to pull out
2988640	2991200	subdomain enumeration, right? That file
2991200	2994640	that I made parsed out all of the kind of
2994640	2997520	subdomain brute forcing tools that have existed for the last, you know,
2997520	3000320	10 years or something like that and put them into one file.
3000320	3002920	But there's some new research by the team at AssetNote,
3002920	3006460	which is an attack surface mapping company.
3006500	3009040	And they did some cool research using
3009040	3012680	Google BigQuery to generate and discover
3012680	3014980	subdomains that were used on, like, the Alexa
3014980	3018320	top 10 or Alexa top 1,000 or 10,000
3018320	3020780	or something like that. Or Reddit. Or anytime
3020780	3024480	someone referenced a URL on Reddit, what was the subdomain in that link?
3024480	3027540	Or Stack Overflow. Anytime someone
3027540	3030420	referenced a URL on Stack Overflow, they parsed these sites
3030420	3032300	with BigQuery, and then they made these
3033760	3036600	subdomain lists that you can use. So they called this project
3036600	3039280	the Common Speak project. I've integrated Common Speak 1
3039280	3042260	into the list, all.txt,
3042260	3045240	but Common Speak 2 came out a couple years ago. It's not in
3045240	3048600	all.txt. It's got some other sites that they decided
3048600	3051400	to target. I recommend checking it out for kind of newer school
3051400	3054760	research on what subdomain names look like.
3055340	3057780	Then there's this idea of alteration scanning.
3057780	3060360	This is a type of brute forcing. So you have
3060360	3063320	dev.company.com, but you could also have dev1,
3063320	3066160	dev2, or dev-1, or dev.1.
3066600	3069580	company.com. And so this technique was
3069580	3072560	pioneered by Shubbs and Nathy when they wrote a tool called AltDNS.
3072560	3075320	It's now been this permutation
3075320	3078480	or alteration scanning, whatever you want to call it, has been built
3078480	3081480	into a mass. So you can use a mass
3081480	3084280	and it will try to find these naming conventions
3084280	3087500	for you. And it will sometimes
3087500	3090460	give you gold because people name stuff predictably
3090460	3092200	in their subdomains.
3093640	3096140	Some of the things I've done with permutation scanning
3096140	3099680	is bypassing web application firewalls.
3099680	3102540	So where I've had SQL injection on a main
3102540	3105680	target and getting blocked by like a web application
3105680	3108400	firewall, I've managed to find a permutation
3108400	3111420	like dubdub2 and then managed to bypass the
3111420	3114820	firewall because it wasn't applied to dubdub2.
3115020	3117420	I've also managed to bypass things like
3117420	3120480	Akamai by finding their origin via predictably
3120480	3123200	named alterations like origin-sub or
3123200	3126520	origin.sub domain to bypass filtering to go to the source.
3126520	3129380	So these are some things you can use that origin scanning to
3129380	3132420	do, as well as just find new surface to
3132420	3133140	attack.
3134300	3137340	All right, so now we're going to go into some other stuff that's related to
3137340	3140640	Widescope Reon. We have seed domains, we've got a lot of seed
3140640	3143520	domains, a lot of subdomains right now, and we should have a pretty good
3143520	3147300	map of the infrastructure that belongs to this company.
3148040	3150080	All right, so one that I didn't talk about,
3150080	3153140	and this is new to this specific talk,
3153140	3155920	that's why I called it 4.002,
3155920	3158760	is favicon analysis. So there is this idea
3158760	3161680	that every page in the tab at the top of your browser
3161680	3164860	has a favicon, right? You see that little Tesla in the bottom left-hand corner
3164860	3167540	of my Chrome tab? That's a favicon.
3167960	3170700	Now, favicons are little
3170700	3173580	images, and what you can do is you can take a hash
3173580	3176880	of that favicon, and then you can search for the hash
3176880	3179720	of the favicon on Shodan. And Shodan
3179720	3182900	will then show you every other site that has that favicon,
3182900	3185740	which will find you some gems, because people tend to reuse
3185740	3188960	favicons on a lot of their sites of domains you might not have seen
3188960	3192200	before. There are some newer tools to do this.
3192200	3194620	I used to do this a lot in my recon testing.
3194640	3197880	I then took it out for a little while, and I recently
3197880	3200800	put it back in. So favicon analysis is kind of
3200960	3203760	a fringe technique, but it's pretty
3203760	3205880	cool. So there's a new tool called favfreakout by
3207100	3209440	Devansh Batham. They're at Twitter at
3209440	3212820	Asmodeus, and basically he does a couple
3212820	3215060	things here. He parses from Shodan.
3215240	3218980	He'll also look for
3218980	3221780	hashes in different places, but he also uses
3221780	3224720	the hash. He also has hashes for
3224720	3228480	common infrastructure, like a scanner would,
3228480	3231020	to look for different types of infrastructure. So on the bottom
3231020	3233140	right-hand side, you can see that he has some
3234160	3237080	fingerprints on different types of things, like Spring Boot pages
3237080	3240040	or Big IP pages or Slack instances. They all have common
3240040	3242920	favicons. And so when you scan a list of
3242920	3246180	URLs or a set of domains on
3246180	3248920	port 80 or 443 and you retrieve their favicon and their
3248920	3251900	hash matches one of those hashes, you know, oh, shoot, I've stumbled
3251900	3255020	upon a Slack instance for this organization. And you
3255020	3257980	may not have found that doing anything else.
3257980	3261080	You can also do the same thing if you scan the cloud
3261080	3264080	and you correlate one of the previous techniques with this one.
3264080	3267000	Let me find things that have my domain and the certificate data and have
3267000	3270120	these favicon hashes associated to them. So this is
3270580	3273300	a fringe kind of analysis technique to find
3273300	3276260	even more kind of esoteric
3277700	3279020	related sites.
3280780	3282080	Okay. So then we're
3282080	3285020	going to go back to a tried and true method
3285020	3288080	is port scanning. We have a lot of domains now. We have a lot of C domains
3288080	3290820	and subdomains and sites that we can work with.
3290960	3294040	We want to port scan them because they may have services that are
3294040	3296960	not 80 or 443 on these pieces of
3296960	3300440	infrastructure. And so the, you know,
3300440	3303140	the tried and true hacker education
3303140	3305900	will tell you to use nmap here. But I use
3305900	3309040	mass scan because I believe it's faster. It has a
3309860	3312160	rewritten TCP IP stack, true multi
3312660	3315520	threading. It's written in C, you know, directly
3315520	3318340	calls, you know, a lot of stuff. So mass scan, in my
3318340	3320820	experience, has been faster than using nmap, even with flags
3320820	3323920	like min parallelism, which there was a huge debate on Twitter
3323920	3326720	the other day of which is faster, min parallelism or using
3326720	3329060	mass scan and people were like, whatever. When I run
3330540	3332820	a scanning tool, a port scanning tool across,
3332820	3335640	you know, 400,000 hosts,
3335640	3337960	I have always found mass scan and it's
3338920	3341720	syntax, it's advantages
3341720	3344800	to beat out nmap. So that's just my personal
3344800	3347600	experience. If you really like nmap to do this, you totally can.
3347600	3350460	And this is strictly for finding ports. It's not to do
3350460	3353260	service scanning, right? Nmap obviously wins in those areas. If you're
3353260	3356400	going to do banner analysis, service scanning, script
3356400	3359360	scanning, like that all pans down. Mass scan
3359360	3362440	only does one thing, finds open ports. That's it. So what I
3362440	3365360	do, also, if you want to learn how to use mass scan,
3365360	3368120	Daniel Miesler, one of my best friends in the whole world,
3368120	3371260	he wrote a study guide on mass scan and all of its syntax.
3371260	3374100	It's one of the best ones I've seen, even better than man page. So go check
3374100	3377220	that out on how to use mass scan and how to set the flags
3377220	3379860	correctly and how to scan. So
3379860	3383120	what I do is I'll take mass scan and then the problem
3383120	3386200	about mass scan is it only scans IP
3386200	3389160	addresses. It won't scan a domain name. So
3389160	3391960	you can use a tool called dnmasscan, which will convert
3391960	3395120	your domains that we have, right? We have subdomains and
3395120	3398200	seed domains and it'll convert them into IP addresses and then scan them with mass
3398200	3401460	scan and tell you all the open ports on each IP address.
3401460	3403860	So what I do is I take mass scan
3404760	3407400	and I scan that over all of my subdomains
3407400	3410480	and then I feed that output, because I know it's
3410480	3413500	open, to nmap to do service scanning. And
3413500	3416500	service scanning will start to give me more information
3416500	3419560	on these services that are open on those boxes. And then what
3419560	3422460	I do is I do a quick default credential spray
3422460	3425380	across everything that has certain services open,
3425380	3429160	which is FTP, SMTP, SSH, Telnet,
3429160	3431880	any type of SQL database, basic
3431880	3434880	authorization and some other stuff. And so this is a tool
3434880	3437940	that I use to do that. It's called Brute Spray. Brute Spray will
3437940	3441000	take the output of your nmap scan, so you feed mass
3441000	3443800	scan to nmap, nmap does the full service
3443800	3446920	scan and outputs an XML file, and then you feed that
3446920	3449680	XML file to Brute Spray to do a quick
3449680	3452920	credential, default credential brute force against all
3452920	3455720	of the services that allow you to do that. And so I've found
3455720	3458460	some good wins doing this, logging
3458460	3461460	straight into SQL databases and SSH
3461460	3464480	and just some horrible stuff that people leave
3464480	3466700	unsecured on the Internet, so on services.
3467800	3470860	All right. So while I'm doing
3470860	3473660	all of this stuff, right, we're in kind of the other category right now. So while
3473660	3476500	I'm doing all of this analysis, and I have most of this automated
3476500	3479620	in a giant ugly shell script that I use, and we'll talk about
3479620	3482780	recon frameworks in a second. But while I'm doing all this,
3482780	3485640	it takes a little while for these tools to run. The conglomeration
3485640	3487980	of tools and techniques we've already talked about takes
3488740	3492000	anywhere between 5 and 15 minutes,
3492000	3495080	maybe a little bit more if I have a lot of subdomains
3495080	3497980	or it's a big project. It'll take to run all of these
3497980	3501000	tools and give me output. So while I'm doing this,
3501520	3503700	I do a technique called GitHub dorking
3503700	3507360	and this has found me many, many great things.
3508240	3509860	And what you do is
3509860	3512800	you basically just go to GitHub and type in your domain as a search operator.
3512800	3515780	So twitch.tv. And you go to GitHub and you type in
3515780	3519020	twitch.tv and then you just start browsing source code
3519020	3522220	that has referenced twitch.tv or tesla.com
3522220	3524140	or whatever, teslamotors.com.
3524460	3527680	And you can find all kinds of sensitive data that former
3527680	3529780	employees or current employees
3531380	3534140	have accidentally put on GitHub.
3534140	3536740	And so I have built a script
3536740	3539800	just to build these search queries for me. These dorks, I call them.
3539800	3542300	And it's right here in a gist. You can grab it.
3542440	3545880	And it looks for things like my domain twitch.tv that I'm looking at
3545880	3548120	right now and password or
3550220	3552120	npmrcauth or dockerconfig
3552120	3554240	or pemprivate for certificates or
3555600	3557620	s3config or htpassword or
3557620	3561720	credentials or bashrcprofiles or sshconfigs.
3561760	3563600	And so it searches these key terms
3563600	3566620	along with the domain. And if it comes up
3566620	3567960	that someone has accidentally
3569400	3572720	put this on GitHub, it's usually automatically a finding and it'll help me
3572720	3575640	get into other systems. Let me give you an example here of something that happened in the real
3575640	3578920	world. I found an admin page for a
3578920	3581620	site. I couldn't do anything with it.
3581620	3584660	I fuzzed the form. Great,
3584660	3588280	didn't work, etc., etc. Then I found
3588280	3590480	some dude who had basically
3590480	3593540	posted a password not for that specific site, but it was
3593540	3596460	related to my domain on GitHub for some other system.
3596640	3599680	That system was internal, so I couldn't access it.
3599680	3602660	But then I tried to use that password and username on this admin
3602660	3605440	portal I had found that I had no luck before. Bam, got in,
3605440	3608380	stole credit cards, millions of records of data.
3608380	3611320	It was game over at that point. So this can
3611320	3614260	help. There's a whole awesome talk here
3614260	3616960	called GitHub Recon and Sensitive Data Exposure on
3616960	3620120	Bugcrowd University by the gentleman. It's probably the best
3620120	3623140	primer on doing this type of dorking to find sensitive
3623140	3626400	stuff on GitHub that I've ever seen. I highly recommend
3626400	3629380	you check that out. And then Gwendolyn also wrote a
3629380	3632280	GitHub search tool that allows you to do this in the command line and
3632280	3641480	not in the browser, which is...
3641480	3644600	Okay, so now you have a bunch
3645160	3647520	of subdomains, and I think I'm running close to time, so I'm going to try
3647520	3650300	to hurry it up here. You have a bunch
3650300	3653360	of subdomains here, and now we want to prioritize
3653360	3656440	which ones we test, right? So what you can do is feed
3656440	3659720	all of your subdomains to a tool that does screenshotting.
3659940	3662300	And there are several tools that do
3662300	3665460	this. Currently, I use Eyewitness. I was using Aquatone.
3665460	3668480	I go back and forth, but there's four tools here. Aquatone,
3668480	3670760	HTTP Screenshot, Eyewitness, and WitnessMe.
3671060	3674140	All are tools that help you take
3674140	3676740	screenshots, and then some do, like, additional analysis
3677880	3680120	on your domains that you feed it, but normally
3680120	3683000	I just use them for screenshotting. It doesn't matter which
3683000	3686000	one you use. Try them out. See which one you like. I like Eyewitness.
3686100	3688660	I feed it a list of domains. It will
3688660	3692140	take those domains, visit them with the headless browser,
3692140	3694940	take a screenshot of the page, and then I will just look at that
3694940	3697820	folder and see, okay, which one
3697820	3701040	of these looks like I want to prioritize first?
3701240	3704020	Does it look like it redirects to the main site? Well, obviously,
3704020	3706000	I'm not looking for the main site right now.
3706720	3709740	It's some kind of back-end admin portal. Okay, I want to prioritize
3709740	3712740	that. So screenshots can help you prioritize your work.
3714900	3715700	When you're
3715700	3718980	given a large list of subdomains like you have now, you can start
3718980	3721960	to look for some vulnerabilities. One is subdomain takeover.
3722180	3724560	There's a repo called CanITakeOverXYZ by
3724560	3727720	edoverflow, and it gives you a list of all of these
3727720	3730960	services and their fingerprints that
3730960	3734200	might indicate that you can take over that subdomain. I'm not going to go
3734200	3737040	crazy into subdomain takeover because it's short on time,
3737040	3739980	obviously. But you can use a tool to check for
3739980	3743080	subdomain takeovers. The best one right now is Nuclei
3743640	3746180	and SubOver, I think, are my two favorites
3746180	3749100	right now. SubOver was an independent tool,
3749100	3751800	has been since ported over into the Nuclei framework that
3751800	3755220	Project Discovery is making. But go check out Nuclei.
3755220	3758120	It has the most
3758120	3761160	subdomain takeover checks that I've
3761160	3764080	seen for any tool. And I would
3764080	3767040	check that out. And you can run it just across a list of large
3767040	3769880	domains, and it'll tell you, hey, possible subdomain takeover
3769880	3772960	at this address. So this is another wide scope tool that I
3772960	3775860	use in the end part of my recon when I have all these subdomains.
3775860	3778780	And you can find some volumes just straight off of
3778780	3781960	using this. Okay, so
3781960	3785000	we're going to blow through automation real quick. Okay, so when you have
3785000	3788020	some of the tools and you have a long methodology
3788020	3790900	like I have, you end up automating it, right? I write a horrible
3790900	3793620	bash script to do my stuff. It
3793620	3796660	works for me, but it could be better.
3796800	3799880	Some of the tools that I use don't do certain things. They're not
3799880	3802860	threaded. They don't take certain types of
3802860	3805040	inputs, like list inputs or
3805920	3808860	glob notation like Nmap does or range notation like
3808860	3811820	Nmap does. And sometimes I need to feed a tool with another
3811820	3814540	tool. And so I can't do that.
3815040	3817780	So Michael Skelton, also known as Kadingo, wrote a tool
3817780	3820920	called Interlace, which basically wraps around other tools and lets
3820920	3823640	them do those things. It will thread them. It'll allow you to take different
3823640	3826880	sources of input. It'll allow you to distribute tools.
3826880	3829920	Here is an example of a blog written by Hackloop,
3829920	3832740	which talks about basically threading Nikto, which
3832740	3836120	Nikto doesn't support inherent threading.
3836120	3838640	So you can use Interlace
3838640	3841800	to basically glue together a lot of stuff
3841800	3844480	that your other tools don't do. Any tool written by
3844480	3846840	TomNomNom is awesome in automation, right?
3847840	3849940	HttpProbe, WaybackURLs, Meg.
3850560	3854140	TomNomNom has several talks out there talking about his tools and how they work.
3854260	3856380	I use HttpProbe a lot for the glue
3856380	3859600	between me finding subdomains and you feed
3859600	3862640	it a lot of subdomains that you have found after your initial analysis.
3862640	3865260	And then it tells you which ones have actual live listening
3865980	3869180	web servers associated to them. So that's HttpProbe.
3869580	3872360	WaybackURLs will find you URLs associated to any...
3872980	3874820	old URLs associated to any
3874820	3878200	site that you're currently looking at. Meg is like a directory brute
3878200	3881260	forcer, but for many hosts. And I use this to find some stuff
3881260	3884360	too. So all of TomNomNom's tools are just
3884360	3887300	amazing. Alright, last part. I promise.
3888060	3890960	Omar, are we okay to finish this last little bit?
3892280	3893160	Absolutely.
3893880	3896940	Awesome. Okay, so maybe
3896940	3900120	recon is not your thing. I definitely run into hackers who are like, this is
3900120	3903000	the most boring part of assessment to me.
3903000	3905960	And then I run into other people who are like, yeah, recon
3905960	3909240	is awesome. It's like my favorite part of the assessment. So it could
3909240	3911780	be that recon is not really your thing. Finding all these sites
3911780	3914760	is not your thing. Hacking the sites is more your thing.
3914820	3917960	And that's cool. There are a lot of new tools out here
3917960	3921100	these days that basically automate all this
3921100	3923340	for you. And they're called recon frameworks.
3923820	3926800	So if you've ever looked at a video game before,
3926800	3929760	like Diablo, every once in a while,
3929760	3933060	content creators will make these tiers of builds for Diablo
3933060	3936140	or something like that. So I've put recon frameworks into a couple
3936140	3939160	of tiers. C tier, B tier, A tier, and S tier.
3939160	3941960	C tier are recon frameworks that are built around
3941960	3944460	scripting up other tools in Bash or Python.
3944820	3947680	They're step-based. They don't really have a workflow. They only have a few
3947680	3950740	techniques and they're not really super extensible. But they work
3950740	3953560	really well. Let me put out a big disclaimer here.
3953560	3957080	My tools that I use personally on all bug hunts
3957080	3959820	are C tier tools. And they work
3959820	3962540	for what I want and they do. And they're great.
3962540	3965440	So there's nothing bad to say about a C tier tool.
3965440	3968560	A B tier tool, in my mind, and this is all
3968560	3971800	very rough, right? These classifications are not
3971800	3974960	in any way super serious.
3974960	3977420	But a B tier tool, in my mind, has
3977960	3981120	some of its own modules it's doing, some of its own sources,
3981500	3983960	maybe has a GUI, maybe has some
3983960	3986880	workflow in it where they're bringing data in from the end of the
3986880	3990000	recon process back to the beginning if they find new stuff.
3990000	3992600	It has more techniques than a C tier.
3992760	3996020	But it still runs that point in time and it's still working on flat files
3996020	3999220	to track the data for recon.
3999280	4001980	A tier is maybe
4001980	4003660	writing all of their own modules.
4003760	4006640	These tools start to have some GUIs, some of them
4006640	4009880	they run iteratively. So like
4009880	4013180	Cron or something like that, they'll run at a schedule.
4013180	4016180	And they start to manage things via database so you can compare
4016180	4018940	scans and recon scans over time.
4019100	4022980	And then S tier are kind of the highest level of what's out right now.
4022980	4025300	They write a lot of their own modules. They have a GUI.
4025300	4028180	They run iteratively. They manage all their data via database.
4028180	4031480	They scale across multiple boxes to make the scanning faster.
4031480	4034100	They send alerts back to the user via email and text
4034100	4037360	and Slack and whatever when they find
4037360	4040320	new things. They have some novel techniques that not a lot of other people
4040320	4043200	are doing. So these are how I classify some of the frameworks. I'm going to show
4043200	4046180	some of them to you now and you can pick one that works for you if you're not
4046180	4048900	into this recon stuff. Warning.
4049000	4051920	I had to put this in here. I don't want to hurt anyone's feelings.
4051920	4055120	I am scaling some tools and giving them grades. It's not
4055120	4057940	serious. It's my rough experience. My gut feel.
4058380	4061100	All these tools are wonderful and I respect the authors of them
4061100	4063560	so darn much of just putting code out there.
4063560	4067340	I can't say enough about the people that open source their code
4067340	4069940	and even C tier. Like I said, my tool is a C tier
4069940	4073000	tool and it works for me. So this is just subjective
4073000	4075920	and based off my own experience. Okay. So
4075920	4079060	some C tier tools that you could look into that are wrapping around
4079380	4082140	a lot of other existing tools.
4082140	4085260	One of the ones called ultimate recon here
4085260	4088220	I like is listed here. And the reason I like
4088220	4090800	ultimate recon is because it's using the new nuclei
4090800	4093340	scanner that's out by project discovery and it
4094360	4097080	implements finding subdomains using a couple
4097080	4099980	different tools. It will then take all of those. It will
4099980	4103140	port scan them and then it will run nuclei templates on
4103140	4106660	them. So it's pretty good. I like it.
4106660	4108540	All of these different C tier tools
4109020	4111840	they're all choosing their own tools to wrap around
4111840	4114540	that they like. So there's no best one really. But
4115340	4117680	I tend to like ultimate recon in this
4117680	4121280	one. There's also other good ones in here too.
4121280	4123900	So check these out. B tier
4123900	4127440	frameworks. This is lazy recon by Captain Milo.
4127440	4129940	And here you can see his workflow. Very
4129940	4132800	similar to what we talked about in this presentation. Amass and sub finder
4132800	4136040	combine those two. You get a final subdomain count.
4136040	4138680	Then you start to do some vulnerability analysis with
4138680	4141760	subjack and core scanner to find some vulnerabilities related
4141760	4145020	around subdomain takeover and cores vulnerabilities.
4145020	4148080	Then you start brute forcing. Then
4148080	4150420	you start port scanning. Then you do some
4150420	4152860	screenshots. And then you have a final
4153820	4156780	output. So a lot of stuff we talked about today is
4156780	4159680	implemented inside of this workflow. So lazy recon is what I like to
4159680	4162700	point out inside of the B tier here.
4164180	4166080	A tier, there's a couple.
4166200	4168300	They all teeter on being S tier, right?
4168700	4171560	I wonder if I should even really have
4171560	4174880	like an S tier and an A tier. Usually
4174880	4177860	the thing that's separating some of these is GUI from the other ones.
4177860	4180540	So the one I have referenced here on this slide is called find domain.
4180540	4183620	And find domain does pretty much everything. It does subdomain
4183620	4186540	discovery and scraping and brute force and all of that
4187220	4189780	via amass and sub finder and asset finder
4189780	4192920	and everything. And then it stores it in a database.
4192920	4195800	It does iterative scanning. It'll text you. If it finds new
4195800	4198900	domains from its last scan, it'll email you, which you see in the bottom left
4198900	4201900	hand corner, which is great. The only thing it lacks
4201900	4204760	is a GUI. It's a command line based tool. So
4204760	4207900	it kind of teeters on doing a lot of the stuff we want to do.
4208040	4210980	And in the enterprise version, it does port scanning
4210980	4213980	and screenshots. And that
4213980	4217040	version is a for pay version, but it's affordable to bounty
4217040	4220160	hunters. Some of those are really good subscriptions to pay for.
4220160	4221880	So find domain, pretty cool.
4222960	4225760	Here's our S tier framework. Some of them you'll never get to use
4225760	4228800	because they're commercial products, but I include them here to give people
4228800	4232040	ideas as to what they should be working towards in their frameworks.
4232220	4235080	Intrigue.io is an asset discovery tool
4235080	4237540	written by Jonathan Cran, who's a friend of mine.
4238800	4241280	He builds Intrigue and he has a
4241280	4244080	GUI for it and it's a SaaS platform and businesses buy it
4244080	4247160	to map their attack surface. But it's doing the same techniques that we've talked
4247160	4249920	about today. The cool thing about Intrigue
4249920	4252940	is even if you don't pay for the subscription of
4253720	4256180	the SaaS version that he's selling to businesses,
4256180	4259180	he open sources almost all of the code for Intrigue.io
4259180	4262180	so you can run your own hosted version or at least
4262180	4265760	see what he's doing and his kind of analysis. So Intrigue is pretty cool.
4265840	4268140	Asset note is another one that's
4268540	4271400	a B2B type play for recon.
4271400	4274260	It's a team of former bug hunters who made this awesome platform.
4274260	4276620	This is probably the gold standard of
4278600	4280420	asset management or attack surface
4280420	4283160	management tools. You can see that it's pretty.
4283160	4286920	It breaks things up by assets that need attention.
4286960	4289340	It's got a lot of graphing. It does
4289640	4292180	a lot of custom vulnerability checking.
4292420	4295340	So asset note is probably, it's S tier, but
4295340	4298440	it's a high price tag so you're probably not going to use it as a bug hunter
4298440	4301220	or a red teamer. But it is kind of the gold standard that a lot of the tool
4301220	4303780	makers should probably work towards.
4305000	4307440	Spiderfoot is another one that does
4307560	4310900	a lot of OSINT type domain correlation information.
4310900	4312560	It does some of the techniques we've talked about
4313340	4316340	in this presentation. They're adding more all the time. I know the author
4316340	4319540	is really strictly interested
4319540	4322580	in adding more bug bounty focused features
4322580	4325540	in it. And it's got a great graphing library and monitoring
4325540	4328160	and everything. So Spiderfoot's pretty cool.
4328320	4331580	The unreleased project discovery, we've talked about project discovery a couple times
4331580	4334560	in the presentation. That team, their unreleased framework looks
4334560	4337580	to be a killer. Looks to be awesome. I'm really excited about it
4337580	4340660	when it comes out. So yeah, you can see here
4340840	4343580	a dashboard of a project, all your discovered domains
4343580	4346540	and seeds, what the scan
4346540	4350100	process is. And then if you dig into any of those, you get screenshots,
4350100	4352680	port scans, technology identification of the
4352680	4356000	site, its activity, its changes over time.
4356000	4358640	So the discovery framework when it comes out will
4358640	4361840	be pretty cool. Jails is a vulnerability
4361840	4364680	scanner written by Jesse, JJJ and team.
4364760	4367680	It is also really good. It is a
4368240	4371120	GUI flow that wraps around some command line stuff.
4371120	4373940	It does vulnerability scanning, which, you know,
4373940	4377840	if you're working with Nessus or, you know, not a lot of red teams are
4377840	4379620	scanning with Nessus. But if you want to look for some
4379620	4382520	very pointed stuff, you can use Jails and look at their
4382520	4386060	kind of CVE checks that they use.
4386060	4388400	And you can see signatures there for
4388400	4391760	the Zoho management page and fuel CMS RCE.
4391760	4394600	So they're looking for some cool volume Jails.
4394600	4397600	I really like it. Osmedias is
4397600	4400560	awesome as well. It's very similar to some of the previous
4400560	4403380	pages we've seen. It'll take all of your
4403380	4407200	domains. It will do... it'll take all your subdomains.
4407200	4409940	It will resolve them to IP. It will do port scanning on them.
4409940	4411760	It'll tell you the technologies.
4412720	4415580	It will do screenshots. It'll do all kinds of stuff.
4415580	4419320	So Osmedias is also pretty easy to stand up and really good.
4419360	4421960	Huntersuite.io, also same kind of idea.
4421960	4425080	You can see technology parsing, services,
4426080	4427880	domain discovery, etc.
4429100	4431220	Bounty.offensive.ai,
4431220	4434180	same type of deal, right? Includes some vulnerability scanning,
4434180	4437960	technology fingerprinting, subdomain finding,
4437960	4439800	and then attack graphing.
4440060	4443640	Reengine is the one I used last week or a week before.
4443640	4446200	Reengine is pretty sick.
4446260	4448800	You can have projects in it that are separate.
4448800	4452060	It'll do the subdomain scanning. It basically takes the methodology
4452060	4455220	that I outlined in this presentation and turns it into
4455400	4458040	a tool, a hosted tool.
4458400	4461160	And they're making some improvements every
4461160	4464420	week. So lots of potential on
4464420	4468020	Reengine, which I highly
4468020	4469560	suggest checking out.
4470240	4473280	And then Scout PDP on the SecApps team. They've been working
4473280	4475720	on tools for well over a decade.
4475820	4479160	And they have built out Scout, which is a paid offering,
4479160	4482300	but it's affordable enough for bounty hunters. Same idea, right?
4482400	4485140	Projects, subdomain enumeration, related
4485140	4488300	domains, reverse DNS. It supports screenshotting.
4488300	4489120	It's awesome.
4490540	4491960	Awesome tool.
4492280	4496000	And then lastly, Nuclei.
4496000	4499080	I talked about it very briefly, but anyone who's not using
4499080	4502420	Nuclei in their bounty scanning right now is at a disadvantage.
4502440	4505060	They're coming out with some epic templates for CVE
4505840	4506800	identification.
4508080	4511660	And you can basically build your own via a YAML file.
4511660	4514080	And it's a scanner that'll go out and scan all your domains for
4514680	4517920	vulnerabilities, subdomain takeovers, all kinds of stuff.
4517920	4521400	So I really have been using Nuclei to great success lately.
4522320	4523280	And that's it.
4523280	4525700	That's all I got. So that's the Bug Hunters methodology.
4525960	4529680	Thanks for giving me the time today. I really appreciate it. Sorry I went over a little bit.
4529680	4532440	No worries. Thank you so much for the presentation
4532440	4534120	and amazing support.
4534780	4537840	Once again, from the bottom of my heart, thank you for supporting
4537840	4540120	DEF CON and the DEF CON Red Team Village.
4540120	4543480	And a reminder for everyone here,
4543480	4546360	please take a look at all the talks and activities that we have
4546360	4549460	in our website. We have the CTF, of course,
4549460	4552400	the Cyber Raid contest as well.
4552680	4555440	Tons and tons of activities throughout this weekend.
4555700	4558040	The link should be in the description at
4558040	4559980	the bottom of your stream. And all that stuff.