start	end	text
0	3220	I like it.
5180	6800	You see my botnet back there?
7240	9980	I got buddhists connected to my botnet.
10220	11400	I got hella buddhists.
11400	12560	Hella buddhists.
12620	13740	Buddhists.
13840	15140	Hella buddhists.
15140	16260	Buddhists.
16340	17720	Hella buddhists.
17820	18680	Buddhists.
18680	20360	I got buddhists connected to my botnet.
20360	22960	Hanging out on hack forums, best believe I got warrants.
22960	25480	Telling all who listened that my botnet's not boring.
25480	28220	Modified Mariah and I'm mixing in the minor.
28220	30480	There's no surprise getting wrecked by a minor.
30480	33020	Cover up his drop there, looking for me everywhere.
33020	35520	No I'm really not there, places I will not share.
35520	38100	Drop a couple backups, I know you won't find me.
38100	40500	Buy a couple more smart balls, won't you kindly?
40500	43060	Screening Captain Phillips, who the hell are you?
43060	45580	Wonder why your toaster's connected to RU.
45580	47960	Why your toaster's toasting, I'm using it for roasting.
47960	50680	Some kid got busy boasting, now his modem's smoking.
50680	53180	Keep the crypto flowing, I need my money now.
53180	55600	But I gotta go, cause my mom said study now.
55600	58040	I'm calling all the shots, and it's time for me to score.
58040	60800	I got a couple spots, hit me up on Discord.
60980	63180	Botnet, I got hella booters.
63600	65700	Botnet, take over your rooters.
66160	68400	Botnet, in your internet of things.
68640	70840	Botnet, man this shit hella stings.
71060	73480	Botnet, I got hella booters.
73640	75980	Botnet, take over your rooters.
76160	78540	Botnet, in your internet of things.
78720	81040	Botnet, man this shit hella stings.
81200	83620	Botnet, I got hella booters.
83800	86100	Botnet, take over your rooters.
88440	103600	Well, that was amazing. Well, thanks so much, Dave, for that amazing intro. I was not really
103600	110960	sure what to think when I just got a PM saying that there was a theme song for my talk. So
110960	116680	hi, everybody. Thanks for coming and hanging out. Let me... hold on. Making sure everybody can hear
116680	122440	me, right? I'll chat. You can hear me? Say hi if you can. Just making sure. I've got a terrible
122440	130200	microphone. All right. Cool, cool. Okay. So let's get going. So, yeah, this is my talk. I'll just
130200	136760	get right into it. So who am I? I'm Ned Spooky, senior reverse engineer at Redacted Company.
136760	143000	I primarily work on embedded devices, firmware, industrial control systems, and taking apart
143000	148880	proprietary network protocols. You know me online as either Ned Spooky or you.
149300	156800	And I contribute OSS tooling and other errata for 3rd Intel, RE, and offensive security.
157320	163560	So, all right, we'll do a little background on this. So why do this talk? So I know that there's
163640	169920	a lot of people that, you know, have seen IoT botnets, whether it be how you've been affected
169920	175300	by it or see people talking about them online. And, you know, I had done a bit of my own research
175300	181680	and I really kind of wanted to add, you know, a bit of perspective from my end on this. Because
181680	187500	IoT botnets are definitely still incredibly prevalent. You know, we are all affected by
187500	193280	them, whether we like it or not. You know, if your work slack is suddenly down because of somebody's
193280	199800	fight on Xbox Live or Minecraft, you know, this is affecting you in some way. But I don't think a
199800	203320	lot of people take them as seriously. A lot of people think it's like, you know, kiddy stuff,
203320	209200	like script kiddy stuff, which is unfortunate because it definitely is something that is,
209200	215480	you know, an issue that we all have to deal with. And so, I spent a good amount of time
215480	222120	collecting malware sources. So I started doing this in about 2018. And I had been collecting
222120	229140	and developing tools to analyze source code and analyze binaries, which I'll get into later.
229140	232420	I studied a lot of the commonly exploited vulnerabilities and wanted to know more
232420	237780	about, like, why they were so prevalent. Like, why can you have 4 million hacked routers
237780	245120	on your botnet? You know, it's insane to me. So I wanted to inform others about the impact of
245120	250960	their technology choices, specifically firmware devs and people who are also end consumers.
250960	256380	And I also wanted to propose some ideas on how to address these. And so, I also, this talk,
256380	262600	I should have given it about a year ago or so. It's just kind of been pushed under the back
262600	267080	burner a bit. But I tried to update it as much as I could. And I also had to kind of cut out
267080	271860	some of the parts, which I'll talk about as I go through the talk. So here's the outline real
271860	276120	quick. So we're going to go over IoT botnet history. We're going to go over the actual
276120	280580	botnet scene a little bit. Talk about the architecture of botnets and how they spread
280580	284920	and propagate. And then we're going to talk about the firmware vulnerabilities that enable them
284920	291020	and steps to move forward for vendors. So starting off just with IoT botnet history. So
291020	297780	what is an IoT botnet? So if you're here watching for IoT Village, you probably are aware of IoT
297780	303640	issues and botnets in general. But for those who haven't seen anything about them, they're
303640	308340	basically a network of hacked IoT devices that are basically internet connected devices like routers,
308340	314920	laptop boxes, webcams, your toaster. This little toaster on here has a little script on how to
314920	321100	scrape Shodan for Linksys routers. But so they're used primarily for DDoS and they're sometimes
321100	327220	used for crypto mining and also tunneling and proxying traffic. And so for this thing, I'm going
327220	331200	to... this talk is kind of more of a cultural as it is technical and I kind of want to be able to
332320	336840	kind of go through a lot of the confusing nomenclature that surrounds this because
336840	342020	there's a ton of it. People call botnets by a million different names, whether you're talking
342020	347700	to a researcher or talking to a person who develops them. There are a ton of different types. So let's
347700	352080	kind of go through a little bit of the history here. So IoT botnets as they're known right now
352080	359140	can be traced back to, I guess, 2014-ish when LizardSquad came out with the, I guess, the
359140	364920	botnet, the malware with the most names available, like any malware. It's been termed a bashlight.
364920	371240	It can also be called LulzBot, Torless, Lizkabob, LizardStressor, Ballpit, Gaffigat, and just a
371240	376000	bajillion other names. And so it was spread by exploiting Shellshock vulnerabilities when
376000	381020	Shellshock came out in BusyBox on a bunch of different devices. So there was... people were,
381020	384760	you know, scanning the entire internet for Shellshock vulnerabilities because they were
384760	391360	all over the place, but they were very, very common in a lot of IoT devices. And so when this
391360	394500	was actually happening, though, there were actually a lot of different botnets or bots that were being
394500	400320	distributed because there hadn't been as many sources as there are now. So you've heard of
400320	406420	KITAN, which is like an IRC C2-based botnet that was spread a lot, as well as like just Perl bots
406420	411660	that are just literally DDoS bots that are written in Perl. So the source code for this, though,
411660	417660	was leaked in 2015, and a lot of people started to work on it. And so collectively, it's hard to
417660	422740	choose one name for them, but collectively I would say that these would be categorized as QBot,
423520	429700	which is unrelated, again, to the KAKBot malware, which people call QBot. And so,
429700	434400	yeah, new devices that are still vulnerable to this exact same vulnerability appear online
434400	442160	like newly to this day. And so fast forward a couple years, so MIRA came out in 2016,
442160	449000	and so it was used in some famous DDoS attacks like the Dyne DDoS attacks, the ones on Brian
449000	454020	Krebs and some other people. But it was leaked shortly after some of the bigger DDoS attacks
454020	459860	happened, and people started to immediately use it because it was a lot more streamlined than the
459860	468060	previous versions of DDoS malware. A lot of the stuff was in, you know, really simple one-file
468060	475140	bots and servers, very, very basic stuff over telnet. So MIRA was a lot more streamlined.
475140	479460	It was very modular, so there's different files. It made it easier for you to plug in new exploits
479460	486360	into, and also made it easier for you to have access control for users that were coming on.
486680	492020	And so it also had a bit better code. It was definitely still not the best, but it's a lot
492020	499800	better than the previous code for LulzBot. It also had a SQL server on there, which made
499800	504820	it running the server a lot easier for them. And so it seems like everybody has a botnet
505380	511280	fork these days, since then. So there's other IoT botnets that are pretty major that have come out.
511480	516980	A big one that you may have heard of was Satori or FBot or Okiru, and it's a pretty well-known
516980	520820	MIRA fork that's a bit different from some of the other ones because a lot of them are kind of just
520820	528760	very copy-pasted stack overflow questions fit into some Golang and C code. But so this one here
528760	532860	had a bit... the person who was doing it definitely knew what they were doing a bit more than most
532860	539100	people, and that person actually just went to jail recently. We've also seen BrickerBot, which is the
540300	545720	botnet that would just basically infect and break IoT devices, and there's been a few iterations of
545720	550540	it. There was one in 2017, and then there was one recently, I think it was like a 13-year-old
550540	555640	kid or something like that that did it as well. A newer one, really interesting, is Kaiji, which is
555820	561360	a Golang-based cross-compiled SSH brute-forcer, and it actually installs a root kit or tries to
561360	565760	to establish persistence, which is really interesting. I'll get into that more later.
566360	569540	AxisR is another one that I've seen. I just threw in there because I didn't hear anybody talking
569540	575520	about that one, but it's more modular, still crappy. Then you have various Bitcoin miner
575520	580000	botnets that you may have seen. It's harder to do Bitcoin mining on an IoT device because they
580000	585560	don't have as much CPU power and no GPU, but they're still out there. And then I also did a
585560	591860	write-up on some Mirai variants that are targeting FPGAs and some really exotic architectures,
591860	597600	which I have a link in citations at the end here, or you can go on my website and see it.
598200	603320	So yeah, botnet activity growth, I mean, like similarly to Qbot, Mirai just started popping
603320	608700	up all over the place once it was leaked, and there became basically a huge marketplace for
608700	613620	people who are trying to sell spots on the botnet, right? There's reseller markets, affiliate
613620	619600	programs, and incentives for having it grow. And also booting itself, you know, DDoSing somebody,
619600	626600	their home router, became a really common thing for people to actually, you know, try and do
626600	630800	because it's just like a way to knock people offline, especially if you're mad at them
630800	635960	in Call of Duty or something. And so yeah, it's basically just like the thing that people
635960	641060	start to do. And so like, you know, though, as these things develop and grow, like a thousand monkeys
641060	645180	at a thousand terminals will eventually take out the intranet, and that's kind of what's been
645180	652640	happening. And so we'll get a little bit into the scene here. So the botnet scene at a glance,
653060	658300	there's entire communities that are dedicated specifically to just one botnet
658300	664780	or one botnet group, and they are usually talking on Discord, sometimes they're on forums or IRC.
665120	669800	There used to be definitely more IRC back in the day where people would have C2s connected to IRC,
669800	675220	but nowadays it's more Discord that people are talking. Advertising is done on literally
675220	680800	every single social media platform you can think of. I think somebody found Pinterest that had
680800	686280	somebody advertising botnets, but yeah, if you go on Instagram and just literally search botnet
686280	691520	or Qbot or botnet setup or YouTube, you will find somebody advertising their latest
692000	700120	slamming botnet. And so bootertime is generally sold people for DDoS or through a web panel
700780	705000	or through a telnet interface, but that's like the main thing people are trying to do is just
705000	710060	sell time on the botnet. And so you'll see here on the bottom, it might be a little small for
710060	716920	some of you, but there's just some advertisements for different botnets and also some videos on
716920	723320	how to boot people offline and how to do it using just an Android and best booters 2020. These all
723320	728900	have hundreds of thousands of views too, so these are people that are really going hard with the
728900	734680	advertising. So the sources, so I talked about the sources that have kind of been modified by people.
734880	741660	They're usually distributed as Zips or RARs or whatever and they are sold for about $500 to $300
742220	748200	USD from just what I've seen. So the authors will typically change very little of the code base,
748200	753580	usually just involves something simple, just changing the ASCII art or changing the variable
753580	759800	names, something like ctrl f and replace. Sometimes they might even add a new exploit,
759800	766080	which is always interesting, but exploits themselves to load bots are sometimes sold,
766080	770220	but a lot of them are literally like you can google any part of the script and you will find
770220	778020	the exploit db link of where they took it from. The ones that are sold though from exploit db or
778020	783700	metasploit modules are usually backdoored and it's really funny they just have like a base64 blob
783700	790560	that just like runs like import os and then just run this or import sys and run this or whatever.
790880	795780	And so when I was going through and finding a lot of these sources though, I would find that
795780	801020	when people would scam each other or rip off somebody or somebody had a fight with them,
801020	804920	they would leak each other's source code, which is great for threat intel people and
804920	810140	reverse engineers who want to figure out what's going on because they would just be like, oh hey,
810140	814940	here's this person, here's everything they've done, here's their botnet and here's their code
814940	821320	and you can just kind of scoop it up and take a look at it. And so selling spots, primary source
821320	828000	revenue, as I said, they're typically sold in weekly, monthly, or lifetime plans. You can see a
828000	833740	breakdown of plans over here. Pretty cheap too. The lifetime is always really funny to me because
833740	838700	it really just means for the duration of the bot's lifetime, botnet's lifetime, and sometimes that
838700	845880	does not last longer than the three days or a month depending on how bad their operation is.
846560	851480	Some more enterprising people, people who are a bit more advanced might use a web stressor and
851480	857820	they can sell access to that with users and everything for a web browser. There's been a few
857820	863360	big web stressors that have been taken down and some that are still up. Web stressor source leaked
863360	867800	volumes in the web stressor. There's so much surrounding that it adds a bit of abstraction
867800	873840	to it that makes it harder to manage. And then yeah, finally, some people act as resellers and
873840	882740	they get a cut of the sales over time. So who runs a botnet, right? IoT botnet operators,
883100	891020	based on what I've seen in the scene, I guess, they're usually pretty young, high school age,
891020	895540	sometimes college age. They're somewhat experienced with computers, but they're usually not
896320	902960	developers. They learn a lot through YouTube and through text files, which I have a collection of
902960	908920	in the GitHub that I'll explain in a little bit, which are just tutorials on how to set them up.
908940	917380	Basically, how to spin up a rel box and how to actually... hold on one second... how to actually
918080	924240	just do the basic things that compile with GCC. A lot of the times, though, they really have no
924240	929180	clue what they're doing. So you'll see people who are trying to get support for different botnets
929180	937840	and they're really confused about GCC or what access control is. But more sophisticated people
937840	943640	might have a web stressor or an API, like I said before. People will use cryptocurrency instead of
943640	947960	PayPal, which is very common for some reason, even though it's tied to your bank directly.
948240	952580	In some cases, though, people will also use botnets for additional purposes like
952580	957040	proxying traffic. So sometimes you'll see fly-by-night VPN operations that
957040	963280	might be doing something shady like, you know, routing their traffic through routers and that's
963280	972300	just their VPN somehow. So why run an IoT botnet? So, I mean, just as much as most malware is,
972300	975860	there's a lot of similar reasons, but there's a lot of stuff that comes with the fact that
975860	980640	there's a lot of younger kids involved in this. So they usually do it for either money,
980640	986040	you know, because people can earn money from the sale of botnet spots. A lot of
986040	990980	people do this for attention. People, you know, seek attention for stuff, even if it's not even
991140	996200	a DDoS, and it's just like a regular production outage. Some people might say, oh, yeah,
997980	1004700	my group, we DDoS these people, and, you know, we're gonna extort them for money. And then you
1004700	1008380	look at their status pages like, oh, yeah, sorry, we had a blip in, you know, updating this thing,
1008380	1014480	and we're back now, which is always awesome. Supply and demand. It's definitely people who
1014480	1020540	want, you know, to DDoS each other. And so that's, you know, definitely wanting to meet that demand
1020540	1026220	is something that, you know, it's good for any young entrepreneur. Revenge is also big. I see
1026320	1030800	a lot of people claiming that somebody, you know, docks them or DDoS them, and they want to get back
1030800	1036300	at them by getting their IP and booting them offline. And then people are also inspired a lot
1036300	1041420	by past attacks, because people have seen what actually happens if somebody DDoS and takes out,
1041420	1047260	you know, the internet. They want to be, you know, doing that. And also, it's incredibly easy
1047260	1053940	to set up an IoT botnet. So let's take a little bit of time to go over the architecture of
1053940	1061260	DDoS botnets. So as I said before, earlier botnets use standalone bot files and C2 files
1061260	1067920	that were just compiled with GCC or UCLibc for cross-compiling. They're very, very simple
1068620	1076520	to set up and deploy. Some of them for C2 itself, they used like IRC for command and control,
1076520	1083160	and they'd have IRC, like very bare bones IRC clients connected to their... or within their
1083160	1090740	bots. But Mirai modernized it, and they have actually a C2 protocol that is used, and they
1090740	1098840	have like a SQL backend for tracking bots and all that. So web stressors will use PHP and some
1098840	1105320	other, I guess, API stuff for managing the bots. But it's definitely evolved a lot more than it
1105320	1111980	used to be like five years ago, which is interesting to see. So the life cycle of a botnet
1111980	1119460	is usually very, very short. You don't see them for very long, not going to be over a month or two.
1120020	1125820	Basically, somebody will set up a C2 on like a lax VPS host, they'll scan for their own devices,
1125820	1131620	they'll get some bots to their botnet, they'll advertise their spots, and then use it. And then
1131620	1137000	the takedown goes one of two ways. Either somebody, like say bad package report, will tweet out
1137000	1147540	their botnet to the web host, and somebody will notice it and get it taken down.
1147540	1152300	Or somebody else's botnet will start kicking their bots from the system, and they won't be
1152300	1156700	able to keep up and they'll lose power. But then it'll just keep happening again. This is just the
1156700	1168880	same thing you see over and over again. Hold on one moment. Okay, that's cool. I didn't think it
1168880	1176580	was the water. So this inevitably leads to a king of the hill game for botnets. They're very
1176580	1180880	territorial. People are, you know, targeting one specific type of device with one specific
1180880	1185060	vulnerability that they've coded into their variants. And then when somebody else gets the
1185060	1191100	same idea, you know, they'll start hacking and doing things like that, getting their bots on
1191100	1196200	there. Anybody who touches the devices usually already has root access, but they might have
1196200	1201500	either like some weird file system, or there's no way to really reconfigure it, or they might not
1201500	1206360	know how to reconfigure the device to kick everybody else out. But basically, every bot
1206360	1211340	will only last as long as it can before somebody else takes its place. And also, there's really no
1211340	1217100	repercussions for this. So everyone's just kind of slamming on different IoT devices,
1217100	1222300	and pictured as an IoT operator, or botnet operator watching their bot count drop.
1223640	1229360	So evasion is definitely an interesting aspect of this. So there's a lot of very simplistic
1230260	1236760	evasion that you'll see here. This one up at the top here is somebody just renaming their process
1236760	1243300	to DropBear, which is, I guess it works, but it's also used by everybody. So then everybody will
1243300	1249280	just kill the DropBear process once they log on. But that's realistically, this is not to hide
1249280	1254680	from, you know, any sort of firewall or any sort of AV or anything. It's only really used to evade
1254680	1259520	other botnet operators. And so they'll do things like, you know, the process masking, they might
1259520	1264620	learn about a different area of the file system that they can put a bot in, they might hide a
1264620	1269960	backup bot and have like, potentially something like a cron job. It's always very, very primitive,
1269960	1276160	and like, very, very bespoke. And I actually had a whole code review section that could have
1276160	1281020	actually been an entire talk, but I had to cut it for time here. But there's a lot of very,
1281960	1288020	very strange ways that people try to do evasion, which I would love to talk about at another date.
1288920	1296060	And so bot killing, as I said before, people will do this, you'll take a look at this, if you can
1296060	1301060	see it on the side here, here's a, you know, an array that's just full of a ton of different bot
1301060	1305620	names. And every time they updated this botnet, which I have multiple versions of it, they would
1305620	1312140	add more and more of these, but you'll see they'll do things like iterate from one to however many,
1312140	1320080	try to kill every process that's called that, or every single version of this specific JackBind
1320080	1324800	MIPS or whatever, TwoFace. So people will like, be aware of different botnets that are operating
1324800	1328880	and what they name them, and then put them into their scripts. And it's like a cat and mouse game,
1328880	1331600	because nobody can fit everything in there, otherwise their binaries are going to be full
1331600	1336940	of strings that ultimately are going to get detected by people who are reverse engineering
1336940	1347020	the malware. And so, not some, not most, nearly all bots and C2s, I would say all, have really,
1347020	1352340	really silly vulnerabilities that make them incredibly easy to knock offline. And I don't
1352340	1357180	really see too many of these techniques really utilized or advertised by people, but in my next
1357180	1364180	slide, I'll show you something interesting, I guess. So here's my non-live demo for a C2 killer.
1364180	1369320	So this is something that I found. I definitely was not the first person to find this, but it was
1369320	1374480	part of my testing when I was testing out these different things when I was researching. It's
1374480	1381320	incredibly easy to kill, you know, Mirai C2s. This is a, you know, take your screenshots or whatever.
1381320	1386260	I already put this out on Twitter at some point. But yeah, this is, if you send this to either the
1386260	1394700	admin port or to the heartbeat protocol port, it just, it just segfaults the Mirai C2. And I've
1394700	1399100	never seen anybody fix this. This isn't every version of Mirai that I've seen. Some people
1399100	1403960	had claimed that it was fixed, but I've still, after reviewing every single one of them that I
1403960	1410280	could find, I've never seen that. And so, take a bit of a second, the last little bit on the
1410280	1415500	botnet scene here. You know, when I was going through and doing this work, I ended up creating
1415640	1420840	a tool to help me track all this stuff. And I put it out on GitHub. It hasn't been updated for a bit
1420840	1428400	because it just, I've gotten too many other projects to do. But it's, it's a static analysis
1428400	1435220	and classification tool for zip files and binaries and things like that. It just feeds it on to a big
1435220	1439720	Elasticsearch database. I definitely, I'm taking the next week off of work. So I'm going to take
1439720	1444420	some time to push my big update to this. But if you want to check it out, definitely do it. I have
1444420	1450180	some new things in API, different symbol hashes and key extraction techniques in there. But it's
1450180	1455800	just mainly for me for fast analysis because I was, you know, I had, it's basically feeding in
1455800	1463060	either new source code or new bot binaries into this and just tracking them. But it works for
1463060	1469460	other malware too. Also, if you want to, I have a Twitter that was deleted by Twitter for some
1469460	1474740	reason, that was called Threatland, but that was the name of the project that I used to track
1474740	1482580	all these sources. So I have like every Mirai and Qbot and other botnet, just even beyond IoT.
1482840	1488380	I tracked them all in a big repo called TLBots and have a few other repos if you want to check
1488380	1492700	them out for like fraud tools and stuff. But yeah, there's literally, clone that,
1492700	1497540	there's like a gigabyte worth of zip files of every malware source code that I could find.
1499460	1502940	Yeah, so now we're going to get into talking about vulnerabilities. And this is kind of,
1502940	1508860	this aspect of it is a bit more about like stuff for devs, because I wanted to be able to
1508860	1514560	give info for developers who are working on IoT devices to take all of what I just said there
1514560	1520380	and put it into context for their actual security architecture. So let me take another sip of water.
1528500	1534380	So okay, so we're here, we're peering into the void here. So here is, if you ever have gone on
1534380	1541000	Green Noise, they have a lot of tags for different either vulnerabilities themselves of the people
1541000	1548020	are scanning for, or just classes of, you know, malicious traffic. If you do a search just for
1548020	1554460	Mirai, you'll see here, it's very, very tiny, but there's four and a half million results for
1554460	1559800	unique devices that have been scanning with Mirai-like traffic. So that gives you a rough
1559800	1564520	example of how many people are, or how many devices are actually infected and actively
1564520	1570640	scanning. And then the other one is a showdown search for this hacked router help SOS had do
1570640	1577160	password thing, which there are still, this is, I think that that hack happened like four years ago,
1577160	1583760	and there's still 6,500 devices that have been hacked and have this hostname. So it's always
1583760	1588820	heartwarming to see, I guess. So what types of vulns are exploited by these botnets? So
1588820	1594540	it's always very basic stuff here. We're talking about weak auth and auth bypass. So there's either
1594540	1601880	admin admin as the credentials, or there's like a page that you can, you know, run OS commands on
1601880	1607500	that doesn't actually need a password to be, you know, interacted with. There's also command
1607500	1614620	injection like shell shock and other really silly command injection stuff. There's also a lot of
1614620	1620640	common exploits and specific services and libraries like the Realtek UPnP SDK, which had a
1620640	1628100	vuln that was like in everything. There's so many different devices. GoAhead webs and ThinkPHP
1628100	1633160	also have vulns that were in a lot of places, thousands and thousands of stuff were affected
1633160	1639940	by GoAhead. And so more rare though, you'll see actual shellcode and binary exploits,
1639940	1643300	which is always interesting to see because, you know, you'll have devices that, you know,
1643300	1650440	are using like the same base address and they can just do a shellcode exploit very, very easily.
1650740	1656900	But if they're not as common as, as you'd think, and I think it might be because people don't know
1656900	1662920	how to code a shellcode or how to inject shellcode, like with an NC, like when they're writing their
1662920	1668060	bots. So who knows, but you'll see them in bot loaders for sure. A lot of other vectors include
1668060	1673460	previously compromised devices. So like if you, people sell lists of compromised devices
1674100	1680000	for a specific category of devices, which there might, I don't actually know if there's any in my
1680000	1685720	repo, but I have seen a bunch of them where people are basically just passing those things around.
1686100	1690140	So we're looking at the most targeted devices here. So if you want to see, you know, what vulns
1690140	1695900	are most leveraged by these botnets, I have a command table or a table here of, basically,
1695900	1701820	I went through every source code that we could find. There's like several hundred unique source
1701820	1708480	codes. And these are the main ones that people are using. A lot of them here don't actually
1708480	1713340	have any like CVE or CPE or any vendor acknowledgement. So you can only really find
1713340	1719020	them by kind of looking up what the traffic is or what the command injection attempt was on,
1719020	1724440	you know, in your log files. Actually, more than half of these don't have any CVE at all.
1724440	1731380	The AVTech one, which I think is being used in the IoT CTF right now, doesn't have a CVE or
1731380	1736860	anything. And it's just a blog post that, you know, people have written about it. Same with like,
1736860	1742220	you know, some of these Netgear ones, the Netgear DGN 1000, that's a huge one that people have
1742220	1749800	exploited. I've never seen a CVE for, you know, the HNAP, Vacron, like Zyxel stuff, even actually
1749800	1755420	go-ahead websites have some. But here, these aren't even SSH or telnet brute force stuff,
1755420	1759920	this is like, like, a lot of the stuff here, like buffer overflow, or like command injection,
1759920	1764060	and some of these aren't even being tracked by anybody. So when a new exploit comes out,
1764060	1769140	though, bot scanners will really just immediately start trying to load bots with like whatever PFC
1769140	1777140	people have. And it's usually IoT bots, and it's very annoying. So the infections spill over from
1777140	1781700	that. So like these malware families are, you know, running on a super diverse array of
1781700	1786320	architectures, like there's every architecture you can think of has a Mirai variant for it at
1786320	1792240	this point. Because of cross compiling, but this means that this can affect other hosts that aren't
1792240	1798380	IoT. And so people will try to use and they'll try to get Mirai onto things like web servers
1798380	1805380	using like Drupal get in or Apache struts, or, you know, CouchDB or whatever thing is running,
1805380	1809920	they're going to try to do that to have that be the scanner as well. So these sort of infections
1809920	1815240	spillover is really common. And you'll see, sometimes like IoT botnets are using Drupal
1815240	1818820	get in and you're like, what router is running Drupal? It's because they're trying to get onto
1818820	1827080	everything. And so why are these devices so easy to exploit? And so we've talked, we were talking
1827080	1832360	about this in the last talk here about, you know, supply chain issues. And so there's,
1832360	1836740	it's very difficult to validate supply chain is a big one. There's vulnerable software and
1836740	1842260	libraries that people use, they might not be able to change or have the people to even, you know,
1842260	1848480	make the changes for it. Easy to guess default passwords is a huge one. Devices by default,
1848480	1853020	doing port forwarding, listening on the internet, giant list of phone devices are passed around,
1853020	1857560	which makes it even easier for people who don't know what they're doing to just start exploiting.
1857560	1862640	And then it all comes down to insufficient or non-existent security practices and development.
1863000	1867680	And so we're gonna get a little bit into firmware bones now and security practices. So
1868440	1875560	you'll see there was a awesome talk, I think two years ago, Shmoo Khan, about firmware bones by
1875560	1882920	CITL. And so there's a lot of stuff like vendor security practices on a binary level are like,
1882920	1887120	they're, you know, almost non-existent. And there's even regression analysis to show that
1887120	1892200	firmware is actually becoming worse and having more vulnerabilities introduced to them in a 15
1892200	1898280	year data set, which is insane to me. So you see here, here's like every vendor that they had
1898880	1903520	looked at. And you see anything that's closer to the edge here is going to have more of these
1904080	1910660	things like stack guards or non-executable stack or railroad, ASLR, things that are closer to the
1910660	1915940	edge are scoring higher. And actually that means that more binaries have these mitigations in place.
1915940	1922520	But if you can see, there's very, very few that actually have anything on the graph. And then
1922520	1928300	the ones that do, they only have like one or two, and there's very few. It's kind of sad. You want
1928300	1932940	this whole, all these things to be blue, all the way blue, and there's like lines of blue,
1932940	1941100	which is really disheartening here. And so why is firmware so difficult to maintain?
1941100	1945200	So there's so many reasons for it. And I used to have done firmware development before
1945820	1954020	for embedded devices. And even the experience that I had doing that is still... I could see
1954020	1958940	all the echoes of this throughout the process, right? Because re-architecting cost is a huge
1958940	1964800	thing. Cost is usually the biggest factor for why things aren't changing. But you can also be
1964800	1969240	locked into a vendor contract. You can be locked into like a middleware contract. So you can only
1969240	1977200	use drivers for this one piece of your kit, and you have to use it for a certain period of time.
1977200	1982160	You might have unsupported chips or hardware to work with. Another huge thing is outdated
1982160	1987100	toolchains. You might be using some toolchain from like 2005, and that's how you build everything in
1987100	1992900	2020. There's also a lot of things like hardware restraints, which sometimes you might not...
1992900	1997680	like your hardware itself might not support the SSL version that you need to. That was an issue
1997680	2003640	that I've had to deal with before trying to figure out how you can, you know, jerry-rig a new SSL
2003640	2010560	encryption scheme and support newer versions of TLS in firmware that's, you know, 15 years old.
2010740	2015160	Sometimes you also need to maintain backwards compatibility, which is a big thing to make
2015160	2019620	this stuff really... it makes it very, very hard. You have to include stuff that you might not want
2019620	2027180	to. A lot of stuff though is a lack of dependable updates for users to update their devices.
2027680	2032740	So even if you have, you know, all the other things in place here, it's sometimes people don't
2032740	2038720	have like a way to actually update the devices without some complicated process. Poor communication
2038720	2044040	channels even tell people about vulnerabilities is also a big thing. And vendors might not have any
2044040	2051020	sort of like channels for reporting bugs or telling people about bugs as well. And then,
2051020	2055940	you know, lack of modern security measures like secure boot or binary hardening we talked about
2055940	2061400	before or code signing are not going to really be in place. And it's hard to get those things back
2061400	2067220	into your, you know, pipeline if you have to do a bunch of testing and you only have a couple
2067220	2074160	people working on the thing. And so why do we see a lot of this older stuff working? So there's some
2074160	2081940	sometimes you'll actually see Qbots or Kytenbots or even Perlbots trying to exploit stuff in your
2081940	2089160	and, you know, if you download binary. And it's because the vulnerabilities are still there,
2089160	2095080	right? And so this is something that I had actually to steal an analogy from Mudge, right?
2095080	2102040	In mining, there's indicator minerals that can prove that there are other things that you,
2102580	2105940	what's it called? There's like a, say you're looking for like diamonds or something.
2107040	2113540	There's like indicator materials that prove that this might be there, the thing that you're
2113540	2122960	looking for might be there. And so the security vulnerabilities that we're seeing are showing
2122960	2129820	that there are not as many security practices that are being followed, which means that the
2129820	2133860	older vulnerabilities are still going to be able to work, right? So like we're seeing like,
2133860	2139660	you know, there's still command injection here in 2020, and then you can still run a Qbot or a
2139660	2146460	Perlbot on this device. This means that there's really not anything that's going into the actual
2146460	2153440	process of making the devices any more secure. And what's interesting is this is rare in other
2153440	2158740	classes of malware, like say for desktop computers, because there's no patch really that you can
2158740	2165140	apply to one specific device or whatever. So each time that there's a new vuln that comes out,
2165140	2169180	there's all these new devices that are added to the pool, but there's still all the routers from
2169180	2174360	2014 that had shell shock vulnerabilities in them, and DVRs that still have auth bypass in them,
2174360	2177840	and those are all just getting added to the pool. So people are just trying the same old
2177840	2182000	techniques and they're still getting the actual devices that they would have exploited, you know,
2182000	2189720	before. So it's kind of frustrating. So moving forward, this is the big thing for
2189720	2195480	vendors and people who are, you know, developers of firmware and embedded devices. So what can we
2195480	2202140	actually do to solve any of these problems here? So we can only really fix them by having better
2202140	2207640	development practices for security by meeting the developers and the vendors where they're at,
2207640	2212060	because we want them to be, you know, on top of their game actually doing,
2212060	2216940	you know, the work that we'd like them to put in so that our, you know, toaster isn't, you know,
2216940	2223520	DDoSing somebody because of some Mirai variant that's run by, like, a 14-year-old kid.
2224420	2228700	So we have to, like, actually talk to them, talk to vendors the way that they...
2228700	2233680	things that they already know in their... the way that they're already developing things.
2233740	2237260	So for vendors, I guess my big advice here is to invest in developer training
2237260	2240820	and to establish best practices and create security testing pipelines
2241420	2245320	and encourage researchers to actually find bones and disclose them properly.
2245320	2250580	We can mitigate some of the existing bones by encouraging safer use of IoT devices,
2250580	2254680	but it only works so much because... imagine trying to explain to your parents
2254680	2260160	how to, you know, turn off port forwarding on their... on their router, right? Like,
2260160	2263700	they're not really going to understand it in the way that you might... I mean, they might,
2263700	2269260	but they... you know, it's sometimes hard to get end users to actually follow your guidelines at
2269260	2275320	all. They might not even be able to be aware of it. But that's one specific way that we can
2275320	2280300	mitigate that. Establishing best practices, though, is a thing I wanted to highlight for a second
2280300	2285860	here. So, like, auditing your development cycle itself. It definitely depends on what you're
2285860	2290200	building, and you have to tailor it to that and be able to audit and say, hey, yeah, we are using
2290200	2298460	C, we are using, you know, this tool chain, we're using either GCC, or, you know, UC Lib C, or
2298460	2305080	whatever, to, you know, develop our firmware. But there are there are best practices for these
2305080	2311900	things. So, OWASP, something that I had when I was doing firmware stuff, I had made developers,
2311900	2317780	I had them look at these cheat sheets for OWASP on, like, toolchain hardening on, like,
2317780	2324280	input sanitization for web apps, and, you know, how to do other things, you know, the best
2324280	2328960	practices that there are. And there's tons of different ones like this. I just post to OWASP
2328960	2334680	because it's, you know, very accessible for a lot of people, and it's free. Another big thing is
2334680	2339620	CIS benchmarks. You know, there's, depending on what you're building, there are benchmarks for
2339620	2344700	security that you can follow, which are super duper useful. You know, you can even automate
2344700	2349600	that. I had done, like, Ansible CIS benchmarks before. You can build those into toolchains pretty
2349600	2355100	easily. And then also, if you really need to, hire a consultant to come in and do all this work with
2355100	2361060	you and work through it with your team. That's definitely a huge thing for vendors. Vulnerable
2361060	2365280	disclosure, though, is probably my favorite one to talk about and the biggest one. So, when you
2365280	2371100	do find people that are actually poking at your stuff, allow them to disclose vulnerabilities.
2371100	2376980	Please, if you are a vendor and you're listening, there are ways now. It's 2020. You can't have a
2376980	2381540	VDP. You can't have a bug bounty. You have to go through the channel, the proper channels, and make
2381540	2387740	sure it's right for you. But there are a ton of resources. Disclose.io has, like, really good
2387740	2392840	legal language and other things, resources for vendors. But yeah, people who do IRT research
2392840	2398400	sometimes either get no response or they get, you know, a summons, you know, get sued for something.
2399040	2404360	Establish security contact, though, and listen to email. There's security.txt is a really easy
2404360	2409020	way on your vendor website to just have a email that somebody who has an issue or a vulnerability
2409020	2413780	can talk to and not feel like they're, you know, trying to chase you down. Because, like, how many
2413780	2418180	times do you see people on Twitter going, hey, does anybody have a vendor contact for, like, this
2418180	2424220	company? And, like, nobody responds. It's like, if we have to go on Twitter to ask about this, not only
2424220	2429760	does it draw more attention to your, you know, the vulnerability, but it also makes you look bad.
2429760	2434840	So definitely, you know, keep up with that stuff. Work with researchers, too, because people who
2434840	2438860	are bringing stuff up to you, they want to help you. If somebody wants to just use your device
2438860	2442880	or crypto mining or, you know, to DDoS, you know, some kid on Minecraft, they're not going to tell
2442880	2447460	you about it. So if you have a researcher that's here and talking to you, they want to help you.
2447460	2451540	You should definitely, you know, heed their advice. And have some open channel with your
2451540	2455380	customers, too, to have the word out about vulnerabilities. Like, if you either you do an
2455380	2459960	internal thing or you submit to CVEs and then you post them on Twitter, whatever you do, just have
2459960	2465960	something so people can know, update their devices. And ultimately, all these are elements of a
2465960	2471640	vulnerability disclosure program. So if you put this all together, you have, you know, the baseline
2471640	2478520	for what you need to actually have one, which is awesome. And so I've done a little quick
2478520	2483640	question out there on Twitter, which you can see, I have the link here. But these are some
2483640	2489200	community suggestions for what vendors can do. You know, everything from automatic updates to,
2489200	2495920	I like the make a security in named persons problem. So like having say, like, I don't know,
2495920	2501180	like Sherry has to deal with the, you know, firmware bones that come in. So, you know, talk
2501180	2506940	to her if there's an issue. You know, other things like minimizing attack surfaces, code signing,
2506940	2512280	all of this stuff is going to be part of other, you know, best practices that you're going to
2512280	2517380	have to implement. But these are all, you know, elements of things that might be good for you to
2517380	2523480	consider moving forward. Default settings definitely should be sane, and with security in mind. And
2523480	2530140	also, don't reinvent the wheel. So final thoughts here, I got two minutes left, just about. So we
2530140	2534620	want to make it less easy for people to run botnets, right? Overall, the supply is already
2534620	2540360	there. The demand is great. Everything is set up. People can off the shelf, you know, get several
2540360	2546500	thousand bots on botnet in an afternoon. Botnets are definitely getting a lot smarter. People are
2546500	2550800	using the messy landscape of this to take control of it. Like if you see with the Kaiji botnet,
2550800	2555980	which is definitely a lot more advanced than previous stuff. People are going to be using this
2555980	2562980	for more nefarious purposes. And because there's so much going on in this space, it's very hard
2562980	2569800	to pick out who is either nation state trying to get access into your router, or just some
2569800	2574000	random kid who has no idea what they're doing. And new devices and architectures are always
2574000	2579600	being targeted. As I said, FPGAs, there's tons of new stuff. You can read that analysis that I wrote.
2580180	2583900	If you don't act soon, like the new products that you put out are already going to be
2583900	2591080	dead on arrival and exploitable once they come out. And so yeah, the Q&A for this is
2591080	2594200	going to be done in the DEF CON Discord, as you might have seen in the Twitch chat. So if you have
2594200	2600560	questions, or you can always hit me up on Twitter, at netspooky. My DMs are open there.
2600860	2605920	I got some shout outs real quick. Shout out to the Safari Zone crew, which is the people that
2605920	2610640	have always been there to look for weird stuff on the internet. Hopefully we'll have a zine coming
2610640	2615540	out soon. Threatland, everybody who helped out with that project to collect sources. And of
2615540	2620440	course, the entire community of Thug Crowd. A special shout out to Hermit for helping me go
2620440	2626660	through so much of this and be such an awesome person for, you know, looking at logs and other
2626660	2634460	weird stuff. I like being able to tell me about a lot of interesting things that she has found.
2634580	2639720	Andrew Morris, thanks so much for letting me use your data set too before it was even publicly
2639720	2645660	available. Thanks to Mudge for coming in hot with some hot takes for me when I was building this
2645660	2651060	talk. Check out Ilya's IoT Village talk on emulating IoT devices and malware, because we
2651060	2656440	actually did a lot of that when I was writing this talk. And then thanks also to Dave for that theme
2656440	2660860	song. So I'll have slides out. I'll tweet them out. Just follow me on Twitter and you'll see.
2661220	2664700	I have citations if you want to read them. But yeah, thanks, everybody.