I like it.
You see my botnet back there?
I got buddhists connected to my botnet.
I got hella buddhists.
Hella buddhists.
Buddhists.
Hella buddhists.
Buddhists.
Hella buddhists.
Buddhists.
I got buddhists connected to my botnet.
Hanging out on hack forums, best believe I got warrants.
Telling all who listened that my botnet's not boring.
Modified Mariah and I'm mixing in the minor.
There's no surprise getting wrecked by a minor.
Cover up his drop there, looking for me everywhere.
No I'm really not there, places I will not share.
Drop a couple backups, I know you won't find me.
Buy a couple more smart balls, won't you kindly?
Screening Captain Phillips, who the hell are you?
Wonder why your toaster's connected to RU.
Why your toaster's toasting, I'm using it for roasting.
Some kid got busy boasting, now his modem's smoking.
Keep the crypto flowing, I need my money now.
But I gotta go, cause my mom said study now.
I'm calling all the shots, and it's time for me to score.
I got a couple spots, hit me up on Discord.
Botnet, I got hella booters.
Botnet, take over your rooters.
Botnet, in your internet of things.
Botnet, man this shit hella stings.
Botnet, I got hella booters.
Botnet, take over your rooters.
Botnet, in your internet of things.
Botnet, man this shit hella stings.
Botnet, I got hella booters.
Botnet, take over your rooters.
Well, that was amazing. Well, thanks so much, Dave, for that amazing intro. I was not really
sure what to think when I just got a PM saying that there was a theme song for my talk. So
hi, everybody. Thanks for coming and hanging out. Let me... hold on. Making sure everybody can hear
me, right? I'll chat. You can hear me? Say hi if you can. Just making sure. I've got a terrible
microphone. All right. Cool, cool. Okay. So let's get going. So, yeah, this is my talk. I'll just
get right into it. So who am I? I'm Ned Spooky, senior reverse engineer at Redacted Company.
I primarily work on embedded devices, firmware, industrial control systems, and taking apart
proprietary network protocols. You know me online as either Ned Spooky or you.
And I contribute OSS tooling and other errata for 3rd Intel, RE, and offensive security.
So, all right, we'll do a little background on this. So why do this talk? So I know that there's
a lot of people that, you know, have seen IoT botnets, whether it be how you've been affected
by it or see people talking about them online. And, you know, I had done a bit of my own research
and I really kind of wanted to add, you know, a bit of perspective from my end on this. Because
IoT botnets are definitely still incredibly prevalent. You know, we are all affected by
them, whether we like it or not. You know, if your work slack is suddenly down because of somebody's
fight on Xbox Live or Minecraft, you know, this is affecting you in some way. But I don't think a
lot of people take them as seriously. A lot of people think it's like, you know, kiddy stuff,
like script kiddy stuff, which is unfortunate because it definitely is something that is,
you know, an issue that we all have to deal with. And so, I spent a good amount of time
collecting malware sources. So I started doing this in about 2018. And I had been collecting
and developing tools to analyze source code and analyze binaries, which I'll get into later.
I studied a lot of the commonly exploited vulnerabilities and wanted to know more
about, like, why they were so prevalent. Like, why can you have 4 million hacked routers
on your botnet? You know, it's insane to me. So I wanted to inform others about the impact of
their technology choices, specifically firmware devs and people who are also end consumers.
And I also wanted to propose some ideas on how to address these. And so, I also, this talk,
I should have given it about a year ago or so. It's just kind of been pushed under the back
burner a bit. But I tried to update it as much as I could. And I also had to kind of cut out
some of the parts, which I'll talk about as I go through the talk. So here's the outline real
quick. So we're going to go over IoT botnet history. We're going to go over the actual
botnet scene a little bit. Talk about the architecture of botnets and how they spread
and propagate. And then we're going to talk about the firmware vulnerabilities that enable them
and steps to move forward for vendors. So starting off just with IoT botnet history. So
what is an IoT botnet? So if you're here watching for IoT Village, you probably are aware of IoT
issues and botnets in general. But for those who haven't seen anything about them, they're
basically a network of hacked IoT devices that are basically internet connected devices like routers,
laptop boxes, webcams, your toaster. This little toaster on here has a little script on how to
scrape Shodan for Linksys routers. But so they're used primarily for DDoS and they're sometimes
used for crypto mining and also tunneling and proxying traffic. And so for this thing, I'm going
to... this talk is kind of more of a cultural as it is technical and I kind of want to be able to
kind of go through a lot of the confusing nomenclature that surrounds this because
there's a ton of it. People call botnets by a million different names, whether you're talking
to a researcher or talking to a person who develops them. There are a ton of different types. So let's
kind of go through a little bit of the history here. So IoT botnets as they're known right now
can be traced back to, I guess, 2014-ish when LizardSquad came out with the, I guess, the
botnet, the malware with the most names available, like any malware. It's been termed a bashlight.
It can also be called LulzBot, Torless, Lizkabob, LizardStressor, Ballpit, Gaffigat, and just a
bajillion other names. And so it was spread by exploiting Shellshock vulnerabilities when
Shellshock came out in BusyBox on a bunch of different devices. So there was... people were,
you know, scanning the entire internet for Shellshock vulnerabilities because they were
all over the place, but they were very, very common in a lot of IoT devices. And so when this
was actually happening, though, there were actually a lot of different botnets or bots that were being
distributed because there hadn't been as many sources as there are now. So you've heard of
KITAN, which is like an IRC C2-based botnet that was spread a lot, as well as like just Perl bots
that are just literally DDoS bots that are written in Perl. So the source code for this, though,
was leaked in 2015, and a lot of people started to work on it. And so collectively, it's hard to
choose one name for them, but collectively I would say that these would be categorized as QBot,
which is unrelated, again, to the KAKBot malware, which people call QBot. And so,
yeah, new devices that are still vulnerable to this exact same vulnerability appear online
like newly to this day. And so fast forward a couple years, so MIRA came out in 2016,
and so it was used in some famous DDoS attacks like the Dyne DDoS attacks, the ones on Brian
Krebs and some other people. But it was leaked shortly after some of the bigger DDoS attacks
happened, and people started to immediately use it because it was a lot more streamlined than the
previous versions of DDoS malware. A lot of the stuff was in, you know, really simple one-file
bots and servers, very, very basic stuff over telnet. So MIRA was a lot more streamlined.
It was very modular, so there's different files. It made it easier for you to plug in new exploits
into, and also made it easier for you to have access control for users that were coming on.
And so it also had a bit better code. It was definitely still not the best, but it's a lot
better than the previous code for LulzBot. It also had a SQL server on there, which made
it running the server a lot easier for them. And so it seems like everybody has a botnet
fork these days, since then. So there's other IoT botnets that are pretty major that have come out.
A big one that you may have heard of was Satori or FBot or Okiru, and it's a pretty well-known
MIRA fork that's a bit different from some of the other ones because a lot of them are kind of just
very copy-pasted stack overflow questions fit into some Golang and C code. But so this one here
had a bit... the person who was doing it definitely knew what they were doing a bit more than most
people, and that person actually just went to jail recently. We've also seen BrickerBot, which is the
botnet that would just basically infect and break IoT devices, and there's been a few iterations of
it. There was one in 2017, and then there was one recently, I think it was like a 13-year-old
kid or something like that that did it as well. A newer one, really interesting, is Kaiji, which is
a Golang-based cross-compiled SSH brute-forcer, and it actually installs a root kit or tries to
to establish persistence, which is really interesting. I'll get into that more later.
AxisR is another one that I've seen. I just threw in there because I didn't hear anybody talking
about that one, but it's more modular, still crappy. Then you have various Bitcoin miner
botnets that you may have seen. It's harder to do Bitcoin mining on an IoT device because they
don't have as much CPU power and no GPU, but they're still out there. And then I also did a
write-up on some Mirai variants that are targeting FPGAs and some really exotic architectures,
which I have a link in citations at the end here, or you can go on my website and see it.
So yeah, botnet activity growth, I mean, like similarly to Qbot, Mirai just started popping
up all over the place once it was leaked, and there became basically a huge marketplace for
people who are trying to sell spots on the botnet, right? There's reseller markets, affiliate
programs, and incentives for having it grow. And also booting itself, you know, DDoSing somebody,
their home router, became a really common thing for people to actually, you know, try and do
because it's just like a way to knock people offline, especially if you're mad at them
in Call of Duty or something. And so yeah, it's basically just like the thing that people
start to do. And so like, you know, though, as these things develop and grow, like a thousand monkeys
at a thousand terminals will eventually take out the intranet, and that's kind of what's been
happening. And so we'll get a little bit into the scene here. So the botnet scene at a glance,
there's entire communities that are dedicated specifically to just one botnet
or one botnet group, and they are usually talking on Discord, sometimes they're on forums or IRC.
There used to be definitely more IRC back in the day where people would have C2s connected to IRC,
but nowadays it's more Discord that people are talking. Advertising is done on literally
every single social media platform you can think of. I think somebody found Pinterest that had
somebody advertising botnets, but yeah, if you go on Instagram and just literally search botnet
or Qbot or botnet setup or YouTube, you will find somebody advertising their latest
slamming botnet. And so bootertime is generally sold people for DDoS or through a web panel
or through a telnet interface, but that's like the main thing people are trying to do is just
sell time on the botnet. And so you'll see here on the bottom, it might be a little small for
some of you, but there's just some advertisements for different botnets and also some videos on
how to boot people offline and how to do it using just an Android and best booters 2020. These all
have hundreds of thousands of views too, so these are people that are really going hard with the
advertising. So the sources, so I talked about the sources that have kind of been modified by people.
They're usually distributed as Zips or RARs or whatever and they are sold for about $500 to $300
USD from just what I've seen. So the authors will typically change very little of the code base,
usually just involves something simple, just changing the ASCII art or changing the variable
names, something like ctrl f and replace. Sometimes they might even add a new exploit,
which is always interesting, but exploits themselves to load bots are sometimes sold,
but a lot of them are literally like you can google any part of the script and you will find
the exploit db link of where they took it from. The ones that are sold though from exploit db or
metasploit modules are usually backdoored and it's really funny they just have like a base64 blob
that just like runs like import os and then just run this or import sys and run this or whatever.
And so when I was going through and finding a lot of these sources though, I would find that
when people would scam each other or rip off somebody or somebody had a fight with them,
they would leak each other's source code, which is great for threat intel people and
reverse engineers who want to figure out what's going on because they would just be like, oh hey,
here's this person, here's everything they've done, here's their botnet and here's their code
and you can just kind of scoop it up and take a look at it. And so selling spots, primary source
revenue, as I said, they're typically sold in weekly, monthly, or lifetime plans. You can see a
breakdown of plans over here. Pretty cheap too. The lifetime is always really funny to me because
it really just means for the duration of the bot's lifetime, botnet's lifetime, and sometimes that
does not last longer than the three days or a month depending on how bad their operation is.
Some more enterprising people, people who are a bit more advanced might use a web stressor and
they can sell access to that with users and everything for a web browser. There's been a few
big web stressors that have been taken down and some that are still up. Web stressor source leaked
volumes in the web stressor. There's so much surrounding that it adds a bit of abstraction
to it that makes it harder to manage. And then yeah, finally, some people act as resellers and
they get a cut of the sales over time. So who runs a botnet, right? IoT botnet operators,
based on what I've seen in the scene, I guess, they're usually pretty young, high school age,
sometimes college age. They're somewhat experienced with computers, but they're usually not
developers. They learn a lot through YouTube and through text files, which I have a collection of
in the GitHub that I'll explain in a little bit, which are just tutorials on how to set them up.
Basically, how to spin up a rel box and how to actually... hold on one second... how to actually
just do the basic things that compile with GCC. A lot of the times, though, they really have no
clue what they're doing. So you'll see people who are trying to get support for different botnets
and they're really confused about GCC or what access control is. But more sophisticated people
might have a web stressor or an API, like I said before. People will use cryptocurrency instead of
PayPal, which is very common for some reason, even though it's tied to your bank directly.
In some cases, though, people will also use botnets for additional purposes like
proxying traffic. So sometimes you'll see fly-by-night VPN operations that
might be doing something shady like, you know, routing their traffic through routers and that's
just their VPN somehow. So why run an IoT botnet? So, I mean, just as much as most malware is,
there's a lot of similar reasons, but there's a lot of stuff that comes with the fact that
there's a lot of younger kids involved in this. So they usually do it for either money,
you know, because people can earn money from the sale of botnet spots. A lot of
people do this for attention. People, you know, seek attention for stuff, even if it's not even
a DDoS, and it's just like a regular production outage. Some people might say, oh, yeah,
my group, we DDoS these people, and, you know, we're gonna extort them for money. And then you
look at their status pages like, oh, yeah, sorry, we had a blip in, you know, updating this thing,
and we're back now, which is always awesome. Supply and demand. It's definitely people who
want, you know, to DDoS each other. And so that's, you know, definitely wanting to meet that demand
is something that, you know, it's good for any young entrepreneur. Revenge is also big. I see
a lot of people claiming that somebody, you know, docks them or DDoS them, and they want to get back
at them by getting their IP and booting them offline. And then people are also inspired a lot
by past attacks, because people have seen what actually happens if somebody DDoS and takes out,
you know, the internet. They want to be, you know, doing that. And also, it's incredibly easy
to set up an IoT botnet. So let's take a little bit of time to go over the architecture of
DDoS botnets. So as I said before, earlier botnets use standalone bot files and C2 files
that were just compiled with GCC or UCLibc for cross-compiling. They're very, very simple
to set up and deploy. Some of them for C2 itself, they used like IRC for command and control,
and they'd have IRC, like very bare bones IRC clients connected to their... or within their
bots. But Mirai modernized it, and they have actually a C2 protocol that is used, and they
have like a SQL backend for tracking bots and all that. So web stressors will use PHP and some
other, I guess, API stuff for managing the bots. But it's definitely evolved a lot more than it
used to be like five years ago, which is interesting to see. So the life cycle of a botnet
is usually very, very short. You don't see them for very long, not going to be over a month or two.
Basically, somebody will set up a C2 on like a lax VPS host, they'll scan for their own devices,
they'll get some bots to their botnet, they'll advertise their spots, and then use it. And then
the takedown goes one of two ways. Either somebody, like say bad package report, will tweet out
their botnet to the web host, and somebody will notice it and get it taken down.
Or somebody else's botnet will start kicking their bots from the system, and they won't be
able to keep up and they'll lose power. But then it'll just keep happening again. This is just the
same thing you see over and over again. Hold on one moment. Okay, that's cool. I didn't think it
was the water. So this inevitably leads to a king of the hill game for botnets. They're very
territorial. People are, you know, targeting one specific type of device with one specific
vulnerability that they've coded into their variants. And then when somebody else gets the
same idea, you know, they'll start hacking and doing things like that, getting their bots on
there. Anybody who touches the devices usually already has root access, but they might have
either like some weird file system, or there's no way to really reconfigure it, or they might not
know how to reconfigure the device to kick everybody else out. But basically, every bot
will only last as long as it can before somebody else takes its place. And also, there's really no
repercussions for this. So everyone's just kind of slamming on different IoT devices,
and pictured as an IoT operator, or botnet operator watching their bot count drop.
So evasion is definitely an interesting aspect of this. So there's a lot of very simplistic
evasion that you'll see here. This one up at the top here is somebody just renaming their process
to DropBear, which is, I guess it works, but it's also used by everybody. So then everybody will
just kill the DropBear process once they log on. But that's realistically, this is not to hide
from, you know, any sort of firewall or any sort of AV or anything. It's only really used to evade
other botnet operators. And so they'll do things like, you know, the process masking, they might
learn about a different area of the file system that they can put a bot in, they might hide a
backup bot and have like, potentially something like a cron job. It's always very, very primitive,
and like, very, very bespoke. And I actually had a whole code review section that could have
actually been an entire talk, but I had to cut it for time here. But there's a lot of very,
very strange ways that people try to do evasion, which I would love to talk about at another date.
And so bot killing, as I said before, people will do this, you'll take a look at this, if you can
see it on the side here, here's a, you know, an array that's just full of a ton of different bot
names. And every time they updated this botnet, which I have multiple versions of it, they would
add more and more of these, but you'll see they'll do things like iterate from one to however many,
try to kill every process that's called that, or every single version of this specific JackBind
MIPS or whatever, TwoFace. So people will like, be aware of different botnets that are operating
and what they name them, and then put them into their scripts. And it's like a cat and mouse game,
because nobody can fit everything in there, otherwise their binaries are going to be full
of strings that ultimately are going to get detected by people who are reverse engineering
the malware. And so, not some, not most, nearly all bots and C2s, I would say all, have really,
really silly vulnerabilities that make them incredibly easy to knock offline. And I don't
really see too many of these techniques really utilized or advertised by people, but in my next
slide, I'll show you something interesting, I guess. So here's my non-live demo for a C2 killer.
So this is something that I found. I definitely was not the first person to find this, but it was
part of my testing when I was testing out these different things when I was researching. It's
incredibly easy to kill, you know, Mirai C2s. This is a, you know, take your screenshots or whatever.
I already put this out on Twitter at some point. But yeah, this is, if you send this to either the
admin port or to the heartbeat protocol port, it just, it just segfaults the Mirai C2. And I've
never seen anybody fix this. This isn't every version of Mirai that I've seen. Some people
had claimed that it was fixed, but I've still, after reviewing every single one of them that I
could find, I've never seen that. And so, take a bit of a second, the last little bit on the
botnet scene here. You know, when I was going through and doing this work, I ended up creating
a tool to help me track all this stuff. And I put it out on GitHub. It hasn't been updated for a bit
because it just, I've gotten too many other projects to do. But it's, it's a static analysis
and classification tool for zip files and binaries and things like that. It just feeds it on to a big
Elasticsearch database. I definitely, I'm taking the next week off of work. So I'm going to take
some time to push my big update to this. But if you want to check it out, definitely do it. I have
some new things in API, different symbol hashes and key extraction techniques in there. But it's
just mainly for me for fast analysis because I was, you know, I had, it's basically feeding in
either new source code or new bot binaries into this and just tracking them. But it works for
other malware too. Also, if you want to, I have a Twitter that was deleted by Twitter for some
reason, that was called Threatland, but that was the name of the project that I used to track
all these sources. So I have like every Mirai and Qbot and other botnet, just even beyond IoT.
I tracked them all in a big repo called TLBots and have a few other repos if you want to check
them out for like fraud tools and stuff. But yeah, there's literally, clone that,
there's like a gigabyte worth of zip files of every malware source code that I could find.
Yeah, so now we're going to get into talking about vulnerabilities. And this is kind of,
this aspect of it is a bit more about like stuff for devs, because I wanted to be able to
give info for developers who are working on IoT devices to take all of what I just said there
and put it into context for their actual security architecture. So let me take another sip of water.
So okay, so we're here, we're peering into the void here. So here is, if you ever have gone on
Green Noise, they have a lot of tags for different either vulnerabilities themselves of the people
are scanning for, or just classes of, you know, malicious traffic. If you do a search just for
Mirai, you'll see here, it's very, very tiny, but there's four and a half million results for
unique devices that have been scanning with Mirai-like traffic. So that gives you a rough
example of how many people are, or how many devices are actually infected and actively
scanning. And then the other one is a showdown search for this hacked router help SOS had do
password thing, which there are still, this is, I think that that hack happened like four years ago,
and there's still 6,500 devices that have been hacked and have this hostname. So it's always
heartwarming to see, I guess. So what types of vulns are exploited by these botnets? So
it's always very basic stuff here. We're talking about weak auth and auth bypass. So there's either
admin admin as the credentials, or there's like a page that you can, you know, run OS commands on
that doesn't actually need a password to be, you know, interacted with. There's also command
injection like shell shock and other really silly command injection stuff. There's also a lot of
common exploits and specific services and libraries like the Realtek UPnP SDK, which had a
vuln that was like in everything. There's so many different devices. GoAhead webs and ThinkPHP
also have vulns that were in a lot of places, thousands and thousands of stuff were affected
by GoAhead. And so more rare though, you'll see actual shellcode and binary exploits,
which is always interesting to see because, you know, you'll have devices that, you know,
are using like the same base address and they can just do a shellcode exploit very, very easily.
But if they're not as common as, as you'd think, and I think it might be because people don't know
how to code a shellcode or how to inject shellcode, like with an NC, like when they're writing their
bots. So who knows, but you'll see them in bot loaders for sure. A lot of other vectors include
previously compromised devices. So like if you, people sell lists of compromised devices
for a specific category of devices, which there might, I don't actually know if there's any in my
repo, but I have seen a bunch of them where people are basically just passing those things around.
So we're looking at the most targeted devices here. So if you want to see, you know, what vulns
are most leveraged by these botnets, I have a command table or a table here of, basically,
I went through every source code that we could find. There's like several hundred unique source
codes. And these are the main ones that people are using. A lot of them here don't actually
have any like CVE or CPE or any vendor acknowledgement. So you can only really find
them by kind of looking up what the traffic is or what the command injection attempt was on,
you know, in your log files. Actually, more than half of these don't have any CVE at all.
The AVTech one, which I think is being used in the IoT CTF right now, doesn't have a CVE or
anything. And it's just a blog post that, you know, people have written about it. Same with like,
you know, some of these Netgear ones, the Netgear DGN 1000, that's a huge one that people have
exploited. I've never seen a CVE for, you know, the HNAP, Vacron, like Zyxel stuff, even actually
go-ahead websites have some. But here, these aren't even SSH or telnet brute force stuff,
this is like, like, a lot of the stuff here, like buffer overflow, or like command injection,
and some of these aren't even being tracked by anybody. So when a new exploit comes out,
though, bot scanners will really just immediately start trying to load bots with like whatever PFC
people have. And it's usually IoT bots, and it's very annoying. So the infections spill over from
that. So like these malware families are, you know, running on a super diverse array of
architectures, like there's every architecture you can think of has a Mirai variant for it at
this point. Because of cross compiling, but this means that this can affect other hosts that aren't
IoT. And so people will try to use and they'll try to get Mirai onto things like web servers
using like Drupal get in or Apache struts, or, you know, CouchDB or whatever thing is running,
they're going to try to do that to have that be the scanner as well. So these sort of infections
spillover is really common. And you'll see, sometimes like IoT botnets are using Drupal
get in and you're like, what router is running Drupal? It's because they're trying to get onto
everything. And so why are these devices so easy to exploit? And so we've talked, we were talking
about this in the last talk here about, you know, supply chain issues. And so there's,
it's very difficult to validate supply chain is a big one. There's vulnerable software and
libraries that people use, they might not be able to change or have the people to even, you know,
make the changes for it. Easy to guess default passwords is a huge one. Devices by default,
doing port forwarding, listening on the internet, giant list of phone devices are passed around,
which makes it even easier for people who don't know what they're doing to just start exploiting.
And then it all comes down to insufficient or non-existent security practices and development.
And so we're gonna get a little bit into firmware bones now and security practices. So
you'll see there was a awesome talk, I think two years ago, Shmoo Khan, about firmware bones by
CITL. And so there's a lot of stuff like vendor security practices on a binary level are like,
they're, you know, almost non-existent. And there's even regression analysis to show that
firmware is actually becoming worse and having more vulnerabilities introduced to them in a 15
year data set, which is insane to me. So you see here, here's like every vendor that they had
looked at. And you see anything that's closer to the edge here is going to have more of these
things like stack guards or non-executable stack or railroad, ASLR, things that are closer to the
edge are scoring higher. And actually that means that more binaries have these mitigations in place.
But if you can see, there's very, very few that actually have anything on the graph. And then
the ones that do, they only have like one or two, and there's very few. It's kind of sad. You want
this whole, all these things to be blue, all the way blue, and there's like lines of blue,
which is really disheartening here. And so why is firmware so difficult to maintain?
So there's so many reasons for it. And I used to have done firmware development before
for embedded devices. And even the experience that I had doing that is still... I could see
all the echoes of this throughout the process, right? Because re-architecting cost is a huge
thing. Cost is usually the biggest factor for why things aren't changing. But you can also be
locked into a vendor contract. You can be locked into like a middleware contract. So you can only
use drivers for this one piece of your kit, and you have to use it for a certain period of time.
You might have unsupported chips or hardware to work with. Another huge thing is outdated
toolchains. You might be using some toolchain from like 2005, and that's how you build everything in
2020. There's also a lot of things like hardware restraints, which sometimes you might not...
like your hardware itself might not support the SSL version that you need to. That was an issue
that I've had to deal with before trying to figure out how you can, you know, jerry-rig a new SSL
encryption scheme and support newer versions of TLS in firmware that's, you know, 15 years old.
Sometimes you also need to maintain backwards compatibility, which is a big thing to make
this stuff really... it makes it very, very hard. You have to include stuff that you might not want
to. A lot of stuff though is a lack of dependable updates for users to update their devices.
So even if you have, you know, all the other things in place here, it's sometimes people don't
have like a way to actually update the devices without some complicated process. Poor communication
channels even tell people about vulnerabilities is also a big thing. And vendors might not have any
sort of like channels for reporting bugs or telling people about bugs as well. And then,
you know, lack of modern security measures like secure boot or binary hardening we talked about
before or code signing are not going to really be in place. And it's hard to get those things back
into your, you know, pipeline if you have to do a bunch of testing and you only have a couple
people working on the thing. And so why do we see a lot of this older stuff working? So there's some
sometimes you'll actually see Qbots or Kytenbots or even Perlbots trying to exploit stuff in your
and, you know, if you download binary. And it's because the vulnerabilities are still there,
right? And so this is something that I had actually to steal an analogy from Mudge, right?
In mining, there's indicator minerals that can prove that there are other things that you,
what's it called? There's like a, say you're looking for like diamonds or something.
There's like indicator materials that prove that this might be there, the thing that you're
looking for might be there. And so the security vulnerabilities that we're seeing are showing
that there are not as many security practices that are being followed, which means that the
older vulnerabilities are still going to be able to work, right? So like we're seeing like,
you know, there's still command injection here in 2020, and then you can still run a Qbot or a
Perlbot on this device. This means that there's really not anything that's going into the actual
process of making the devices any more secure. And what's interesting is this is rare in other
classes of malware, like say for desktop computers, because there's no patch really that you can
apply to one specific device or whatever. So each time that there's a new vuln that comes out,
there's all these new devices that are added to the pool, but there's still all the routers from
2014 that had shell shock vulnerabilities in them, and DVRs that still have auth bypass in them,
and those are all just getting added to the pool. So people are just trying the same old
techniques and they're still getting the actual devices that they would have exploited, you know,
before. So it's kind of frustrating. So moving forward, this is the big thing for
vendors and people who are, you know, developers of firmware and embedded devices. So what can we
actually do to solve any of these problems here? So we can only really fix them by having better
development practices for security by meeting the developers and the vendors where they're at,
because we want them to be, you know, on top of their game actually doing,
you know, the work that we'd like them to put in so that our, you know, toaster isn't, you know,
DDoSing somebody because of some Mirai variant that's run by, like, a 14-year-old kid.
So we have to, like, actually talk to them, talk to vendors the way that they...
things that they already know in their... the way that they're already developing things.
So for vendors, I guess my big advice here is to invest in developer training
and to establish best practices and create security testing pipelines
and encourage researchers to actually find bones and disclose them properly.
We can mitigate some of the existing bones by encouraging safer use of IoT devices,
but it only works so much because... imagine trying to explain to your parents
how to, you know, turn off port forwarding on their... on their router, right? Like,
they're not really going to understand it in the way that you might... I mean, they might,
but they... you know, it's sometimes hard to get end users to actually follow your guidelines at
all. They might not even be able to be aware of it. But that's one specific way that we can
mitigate that. Establishing best practices, though, is a thing I wanted to highlight for a second
here. So, like, auditing your development cycle itself. It definitely depends on what you're
building, and you have to tailor it to that and be able to audit and say, hey, yeah, we are using
C, we are using, you know, this tool chain, we're using either GCC, or, you know, UC Lib C, or
whatever, to, you know, develop our firmware. But there are there are best practices for these
things. So, OWASP, something that I had when I was doing firmware stuff, I had made developers,
I had them look at these cheat sheets for OWASP on, like, toolchain hardening on, like,
input sanitization for web apps, and, you know, how to do other things, you know, the best
practices that there are. And there's tons of different ones like this. I just post to OWASP
because it's, you know, very accessible for a lot of people, and it's free. Another big thing is
CIS benchmarks. You know, there's, depending on what you're building, there are benchmarks for
security that you can follow, which are super duper useful. You know, you can even automate
that. I had done, like, Ansible CIS benchmarks before. You can build those into toolchains pretty
easily. And then also, if you really need to, hire a consultant to come in and do all this work with
you and work through it with your team. That's definitely a huge thing for vendors. Vulnerable
disclosure, though, is probably my favorite one to talk about and the biggest one. So, when you
do find people that are actually poking at your stuff, allow them to disclose vulnerabilities.
Please, if you are a vendor and you're listening, there are ways now. It's 2020. You can't have a
VDP. You can't have a bug bounty. You have to go through the channel, the proper channels, and make
sure it's right for you. But there are a ton of resources. Disclose.io has, like, really good
legal language and other things, resources for vendors. But yeah, people who do IRT research
sometimes either get no response or they get, you know, a summons, you know, get sued for something.
Establish security contact, though, and listen to email. There's security.txt is a really easy
way on your vendor website to just have a email that somebody who has an issue or a vulnerability
can talk to and not feel like they're, you know, trying to chase you down. Because, like, how many
times do you see people on Twitter going, hey, does anybody have a vendor contact for, like, this
company? And, like, nobody responds. It's like, if we have to go on Twitter to ask about this, not only
does it draw more attention to your, you know, the vulnerability, but it also makes you look bad.
So definitely, you know, keep up with that stuff. Work with researchers, too, because people who
are bringing stuff up to you, they want to help you. If somebody wants to just use your device
or crypto mining or, you know, to DDoS, you know, some kid on Minecraft, they're not going to tell
you about it. So if you have a researcher that's here and talking to you, they want to help you.
You should definitely, you know, heed their advice. And have some open channel with your
customers, too, to have the word out about vulnerabilities. Like, if you either you do an
internal thing or you submit to CVEs and then you post them on Twitter, whatever you do, just have
something so people can know, update their devices. And ultimately, all these are elements of a
vulnerability disclosure program. So if you put this all together, you have, you know, the baseline
for what you need to actually have one, which is awesome. And so I've done a little quick
question out there on Twitter, which you can see, I have the link here. But these are some
community suggestions for what vendors can do. You know, everything from automatic updates to,
I like the make a security in named persons problem. So like having say, like, I don't know,
like Sherry has to deal with the, you know, firmware bones that come in. So, you know, talk
to her if there's an issue. You know, other things like minimizing attack surfaces, code signing,
all of this stuff is going to be part of other, you know, best practices that you're going to
have to implement. But these are all, you know, elements of things that might be good for you to
consider moving forward. Default settings definitely should be sane, and with security in mind. And
also, don't reinvent the wheel. So final thoughts here, I got two minutes left, just about. So we
want to make it less easy for people to run botnets, right? Overall, the supply is already
there. The demand is great. Everything is set up. People can off the shelf, you know, get several
thousand bots on botnet in an afternoon. Botnets are definitely getting a lot smarter. People are
using the messy landscape of this to take control of it. Like if you see with the Kaiji botnet,
which is definitely a lot more advanced than previous stuff. People are going to be using this
for more nefarious purposes. And because there's so much going on in this space, it's very hard
to pick out who is either nation state trying to get access into your router, or just some
random kid who has no idea what they're doing. And new devices and architectures are always
being targeted. As I said, FPGAs, there's tons of new stuff. You can read that analysis that I wrote.
If you don't act soon, like the new products that you put out are already going to be
dead on arrival and exploitable once they come out. And so yeah, the Q&A for this is
going to be done in the DEF CON Discord, as you might have seen in the Twitch chat. So if you have
questions, or you can always hit me up on Twitter, at netspooky. My DMs are open there.
I got some shout outs real quick. Shout out to the Safari Zone crew, which is the people that
have always been there to look for weird stuff on the internet. Hopefully we'll have a zine coming
out soon. Threatland, everybody who helped out with that project to collect sources. And of
course, the entire community of Thug Crowd. A special shout out to Hermit for helping me go
through so much of this and be such an awesome person for, you know, looking at logs and other
weird stuff. I like being able to tell me about a lot of interesting things that she has found.
Andrew Morris, thanks so much for letting me use your data set too before it was even publicly
available. Thanks to Mudge for coming in hot with some hot takes for me when I was building this
talk. Check out Ilya's IoT Village talk on emulating IoT devices and malware, because we
actually did a lot of that when I was writing this talk. And then thanks also to Dave for that theme
song. So I'll have slides out. I'll tweet them out. Just follow me on Twitter and you'll see.
I have citations if you want to read them. But yeah, thanks, everybody.