I have pushed the button. Now we're waiting on it to kick over.
And see, I learned I have to actually refresh this thing, because for some reason sometimes
the stream will just continue playing whatever source it was playing. And now it's music.
It's still music.
Let's do it. Oh, oh, oh, oh, that's looking good. Hey, there's a bunch of faces.
All right. Hello, DEF CON. Welcome to your early morning Sunday content.
My name is Fallible. We have got Jethro here. Joining us from the speaker side is
Mickey and Jesse with the Q&A portion for their talk of Bytes in Disguise.
Oh, hold on. Before I do this, I have one more button I have to push on my side.
Okay. So before we get going here, I guess I'd like for the two of you to tell us a little bit
about yourselves and your DEF CON history, because I know that neither of you are strangers to
the hanging out in the desert with a bunch of geeks for a weekend. So
tell us a little about yourselves.
Go ahead, Jesse.
Okay, I can start that. So I first started coming to DEF CON in 2003 when I was still back at the
Alexis Park and was playing CTF back then and ended up winning a couple black badges. So I've
been coming to DEF CON every year since 2003 and enjoying myself a lot and having a lot of fun
and ended up starting giving talks a few years ago. So I think this is number five, maybe. I
need to go count how many talks we've actually given, but I've had a lot of fun here.
Casual drop in there, a couple black badges.
Yeah, I'm not as acquainted with DEF CON as Jesse is. I started coming to DEF CON
since DEF CON 19 and been speaking since 20, except 21, I think.
I didn't get a black badge. I had to make my own.
Give a talk about that, too.
Yeah, I love the vibe, love the community. Great experiences, horrible stories.
Yeah, it sounds about right. Well, excellent. I know that Jethro over here has a lot of
interesting things to say about your talk. I have some interesting things to say about your talk,
but I'm not into the niche that you occupy. So I will be watching for some of the questions
that come in and maybe we'll have my fellow goon ask our initial question while I watch
channels. Sure. Yeah. So for me, I watched the talk and I thought it was kind of cool,
just the number of different places that you could actually hide information. And the joke I was
saying is like, you've literally covered everything from like, hey, you need to wear a wrist strap
when you go into this to, you know, Google hacking, like just run these searches and get this
information to, you know, here's how to write random chips and, you know, code release. I guess
like so quick 30 seconds, like what I guess what's the someone hadn't seen the talk yet? How would
you recap everything for them? What are the high points? Not wearing a wrist strap. Let's, let's,
let's do a 30 second recap for an hour talk. What's the best part they should look out for?
Why should they go and watch it? The demo music. Yeah, the bossanova thing. Well done.
No, no. Yeah. So, so really, I mean, like guys, what inspires you to take like such a complete
look at the attack surface, right? Like, how did you think to start to look at all these different
ways that you could go after devices? We both have similar perspectives, but they're slightly different.
What got us into this? Well, I'd say we got tired of getting caught.
So when we do the red team exercises or whatnot, and we want to hide our payloads and
we always get caught and, you know, adding in a memory and on disk and all kinds of locations.
And we've had a lot of experience dealing with hardware and firmware. And we thought,
why the hell not? I mean, there's so many places that everyone thinks that once you go,
the lower you go in the stack, the more complicated things get. They don't actually,
they're way simpler, but they're just a little bit low level than most people are comfortable
with. But they're not too complex. And once you get over that hurdle, you go,
oh, I can just do that. That's awesome. I'll just write firmware to this thing.
As simple as that. Yeah. I mean, there are a lot of mitigations in the operating system
and application level that there's been a lot of attention there. So there's all kinds of
OS level application security stuff, looking at the file system, doing scans of the file system
are really common. And there's been a move towards like fileless malware and doing stuff only in
memory. But a lot of that is just, you get code execution on the box and run fileless in,
or just in memory. But being able to store stuff in places that people aren't looking
gives you that additional persistence where the loader that you actually have on disk
looks completely benign because people don't understand how these access mechanisms work
in most cases. So being able to have something that looks benign on disk and it'll pass all the,
like you can throw it up on VirusTotal and everything is 100% clean, but it still can go
read the actual malicious payload out of the GIGI region or from some random USB controller
firmware that's unsigned. And maybe it has some encryption key or something like that, where
the only thing that happens in memory is malicious, but the thing that actually happens on disk is
completely benign. And it's an interesting mechanism for persistence. And we wanted to
provide more visibility into that and let people know that this type of capability exists in the
system. Were you inspired? Go ahead. I was going to say, yeah, if you were inspired by any particular
researcher attackers using similar techniques. I mean, there's generally been, like I said,
there's been all this research and attackers in the operating system space and attackers have
started moving down in the stack. Like we've seen UEFI implants. There was like a Lojax implant.
There were some other ones were found in the wild. There was something in the Vault 7 leak that was
talking about an EFI implant that they were using there. So I think there's a trend towards people
moving further down into the stack. And a lot of these firmware components at those lower levels
in the stack don't have all those same hardening protections that the application and operating
system has. So you don't have things like code signing protecting your firmware in the stack
or firmware in some of these devices. A lot of these are lower cost, lower power devices that
maybe they don't have like a nice crypto engine built into this chipset. So a lot of that is
lagging behind just because there isn't this additional visibility and people going after these
components. And it's an area that's ripe for exploration and exploitation. And like we
talked about in our talk, some friends of ours gave a talk about doing this type of thing using
UEFI variables. And we were like, but wait, there's more. So let's go out after some of these
other areas. So that was kind of the inspiration for this talk. So a lot of what you're saying
is really interesting. I work way higher. I'm kind of in the website usually, and there's a lot
of things that have happened over the years in the website that make it harder-ish to do the web
hacking world. It sounds like there's maybe less... because there's less people working in the space
of attacking low level, are there less mitigations that I would run across if I got into it?
That depends. I mean, it is lagging behind to some degree. There are some... so people,
depending on what people's understanding, like you said, you work in the web space. So you
may be familiar with web stuff. People can talk about a full stack. And there's the full
stack of the web stuff that's externally facing, going down to a lot of these
underlying levels of the operating system. And you can get down to the point where you end up
looking at bugs in CPU microcode that have side channel implications. And there's a lot of
different vulnerabilities all throughout that stack. And there's been a lot of effort to do
the easier stuff that's more directly exposed to the user. But there are some of these underlying
things, some of these foundational technologies that everything is built on top of,
that are kind of lagging behind those exploit mitigations. Where like when the system
powers on and boots up, the processor starts executing code at the reset vector.
It used to be the case this was BIOS, where there was no protections whatsoever there.
Then people have standardized on UEFI, where there's the UEFI firmware that executes.
And a lot of that is they're adding protections there. But a lot of the times when the UEFI
firmware is running, there aren't things like address randomization. Sometimes the stack and
heap are fully executable. A lot of those mitigations that you're totally used to in the
operating system and application space aren't there. And there is some effort to put some of
these mitigations into the reference code. So there's a... UEFI has a reference specification
EDK, EFI development kit, development environment. And people take that specification. There's a
reference implementation called TianoCore. Some of these exploit mitigations are going
into TianoCore, but there's this whole big supply chain for getting those out to
actual customer systems. So there's reference code in TianoCore that then independent
vendors like AMI Phoenix inside take that code, add their own special sauce. Then the OEMs like
HP Lenovo Dell take the code from the independent virus vendor and do their own
value add. And then quite a bit later, that actually ends up shipping in production firmware.
So there's kind of this... there's a lot of components that are going into it. And it's
just lagging behind a little bit. So something going into the reference code in TianoCore
it might not be enabled by default. And even if it is enabled by default, it's going to take a
little while before it actually gets out into the systems that are shipping to customers. So
the other problem is that updating firmware is a lot harder than just pushing out a patch
through some Windows update in general. So there's... they've tried to standardize on
firmware updates for system firmware to some degree, but a lot of different vendors are doing
things differently. And that isn't fully easy to do yet. And so there's the system firmware
for your device, the UEFI firmware, but there's also all the firmware for all these
embedded controllers like the S-Media controller that Mickey talked about. And there's a
GigE controller that's as part of the Intel PCH chipset that we can write to that region.
We're not reading and writing to the firmware directly for that GigE region, but there's other
controllers on the system like there's a lot of servers have a Broadcom 5917 chip
that we talked about in some research a while back that also has unsigned firmware that
that's a common server chipset where maybe an HP DL380 will have that chipset and you can
write firmware there as well. Sorry, I kind of rambled a lot. Did you want to say anything?
I need to change my background after that.
Yep, that's pretty much it summarizes everything Jesse talked about.
Yeah, so you have like a operating system security which maybe is updated through
Windows Update or maybe Debian or Canonical or Red Hat SUSE repositories where
they're providing updates there, but there is a little bit of an effort to update firmware,
all of these different components are out there and some of them just don't have the capability
to have signed firmware. So there's a lot that you can do there and there's not a really good
easy way to fix it other than just waiting for the next generation to come out and hopefully
it has the capability to have signed firmware as well. Sometimes they'll have checksums and
stuff like that, but that's not a security mechanism and it's pretty easy to bypass
and recalculate a checksum. Excellent, there's a lot there and I'm sure that
I'll be watching through this talk later to make sure I get more of what you just said.
We do have some questions coming in from the audience, so I'm going to get to one of those.
So RPTK2015 says, question, do you know of a way to write some code into one of the various
EEPROMs that will be capable of executing code by itself without code in the disk?
Yeah, I mean that depends on the capabilities of the specific device, but as an example,
there's all the rubber ducky bad USB devices where a lot of flash controllers or flash devices
use an 8051 processor as part of the controller and you can write firmware to that and have that
essentially act as a keyboard and inject keystrokes, mass storage, stuff like that. So
if you have the ability to update firmware depending on the capabilities of the device,
if it's an internal USB device, it might have the ability to essentially act as a rubber ducky and
execute code and basically restage back from the device. We had given a talk a number of years ago
also at DEF CON about an internal Huawei LTE modem that we were able to update the firmware for that
and this was an internal device. Basically it was an M.2 card that was connected to the rest
of the system over a USB interface and it essentially was running a version of Android
with the Android gadget interface inside of this little M.2 card. So it was a complete Linux
distribution inside of the USB device inside of your laptop or tablet. So you could essentially
update the firmware there and have your payload there. The cool thing about that one is that
you can have that inject keystrokes back to your tablet or laptop and it also had a cellular
connection so you could do a CNC out to AWS. So that was a fun talk also that was a few years
ago at DEF CON. So yes, you definitely have the capability where if you can update the firmware
for the device, depending on the capabilities of the device, you might have the ability to
directly attack it again without needing that loader within the operating system
that's on disk in the file system. Additionally, if you have like a PCI device, you could potentially
do DMA attacks against the system and do that same type of capability. Sometimes the
operating system has DMA protections but the boot process doesn't and you can do an injection into
the boot process while the system is booting up. There was a quick follow-up question on
that from the same user. Did you use this in an assessment?
Well, we don't work as consultants or red teamers.
Not currently. So we were on a red team and we've since moved to different positions doing more
hardware and low-level security. And it was like this is the type of thing that we would have
wanted while we were still actively red teaming. But we haven't specifically used this on an
engagement, just proof of concepts and playing around with our own systems. We've heard others
use the high trick in an engagement. But not the rest of it. The rest of it's just so
many options we haven't had a chance to try them all. That's kind of the problem across this
space, isn't it? There's a target-rich environment.
While I read this next question in my head...
We'll just go right into it. It was like this when I found it. So the demo jumps right into
and here's this stuff and we can write and read to it. Isn't this cool? And the natural is it is.
It totally is. Plus one for elevator music. Any more pointers on enumerating access methods for
these things? White papers? I presume you're not using physical methods, JTAG or in-circuit
programming. So, point three, what APIs functions are you using for this kind of thing in general?
It seems like you could drop stuff into legit update image and flash it, but it looks like this
is on the fly. So a couple of... I can go through the TLDR of that. We can answer that. Well, it's
going to take 15 minutes, but I'm going to jump right at it. So there are certain tools you can
use to flash a pre-infected image. Like you can modify the legitimate image and then use the
provided tool by the vendor to send it to the device and have the device flash it to its image.
There was another way where we use one of the code releases was actual
talking directly to the spy controller in the computer. So, Jesse, you're itching to
say something, so go ahead. I just wanted to point out that if you haven't taken a look at it, we do
have a sample code up on GitHub now. Basically, GitHub hacking things slash bytes in disguise
that shows how some of this code works where... I will beg you after this thing to post that
GitHub to the TrackOne channel, but go ahead. Okay. So, yeah, basically we have some examples
to show off doing things like reading and writing to CMOS. Essentially, the low-level hardware
interfaces that the processor uses to talk to some of these resources are things like IOPorts,
where there's an x86 command called in and another called out that are literally for doing
reads and writes to these IOPorts, and that's an architectural feature of the system. There's also
a PCI access where you can do IOPort read and write to IOPorts CF8 and CFC in order to do
what's called legacy PCI access. And then there's also what's called memory mapped IO, where if you
can read and write to specific physical memory addresses in the system, you can basically talk
to these PCI devices through the PCI controller. And generally, in order to talk to these interfaces,
IOPort, you can basically do a privilege access like IOPL in Linux in order to
set your IO privilege level, so then you can do IO reads and writes from ring three
and talk to these ports. But in order to do reads and writes to the memory mapped IO,
you generally need a ring zero access, kernel access, or a privileged proxy that can do that
for you. So we gave a talk last year about a lot of drivers that provide this access for you. Basically,
you can do IO controls to talk to the driver, and then the driver will go do reads and writes to
these IOPorts, or PCI access, or reads and writes to physical memory. And we found a number of
these drivers that can provide this type of access. And we took one of the specific examples that's
pretty commonly found, the PMX driver from Intel. And it has a lot of capabilities, like
you can do an IO control to talk to the driver, and it'll do an arbitrary read write to a port,
or arbitrary PCI access. So we have some code that shows how to do that, including things like
talk to the CMOS, read and write CMOS, read and write
different areas of memory, talk to PCI devices. So one complication is that
if you use the direct PCI access mechanism in that driver, it goes through the IO,
or it goes through the Windows API functions to do that IO, which is how get bus by offset,
bus by offset. The interesting thing is that the spy driver, or the spy device,
when the Windows system is booted up, is actually hidden. It's a hidden device. So
if you try and go through the API in Windows, you won't be able to talk to that spy device
at bus 0, device 31, function 5. But if you do legacy PCI access by going through that IO
read write of CF8CFC, you can directly talk to the spy controller in that device with no problem.
So we have some code to demonstrate that. And using what's called hardware sequencing,
where you basically say, here's the address within the spy chip that I want to talk to.
Go read the 64 bytes from that and give me that data back. So we have an example of doing that.
We have an example of basically mapping the physical address, the last page of the four
gig region, and printing the bytes from the reset vector in the physical memory region. So all of
that is using this driver and DLL that was included in a number of software packages for update
mechanisms. And if that answer went way over your head, you can just go to the GitHub. No, that's
fine. That's fine. Go to the GitHub and look at the code. It mostly is in C sharp, which is easier
to understand than most things. It goes a little bit low level at some places. But if you
go there and you look at the code, it's not big. It's not scary. Just take your time.
If you have any questions, you can ask us later on. But whatever, everything that Jesse just said,
and if you completely went over your head, it's not that complicated once you spend an hour
reading on the internet. It's not that big of a deal. I promise you. It should be fairly
straightforward. There are some defines and register offsets that are specified for certain
things like the spy controller. If you look for the Intel PCH chipset data sheet, you'll be able
to find the definitions for those registers and offsets. This was specifically put together on a
Kaby Lake system, at least from my side. So some of the registers, like the fields, are a little
bit different sizes between a couple different generations. But this code should work on
everything from Sandy Bridge onward, I believe. All right. So I love where you're heading with
this. And this is particularly the way that you're explaining how to... for some of us who don't have
the depth of experience in the low level stuff, you're giving us some entry points so we can go
look at your code. We can try and replicate some of this on our own. Are there other thoughts for
those of us who might want to do this on our own? Do you have other places you would like us to
maybe go point and get some base experience to start playing with you? Yeah, I mean...
Before you do anything, before you start playing with anything, you got to remember too, if you're
going to do something with something, back shit up. If you're going to experiment with
non-volatile memory, first learn how to physically back it up and flash it back on.
That said, go for whatever you see has a chip on it that contains data. Go wild.
Yeah, to follow up on that, there are some things you can do, like writing to some locations
in your system, where the system won't boot anymore. So you'll need to physically open the
case. In the case of writes to just bare SPI chips that are easily accessible,
like the BIOS or UEFI firmware or BMC firmware in a server, you could use something like a
Dediprog with a chip clip and physically clip onto the SPI chip and read that, make a backup copy
using physical access before you start messing with it. There's also, you can use a bus pirate
to do SPI read and write if you don't have a Dediprog. But you can often find a Dediprog
on eBay for like 50 bucks or so. What I recommend is if you don't want to do that, I have an
alternate method, which is you have a Thunderbolt capable device, you just plug in a PCI Thunderbolt
adapter you can pick up for 100-150 bucks off Amazon. And then you just plug in one of these
PCI cards. This is one of the cards we worked on for the demo. This is a USB controller PCI
device. It has its own EEPROM. And if you mess it up, it's 15 bucks. Isn't that the thing too,
but was not the controller you actually writing to? This is the same controller that I thought
I was writing to, but apparently I had one in my motherboard that I've been reading and writing to.
So yeah, be careful about doing that. It's nice if you can just pull out the PCI card
and your system boots again, but sometimes it's more difficult. Remember to check you
don't have your target already in your computer before you start flashing.
This is all great advice. All right, gentlemen, we are right at the end of our allotted time.
You two are incredibly easy to talk to about this stuff, so we appreciate that. I like to
provide everybody an opportunity to distill a takeaway. What is the call to action you
would like to provide for the community who is watching this? I would just start out by saying
this might seem intimidating and difficult to get into, but it's really a lot easier than you think.
And if you just start experimenting with this case and do some of this yourself,
it's actually a lot of fun and it's not as hard as it might initially seem. So
we have some code examples you can get to. Like I mentioned, the Intel PCH chipset data sheet.
It's a pretty big document, but if you just look at certain things like the SPI controller
registers, it can help guide you and get you started. So I totally recommend people to go.
Don't be intimidated by this and just try it out and see what you can do. I'm just going to add that
feel free to ask other people for advice. And I'm just going to caveat this with if you ask
me for advice, I will gladly help you under the condition that you help others if they ask you.
So don't be an asshole. Help others. This is the point of community.
Yeah, that is one of the best takeaways that I've heard so far. So you have a whole series
of people in chat wanting to buy you drinks next time they see you in person. Well, thank you two
very much for coming and providing another deep load of content for DefCon and all that you do
for us. So if anybody has additional questions, I'm going to see if I can get these two to post
more of their contact information in the TrackOne channel so that people can find you later.
Other than that, I hope you have a wonderful rest of your Con and that's pretty much it.
Cool. Thank you. Thank you both. Thank you.