All right, and welcome back, everybody. We have another great speaker here today. We've got
Eyal Iqin here to talk about light bulb security. How's it going, Eyal?
Fine, how are you?
I'm doing okay. We also have my co-goon here, Siget. Hey, Siget, how's it going?
So, Eyal, can you kind of just take a minute and maybe give like a brief introduction of
yourself and the presentation that you gave that people can go watch on DEF CON?
Yes. So, hi, I'm Eyal. I'm a vulnerability researcher. I usually focus on embedded devices
and network protocols. And this is a classic research in which I combine them both together
to hack some, in this case, smart device. Essentially, in our research, we continued
the previous research by two great researchers, Eyal Ronen and Colin O'Flynn,
and they showed how they can take over a smart light bulb and propagate from one light bulb to
the other. And our goal in this research was to take it one step further. We wanted to take
over a light bulb and then from the light bulb take over the smart controller. And the controller
is connected to both the radio network, the ZigBee network, and to the regular computer network. So,
we could manage our smart light bulbs from our mobile app, for example. This basically means
that using a malicious light bulb or an antenna, we could take over the controller and then
infiltrate the network and attack computers inside it. So, we only need a radio antenna and
we can infiltrate your home or your office network. So, that was the goal and this is what we
demonstrated. That's excellent. All right. So, I know in your presentation, you kind of mentioned
a whole bunch of the people's research that you based yours off of. What got you interested in
this kind of thing? What made you want to look into trying to take things one step further
from what the other research that previous people did? Okay. So, it was quite simple. One of
the researchers, Eyal Ronen, is an old colleague of mine. And when he started working on his initial
research roughly four years ago, it was the first time I actually heard of smart light bulbs.
And I really didn't know why people need smart light bulbs because I really only need the light
when I'm physically in the room and I'm physically capable of toggling on and off the switch. So,
I don't really know why I need an app in order to control the light in the room that I'm currently
in. And I figured out that smart light bulbs won't be a real thing. People won't buy it and
forgot about it and continued on with my life. But it turns out that people actually buy
smart lighting solutions. It turns out a lot of people specifically in the UK
use it. It's not so popular here in Israel. So, Eyal Ronen talked with me again and he said,
we have an opportunity to continue on our research because we had an additional task that we
couldn't complete in our research. And since it turns out that people actually use smart light
bulbs and in previous DEF CON talks, we infiltrated the network using a fax machine, it turns out that
it could be interesting to do that over the radio as well, over ZigBee. So, this was the goal.
We collaborated with the previous research team to get their equipment and knowledge from their
results. And then we started on our way. Awesome. That's all great stuff. So,
again, do we have any questions coming in yet or did you have any?
Not sure if he's hearing me at all over there. Okay. Go for it.
No, there's a comment that says, I hope you got a bounty for showing that the
weren't completely implemented. The second part of the patch that you tested, right?
Yeah. We didn't get any bounty. We usually don't get any bounty.
I never worked on an embedded research which we received any bounty on any vulnerability
whatsoever. Usually even when we report the vulnerabilities, they are only partially fixed
or only the crucial vulnerabilities are actually patched. So, for example, our vulnerability
was that we were able to show that in previous versions there were arrays which weren't parsed
correctly. And in newer versions, they migrated the strings. But behind the scenes, we're still
using them as arrays with the same vulnerability. So, the patch was to check the sizes of the
arrays and handle them correctly instead of treating them as strings because strings are
handled correctly and we only can send strings over the radio. So, there's no reason to treat
them internally as arrays just because someone did it three years ago and that's the code. But
it's harder to fix it on a big design issue than just adding an if in the code and that's it.
So, one of the questions that we see coming in from looks like Jiska. Did you get any further
with the ZigBee modem itself? Like, were you able to extract its firmware?
It's a good question. The problem we had with the ZigBee modem is that the firmware update is
encrypted. And in a previous research, what they did was they had a power analysis attack and they
extracted a key and decrypted a firmware update and signed their own firmware update. But it was
only applicable for the firmware to the light bulbs themselves. It turns out that the modem
uses a bit different cryptographic scheme. So, we had to, if we wished to get the firmware of the
modem, we had to reiterate all of the research that the previous research team did. And we
didn't have the resources for that at the time. So, we tried to live with it as a black box and
we managed. It could be interesting to look on it because it's more advanced than the light bulb
itself. It does more functionality. But no, we don't have the firmware, sadly.
Okay. Yeah, I think in your presentation, you mentioned that previously, before your research,
it was kind of thought the worst things that you could do was turn all the light bulbs on or turn
all the light bulbs off or flicker them or anything like that. So now, based on the research that you
did, what do you see as kind of a worst case scenario if somebody wanted to do bad things
with your research? The classic scenario we thought about is either a neighbor attacking
your network or someone parking a large van across the street and deploying an attack. And then
we can easily hijack a given light bulb and convince it to connect to our network. And then,
as you saw in the demo, if we turn off the light bulb, you think that it's broken and you just
toss it away. And instead, we change the color so you'll know that something is wrong with your
light bulb and try to reset it. But there's no reset button. There's no buttons at all to the
light bulb. So the only way to reset the light bulb is to delete it from the app and then search
for new light bulbs. And when you search for new light bulbs, we can actually attack your
controller and infiltrate the network. So once we take over the controller, we are inside your
home network or your office network. We can attack whichever computer we wish. We can sniff the
traffic. And if inside your own network you use HTTP instead of HTTPS, we can sniff all of your
traffic. And essentially, no one will notice an intruder on the bridge. You don't have an
antivirus on your bridge. You don't have any security solution on it. So we can have a persistent
foothold inside your network and attack additional computers. And just from being near your network
with an antenna for five minutes. Wow. So knowing what you know now from this exploit, what would
you suggest to the makers of IoT devices? Specifically in this case, there were some
security mitigations in place, which is sadly quite rare. Usually we get none. And still,
although there were some security mitigations, we're still able to exploit the vulnerability
because there weren't enough mitigations. And actually, after our research,
we looked again on the heap implementation that was used. And it was really weird for us that
there were some security mitigations, but because the vendors didn't compile their own binary
correctly, we were able to bypass the mitigations. Because it turns out that if you don't have
security on by default, the vendor won't compile correctly their firmware. And they won't get
anything because on their compiler, it's off by default. So in our team, we designed a new
security mitigation called Safe Linking. And we integrated that into both glibc and uclibc-ng so
that it will be a new security mitigation in popular malloc implementations. And it's on by
default. So just this week, glibc introduced a new version that includes Safe Linking. And uclibc
already have versions with Safe Linking. And this security mitigation would have blocked our attack.
It would have demanded us to have an additional infleek vulnerability, which we couldn't find.
So nowadays, vendors just need to use the most updated versions of libraries that they use.
The most updated Linux version, glibc or uclibc version, and preferably even a good compiler
with a good toolchain to actually compile with security mitigation. But just updating your libc
implementation will be a huge step forward. Excellent. Segev, are you seeing any other
questions for him coming through the Discord channel at track one live QA?
So we have one. So your discovery was a modern firmware security problem,
or just a ZigBee protocol issue?
It turns out that ZigBee is complex, which wasn't a big surprise. But the vulnerability itself
was an implementation vulnerability specific to that product. So Philips, now Signify,
had an implementation vulnerability over the years. It changed a bit over the firmware versions.
But essentially, it was an implementation vulnerability on their side. It wasn't a
vulnerability on ZigBee. That said, ZigBee defines data types with variable lengths
that should be received over the network. And usually, embedded devices and embedded developers
are not that good with handling variable-sized objects. Although there was no vulnerability
in ZigBee, but only the implementation, I can guess that if five developers will have to
implement this model, at least two of them will get it wrong. So it still should be done correctly.
So have you checked the updated bridge firmware, like if they used more mitigations?
We checked the updated firmware version for the patch, so we would know how did
they actually fix it. And to check that they fixed it correctly, so we could continue on
with the publication. We only saw a small patch, as I said earlier, just an additional lift check.
We didn't notice any change in the compilation or anything of this sort.
If they are hearing this switch, I really urge them to update their libc version and compile
to use dynamic addresses instead of static addresses. This will be a huge step forward
in the security of their device. That's great. Let's see, what else do we have coming through
over there? Anything else, Sighad? No, I'll ask a question. I noticed in your talk,
you spoke a lot about your sort of false starts, right? And you tried this, you tried that,
and these didn't work out, and those didn't work out. And then you also kind of mentioned hitting
a wall on the erase issue before you were able to find the second implementation that still
contained the vulnerability. So, two questions. One was just, you know, what about those false
starts did you think were important enough to include in the talk? And secondly, you know,
what got you to the thinking that got you past that wall? Okay. So, I started the first one.
Without the false starts, I wouldn't have been able to actually find the vulnerability that
eventually I found, and to reliably exploit it. Because even as in the slides, I didn't even have
enough time in the blog post, which was just uploaded, you can see that there was an additional
false vulnerability, but there is a vulnerability, but it isn't exploitable. And even if I wouldn't
have encountered this code module, and the treatment of ZigBee addresses and the ZigBee
phone book, which initially had a vulnerability, but it wasn't exploitable, I wouldn't have been
able to understand where to store the shellcode and how to construct my shellcode and how to use
it. So, every part in this research, even if I didn't eventually find a vulnerability in the
specific model that I tried to, I learned a few things about the device and about the memory
layout. And every one of them was important in the overall attack at the end. So, that was the
main reason I decided to talk about all of the steps in the research, even if there were some
dead ends, because it gives additional information that we later on use during the exploit. So,
it was important about hitting the wall. Then, when you start a research with
an interesting attack vector, and you're already into it, you got the firmware, it's loaded into
your disassembler, you bought a kit with the hardware, you have everything, you really want
to find vulnerabilities. And in most of the projects that I worked on, there are critical
vulnerabilities. We just need to find them. So, you check one module, it's a dead end. You go to
the other one, it's a dead end. You go to the third one, it's a dead end. And then you go back
and you check your initial assumptions. Maybe there is some assumption that we thought about,
but it's wrong. The device might behave differently. Maybe we skipped a module because it looked
well written, but we didn't check it deeply enough. So, we can go back to that module
and test it again. So, essentially, in a security research, we attack the device
in layers. We want to find the path of least resistance. And it's really common to reiterate
over the same module again and again, because now we want to go deep into that module after we had
a shadow look on all of the modules we initially marked. So, we have a few iterations.
We most of the times we find critical vulnerabilities. It's not easy to continue on
when you get to a dead end and when you get to multiple dead ends. But you're already into it
and you don't want to raise a white flag and surrender and to the research team in the office
to laugh about you. So, you want to continue on. You want to prove that you actually can
break this device. And it's really important not to never to give up.
That sounds like there might be a little bit of an extra story there about being
laughed at in the office, but certainly not one that we can need to talk about today.
Were you ever able to find a way to overcome the 60 second limit in the exploit?
Sadly, no. Technically, there is a grace period that once the user requested a bridge to
search for new light bulbs, we have roughly 60 seconds. In some experiments,
it wasn't exactly 60 seconds. It looked more like two minutes. But essentially, there's a
timer. Most probably some thread on the main CPU initiates a timer. And when the timer expires,
a message is sent to the modem and the modem shuts down the communication. So, unless we starve
the thread that is responsible for the timer, the timer will expire and we will be blocked out.
And it is a good security design. We can't send messages to the bridge
unless the user requested a bridge to look for new light bulbs. So, it is a good design.
It blocks most of the attack vector. However, there's no reset button for the light bulbs.
And there's no indication of please just pair this light bulb again, according to the addresses
of the light bulb. So, essentially, the design started quite good, but broke somewhere in the
middle. We did find out that if we bombed the controller with multiple messages, we can starve
the rest of the threads. But it has some disadvantages as well. Because if we starve
the rest of the threads, then the watchdog will restart the bridge. And we mostly don't want that
to happen. So, the 60 seconds is... it is a hard limit. We can't really do a lot about it.
Excellent. So, yeah, do we have anything else currently coming through the
Discord chat? Yes, we have a question. Did you start working on IoT exploitation at Checkpoint,
or is this an area you've been working on beforehand?
That's a good question. It really depends how you define IoT. If IoT is anything that is not
Windows, Linux, or mobile, then I started working on embedded devices practically from my first
year on work. If we rule out network equipment, for example, then actual embedded devices
were from after three or four years of work. So, it wasn't the first device that I encountered.
But I did research embedded devices way before I started working on Checkpoint.
Excellent. And did you ever think of attempting to brick the other bulbs or possibly the controller?
There is a sad story about that. Because if you break the device,
you also lose your own device. And then you need to go and purchase a new device and convince some
manager to once again buy it. And then you need to tell the manager, yeah, we need to buy it again,
because we bricked the earlier one. And you can't do that often. We had that on the research in the
previous year. We presented a DEF CON ransomware on DSLR cameras, and we had to test the patch from
the vendor. But if the patch will work, we'll get locked out from the device, and we won't be able
to demonstrate the attack, which we wanted to demonstrate on a live demo. So, once we noticed
that it will be some issue about it, we went to buy an additional camera. So, if we lose one of
them, we still had a backup. So, it is quite easy to break the devices. And most of the times,
you don't want to do that, because you lose your own research device and you need to buy a new one.
A few years ago, I actually bricked my device on a different research project. And if you break
your device, then people really laugh. Because, yeah, he bricked his device, we need to buy a new
one, and you don't want to be that researcher. So, you really don't want to break your own device.
Sounds like quite the environment at work that you have there. So, it seems like a lot of times,
a lot of years at DEF CON, you know, the big story comes out about this big thing is insecure,
there's going to be these problems. And you're talking about how that people could get into a
home network, or maybe even a business network through a light bulb. So, what do you think about
overall security concerns based on your research? Is this something that people at large should be
really concerned about? Or is it being handled really well by the manufacturers?
I wouldn't trust the manufacturers at all, specifically in IoT devices. It doesn't look
like security is a top priority in their devices. If we look specifically on the light bulbs case,
you would be more worried about a neighbor hijacking all of my light bulbs and turning them
to whichever color they want. Because they could have done that years ago, even from the conclusions
of the previous researchers. And it's a flaw in ZigBee, you can't fix that. So, specifically in
light bulbs, anyone can hijack your light bulb and steal them to control them from their own
network. And there's nothing really you can do about that. If you're worried about someone
attacking your bridge and infiltrating your network, most probably it won't happen a lot.
We are not releasing our full exploit, and we work quite hard on the exploit,
it will be reliable and will work correctly. If you're a real target and some high
authority wants to hijack your network, they could do that. Will someone do that to your
neighbor? Most probably not. That said, on the light bulbs, two years ago, we showed you we can
infiltrate the network from a fax machine. And then the attack was easier, more reliable,
and I can be in China and still attack your all-in-one printer. So, that one is more concerning.
And I would worry more about using a fax than using smart light bulbs.
Excellent. Sighed, how are we doing with that Discord channel there? Coming up with anything
else, or should we kind of start giving him some wrap-up questions? So, we had one question about,
did this research ever give you the kind of insight you started out with on why people
want connected light bulbs? It actually did. It turns out that smart light bulbs are actually
quite cool. I read about a few people in the north which use the light bulbs and calibrate them
to match the season and the clock. So, they will still get sunrise and sunset, even if they don't
actually have that way in the north because of the season. You have quite a lot of control over
the light bulb, both the intensity and the color itself, and you can sync multiple light bulbs to
get a scene. And it is really cool. It isn't very practical because, for example, if there's a power
shutdown, then all of your light bulbs go reset, and then you lose all of your scheme, all of the
colors, and all of the intensity, and you get all of the light bulbs go white, go to the maximum
lighting, and it happens quite a lot. The power has some sort of glitch, and you're not supposed
to connect your light bulbs to a backup system or to a UPS or anything of this sort. So, if you're
into the colors of the light bulbs, you need to have pretty good electricity in your city or in
your state. Otherwise, you need to recalibrate your light bulbs again and again. That's quite annoying.
That's great. So, what would be your one main takeaway from your research or call to action?
To happen after this? What should people take away from your research and your presentation?
So, if I have a message to manufacturers, then please compile with security mitigations and use
the latest versions of each library. It really adds a large amount of security. If it's for
researchers, then it turns out that practically any device that you can think of using any
protocol will most probably still be vulnerable. On ZigBee, we have a maximal transmission unit of
127 bytes, and we still reliably exploited the vulnerability. So, even with really harsh
constraints and really esoteric protocols, we can still consistently, year over year,
present in DEF CON that we broke an additional device. So, everything practically could be
broken. And even if you have harsh constraints, you most probably will still be able to exploit
the vulnerability once you find it. So, if you pick a target, don't give up. Most probably,
you could get a win out of your research. So, just try hard enough.
Let's slide one last question in there for you, from Brief Halo. What recommendations
would you give to someone interested in studying IoT exploitation formally, like at a university?
There are some good courses in several universities, but usually reading write-ups
from researchers that published their research, and you read about it, and it was interesting.
Reading the write-ups usually gets you a lot of tips. So, if you read the publications from
Colin O'Flynn, you get a lot of new knowledge about hardware, and how UART works, and how
UBIT works, and you can learn a lot from practical write-ups of researchers. It will be more
practical than most of the courses in many universities. That's great. All right. Thank
you so much for doing this. This was a great talk with you. I really enjoyed your presentation as
well. Thank you so much for doing this. Thank you. All right. Have a good night,
and we will be back in about 30 minutes with another speaker.