I'm just here to destroy. So, it's definitely a learning curve for me, where I'm just used to breaking the thing down, just to break it down, versus breaking it down to actually get some value out of it and turn it into something cool. So, yeah, I really appreciate all the support and all the help so far. It's been a great journey. Outstanding, outstanding. So, we're getting close kind of to the start point, just a minute or two out. Again, for those joining in the actual Zoom webinar, there is a button down there where you can actually post questions. I encourage you to please do so. The goal is, is we will try to answer any questions that come up. If we have the answer. U-Boot's kind of an amazing topic, and none of us claim to be, know possibly everything out there. But we'll do our best to be able to give you some good feedback on your questions. Also, after this is all over, I'll be jumping over to Discord. I think Garrett will be there also in the IoT Village Discord in the speaker area. So, if you have any follow-up questions after that, please reach out to Garrett or me or Jonathan, and we'd be more than glad to help you out. So, we're kind of at the go point. So, I'm going to turn this all over to Garrett to go ahead and get started. So, it's all yours, Garrett. All right, right on. Well, appreciate everyone taking some time out on Saturday. I know that normally we're in Vegas right now. So, it would be no harm and no foul to be taking time out on a Saturday. But appreciate y'all, you know, taking some time and tuning in today. So, without further ado, let's get this party started. So, you should be able to see my screen. We're looking at the U-Boot DC28 IoT Village is what we're going to be going over today. So, we got a couple agenda items to go over real quick. You know, what is U-Boot? First off, we're going to get that out of the way. Then we're going to go into pin glitching, a Philips Hue device, and then potentially getting root on that machine. And then we're going to switch gears a little bit and go over some common problems with Daryl Hyland. And it's common problems related to U-Boot and how U-Boot can solve them. Then we're going to be rooting a Luma Wi-Fi access point. It's like a mesh Wi-Fi device utilizing TFTP. And then following that, we're getting faded. So, that's the agenda for the day. And without further ado, let's get this party rolling. So, what is U-Boot? So, U-Boot is basically a universal bootloader. It's loaded by the system's ROM or read-only memory or BIOS. And for many one of these supported devices like, you know, SD cards, Flash, you know, using SPI or NAND Flash. And basically, it runs a command line interface over a serial port. And, you know, people like us can load and boot a kernel, possibly changing any of the parameters in there, you know, from the default to make it do what we want to do. Kind of take over and give it our own instructions versus something that comes from the manufacturer. You're also able to read device information, read and write Flash memory, you know, download files. There's a lot of different things that you can do with this. And, you know, basically, you can choose the memory locations of the kernel and other boot information and explicitly, you know, tell it, hey, I want to look at this destination, and I'm going to copy this information. So, it's really good at extracting data and pulling that out for further analysis, you know, outside of that U-boot connection. So, it's loaded from Flash memory. It's often accessible over a serial connection. And that's what we've used today is we're using a Chakra utilizing UART to connect to these devices. It also comes with a limited set of commands. So, several commands that we're going to be going over that come with U-boot. So, moving on. So, ping glitch on the Philips Hue. So, that's what we're going to be doing first. I'm going to get out of these slides. Boring, right? Let's get out of here. So, we're going to go up to my other screen over here. And just to show you some of the devices we're working with, we got the I'm looking all around me. It's right in front of me. The Philips Hue. It's basically it's been stripped down. You know, I can see the underlying form factor here. But, you know, this thing is to connect your lights in your house. So, if you wanted to, you know, have a bunch of different colored lights and then control them from your phone, you'd buy one of these devices. So, you can imagine a lot of people have these in their houses. I think you can connect up to, like, 50 light bulbs with this thing. And then we have the Luma, which is looks like a little Wi-Fi access point or something like that. But it's exactly what it is. It's a mesh Wi-Fi access point. It's got Ethernet on board. It's got a USB. It's got a bunch of other stuff in there you can't really see because camera's not very clear. But that's what we're going to be breaking down today. So, let me get my other situation going on here. And first off, got to set this up. So, bear with me here. And if you already have questions, you know, don't hesitate. Throw them out there while I get this thing set up here. Okay, that's in there. Perfect. So, just get some wires out of the way. I should remember that choice from last time. All good. So, just to break down what is on this now. So, we have the UART connection over here to my Shikra using ground over here, receive and transmit over here. So, we're connected to some headers that are on this motherboard. Then we have another set of headers over here that's set up for my ground out wire. So, I have a wire that has a paper clip shoved in the end of it that we may be using to glitch this thing out. So, that's a little bit about what's on this thing. You also have the two flash chips that we're going to be looking at. This one down here and this other rectangular guy right here. So, those are the two chips that are communicating that we're going to be disrupting. So, on the left-hand side, I have a terminal called... software called Cool Terminal. And it's free. You can download it. It's pretty versatile on Mac from, you know, the few times that I've been using it here. And so far, so good. You can actually see when it's transmitting and receiving. You can see when it's connected. A lot of good detail here. So, I'm going to disconnect and I'm going to clear the data. And I'm also going to look at these options. So, there has been some things that we did prior to this call to kind of set this up for success. So, I did change the port to my USB serial because we are using Ashikra. I did change the baud rate to 115200. And I also changed some of the terminal settings so that we can actually interact with this terminal and not pull our hair out because you can't backspace, you can't delete, and things start getting crazy. You know, if I can't go backwards, then how can we go forwards? So, a few of the options I just wanted to highlight. So, now that that's all said and done, let's get some power to this thing. So, you can't see it, but I have a power strip with my, controlled by my foot over here. So, let me just get power to this real quick. Okay, so we got power there. Got to take my shoe off because I can't do it with my shoe. I totally forgot I have to use my toe. So, all good. We're going to connect to this thing once we get some power on it. And we're in. So, this is just the normal boot up of this device. We're just looking at it from a terminal perspective. And with this, we can already get some good information, good value, because we all know when things boot up, there's a lot of information that's kind of sprayed out there. And sometimes it's machine communication, sometimes it's detailed information that we can actually use and take back and, you know, learn more about this system. So, you know, as it boots up, I think it takes about 32 seconds. So, we're almost there. This is a seconds timer on the left-hand side, if that wasn't clear. So, once we get to the login, I think it's... So, we're here. So, here's the login prompt. I don't know the password. I tried to look it up and I didn't find it. Let's see. Roots admin. Just try one. All right. So, it's not working. Okay. So, we cannot log in at this stage. So, let's take it to the motherboard, right? So, I'm going to clear this out. Okay. So, we're going to start fresh. I'm going to turn the power on this guy. My trusty index toe. There it is. So, I'm going to clear it. And let me get my wire ready. Hopefully get this on the first shot here. So, clear the data. We're connected. Power is on. And that's the pin. Oh, I'm shaking. Oh, there it is. Perfect. So, we got the command line. So, if you didn't see it, it was pretty quick. But there is a pin that I just lightly touched and pressed against this little paper clip thing right in there. So, if I can keep my hand steady, it's right around there. So, that is what got us to this point. And what can we do here? So, a few things. We can just ask for help. I mean, where are we? Help me. So, we get a lot of different commands. You know, there's boot commands. There's echo commands. We can echo arguments to the console. We can loop, infinite loop on an address range. We can ping. I think there's a ping option in here. Yeah, ping. So, we can even try to connect to other network assets, you know, that may be able to communicate to this device if we want to. We can print the environment variables. We can set the environment variables, save the environment variables. So, we have a lot of options here. We can even boot, TFTP boot if we wanted to. So, a lot of different things we can see at this point. So, we're just going to print the environment variables and see what else is in here. So, these environment variables are somewhat of the command and control area of this device. So, what we change in here does affect the entire system and the device. So, you know, we can see the baud rate there is set to 115200, which is how we're able to communicate. We see some more flash boot up details with the kernel. We also see some... what else is in here? We got some version information, production information. Oh, we got an IP address. So, we got some IP information. We got a server IP information. A lot of information in here. Boot delay. Oh, that looks good. Oh, some security thing with a hash. That's probably something. Let's see. So, I'm gonna just... so, we don't have to do that ping glitch again. I'm just gonna set the boot delay right now because I don't know about you, but my head got shaky hands and I don't want to have to do that every single time I reboot this thing. So, we're gonna set the environment variable for boot delay to five seconds just to make sure that that held. We're gonna check it again and it's there. So, we got five seconds on that boot delay. We have to save it. Otherwise, it just goes away. So, we're gonna save this. And that's all the jibber-jabber of saving that environment. Let's just print it one more time for safekeeping. And boot delay is five. So, we are there. So, I'm just gonna reset. Let this thing boot up again, except this time we should see a boot delay. Ah, there it is. So, I am going to interrupt. Rude, right? So, now that we're back in, what else can we do with this thing? So, let's take a look around back in those environment variables. And the one that kind of jumped out to me was the security and this weird hash following it. 1984, maybe there's some type of underlying message in here, but the security is something we can mess with. So, let's just set the environment of security to nothing. And nothing can be just two single ticks. Okay. So, if I save that or set that, let's check it. Ah, security equals nothing. Perfect. So, we do need to make sure we save this, save environment. No matter what, you always have to save. So, make sure that that's a, you know, nailed down. And we'll just try resetting. See if that did anything for us. So, essentially, what the manufacturer did with this is they hard-coded the password into the device. Not a good idea, in my opinion. But that's what they did. And that's why we're able to mess with it right now. So, we did let it go past the boot delay because we want to see what happens when it actually boots up now. Because before, we got hit with a login prompt. And I didn't have the password. So, we couldn't get anywhere with that. So, as this thing kind of boots up, same kind of song and dance with the boot up information. You know, it's writing, it's opening, it's loading, it's doing all these things. And any of those environments that we've changed may affect, you know, this time right now. So, if we just wait a few more seconds, the magic of television. And we are root. So, just by eliminating that environment variable security, it actually cleared out the password. And that was actually the password in a hash. So, removing it gave us root access. So, right on. High fives, air fives, you know, because the timing. But air fives all around. We got this thing. So, what can we do now? Well, lots of things. We can start looking at the mounted drives. We can dig into the root file system. We can look at the kernel. A bunch of different root drives in here. The root file system data drive. Maybe there's some good detail in there. You know, we're clearly at the helm here. We clearly have the reins of this device now. You know, we can look at other information on this thing and see if there's more details that we can pull out of it. More user details. Maybe some hashes. Who knows? But we are in like Flynn here. So, any questions on that so far? Hey, Garrett. Yeah, looks like we have a couple questions here cropping up. So, Strikeout looks like he's trying to do possibly some U-boot bypass through glitching on a wise came out door. And this individual is asking where did you actually put the paperclip out on the chip? A data line? Yes. So, that is a great question. It is a data line. And it's in between these two flash chips on the... I don't know if I can get a better view of this thing. But that's looking terrible. So, I can't... I don't have a micro... I don't have a magnifier to get you a better view of this. But basically, there is a pin in between the two chips that is sending data. It's basically this pin right here. It's a little silver flat pin there. And if you look close on the motherboard, you can see the tracks going between those chips. You can kind of see it, but now it's all blurry, of course. Yeah, to kind of add to that, this is real quick. Often, one of the best things to do is once you've identified the flash chips that you're going to attempt this on, is to track down the data sheets on those. And from the data sheets, in this particular case, I traced out the data zero line. So, between those two chips, this particular glitch is a little easier timing-wise because the smaller chip that's toward the bottom actually has U-boot on it. The other chip has the kernel on it. So, the goal is to find the data line for the chip containing a kernel and interrupt that by taking it to ground. And the attempt there is U-boot's loaded, and before it has a chance to actually load the kernel or call the kernel, you literally take one of those data lines to ground, causing it to fail to read it, as Garrett was talking about and as he showed there. One other question here, Garrett, that is cropping up. Individual asks, what type of device are you using to interact with the device here via UART? Are you using something like a Shikra? Yeah, let me show you. I can show you what that is here. We got one right here to show you. So, let me kill out this thing real quick. So, we're going to disconnect and pull this thing right out of my laptop for you. Hold power. All right. So, to give you a better idea of what this thing is, it's basically just a little USB with a chip attached to it with several different header connections on it. It comes with a data sheet that kind of breaks down what these connectors are, but that's literally all it is. It has three cables. I don't know where my camera is. It's got three cables, one for ground, one for receive and transmit. So, that's how I'm connected directly to this device. So, hopefully that helps. But you can pick these things up off the internet. I think they're 30 or 40 bucks. I can't remember how much they cost, but they're doable. And there's some other devices out there like Pirate. There's one called, I think, UART Pirate or something like that, or Pirate Bay. I can't remember what the name of it is, but there's several tools out there that do the same, essentially the same thing. This is just the one that we use today. Cool. That's, I think, the last of the cute questions that we have here. Bus Pirate, I think, was the word you're looking for. That's what it was, Bus Pirate. Oh, man. Yep. I knew it was something to do with the pirates. Pirate would be pretty good too, I like the ring to none. Right on. Cool. Well, let me, if that's all the questions, let me just get back to the slides here and move on. So, now we're going to switch gears a little bit. We're going to be going over some common problems with embedded devices and then using U-boot to solve them. So, I'm going to switch gears and pass it over to my colleague, Darryl Hyland, and take it away from there. Let me just stop my share. Thanks, Garrett. Let me go ahead and share my screen. Hopefully, everyone can see my screen. So, kind of get back to the actual hue. So, I went and configured this hue to kind of exhibit some of the common problem that you will often encounter. And I know I've hit it two or three times working on IoT gear in the last year or so. And I know Jonathan has also encountered it. And again, this here is just a standard hue. And then this here happens to be a quad UART device. I kind of like this one. It's a little different than the Shikra. This one actually takes four different UARTs and you can switch between three and a half and four volts. And it's all USB-based. So, it's kind of sweet. I like using that if I need multiple UARTs at the same time. Sometimes... let's go ahead and plug this device in. Make sure everything's going to come up here. So, again, often when you're encountering devices, they'll come up. And if you noticed, it had the boot delay. I put the boot delay on here just to avoid having the glitch again. But the ping glitch to force into a U-Boot console is a literally very common method. But as you see, we hit a point here where it's literally disabled. We no longer have a console. And this is not uncommon. You'll have a device where you're actually able to establish a UART connection on the device and start seeing it boot up. But then inevitably when it's booted up, there's literally no console whatsoever on that device. I've had it go up to this point. You'll see console disabled. I've seen it come up and not tell me console disabled, just say loading kernel. And then you see nothing after that. So, whether you need to use a ping glitch or they've enabled the boot delay or some method you can gain access to it, we're going to go ahead and go through that. And often, the fix for this is typically very simple. Let's see if I can catch this here. Okay. Actually, which is funny, that did not work for me. So, we'll see. This is going to be funny if this doesn't work. It wouldn't surprise me if Zoom caused the problem. Because, okay, we're able to get into it. So, back to those common commands that Garrett had mentioned, like print environment. So, we're going to come over here and print environment. And again, there's literally tons of environment variables. A lot of these environment variables are part of the boot process, like Garrett said. Some of these actually pass commands to the kernel. So, the kernel knows what to do as it's loaded up. And often, where you see alterations that need to be made, the most common one is bootargs. You'll see bootargs showing the configurations. But when we come over here, we see bootargs console is actually set normal. So, maybe it's somewhere else. But here, we found out they have another environment variable called standard bootargs. So, apparently, this is what's being called at some point. Thus, we see the null taking place in here. Sometimes, this will involve very experimental. First time, when I was setting this up to identify it, that's when I found out that this bootargs segment doesn't work. And then you need to do it some other way. And then turns out it was the standard bootargs. So, to go ahead and do this, again, like you did, you go, you set the environment, which was standard std bootargs. Something always to remember also. In this case, I'm going to rewrite the entire one. And we're going to set the console here. And look at our standard bootargs. Make sure there's not another one. You would be surprised how many times I've misspelled something. And it's easy to do. And what you end up doing is just creating another argument that isn't read by the system or used. So, then you boot the system. It goes, well, it didn't fix anything. What went wrong? Often, it's just a typo. You've seen where people have made typos up here. Here's an example. Security, security. That's kind of weird. So, again, that's probably a typo that took place there. So, at this point, we'll go ahead and go save environment variables. Write the environment variables back out. And then the reset command will cause the CPU to reset on the device from the U-boot console. So, let's see if this will actually work. And it's kind of amazing. Out of the number of times I've done this, more than three quarters of the time when the device did boot up. Oh, that's interesting. We seem to have a crash on the system. We had a kernel panic. That's interesting. So, demos do go bad. So, we will see what we have here. This doesn't come up. We're probably just going to move on. Typically, you make these changes on the system. Yep, we got another kernel panic. I'm going to take one more check and see what corrupted. Yeah, that's correct. So, we're not going to waste much time on this. And we're just going to move on. Typically, there may be something in here. Sometimes these things go bad. But normally, just by fixing the console setting, for some reason, my kernel's messed up. I'm not sure how that happened. Maybe from constantly rebooting this thing over and over and over. Maya finally said, hey, I'm not going to play anymore, corrupted something. But typically, when you go into the uBoot console, and you've broke into it, and you have a device that will not give you a console after boot up, a lot of times you can go in here and alter either boot args or some other reference that's used for the boot arguments that tell the kernel what console, what bauld rate, all those typical things, and set that, and it'll come up. Usually, three quarters of the time, when I've encountered this on commercial gear, where they've actually turned the console off trying to hide access, when I do allow it to boot up, it's giving me root access from the very get-go versus giving me a password prompt. It's giving me root access. So, again, the best way to do this is check the console settings, make sure they're correct, make sure they haven't turned initialization off. I've seen this messed with, which would cause weird things in killing consoles and stuff like that. But typically, nine times out of ten, it's literally going to be the console setting. If it's not set, often when you identify the processor that may be used in this case, you can do a little Google reference and identify the most common TTY setting for that. A lot of times it's in the data sheet and stuff like that, to identify how it is naming its primary console TTY. So, often data sheets, a little Google on the device or the processor in case, to be able to identify that, and that helps you to literally set the console where it actually needs to be. Unfortunately, that demo failed, even though it's worked 40 times up until today. And I will stop sharing and turn this over to Garrett, so he can attempt his next demo. Is there any questions related to this particular failure? Let's see. It looks like there's a couple folks that are helping to troubleshoot, and I saw one of the issues too, I think. I guess it looks like bootargs was double set. Also, the board equals appeared to be missing. The bootargs, I think, for sure, might have been one of them. The board equals might take a little troubleshooting, but that's a couple questions, I guess, that came up. But another... Oh, yeah, got it right there. I see it. Yep. Yeah. Good catch, guys. Yep. Often when you're cutting and pasting, it's very difficult to do that. Since I didn't define the board, that obviously called it. So, good catch out there. Yeah. Props to folks on Twitch as well as here in the panel as well that kind of caught that. But there is a couple questions that did crop up. One individual asks, how often do you find that IoT devices are actually using U-boot? Like, would you say about 75%, 50%? I'd have to say it's still probably a little higher than 50% that I engage devices that U-boot used. Now, if they're running the latest version of U-boot, this typical pin glitch will not work. So, what happens is the newer version of U-boot that is out there, if anyone is actually using it, is supposed to, when it can't learn, it can't load the kernel, will actually restart the U-boot process. So, it'll just go into an infinite loop until it can load the kernel in that case there. That's how they kind of started to fix this problem. I assure you there's probably attack methods in that case with that to get around to it. But the best fix, what we always recommend to most vendors on how do you solve the problem with U-boot being an issue is start using secure boot. That actually answers the last question that I had here was, what's the best way of overcoming some of these issues? And it sounds like secure boot's the best method for that. Yeah, I would say secure boot is the best solution to these issues that we're running through today. And I think we had another question just popped up here that we can kind of squeeze in before we move on. Because there's one more demo, is that correct, Garrett? Yeah, yeah, we still got one more. But yeah, more questions, the more the merrier. Okay, yeah. So, Asher asks, do you know why sometimes the environment variables cannot be modified even with root access to the U-boot console? This apparently has happened for this individual. The only thing I can think of off the top of my head is that the manufacturer hard set some of those environment variables to not allow change even at a root level. Or there's just some underlying permissions that, you know, even though we may feel we're root, there may be some hard coded just permission set there that we just can't get into. I don't know, maybe someone else has more to speak on that. But that's just my initial gut feeling. I'd have to agree with that. I actually had a device I was playing with the other day. And there was probably a half dozen arguments in the environment variables that when I changed them and saved them, they never saved nothing. So, U-boot's a funny beast. Everybody compiles it differently. So, the amount of features, functions, capabilities, and how it works is very configurable. And every vendor will compile it and use it different. Also, I've seen some where boot args and all of those settings are not even used by the kernel. That stuff is actually hard coded into some kernel configurations. So, alterations with boot args don't actually work, which can be problematic too when you're trying to carry out some level of tax. Based on that, I think what Garrett's going to demo here just shortly will actually open up some other possible opportunities. If you're able to break in to the U-boot console, some of the stuff he's going to show may be very helpful in gaining a level of access into the system. Awesome. Any other questions out there, John? I think we're kind of on the queue. I think we're good to go. All right. Right on. Moving on. So, next, we're going to be going over the Luma. The Luma is that mesh device that we were looking at. It's got a little bit more to it on the motherboard itself. There's just a lot more going on. It has onboard Ethernet. It's got a USB port. It's got several different places to connect wires. There's some headers. A lot of stuff going on. I think it's got a built-in Wi-Fi chip on it, possibly. I'm not sure. But a lot of stuff going on there. So, we're going to boot to root here real quick out of these slides back into the live action. Here we go. So, now I got my Luma set up here, broken down, ripped it apart. I've got my UART connection over here with my ground, my receive, and my transmit. We're all good. I have my Ethernet actually plugged in because I'm just going direct line to this thing. And then I also have a ground wire that you can't really see. But there is a ground wire soldered onto the board over here. So, the way that I did get into this thing initially was a pin glitch. And I'm not going to show that today. But if it's something you want to see, I can demo it later if we have time. But basically, this particular chip right here is the chip that we are compromising to break into this thing. And it's actually this pin right here, this data pin right over here, the second one up. I don't know if you can even see that. But there's eight pins around that square chip. And we're going to short the data line or the data pin on that thing. So, that's how we would have got into this thing. But since we already did that, the magic of television, we're just going to plug some power into this thing. Okay. We got power now. And a little bit more slack in that line. The trusty toe. Come on, toe. There it is. And we're connected. And there it goes. So, what's funny about Cool Term is you have these receive and transmit lights going on in the bottom. So, we know that we're getting data in some way somehow. And again, with the boot up process, a lot of information here. And obviously, we can all read extremely fast. That's how we're able to read everything that's going down the screen. No, probably not. There's ways to extract this if you wanted to. Kind of if you recorded this and just played it back slow. And you could be taking like screenshots as it goes through. There's plenty of ways to kind of extract everything that's going on here. This one actually takes a little bit longer to boot up than the Hue. I think just because it has more functionality to it. It's not just going to turn your lights blue or green or red or whatever the ambience for the evening happens to be. This time, it's actually going to provide internet connection. So, when people come over or you have parties, not so much anymore. But when we did, you want to have everyone be able to have internet. We got to be staying up to date on our socials. We got to be, you know, staying up to date on everything. So, internet is good. So, we may not wait for this whole thing to boot up. I think it takes about a couple minutes. And my patience is weak. So, as this thing kind of rounds the bend here, it looks like it's doing stuff with Wi-Fi. Looks like some connections were going on, some different channels. Let's just see what happens. Oh, okay. So, we got a login prompt. We can try to log in. I don't know the password, though. Let's give it a shot anyway. Login incorrect. So, no luck again. We're 0 for 2 there on trying to just log into these things. So, what can we do? We're just going to power cycle this thing. All right. So, clear. Clear. All right. Pinky toe in action. All right. So, like I said before, the, you know, how the cooking shows do it, they don't cook that whole turkey right in front of you. That would be crazy. But they have one already in the oven. So, that's kind of what happened here is I pin glitched this thing prior to this call so that I did set this boot delay so that I could break out of it and get in there and actually show you some stuff. So, similar to before, print environment variables. We can just ask for help, though. Just see what's different about the Luma versus the Hue. So, some slight differences. This one's not as long as far as the list of items that we can do. So, just kind of piggybacking on that final question before is, you know, U-Boot is configured differently on every device, or essentially it can be. So, I was even reading up on this company called Emac. I think they do, Skada devices. And the way that they configure U-Boot was completely different than another person who configured U-Boot. Because when I was looking things up on this stuff, I would find several different explanations. Like, well, wait a minute. Is this even the same thing? Like, yeah, it is. It just can be completely configured, you know, in several different ways. Or compiled, I guess is the right word to use. So, if we look at some of these things, what jumps out to me is the boot arguments, the boot IPQ, boot from a flash device, boot from memory, boot from a network connection using TFTP. Spoiler alert, maybe. We got ping, so we can actually test ping connections. Print environment, like we already know. Save environment, set environment, that's all there. Print the memory flash information. There's a TFTP boot command right there. Oh, and a USB boot. So, a lot of stuff. This thing has a USB onboard, so, you know, there's more opportunities here. So, today, let's get back into the print environment. Let's take a look back at this. Today, we already have set the boot delay. That's already done. Boot delay is set. This boot command, though, is set to that boot IPQ. And if you don't remember, we can just quickly look back at that boot IPQ. What the heck is that? So, here it is. Boot IPQ. And it's to boot from flash device. So, that's what it's set out of the box from the manufacturer there. So, if we get back into those environment variables, what else can we mess with? Let's see. So, IP address. Ah, IP address is important. And we also have server IP. So, what can we do with those? So, basically, this IP address, this Luma device does not have networking configured out of the box. So, when it turns on, it just has a random IP in there. And I just set it to an available IP on the network that isn't already being used by the Xbox or the PlayStation or the 10 Raspberry Pis or the N64. You know what I'm saying? All these devices out here. So, I just chose a random IP. Doesn't really matter. What does matter is the server IP. The server IP is going to be the TFTP server. So, we're gonna be using TFTP to boot this thing or to boot from TFTP. And that's where this extra server comes into play. So, I got my just a Ubuntu 16.04, just fresh ISO I just pulled down the other day. And just found a couple articles online on how to set up a TFTP server. It's really straightforward. You know, if I can do it, I'm sure anybody on the call can do it. Because it's just creating a file, setting the right parameters in that file, and then making sure the service is up and running. I can't tell you how much pain I was having literally yesterday about trying to get this thing to work. And literally, I didn't really change much. And it just started working again. So, we're gonna see how well today goes. But anyway, it looks like it's up and running. We can just see if this is still good to go. All the passwords, super secret. All right. So, we're still running as of 1529. So, right on schedule there. Cool. And if we look at the IP of this device, that might help, right? I have config. Let's see. So, this is 192.168.118. So, that's where that IP comes into play. All right. Cool. So, jumping back to this now. So, this boot command is set to this. We can actually override this particular boot command here. And these boot arguments. Sorry, there might be a neighbor calling for their cat. So, I've actually already gone ahead and wrote out this command just for sake of typoing this thing a million times. I'm just gonna copy it over. And if you notice, I didn't show it, actually. But if I go back to my VM, I have a kernel image over here that has been compiled using open WRT. So, it's this is the package or the image that we're gonna be calling from my TFTP server. So, I just dropped it on the desktop here. I have it in several locations just because I was testing. So, I think it just needs to be on the root drive, just accessible. So, good stuff there. And now that we're back here, I'm just gonna paste this command. Cool. Just like that. So, I'm using TFTP boot, which is one of those help arguments, if you remember. There was a TFTP boot argument that we could use. And then I'm telling it, which piece of memory do I want to boot from? And I want to boot this particular image. And then boot at this particular address in memory. So, that's what this command is meant to do. So, without further ado, let's see how it goes. Let's just hit the big red button. And right away, you can see that it's using an Ethernet device, eth0. So, it is connected. It's from the server, my TFTP server at 1.18. And then from our address, .25. So, looks like we might be in good shape here on the first go around. How about that? Looks like it's uncompressing the kernel image. It's booting it up. You can see the OpenWrt right here, if you catch it. And then it's off to the races. So, as this thing kind of boots up, we'll see if we got any further than that login prompt that we had before. And if you're not familiar with OpenWrt, it is open source. And those can also be kind of compiled a little bit differently depending on, you know, what you want to do with it. So, by the way, the devices, if you wanted to buy these devices, very cheap on eBay. You can pick them up here. Very cheap. Just search eBay. But this OpenWrt, it's basically just an open source project that you can compile different, you know, different things and connect to different devices. It's just fully customizable. So, that's a little bit about what's going on there. Let's see. So, as this thing kind of boots up, see if we just hit one of these random buttons over here. Boom. So, that did it. We are in a built-in shell on the Luma device. So, this technically is not the Luma OS. So, I want to be clear on that. We didn't just root the whole Luma device. What we did is we created a kind of a landing place for kind of like before the OS hits the ground, we created our own OS and said, hey, we want to boot to our OS before you even get to yours. Okay? So, that's what this is. And so, it's not the direct file system on the Luma. But we can still do things similar to the Hue where we can, you know, dig into the root file system. We can look at the kernel, the firmware. There's a lot of things that we can potentially get into. There's several different techniques to do that. One of them would be mounting these drives. So, assuming that we had full permission, then we would just mount these drives and start extracting data from there. If we don't, you know, in a situation where we don't have access, say like for right now, like we have all zeros here. So, it's possible that we don't have access to this right now. So, what we could do is use something like DD to extract the data that way. It's a command line tool or something like TFTP and just siphon the data to another medium and then perform, you know, additional analysis there. But as far as from where we are right here, there are some things that we can do. But it is kind of in this open WRT console now where we'd have to get a little bit creative and kind of use our noggins and find another path to the Luma OS. But it's possible. So, that's a little bit about what we got going on on this thing. We wanted to show, you know, booting to an OS that we compiled on the device itself. So, that's a little bit about what we got going on here. Any questions on how we got to this point? Yep. A couple questions here. Let me take a look. One individual asks, I assume all of these attacks require physical access. There's no way of performing this attack remotely. Is that correct? Yes. So, unfortunately, if you did want to, you know, start doing some war driving or something or mess with your family members, whatever, it's all ethical, right? You know, you would have to have physical access to these devices. I imagine you could get physical access one time and then set up some type of logic bomb or some type of, you know, hook into the device so that if it were online, it could call to us in some way or somehow. Probably some of the devices that have internet connection more so than not. But I know that I have the light bulbs from Costco. I think they're called... oh, man, now I'm drawing a blank on it. But it's basically the same thing as the Philips Hue. And it's the Costco version. I can't remember the name of it now. It's driving me nuts. But basically, they were on the internet, and they didn't have very much any security. And I was just afraid to even mess with it at that point. I'm like, physical access to this thing. But, you know, you never know. So, good question. All right. Taking a look at another question here. As far as... I think this was touched on a little bit earlier as well by Daryl. But how do you know which pin to take to ground when glitching? I know earlier we were talking about data zero, data one. But as far as testing is concerned, how would you know about taking which pin to ground whenever you're performing this type of glitching? Yeah, that's a good question. I do have the breakdown of that flash chip on another screen somewhere, somehow. Bear with me. Can you still see my screen? Is it still showing questions? Yes. Okay, give me one second and I can get that readout for you. But basically, it is a data line, and it is the one that we would want to break into. So, when you look at the data sheets for these devices, you do want to look for the data lines is kind of the idea in finding out where those actually exist. Let's see if I can't get a good shot of this. Give me one second. Okay, got that now. Let's just minimize some things. Okay. So, see if I can't get a better view of this device here. Okay. So, you can still see my screen. What I have on there now is basically the top down view of that exact chip. So, if I look back to the camera, this is the chip breakdown, this one right here. I don't know if you can see that, but the one with the little orange paint drop. So, that's what we're breaking down in this diagram here. If I can get my pen out. There we go. So, the line that we're shorting is this SO line. This SO line over here, that's the data line. I can't remember exactly what it stands for. Maybe Daryl remembers, but that's essentially what we're shorting in this example. Yeah, exactly. There's also an interesting question that came up. When glitching, would it be a little safer to use a resistor in lieu of a paper clip? There's a possibility you could short a 3.3 or 5 volt trace pin and burn a register. Yeah, there's always a chance. That's why it's very important to... I'm not against using a resistor. I mean, there is some value to that. If you'd have to experiment around, because obviously you want to interrupt the data significantly enough, and a resistor may or may not drop the voltage enough based on the resistor size. But in reference to doing that, again, like he's showing here, go ahead and get the data sheets, figure out what you're grounding out. So, literally, you don't like just smoke something. To randomly go into a device here and go, oh, I think I'll just start grounding stuff out. It's almost a guarantee that you will brick the device permanently. Now, I've had on these glitching attacks, probably successful. Anytime there's two chips used, where U-Boot is on one chip and the kernel file system is on another, I've had a hundred percent success every time in actually doing that. I've only had a problem on one device, and it was using... I can't remember the chip it was using. It was very much more high speed, so getting it at the right time was difficult. We never destroyed the chip, but during the attack, even though I was taking the data line down the ground, I ended up actually causing it to screw the U-Boot up, and it actually damaged the U-Boot so that the chip would not boot up anymore. It would come up actually giving you an error in the U-Boot. Now, how it altered part of the data in the U-Boot during this process, I have no clue. Maybe it did fry register in there or smoke something, but we were able to pull the chip and then successfully pull the entire operating system off and recover it all anyway, so problem solved. Yeah, and just to speak on that note too, I've definitely broken enough devices just tinkering just like that and not really looking, you know, doing the right diligence of looking at the data sheet, looking at what am I actually messing with. It was more like, it's how it works in the movies, so it's how it works in real life. You know, I'm just going to touch a bunch of stuff and now I'm in. So, I definitely have a bunch of devices that just are paperweights around, scattered around. Yeah, that's funny, dude. Good stuff. So, that is getting into the Luma and being able to boot up, another operating system such as OpenWrt. So, if those devices aren't readily available to you, like the Philips Hue and the Luma, things like this, or they're the ones that you use in your house and you're not trying to lose internet in your house, totally get that. But if you have a Pi laying around, you can do the exercise with a Raspberry Pi. There's several articles over the internet that you can find how to do this, how to set it up, what you can do with it, and it's pretty wide. So, this is just an example of a Raspberry Pi that we had connected, one of my colleagues set up for me for this demo. So, then you have Uboot going on on the right-hand side. It's doing the same kind of song and dance of getting into the flash memory, the kernel, and actually printing the environment variables so that we can tweak them and adjust them and then tell it what we wanted to do. So, it's just another example of a Raspberry Pi. Cool. So, that is all, folks. I really appreciate everyone who was able to join this session on a Saturday when we're not in Vegas. I'm assuming we're not in Vegas, but normally hungover at this time of the day and, you know, completely, you know, con funk is in full force. You know, I don't know what time it is. I don't know where I am. That's normally the status of this day for me. So, it's a little bit different doing this thing virtual. So, I appreciate everyone who was able to join and all the questions and, you know, really happy to be here. So, if you have any other questions, throw them out there. But that's the end of the show. Yeah, there was one more question I saw come in in the chat. It said, did you download someone else's build for the Luma or did you do it yourself? In this particular case, this build was... I actually built this a while back while I was doing some testing on the Luma. So, I used OpenWrt's program. This is built off an IPQ40 series processors, which is what the processor is on that particular Luma. And it's one of the chipsets that's actually supported within the OpenWrt. Now, in this particular case, the success of this wasn't 100%, as Garrett pointed out, that the root FS was showing nothing there or zero location. And the reason why that is, is because, remember, this is a two-chip system. And when you're building an actual firmware package, you have to build out a device tree. Now, most of all of the data available on OpenWrt was very well available dealing with one-chip environments. But the device tree for the second chip has not been built into this yet, which potentially made that an issue. But once you compile it and compile it with the right complete device tree, complete it, then what he showed with PROC MTD, which is the memory technology devices, kind of an abstract layer between the hardware, the physical flash hardware, and the other part, you can literally from those potentially mount those file systems up, or you can also DD them off by actually calling them directly through dev MTD 0 through 9 or 10, which is one of the methods. It's really easy once you get some kind of console, which would be the ultimate goal of bringing in a new kernel via TFTP, because you would come in, it would have the device tree, you would see the chips, even though they're not mounted up, you can easily DD all of those images off their file system, the root file system, and everything from a separate booted kernel like this, giving you the ability to do some offline binwalk and extract the data, hopefully find the root. Or inevitably, you could even, when you find the root, you could easily in this particular case, DD it off, explode it out, and then alter it, rebuild it, repack it back in, and then DD it back over that partition area, and then reboot the device. And in that fashion, actually change the root password. So this particular attack, as he addressed and talked about, will really open up a lot of new avenues to some hardened devices. But it all comes down to, can you get access to that UBoot console? And also, the presentation or the demo I ran, and the person pointed out where I missed board, that was correct. As soon as we jumped off and he moved on, I went ahead, fixed that typo, failed to copy and reboot it, and it came up to the root prompt. So that worked out pretty good. So good shout out to that person, was able to visually, quickly capture the idea that I missed that little piece. So totally cool. Nice. Yeah, I just wanted to add on that. Sometimes on the devices, if you can still see my screen, the prompt, the command prompts on the Luma is actually the name of the chip, the IPQ40. So sometimes you can get even more details from just the device itself, just telling you like, hey, this is what I am. Hope this helps, you know. But yeah, I just wanted to call that out. I didn't mention it earlier, but IPQ40, like what the heck is that? It's the name, it's the type of chipset that's on this thing, type of processor. So cool. Well, really appreciate everyone, if that's all the questions. Not sure if there's any more, but we're good to go, right on time. Yeah, Jonathan, do you see anything else in the Twitch we might have missed? I'm taking a look here. I do see that in Zoom, someone asks, it looks like it identified the attached flash as well. So I guess from the U-Boot output there, it identifies the flash. Pulling up Twitch, I think from Twitch, we're also good to go. So the last question, I guess, was there and the question was, it looks like it identified the type of flash. And that might be kind of equally a statement as it is a question. It did. It identified the one of the flash chips. So could you actually from that booted kernel potentially get to those current chips? Yeah, it's highly possible. You may have to do some manipulation. I haven't tried it yet. But typically, the easiest way is to build it into the device tree for the kernel that you're building for that processor. That's the most effective way. Once that's built in there, it makes it 10 times easier for gaining access and basically altering this data or extracting that data off there versus not necessarily have to figure out how do I access a chip when I don't have a device tree for it. It's going to be very difficult. So yeah. Sorry, I had someone that was playing the Linkin Park at the full capacity of the volume there. So I apologize as I drove by. Okay, yeah, you have one person out there says you need to put that cowboy hat on behind you. So you look like Billy Ray Cyrus. Oh, y'all are cracking me up. Oh, man. I appreciate y'all. Yeah, let's see what we got. Here we go. Hats on. We have to take this off. But there we go. We're locked and loaded now. Pretty great. This is how you find the real bones. Nice. I mean, the people in the audience, unless the presenters need to leave, we still have time. There's no rush to close out quite yet. There's 30 minutes before next talk comes on. So we still have just a little bit of time probably we could do another five minutes if the presenters are still available. Hey, let me go ahead and let me go ahead and boot my failure. So let's switch over to that. So let me go ahead and share my screen. Let's get that going. Okay, hopefully everyone can see everything fine now. So let's go ahead and power this. Watch this thing fail again. I should know when to walk away to a demo failure. So let's go ahead and stop here real quick. Let's go print environment. So we can see what I altered. So I altered standard boot. When I copied it over, I missed this word right here, board equals. So it ended up specifying this board. And what's kind of strange is often when you do boot args, and I've mistyped things before, I'd never had it cause a kernel panic like that before. So it's kind of interesting. I've had it just skip over that functionality or whatever. So that was kind of interesting. So the change I did this originally said null, we're able to identify with a little legwork watching the system normally boot up, looking at some of that stuff, identify the type of TTY and replace the console null out of there. So it's common to see these options also on devices that are outside of the U-boot where it says press F key, enter into failsafe mode. Sometimes these will put you into a single user mode when the vendor's crazy enough to leave them available. Something to think about from when you're watching a boot option, it's not part of the U-boot stuff, but it can be very helpful. It will put you in a very locked down system. There won't be network, there won't be device drivers, there won't be nothing on there. So inherently, there you go. So instantly you're able to get past that. And like I said, this is kind of a common issue that you can run into a number of times where they've just basically said, turn the console off before you boot kernel or turn the console off when kernel, before kernel runs. And because of that, you don't get a console. If you can gain access to U-boot, probably eight out of 10 times, you can change those settings to be able to gain access to a console. Now, whether you'll have full root access after boot is to be found or figured out. But believe it or not, I think over the last so many years, I've encountered three or four devices that had it set up to go ahead and halt the console so you can have it. And when I changed the argument so the console was enabled, at least half the time, I had a full root console at that point. And these settings here that I mentioned, the press one, two, just select the debug mode. Never had much luck with those. Sometimes they do things, sometimes they don't. But the F key, if you ever see that, takes you to failsafe mode. This will give you root level access, but it's amazing how complicated with root level access and no ability to move data on or out of the device. It's doable, but it becomes very problematic because often you actually have to build the device nodes. And if they harden the device and turn certain programs off and remove them, it becomes more difficult. From the Luma, just out of curiosity from the Luma, this had this and I was working to try to get data off of it. In a way I originally did it was go into the F mode and actually build, manually build all the device, not totally manually, but very close to manually reconstruct the device nodes, naming them all the correct pieces and parts to build a USB. And then from there was able to move the data off to the USB. Just some stuff to think about. A small question just came up, Daryl. I know you had mentioned this yesterday during your building the IoT lab. What is your FTDI device you're using for that one? Its name escapes me. Someone was asking on Twitch. Oh, gosh, this one I have right here. Yeah, it was that cool one that could do like more than just it had like it supported like SPI and others in addition to UART. Yeah, that's the one. Let's power this off. Let's see if this has this name, has a name on it. I don't know if this even has a name. It's very generic and I do not even have the box for it anymore. I bought this off Amazon. All I did was search for multi UART connector or multi port UART connector and this came up. If I remember the price of this was like twenty seven dollars. But to to the life of me, I do not know the name. This is a generic build device coming out of China and it works pretty good. So cool. But sadly, I do not know what its name is or who makes it. Gotcha. Another question cropped up. Is there a URL or GitHub of some type where one could get ROMs to access a processor or access to a processor to play around with and do this type of stuff? I think they're asking, is there, I guess, any like I guess I think I think what's being asked with it is are there maybe like a list of devices that that could be purchased in order to do things such as U-Boot exploitation? I would actually literally exactly what Garrett said, go to go on to eBay and look at buying some of the devices on eBay. I think would be bright. You get the Lumos. You can get the you can go ahead and get the Hughes out there. He actually had a screen where he was showing some that were available. They're not expensive. I'm not sure how this would work. This is a product that that I was playing around with just recently, and it's like 19 or 20 bucks. And basically, it has OpenWRT running on it right now. It has serial connections. There's a flash chip so you can practice pulling the chip off if you want reading it or using SPI chip reading techniques. You can probably trace out versus some of these headers or connectors. You may be able to trace out the actual JTAGs on this for the chip. It has Ethernet. It has Wi-Fi. It has USB. And literally, it's like 20 bucks. So this would be an easy, cheap device to possibly play with. And that's Vixby 300 wireless travel router. I would try playing around with that. But I have to admit, this was kind of footsy when it came to the actual U-boot arguments. I had a couple arguments that didn't work, and I couldn't alter and stuff like that. Here's another device. I think it's the exact same product. They're just white-labeled different. GL-iNet and Mini Smart Router also has all the same functions and features on it. I'm not sure if the Mango is still on the market. May be able to buy one used. But a lot of these cheap devices are out there, and it gives you a chance to experiment with UART and JTAG and SPI chips and sometimes NAND chips and flash chips. I mean, the list goes on and on and on, and they're often cheap. Like I said, Luma I like a lot because it has all of those same features, and you can get them for 20-30 bucks used. It has the USB. It has the Ethernet. It has UART connectivity. It has NAND flash chips on the device. It has SOIC8 SPI flash memory chips on the device. So it's a perfect platform for hacking, attacking, experimenting. The pin glitch works on it also as an example. Awesome. I was just catching up in the chat. It was a F-E-I-T, however you pronounce that. It was a fight bulb that I had from Costco that I just decided to put and leave in the drawer because just didn't feel very secure about it.