WEBVTT 00:00.640 --> 00:08.200 NAND flash chips, a little different than standard flash chips, be a little more problematic. 00:08.200 --> 00:13.040 So we're going to talk about that and kind of step through with a number of demos. 00:13.040 --> 00:15.100 So great, let's go ahead and get started. 00:15.900 --> 00:21.360 So thanks everyone this evening or this afternoon based on where you're at in the United States. 00:22.140 --> 00:26.860 Presentation is NAND chips, introduction to NAND chip extraction and data reconstruction. 00:27.020 --> 00:29.760 This is an intro primer. 00:30.000 --> 00:34.800 We're going to cover a number of different topics and we have like three different demos. 00:35.020 --> 00:37.420 I think you'll get a lot out of this. 00:37.620 --> 00:38.680 So my name is Daryl Hyland. 00:38.680 --> 00:43.560 I'm the Principal Security Researcher for IoT at Rapid7. 00:43.640 --> 00:51.080 So section one, we're going to just give a basic introduction and look at structure. 00:51.240 --> 00:55.440 So what is a NAND flash memory? 00:55.580 --> 00:56.880 Simple, you can read it. 00:56.880 --> 01:03.020 It's non-volatile storage technology that doesn't require power to retain the data. 01:03.020 --> 01:04.340 Pretty straightforward. 01:04.700 --> 01:06.300 See those references there? 01:06.300 --> 01:15.620 If you're interested in learning more in the nitty-gritty, take a screenshot of that real quick because that is great references. 01:15.680 --> 01:27.020 I go back to these quite regularly to kind of refresh myself because some of this stuff can get pretty deep as you're trying to reconstruct data in a number of cases. 01:27.140 --> 01:29.680 So go ahead and get a screenshot of that. 01:29.680 --> 01:31.720 I'm getting ready to jump off of it now. 01:32.100 --> 01:34.580 So a couple of things. 01:34.580 --> 01:41.060 When we start thinking about NAND flash memory chips, you need to go to the data sheet. 01:41.060 --> 01:45.880 You really need to understand how things are laid out. 01:45.880 --> 01:49.740 This will play a part in recovery a number of times. 01:49.780 --> 01:56.780 So this is the memory array structure that lays out the page, the blocks, and the planes. 01:56.920 --> 02:04.040 In this case, the page is 2K in size or 2048 bytes. 02:04.040 --> 02:09.700 And then there's what's referred to as an OOB area, a 64-byte area at the end. 02:09.780 --> 02:11.020 And this is critical. 02:11.020 --> 02:12.080 So what is OOB? 02:12.080 --> 02:12.980 It's out of band. 02:12.980 --> 02:14.440 It's a spare area. 02:14.440 --> 02:24.500 It's used for containing error correction code information, bad block information, and it exists in every page. 02:24.500 --> 02:26.000 So what does that mean? 02:26.000 --> 02:35.740 Generally, what it means is you have all your data jammed full of potential garbage when you're trying to parse it out and rebuild the data. 02:35.740 --> 02:41.060 And this could be impactful and can cause problems in some cases. 02:41.060 --> 02:46.400 So we're going to talk about how we're going to fix that, and we're going to show that in a couple examples. 02:46.520 --> 02:51.760 The OOB area, the out-of-band area, is also laid out in two formats. 02:51.760 --> 02:57.040 One is known as adjacent, and one is known as separate. 02:57.160 --> 03:00.700 If it's laid out, the OOB is laid out adjacent. 03:00.700 --> 03:11.800 So a 2048-byte page would be laid out with a 16K or 16-byte of OOB after every 512 bytes. 03:11.800 --> 03:14.200 This would be an adjacent layout. 03:14.920 --> 03:18.540 The most common I see is what's known as separate. 03:18.540 --> 03:26.160 A separate area is the 64 bytes of data exist at the end of the 2048-byte page. 03:26.160 --> 03:27.960 This is the most common. 03:27.960 --> 03:36.080 But often you need to verify that as you're working through this and trying to remove the data or correct the problem. 03:36.080 --> 03:45.220 Another thing that comes in handy, and this is in reference to a number of the tools that we're going to use today, your chip ID code. 03:45.220 --> 04:07.840 An example, anytime you use a flash memory reader, so you pull a chip out, drop it into a memory chip reader, often it will, when you set it up for that chip, it associates that with a ID, and it'll read the chip, compare the ID on the chip to make sure it matches before it tries to read the data. 04:07.840 --> 04:15.620 Now, a lot of applications out there have a use for this type of chip ID data. 04:15.620 --> 04:18.020 This is out of the datasheet also. 04:18.020 --> 04:28.260 And in this example here, it's showing the datasheet for a S34MS02G1 chip, which is a 512-megabyte NAND flash. 04:28.260 --> 04:32.840 And you can see at the bottom, it shows what that chip ID is for that chip. 04:32.840 --> 04:40.580 And you'll see that we'll use this later on, and we'll probably come back and look at it in the datasheet again when we get ready to do that. 04:41.860 --> 04:44.820 So, at this point, are there any questions? 04:45.300 --> 04:52.340 So, Jonathan, has anyone posted any questions or anyone not have any questions about some of these basic concepts? 04:53.560 --> 04:54.780 Let's see here. 04:54.780 --> 04:58.060 So far, it looks like no questions posted here. 04:58.380 --> 05:02.360 One quick question that is actually coming up here that I'm seeing. 05:02.520 --> 05:19.780 Whenever you're, I guess, pulling firmware from NAND flash, do you have like a consistent philosophy that you follow, such as like always first see if you can, you know, first pull, maybe if you can like log into it via like serial port through UART? 05:19.780 --> 05:24.280 Or do you typically go straight for attempting to dump the NAND flash content? 05:24.280 --> 05:34.320 Yeah, usually the first way if I want to get the data, the most effective way is get the data when it's in a constructed fashion. 05:34.320 --> 05:52.620 That would mean if I can get a console on the device with the entire operating system running, and the entire partitions for all the data laid out, that's easier because then I could easily DD the partitions off the chip in their constructed fashion. 05:52.840 --> 05:56.680 And that would avoid me almost always having to do what we're going to talk about here. 05:56.780 --> 06:15.640 What we're going to talk about here is in that case where you actually have to read the data raw, right off the flash chip, either chip off where you remove the chip and drop it into a reader , or some other methods where you may be able to clip on or hook onto the device and read the raw data out of the chip. 06:15.780 --> 06:22.960 And that's when you get this structure that we're going to be dealing with today, and how to clean it up and a few methods of recovering that. 06:25.550 --> 06:39.110 So I shout out to everyone, if you're interested in interacting or asking questions during this, join the Zoom chat, the Zoom channel that we have set up for the webcast. 06:39.150 --> 06:46.950 Once you're into there, you'll be able to post questions up during the presentation, and hopefully we'll have time and we'll be able to answer them. 06:46.970 --> 06:48.990 So please do that if you can. 06:48.990 --> 06:51.050 It makes this way much more interactive. 06:51.050 --> 06:58.690 Instead of just being a data dump, we have a chance to actually interact with the audience and the people and learn more in this process. 06:58.950 --> 07:01.310 So I'll go ahead and move on. 07:01.410 --> 07:05.370 So section two is a general chip readers. 07:05.370 --> 07:13.330 Since I talked about chip off from the question Jonathan gave us, we get into chip readers. 07:13.910 --> 07:21.310 So when you're interacting with a NAND flash chip, here are the signals you're typically going to need to interact with. 07:21.310 --> 07:32.570 And they go everything from the 8-bit data channel, the bi-directional channel, all the way down to input signal, output signal structure, and some other various pieces. 07:33.050 --> 07:38.130 So if you end up buying an off-the-shelf chip reader, this kind of solves all these problems. 07:38.130 --> 07:41.930 But there was some work done out there by this gentleman here. 07:41.930 --> 07:45.310 I would recommend taking a look at this write-up. 07:45.310 --> 07:54.050 If you're interested in building your own chip reader, this one actually works against a TSOP48 NAND flash chip. 07:54.050 --> 08:01.510 And he uses a technique known as bit banging using this FTDI board that he has here. 08:01.510 --> 08:06.130 He's able to bit bang the data off of the actual chip. 08:06.210 --> 08:15.210 So I would look at looking at what bit banging is, because I'm not going to explain it here, and look at how he's set this thing up. 08:15.390 --> 08:20.770 If you're the hacker mentality and you like the idea of building things your own, I would check this out. 08:20.770 --> 08:21.950 It's kind of cool. 08:22.950 --> 08:30.190 If you're like me and you just like to throw money at it, these chip readers are reasonably inexpensive. 08:30.270 --> 08:35.050 The TI-866 Plus, you get all the sockets with it. 08:35.050 --> 08:36.590 It's like $130. 08:36.750 --> 08:40.370 This will read a number of NAND chips. 08:40.590 --> 08:41.530 Pretty good. 08:41.530 --> 08:43.630 And it uses a TSOP48. 08:43.710 --> 08:48.810 Right below it is a socket that doesn't come with the set that you need to buy. 08:48.810 --> 08:53.070 This socket is for from TSOP32s to 48s. 08:53.070 --> 08:54.770 It's known as NAND08. 08:54.770 --> 08:57.790 This is for larger memory chips. 08:57.790 --> 09:07.810 So anytime you encounter larger NAND flash memory chips in the TSOP48, the one that comes with the kit will not work and only work with smaller memory blocks. 09:07.810 --> 09:10.810 So you need to get the socket for that. 09:10.810 --> 09:19.950 The ProMan isn't made, this model wasn't made anymore, but they do make a similar version. 09:19.950 --> 09:24.050 So if you search for ProMan, you'll find a similar version out there. 09:24.050 --> 09:29.750 The socket below that is designed for 1.8 volts. 09:29.890 --> 09:33.870 The ProMan is typically a 3.3 volt device. 09:33.990 --> 09:38.890 And we're starting to see way more chips in the 1.8 volt arena. 09:38.890 --> 09:40.630 So it's good to have that add-on. 09:40.630 --> 09:44.950 Most of these add-ons are $8, $9, $10 on both of these chip readers. 09:44.950 --> 09:47.110 So they're fairly inexpensive. 09:47.570 --> 10:01.650 The thing I like about the ProMan is if I can't identify a TSOP48 , I can't find the data sheets or not, well, let's say I do have the data sheets and I can't find any information on none of my chip readers. 10:01.650 --> 10:04.170 They don't identify it, they can't detect it. 10:04.170 --> 10:18.330 This will let you hard set page OB block sizes into the actual tool that comes with the application that comes with this, so that you can attempt to read it using that kind of raw information. 10:18.430 --> 10:31.210 And it's been successful several times with me, making it possible for me to read a NAND flash chip where I had the data sheet to identify that structure, but none of my chip readers would recognize the chip. 10:31.770 --> 10:34.730 And the last one's RT809. 10:34.730 --> 10:42.030 And that one's a little more pricey if you buy it Amazon, but I think you can track that one down on AliExpress cheaper. 10:42.270 --> 10:43.690 I like this one. 10:43.870 --> 10:49.910 It's not a bad chip reader, and it has, you can buy a lot of different sockets for it. 10:50.610 --> 10:59.850 Most of the sockets that come with the TL866+, the one on the left, will work on this, all but the TSOP48 sockets. 11:00.310 --> 11:06.310 Because the TSOP48 sockets have a circuitry built into them, so they won't work on this reader. 11:06.310 --> 11:08.450 You have to have a straight pin through. 11:08.650 --> 11:14.930 And the one I have below shows that TSOP48 socket that's a straight pin through. 11:14.930 --> 11:24.410 I also recently had to pull a NAND flash that was a BGA ball girder array, 63 balls on the bottom of it. 11:24.410 --> 11:33.190 And I was able to buy this off AliExpress for the RT809 for about 40 bucks, which isn't bad. 11:33.350 --> 11:48.890 If you get into the high-end chip readers that run, start off at like $1,600, $1,700 and go up for them, the sockets for them start off at $100, $200 and go up into the thousands for BGA sockets. 11:48.970 --> 11:52.010 So, these things are very cost effective. 11:52.150 --> 11:55.930 The sockets are not high quality, so you have to take care of them. 11:55.930 --> 11:58.910 But they're overall, they're pretty good products. 11:58.910 --> 12:05.670 And I use them quite regularly in my lab without issues working with NAND chips. 12:06.070 --> 12:14.530 So, any questions in reference to the chip readers and chip reading process? 12:16.550 --> 12:18.310 One question here. 12:18.310 --> 12:23.830 What are the most challenging NAND packages or pitches to work with that you've encountered? 12:23.830 --> 12:28.770 Is there any package or pitch you would not attempt messing with? 12:28.850 --> 12:33.390 I would never say there's nothing I wouldn't attempt messing with. 12:33.390 --> 12:36.110 I would literally mess with everything. 12:36.110 --> 12:43.310 Probably some of the biggest problem is, obviously, if you're using a chip reader, try to read this. 12:43.310 --> 12:54.220 The BGAs or ball grid array devices are the more problematic, because there may not be a socket available for some of these devices. 12:54.440 --> 13:03.360 In cases like that, if you have the data sheets, you can always go to the process of dead bugging. 13:03.360 --> 13:22.800 That is turning the ball grid array chip on its backside and soldering small 40-gauge wire to the actual balls, and then feed that into a connector or a chip reader or a device for doing bit banging to actually be able to do that. 13:22.800 --> 13:27.280 I have been successful doing that a couple times, but it's a pain in the butt. 13:27.280 --> 13:37.200 It's way much easier to pull the chip with hot air or IR if it's a BGA, clean it up, drop it in some device, close the lid, and have it give you all the data. 13:37.480 --> 13:48.680 But typically, if I need the data off it, and I can't get it any other way other than chip off, I will chip off from the board. 13:48.680 --> 13:52.480 And if necessarily, I will dead bug one. 13:53.040 --> 14:00.300 I mean, yeah, me and you, Jonathan, we worked together at one time where we dead bugged that one weird device. 14:00.440 --> 14:13.520 We had a 0.25 millimeter pitch BGA, and we had to solder 40 wires to it or 20 wires to it to get it hooked up. 14:13.520 --> 14:15.220 So yeah, we're going to try anything. 14:15.220 --> 14:20.160 I'm not going to pass up any potential target if it has data on it that I want. 14:21.420 --> 14:22.340 Makes sense. 14:22.340 --> 14:25.080 And you're definitely bringing back some crazy memories with that. 14:25.080 --> 14:28.140 And actually, you partially answered the next question as well. 14:28.140 --> 14:32.980 The next question was, what hot air tools do you recommend for BGA removal? 14:34.120 --> 14:40.060 Actually, I have both hot air and I have a IR oven. 14:40.520 --> 14:42.680 I like the IR oven. 14:42.740 --> 14:52.900 Once I got it, it's a small it's a small device, it'll take a board that's probably eight by nine, eight by 10 inches or something like that. 14:52.900 --> 14:55.040 To me, that's the sweetest thing in the world. 14:55.180 --> 15:01.040 I usually just stick it in the oven, run it through the cycle that I brings it up to the temperature. 15:01.040 --> 15:06.240 And I take a suction tool, I pull the chip off of it, and then I let the whole board cool down. 15:07.340 --> 15:13.520 Because, you know, it heats up the whole board, you have to watch out for things like plastic components that are on the board. 15:14.060 --> 15:17.080 So you may have to remove those prior to putting them in the oven. 15:17.080 --> 15:20.400 But you can usually pick up one of these IR ovens. 15:20.400 --> 15:26.160 If I remember what was the name of the one I got, it's spelled P-U-H-U-I. 15:26.560 --> 15:31.520 It's a Chinese made product or Asian made product oven that I have. 15:31.520 --> 15:36.460 I think it's a model 2962 is what it is. 15:36.480 --> 15:37.420 Small oven. 15:37.420 --> 15:43.320 I think I only paid like $130, $140 for it, somewhere around there. 15:43.320 --> 15:45.860 And there's a number of products out there like that. 15:45.860 --> 15:50.240 But an infrared is way much better, in my opinion, than hot air. 15:53.520 --> 15:55.120 So... Another question. 15:55.120 --> 15:56.240 Sorry, Daryl. 15:56.780 --> 16:02.140 Elvis Cousin asks, is it hard to reball BGA without killing it to re-solder? 16:03.040 --> 16:03.780 Yes. 16:03.920 --> 16:09.300 It's not that hard. 16:09.400 --> 16:11.920 It's just a process of learning how to do it. 16:12.260 --> 16:17.680 Not to drag this all out, but I had a project where I wanted to learn how to do this. 16:18.040 --> 16:20.660 Because I wanted to get root access on a device. 16:20.660 --> 16:26.560 It had 153 ball BGA embedded multimedia controller on device. 16:26.740 --> 16:28.200 So I used hot air. 16:28.200 --> 16:29.440 I pulled it off. 16:30.340 --> 16:34.160 And I had a couple of devices because I wanted to, in case I killed one of them. 16:34.160 --> 16:38.540 The first one, when I put the balls back on, there's a couple ways of doing it. 16:38.540 --> 16:43.200 You can use screens with paste, or you can use just the balls. 16:43.320 --> 16:45.340 In my case, I just used the balls. 16:45.620 --> 16:51.440 I floated on there correctly, but I overheated the chip and it looked like a chocolate brownie when I was done. 16:51.640 --> 16:56.880 That's when I learned that you don't want to go on these BGA chips. 16:56.880 --> 16:59.200 You don't want to go over about 210. 16:59.320 --> 17:04.660 Well, melting temperature is about 183 degrees Celsius, if I remember right. 17:04.680 --> 17:11.240 And you don't want to go over more than 25, 27 on the temperature in Celsius. 17:11.240 --> 17:18.800 If you hit 215 or higher, it starts causing the fiberglass substrate of the chips to start breaking down. 17:18.800 --> 17:26.740 But if you keep it within those range and you don't shock it by overheating it too quick or cooling it too fast, it'll work. 17:26.740 --> 17:28.260 And I was successful. 17:28.260 --> 17:30.320 The first one I burned up and I failed. 17:30.320 --> 17:32.060 The second one was perfect. 17:32.060 --> 17:40.120 Since then, I've never had a problem ever working with BGAs, either pulling them or putting them back on without damage. 17:40.120 --> 17:43.460 And once I learned those temperature parameters. 17:46.390 --> 17:50.850 Okay, so the next section we want to move into is going to be demo. 17:50.850 --> 17:57.330 So, we're going to get off into the demo area and see how that works out. 17:57.530 --> 18:01.110 Hopefully, we won't have any technical problems. 18:01.270 --> 18:09.550 So, what we're going to do here is we have a flash memory that's been pulled. 18:09.550 --> 18:13.770 This has been pulled from... let's go ahead and look at a couple of them here. 18:13.770 --> 18:19.950 It's pulled from an Arlo device, which is a doorbell, if I remember rightly. 18:19.990 --> 18:24.790 It's actually a kind of like a ring doorbell type product. 18:24.850 --> 18:28.730 Its memory chip was a W25N01GV. 18:28.750 --> 18:33.550 So, typically when you pull this thing, what's the first thing you want to do is binwalk. 18:33.550 --> 18:39.190 And again, binwalk works many times, but sometimes it does not. 18:39.190 --> 18:43.390 So, we'll give this a try and see what happens here. 18:43.390 --> 18:46.470 Hopefully, we won't run into any real technical problems. 18:46.590 --> 18:48.490 And this is kind of the common flow. 18:48.490 --> 18:51.190 We see we hit a squash file system. 18:51.790 --> 18:57.830 We should have at least two squash file systems on there if it's coming off a standard embedded device. 18:57.830 --> 18:59.390 And there's a second one. 19:00.150 --> 19:02.470 And then it kind of hangs right there. 19:02.630 --> 19:08.310 If we sit here, this thing would sit here in idle for hours and not produce anything. 19:08.310 --> 19:09.930 So, we're going to stop that. 19:10.190 --> 19:20.550 So, if we change over to the extraction from binwalk, we can see that it opened up a bunch of stuff, but then there's a squash file system. 19:26.010 --> 19:30.030 So, as we notice, there's nothing in there. 19:30.870 --> 19:37.330 And truthfully, because it went past the squash file system file, it should have put something in there. 19:39.920 --> 19:43.680 So, as we can see, literally nothing in there. 19:43.680 --> 19:47.760 So, for the most part, this didn't work. 19:47.900 --> 19:54.140 This is actually a fresh rebuild of binwalk. 19:54.140 --> 19:59.600 And also, I have a binwalk pro account. 19:59.600 --> 20:06.560 The results are pretty much the same in this particular case because there's some things that have to be dealt with here. 20:06.560 --> 20:17.300 So, let's go ahead and remove this structure just so I don't start running into space issues by blowing all this stuff out of here. 20:18.480 --> 20:20.520 So, what do we want to do? 20:20.920 --> 20:26.640 Well, as we talked earlier, we talked about the actual OOB area. 20:26.740 --> 20:30.220 So, we want to be able to remove that area. 20:30.220 --> 20:33.940 So, we have a tool in here called NandDump. 20:34.300 --> 20:40.320 And at the end of this presentation, I'm going to have links for everything, which will cover all this stuff. 20:40.420 --> 20:44.200 So, here we can actually do input files, output files. 20:44.200 --> 20:46.660 Remember that ID I mentioned to you, chip ID? 20:46.660 --> 20:53.360 We can use chip ID, or we can specify actual page size, OOBs, and the layout. 20:53.360 --> 20:55.300 Remember, adjacent, separate. 20:55.460 --> 21:09.500 So, in this particular case, we are going to... well, before we go ahead and separate the OOB out of here, let's go ahead and look at the data sheet. 21:09.860 --> 21:16.980 So, as we look at the data sheet, we see that it is 2048 bytes by 64. 21:16.980 --> 21:19.980 So, it is in a separate format. 21:19.980 --> 21:23.400 That means the 64 bytes is at the end of the page. 21:23.400 --> 21:24.400 That's a good sign. 21:24.400 --> 21:25.960 That tells us the structure. 21:25.960 --> 21:28.180 You always want to go to the data sheet. 21:28.420 --> 21:36.220 But typically, I always like to dig a little deeper and make sure that the chip hasn't been kind of messed with. 21:36.680 --> 21:38.120 And why I say that? 21:38.120 --> 21:49.660 Because just because the data sheet says this is how the chip's going to be laid out, the manufacturer that actually used the chip, I've seen them do some strange things, to say the least, to the data. 21:49.660 --> 21:59.320 Meaning that, you know, you go and remove the OOB area, thinking you know where it's at, and literally have it not be laid out the same way. 21:59.320 --> 22:02.460 So, we're going to look at this with HexEdit. 22:02.460 --> 22:08.600 So, we're going to look at the raw data of the actual device so we can see what it looks like. 22:25.150 --> 22:26.350 There we go. 22:27.090 --> 22:40.030 So, at this point, we should be able to go to 2048, which would be 800 in hex, and it should be the beginning of the second block. 22:40.030 --> 22:43.090 So, we can see that there's something here. 22:43.090 --> 22:47.750 It looks a little different, but it may not be. 22:47.750 --> 22:49.490 We're not quite sure. 22:49.810 --> 22:51.310 It doesn't really tell us a lot. 22:51.310 --> 22:53.270 So, we could go to the next block. 22:53.590 --> 23:07.370 So, for example, if hex is 800, and we want to, of course, we want to go, remember, is 20, gosh, we get it right here. 23:07.470 --> 23:11.610 So, if we go 2048, we'll get it cleared here in a second. 23:11.610 --> 23:12.270 Try it again. 23:13.190 --> 23:23.950 2048, which would be 800, and we want to go up to the next beginning, which should be 2,112 bytes. 23:23.950 --> 23:33.310 Remember, 2048 plus the 64, and we look at this, it should be at 1040. 23:33.310 --> 23:34.870 So, if we go to 1040, 23:40.570 --> 23:54.250 we still see some interesting things, and this may tell us that this is correct, but I've seen a lot of devices where the actual OOB area was nothing but all Fs and wasn't actually configured. 23:54.350 --> 24:02.770 So, what we're going to do is let's look a little deeper so we can see some patterns, because I'm always a big fan of spotting patterns. 24:04.810 --> 24:12.770 So, we come over here, and get it going, and we're going to search for strings so we can get some data. 24:12.770 --> 24:13.990 So, here we are. 24:13.990 --> 24:19.050 So, we're in an area where there's a lot of strings, and let's scroll down until we see some patterns. 24:21.370 --> 24:23.890 And quickly, we see a pattern right here. 24:24.530 --> 24:26.850 So, got a pattern right there. 24:27.450 --> 24:28.450 So, if that is... 24:31.670 --> 24:37.570 So, we go, hold up, 2112, and go. 24:37.570 --> 24:38.570 So, it'd be 840. 24:38.570 --> 24:41.870 So, we go 840 plus E880. 24:43.710 --> 24:47.870 E880 equals F0C0. 24:47.870 --> 24:52.430 So, the next pattern should be an F0C0. 24:53.670 --> 24:57.390 And we can see that we're starting to see that pattern there also. 24:57.850 --> 25:07.970 So, as we can see, more than likely, we are witnessing the 64 bytes at the end of the 2048 as the OOB area is showing up. 25:07.970 --> 25:15.550 And again, if we look over in the string area, there's no common words, because I would not expect that to be in the area. 25:15.890 --> 25:17.630 So, let's scroll on down to the next one. 25:17.630 --> 25:25.970 There's another thing I like to do sometimes, especially if I'm having problems positively identifying this location. 25:25.970 --> 25:28.770 So, this will involve a little bit of math. 25:29.510 --> 25:32.070 So, I like to use a calculator for that. 25:32.070 --> 25:43.330 So, if we come over here, and we put this in here, like that, and we turn it to decimal . 25:43.710 --> 25:53.230 And then, because the programming calculator does not round off or have decimal points, we want to do this basic. 25:53.230 --> 25:55.870 So, what do we know about the overall structure? 25:56.150 --> 26:02.830 The first OOB area or OOB area showed up after 2048. 26:02.830 --> 26:04.590 So, we had the first 2048. 26:04.610 --> 26:09.230 And then, to get to the next leading ones, we had 2112. 26:09.230 --> 26:10.910 So, we go ahead and subtract. 26:11.570 --> 26:14.370 And hopefully, my math works here and doesn't mess it up. 26:14.770 --> 26:15.850 2048. 26:16.350 --> 26:21.750 And then, divide it by 2112. 26:22.090 --> 26:23.790 And hopefully, we'll get an even number. 26:23.790 --> 26:24.590 And we did. 26:24.590 --> 26:31.550 If you do not get an even number, then this is not mathematically correct for an OOB area. 26:31.550 --> 26:45.830 So, what we're seeing here is 0500F900 is actually the 39,749th page of data in this particular device. 26:45.930 --> 26:46.570 Okay. 26:46.570 --> 26:57.380 So, now that I punished you with some math, we're going to go ahead and strip that out of it. 27:01.280 --> 27:05.800 So, we have this command called NAND Dump PI. 27:05.800 --> 27:09.560 We're going to use .I Arlo for the input. 27:09.560 --> 27:11.560 The layout's separate. 27:11.800 --> 27:14.640 Page size happens to be 2048. 27:14.640 --> 27:16.720 The OOB size is 64. 27:16.720 --> 27:20.740 And then, the output will be Arlo No OOB. 27:20.860 --> 27:22.700 Hopefully, this runs. 27:22.700 --> 27:24.520 And it finished up. 27:24.820 --> 27:33.760 So, there, once we finished up, we have a file that's a little smaller, as you can see, where the OOB's been stripped out of it. 27:37.730 --> 27:45.970 And let's go ahead and run this and see if this works for us. 27:53.090 --> 27:54.790 The first squash system. 27:54.790 --> 27:56.850 You see, the pattern's different now. 27:56.950 --> 28:03.650 And then, we also see that there's an OOB structure also starting to show up. 28:05.050 --> 28:13.610 Often, you'll see inside an operating system on embedded device, you'll see squash file systems, which are typically read-only. 28:13.810 --> 28:17.130 OOB file systems allow read and write. 28:17.130 --> 28:25.350 So, it may be an area that's used dynamically that's being created that can use to read and write data into the device. 28:25.350 --> 28:25.870 Okay. 28:25.870 --> 28:27.850 So, that one actually finished. 28:27.850 --> 28:33.610 So, let's go ahead and jump over here and see if it worked for us. 28:34.210 --> 28:35.370 A little different there. 28:35.370 --> 28:36.750 We see we got three folders. 28:36.750 --> 28:38.590 We have an OOB root system. 28:38.690 --> 28:41.890 So, we go to squash root. 28:43.570 --> 28:45.170 And there you go. 28:45.170 --> 28:53.870 We were actually able to extract this data by effectively removing the out-of-band data within the NAND flash dump. 28:54.790 --> 28:57.690 So, do we have any questions? 28:59.110 --> 29:01.410 Yeah, we got a couple of questions here, T-Dub. 29:01.410 --> 29:05.890 So, Elvis Cousin asks, I have two BGAs on a daughterboard. 29:05.890 --> 29:13.190 Is it likely I can read each individually, or will I need to reassemble them somehow to try to get a squash FS? 29:13.190 --> 29:15.990 And they're also both Intel-based flash. 29:16.990 --> 29:22.690 Oh, you're saying you have two flash memory chips on the device. 29:23.270 --> 29:26.670 Really comes down to how they lay those out. 29:27.610 --> 29:32.770 I would not expect them to stripe across separate chips like that. 29:32.770 --> 29:34.550 I've never seen that. 29:34.550 --> 29:43.330 Often, when I've seen two chips done up like that, each one will contain a different partition, or they may be redundant of each other. 29:43.330 --> 29:47.790 But typically, they'll be partitioned separately on embedded. 29:47.790 --> 29:49.570 That's the way I've seen it. 29:49.570 --> 30:03.590 And I've seen everything from where one chip actually contained the boot, U-boot, and the kernel, and the other one contained the file system, as an example. 30:03.590 --> 30:10.630 So, the only way is to pull them off, examine the data, and make a determination from there to see if you can do it. 30:10.630 --> 30:19.550 Of course, the best solution is if you can get a root console on the device somehow, then obviously, you could figure that out more quickly and DD the partitions off that way. 30:19.550 --> 30:22.490 Which is also the sweetest way of doing it. 30:23.130 --> 30:28.410 But if you have to chip off, yeah, you still may have to do some analysis of them. 30:29.610 --> 30:39.390 Yeah, you kind of pivoted off that, said, both are connected to a common connector, but I don't have a pinout to try to recover a cross connector by soldering something. 30:40.730 --> 30:42.810 Yeah, it's kind of hard to say. 30:43.330 --> 30:57.090 If you have some kind of pictures or images of the data, or not the data, the chips and the layout and the board, shoot them to me sometime, and maybe I can take a look at it and help you come up with a plan of attack. 30:57.550 --> 31:00.810 Like I said, anyone's feel free to reach out to me anytime. 31:00.810 --> 31:05.850 I may or may not have an answer for you, but I'd be more than glad to give you a hand if I can. 31:06.850 --> 31:09.270 One final quick question here that I'm seeing. 31:09.570 --> 31:13.550 So, in this particular case, it looks like you're able to find the data sheet. 31:13.590 --> 31:20.110 Have you ever encountered a NAND chip that you wanted to dump but couldn't identify it or could not locate a data sheet for? 31:20.110 --> 31:22.410 If so, how did you approach that? 31:23.170 --> 31:26.070 Yes, guessing. 31:27.830 --> 31:46.250 Often, if I can't find a data sheet actually for that, I've actually gone out and looked at other chips that may have data sheets that are in that same series or produced by that manufacturer and figure out how they do their layout. 31:46.290 --> 31:58.390 Even though I may not be able to quickly identify the chip ID, there are ways to actually enumerate those, and it usually typically need a data sheet to do that. 31:58.530 --> 32:18.630 But guessing does work, and crazy as can be, I've actually dumped a lot of chips by going, hey, here's a manufacturer, and it seems like most of their chips, or a large portion of them within this structure range, are laid out this way. 32:18.630 --> 32:21.430 They have this page structure, this size. 32:21.510 --> 32:32.110 Often the naming conventions, like here on the screen, W25N01GV, that's one gigabit chip. 32:32.730 --> 32:37.390 And often in the name, you can figure out what the chip is a lot of times. 32:37.430 --> 32:45.150 So, that would be eight bits, 128 kbytes or whatever. 32:45.690 --> 32:54.770 So, it's doable just looking at the name sometimes to pull some typical information out of it, and then guessing on some of the other stuff. 32:54.770 --> 33:00.930 But again, I've also smoked chips by doing weird things to them that obviously didn't work. 33:00.930 --> 33:04.810 So, got to take some caution and take some risk. 33:05.610 --> 33:08.970 But amazingly enough, it can be successful. 33:08.970 --> 33:25.390 One of the things to do also is kind of look at, if you can get a UART console onto the main processor, let's say the main processor has a console, even though you can't get root access to it, capture all the debug data from the boot up. 33:25.810 --> 33:29.690 That can also be very revealing sometimes. 33:32.050 --> 33:34.030 Okay, time to move on. 33:34.350 --> 33:36.970 So, we've kind of done that. 33:36.970 --> 33:38.730 We're able to get the data off that. 33:38.730 --> 33:41.370 I'm going to go ahead and clean this up. 33:42.310 --> 33:47.130 Because we have a couple exercises to go to and I don't want to run out of time. 33:47.130 --> 33:47.730 So, 33:52.460 --> 33:54.640 okay. 33:54.640 --> 34:00.240 So, the next one we want to look at... I'm also going to RM the article 34:03.530 --> 34:04.010 . 34:04.210 --> 34:07.230 So, the next one we want to look at is the Nest. 34:07.770 --> 34:13.710 This actually came from a friend of mine who had his Nest thermostat crash out. 34:13.710 --> 34:15.190 It just died. 34:15.490 --> 34:20.070 He threw it my way, said, hey, can you get any interesting data off of it? 34:20.070 --> 34:25.370 So, and I struggled with getting data off of this, but it was eventually successful. 34:25.370 --> 34:28.670 We're going to show some of this. 34:28.990 --> 34:33.670 There was no confidential data actually extracted from the device. 34:33.670 --> 34:36.670 So, there's no problem with us using it here. 34:37.070 --> 34:44.150 But so, what we want to do is what we're going to use is a process where we're going to create a NAND sim. 34:44.570 --> 34:45.810 So, what does that mean? 34:45.810 --> 34:51.170 We're actually going to simulate a NAND chip in memory. 34:51.190 --> 34:54.230 We're going to use like ModProbe. 34:54.410 --> 35:01.990 If you haven't used ModProbe, ModProbe will let you load kernel modules onto the system. 35:01.990 --> 35:09.290 So, we're going to build a block device using ModProbe kernel modules. 35:09.290 --> 35:13.750 We're going to load those up and then we're going to write the data out to the chip. 35:14.010 --> 35:25.990 We're actually going to tell the data with NAND write to use the OOB data to reconstruct any error corrections and that type of stuff. 35:25.990 --> 35:28.410 And then we'll mount it up. 35:28.510 --> 35:37.530 So, this particular one, I actually ran Benwalk on this and it never recovered anything. 35:37.530 --> 35:41.770 I actually shot it out to good friends at 35:45.970 --> 35:47.890 that do the Benwalk Pro. 35:48.030 --> 35:53.770 And unfortunately, I didn't get back to the system for a couple weeks and it churned out there. 35:53.770 --> 35:58.390 So, I apologize to them again for eating up several terabytes of data. 35:58.390 --> 36:06.050 So, this thing was constructually a pain in the butt and running Benwalk was potentially catastrophic. 36:06.150 --> 36:09.270 It would eat up all your memory and produce nothing. 36:09.950 --> 36:30.430 And I'm not going to run it here, but if you run it, every page or every header for a GIFs to file system or every header for one of those, it would create a folder for it, put nothing in it, and literally eat up terabytes of data in a couple days quickly. 36:30.470 --> 36:32.950 So, let's go ahead and get going here. 36:32.950 --> 36:37.890 So, we're going to load ModProbe MTD, ModProbe Block. 36:37.890 --> 36:40.710 This will help us set up a block device. 36:41.330 --> 36:43.630 And then we're going to go ahead and 36:48.700 --> 36:50.340 we're going to load this up here. 36:50.340 --> 36:58.020 So, the way this works, ModProbe NANDSim, BCH is error correction functionality. 36:58.460 --> 37:02.140 This was the chip, it uses a 8-bit structure. 37:02.300 --> 37:07.120 There's a BCH16 for a 16-bit structure, if that's actually being used. 37:07.120 --> 37:10.640 And this here happens to be the chip ID. 37:11.000 --> 37:19.720 So, if we go over to this particular chip, that is not the one I want. 37:19.980 --> 37:22.560 So, this is the actual chip. 37:23.040 --> 37:26.860 And I believe we go to page 25. 37:27.380 --> 37:29.240 I have this memorized. 37:30.360 --> 37:31.280 Okay. 37:31.940 --> 37:35.540 And this would be the read ID, the ID of the actual chip. 37:35.540 --> 37:38.180 We know this chip is a 2-gig chip. 37:38.740 --> 37:41.400 It's organized in 8-bit structure. 37:41.640 --> 37:45.100 It is a 1.8-volt bit chip. 37:45.260 --> 37:49.880 So, 01 is the first one, AA, 90, 15. 37:50.060 --> 37:56.400 44 isn't used in this case, but it deals with error corrections and multi-plane information. 37:56.920 --> 38:17.800 Multi-plane information, just to make note here, if you have a chip that has multi-planes on it, and you'll see it in the datasheet, it may or may not stripe across the two planes, meaning it puts one block on one plane and one block on the next plane versus putting them contiguously on the same plane. 38:17.800 --> 38:21.160 I have not encountered this, but I have read about it. 38:21.160 --> 38:22.800 So, it's something to be noted. 38:22.860 --> 38:25.980 So, at this point, we're going to go ahead and load this up. 38:26.280 --> 38:42.380 So, now we've loaded up that particular module, kernel module, and now let's go ahead and just look at some information, mtdinfo, and this will tell us a little bit about what we just created. 38:44.300 --> 38:46.600 So, we created a simulated. 38:46.600 --> 38:48.380 It's how many bytes? 38:49.860 --> 38:57.200 256-megabyte chip, 2048, subpage size, 64 bytes, pretty straightforward. 38:57.200 --> 39:00.680 And then we go ahead, and this one may show some errors. 39:01.400 --> 39:04.020 Sometimes it does, sometimes it doesn't. 39:04.180 --> 39:08.600 And we're going to look at this within DMessage. 39:09.000 --> 39:18.360 And if we come down here, we can see that it took the manufacturer ID, chip ID, and information, and actually have identified the actual chip. 39:18.360 --> 39:23.360 That is the chip ID that does that and helps lay this out. 39:26.670 --> 39:30.650 So, the next thing is we want to be able to write into it. 39:30.670 --> 39:36.350 So, we're going to use NanWrite, which is another tool. 39:38.370 --> 39:40.450 Hope you don't mind me not typing. 39:40.490 --> 39:42.490 It saves you guys a lot of headaches. 39:42.890 --> 39:46.630 So, let's cancel out of that. 39:47.110 --> 39:48.110 NanWrite. 39:48.530 --> 39:50.350 I want to see the command. 39:50.390 --> 39:51.930 So, here's all the functions. 39:51.930 --> 39:55.490 So, what we're going to do is we're going to input contains OOB data. 39:55.670 --> 39:59.830 So, we're telling it to utilize the OOB data. 40:01.250 --> 40:03.550 And we're going to go ahead and let this run. 40:03.550 --> 40:05.830 And hopefully, it'll run through clean. 40:05.910 --> 40:06.510 Boom! 40:06.510 --> 40:11.070 There we wrote all of the blocks out onto the device. 40:11.110 --> 40:14.970 So, the next step is... we'll come over here so you see this. 40:14.970 --> 40:16.370 We go ls. 40:22.670 --> 40:25.010 And we can see there's nothing there. 40:25.350 --> 40:28.790 And then what we're going to do is cut and paste. 40:28.790 --> 40:30.790 So, I don't have to type. 40:34.300 --> 40:37.580 We're going to mount this up on block zero. 40:39.300 --> 40:41.880 Change directory to gifs. 40:43.200 --> 40:45.760 And voila, we have data. 40:46.220 --> 40:50.120 So, if we look through here, it starts looking pretty good. 40:50.120 --> 40:52.860 Hey, we've recovered all this data. 40:53.120 --> 40:58.720 The sad thing is most of this stuff is system logs. 41:01.700 --> 41:03.240 That we recovered. 41:03.240 --> 41:08.480 But if we get up here toward the top, we find some weird things. 41:08.540 --> 41:10.300 Here's bin. 41:10.380 --> 41:11.960 It's a file. 41:12.260 --> 41:14.500 etc is a file. 41:15.120 --> 41:18.720 Well, truthfully, this is bullcrap. 41:19.560 --> 41:21.660 We know that's not a file. 41:21.660 --> 41:26.120 So, if we cat etc, wow, look what we get. 41:26.120 --> 41:28.800 Now, I wouldn't expect to see that in etc. 41:28.800 --> 41:30.700 So, we recovered a lot of it. 41:30.700 --> 41:34.400 But some of it's corrupted in a weird fashion. 41:34.740 --> 41:36.800 So, how did I recover from this? 41:36.800 --> 41:38.340 I experimented. 41:38.340 --> 41:39.920 And we're going to go over that a minute. 41:39.920 --> 41:45.060 But before we go forward, I actually want to see if there are any questions. 41:49.890 --> 41:51.250 Jonathan, any questions? 41:51.910 --> 41:53.870 Yep, taking a look now. 41:53.870 --> 41:55.410 No updates here. 41:55.410 --> 41:57.430 I'm checking Twitch real quick. 41:59.050 --> 42:00.910 Yes, here's a question. 42:00.910 --> 42:06.490 In IoT devices you've examined, do you ever come across NAND storing non-file system data? 42:06.490 --> 42:12.810 If so, do you just dump and then use binwalk and other usual tools and methods for poking around? 42:13.530 --> 42:20.270 Yeah, I've had them storing non-file system data on NAND flash. 42:21.150 --> 42:32.730 So, yeah, typically I get into that and when I get those and there's no structure to them, I start off with throwing things like strings to the thing. 42:32.730 --> 42:41.370 With strings, can we identify any known structure in there? 42:41.930 --> 42:45.290 I will run binwalk just to see what it'll do. 42:45.290 --> 42:50.590 Sometimes it'll point out some structure headers that I can more quickly find. 42:50.590 --> 42:52.810 So, I think that plays a big role. 42:52.850 --> 42:59.670 But yeah, sometimes you'll have them storing things like blocks of configuration data. 43:00.130 --> 43:09.550 And I've seen them just blocks of encrypted data, which may be a file system structure, but they're not framed as such. 43:09.550 --> 43:14.310 So, it's just encrypted blocks of useless information. 43:14.430 --> 43:17.290 So, yeah, that's not all that in common. 43:17.370 --> 43:26.310 Well, most of the time when I get a NAND chip, it's a file system, but you will encounter those, to be honest. 43:27.470 --> 43:39.690 So, right now, before we move on to the next section, I want to remove the kernel modules so they don't conflict with what we're doing. 43:45.320 --> 43:46.840 Okay, and we are down to... 43:48.160 --> 43:51.020 we still got some time, doing pretty good. 43:51.360 --> 44:00.340 So, the next section I want to get into, we are going to use MTD RAM to actually do this. 44:00.520 --> 44:15.380 So, we're going to take that same system, that same dump that we just did, where we mounted up the file system, it did not come out exactly the way we want it. 44:15.540 --> 44:17.160 And we're going to mess with that. 44:17.160 --> 44:23.640 So, we're going to make a make directory, and we're actually going to create our own block structure. 44:29.350 --> 44:33.270 And then we'll go, and then we're going to make a node. 44:35.170 --> 44:44.150 If I learn how to spell it, I should just copy this over, make node dev slash MTD block. 44:44.710 --> 44:46.510 And we're going to create zero. 44:46.510 --> 44:49.130 We're going to create it as a block device. 44:49.130 --> 44:55.350 We're going to give it a master number, a primary number, and a minor number of zero. 44:55.350 --> 44:57.710 So, we're going to create that device in there. 44:57.930 --> 45:07.350 And then what we're going to do is we're going use modprobe, and we're going to bring in a kernel module called MTDRAM. 45:07.350 --> 45:10.470 So, this kernel module's MTDRAM. 45:10.470 --> 45:15.250 We're setting the total size to 274432. 45:15.250 --> 45:20.970 That is bigger than a 256 megabyte RAM area. 45:22.090 --> 45:29.370 256 megabyte RAM area would be 262144, and that would not contain OOB data. 45:29.370 --> 45:33.950 In this case here, we're bringing in the OOB data also. 45:34.430 --> 45:41.150 Because if I bring in without the OOB data, I noticed that we get the same results as the other one. 45:41.150 --> 45:46.310 So, experimenting, and I encourage you to experiment when you're dealing with these NAND chips. 45:47.030 --> 45:55.110 It can be quite fascinating what you can find and figure out and make work to get you data out of this system. 45:55.110 --> 45:57.010 I'm currently working on one. 45:57.010 --> 46:07.070 It has me slightly befuddled, but I will master it eventually, where the actual entire data on the chip is shifted 10 bytes. 46:07.190 --> 46:24.110 They even shifted all the OOB data 10 bytes, and it took looking at it manually to figure this all out and doing that math and looking at the data in the structure to determine that they had also shifted that. 46:24.110 --> 46:30.250 Because I was getting block errors saying that block is 10 bytes longer than it's supposed to be type thing. 46:30.250 --> 46:35.330 That's when we found out that everything was shifted and was able to confirm that the OOB was shifted too. 46:35.330 --> 46:36.450 But I digress. 46:36.450 --> 46:41.850 So, what we're going to do is we're going to load this up, and it's supposed to contain the entire thing. 46:41.850 --> 46:44.570 And then we want to load one more module. 46:47.250 --> 46:50.510 MTD block for kernel module. 46:52.210 --> 46:54.890 And then we're going to dd. 46:54.930 --> 46:56.410 Excuse me. 47:00.420 --> 47:03.280 And dd is a bit copy. 47:03.280 --> 47:04.880 I use this all the time. 47:04.880 --> 47:11.200 Matter of fact, if you get access to almost every embedded IoT device, you'll find dd is actually installed on it. 47:11.200 --> 47:16.600 Because most of the time, dd is actually used for doing firmware upgrades. 47:16.600 --> 47:21.940 You'll have duplicate root file structures, duplicate kernels on the device. 47:21.940 --> 47:24.540 And what happens is one is primary and running. 47:24.580 --> 47:38.800 So, what will happen is when the firmware comes down, it'll dd it over the secondary, and then it'll change the U-boot or secure boot arguments to make the new one primary and the old one secondary. 47:38.800 --> 47:44.200 That way, if you have a failed firmware update, it does not break the device. 47:44.200 --> 47:48.700 It just goes back to the original one, which was not overwritten. 47:49.220 --> 47:54.260 So, again, input file here is the nest file. 47:54.260 --> 47:57.560 And then output is to our block device that we created. 47:57.560 --> 48:02.080 And if I did everything right, this should start writing data. 48:02.080 --> 48:03.320 Should be fairly quick. 48:03.320 --> 48:14.680 And as you see, we're able to write out 277 megabytes of information fairly quick, because it's all internal to the system. 48:15.120 --> 48:21.860 So, from here, we're going to go ahead and mount this up. 48:22.020 --> 48:24.000 Same way we did on the other one. 48:24.360 --> 48:28.100 And we're going to mount MTD block zero up. 48:32.780 --> 48:40.360 Now, if someone wants to explain to me why this worked, because I really not sure why. 48:40.360 --> 48:45.920 As I said, often dealing with NAND chips, little quirky things have worked for me. 48:45.920 --> 48:48.880 So, I'm not going to speak bad of them. 48:48.880 --> 48:53.800 And now we can see etc and bin are actual files. 48:53.800 --> 48:55.780 We do have some corruptions in here. 48:55.780 --> 49:01.520 So, there's a few files actually corrupted within this. 49:01.520 --> 49:09.060 But for the most part, a large portion of them were actually able to be recovered properly. 49:17.990 --> 49:23.970 And so, these are methods that I use quite regularly for recovering. 49:24.250 --> 49:39.790 When you start working with UBI file systems, if I ever get some good methods, methodologies hammered out for working with some of the strange UBI file systems, like the one with the 10 bytes that's shifted. 49:39.790 --> 49:53.730 I did one on an engagement here earlier this year, where again, nothing are large portions of it were not laid out contiguously and trying to build good solid methodologies around UBI. 49:53.730 --> 50:00.090 I'll probably come back around and put together a training session presentation for that down the road. 50:00.090 --> 50:07.850 That's what I'm currently working on, trying to work out those kinks and bugs and build that knowledge level to build to work with us. 50:07.850 --> 50:18.710 Now, probably half the chips that are UBI file systems, I'm successful recovering them using various techniques and UBI tools that are available out there. 50:18.890 --> 50:35.990 But typically, GIFs, squash file systems, things like that can easily be quickly recovered from NAND flash chips using any one of these three methods that we covered today in this presentation. 50:36.210 --> 50:39.250 So, hopefully there's some questions out there. 50:40.550 --> 50:42.670 Yeah, I got a couple here. 50:42.990 --> 50:52.850 So, one question here asks, have you encountered an IoT device that was encrypting the data it was storing in the NAND flash? 50:54.330 --> 50:59.570 No, I haven't, which is kind of amazing to me. 50:59.850 --> 51:08.670 I mean, the only time I've ever seen IoT technology using any kind of encryption, if it was running an Android operating system. 51:08.670 --> 51:16.890 And by nature, Android can support the data section being completely encrypted. 51:16.890 --> 51:20.690 So, that's quite common. 51:20.690 --> 51:41.390 I do have some other tech that I consider IoT, but it's more enterprise IoT and not necessarily the normal consumer grade, which actually is very much encrypted and most of the, not all the file system, but most of the file system that contains user data, 51:41.390 --> 51:46.690 similar to the Android file system, is also encrypted. 51:46.950 --> 51:52.030 So, typically, if I see it's an Android, it's usually 50-50. 51:52.030 --> 52:00.490 If it is a standard phone-style Android operating system dumped on there, then most always it has encrypted. 52:00.490 --> 52:08.570 But yet, if it's an older Android style that's put on there, there was no requirement or typically that's not always encrypted. 52:08.570 --> 52:21.350 So, it's a mix and match, but it's pretty rare to see encrypted, at least from a lot of the IoT I've looked at, unless you get into more enterprise level technology. 52:23.690 --> 52:26.770 Anonymous attendee asks, Hi, I am new to this. 52:26.770 --> 52:29.410 Sorry, you already have the binary file. 52:29.410 --> 52:32.190 How did you extract it from the hardware device? 52:33.830 --> 52:38.030 In the cases of these, both of them were chip off. 52:38.750 --> 52:55.330 So, early in the presentation, and so you can rewatch this presentation, or in the presentation earlier today, where we talked about building an IoT lab and we showed a number of devices, I have a number of chip readers. 52:55.330 --> 52:57.750 So, I actually use those. 52:57.750 --> 53:10.130 Other ways are often if the chip that's being used has some kind of debug function, you can put it in debug and pull the data off of it. 53:10.130 --> 53:17.330 Also, you may be able to check, connect in with JTAG or SerWire debug and actually read it off the device. 53:17.330 --> 53:25.990 But if those fail, I often rely on just desoldering the chip and dropping it into a reader and extract the data from there. 53:25.990 --> 53:44.450 If I need to put the device back into functionality, then I'll go through either resoldering the chip back on or reballing the chip and reflowing it back on an IR oven, which are fairly cumbersome, at least the reballing BGAs, but it's still doable. 53:44.450 --> 53:47.370 And it's not all that complex. 53:47.430 --> 53:54.470 I did a presentation not last year, but the year before last at Nuit de Hacq. 53:54.470 --> 53:55.470 I did it there. 53:55.470 --> 54:04.030 I also did it, oh gosh, I did it in Perth, Australia, and I also did it at DerbyCon. 54:04.030 --> 54:07.770 And it was, I'm trying to remember the name of it. 54:07.770 --> 54:12.410 Basically, it was how to get root access the hard way. 54:12.570 --> 54:20.390 And it talks about slashing, smashing, cutting the device apart and rebuilding it to get root access or something to that effect. 54:20.390 --> 54:29.270 So you can look at that, and it goes through some of the concepts in my learning process on learning how to manually reball BGA chips. 54:31.050 --> 54:34.290 Elvis Cousin asks, these are all variants of Linux. 54:34.290 --> 54:40.150 On a Windows CE device, might it still use SquashFS or some other Windows thing? 54:40.310 --> 54:43.850 Ooh, probably some other Windows thing. 54:44.230 --> 54:49.390 I have not looked at a Windows CE device. 54:51.230 --> 54:55.350 I have been looking at some Chrome devices. 54:56.630 --> 55:11.550 In those particular cases, this has an SD drive, solid state drive, where you're able to get the data off of it and just mount it up on your system. 55:11.770 --> 55:14.510 I'm not sure if they have NAND chips. 55:14.510 --> 55:19.130 Is it going to be a Linux file system? 55:19.130 --> 55:19.910 Probably not. 55:19.910 --> 55:22.510 It's probably going to be some variation of Windows. 55:22.510 --> 55:30.010 But I have not tore into any Windows or Windows CE type devices to be able to give you a solid answer on that. 55:30.110 --> 55:32.110 Let me know if you find out. 55:35.970 --> 55:37.550 Any other questions? 55:38.130 --> 55:39.570 Nope, all caught up. 55:40.410 --> 55:41.150 All right. 55:41.150 --> 55:43.030 Well, I think we're... what time we start? 55:43.030 --> 55:43.770 Six o'clock. 55:43.770 --> 55:46.670 So we're probably right on time. 55:46.670 --> 55:50.090 So let me jump over here. 55:51.050 --> 55:55.230 Let's do a final stop to share. 55:55.510 --> 56:01.230 So let's go ahead and ask question one more time. 56:01.230 --> 56:06.510 Anyone have any questions, any requests for questions or information? 56:06.510 --> 56:16.850 If not, you'll probably catch me on Discord either here in a little bit after I've eaten or late tonight or tomorrow. 56:17.170 --> 56:20.110 Feel free to hit me up for any questions. 56:20.710 --> 56:22.850 Also, oh gosh, yeah, I was promising that. 56:22.850 --> 56:24.750 Hold on before you all drop off there. 56:24.750 --> 56:25.950 Let's get this up. 56:32.020 --> 56:36.840 So I had promised... slideshow. 56:46.460 --> 56:49.760 So I want to be able to get some of these things captured in the video. 56:49.760 --> 56:53.040 So here's the NAND sim example that I use. 56:53.040 --> 57:00.220 The mod probe commands, the NAND write commands, and the mount to be able to mount this particular GIF file system up. 57:00.840 --> 57:06.560 And here is the commands that I use for mtd-dram example. 57:10.460 --> 57:13.040 Quick note on some of the commands we use. 57:13.180 --> 57:20.900 There is the link for NAND dump tool, concept of mod probe, make node, NAND write, DD. 57:21.040 --> 57:24.440 And then here is the reference. 57:24.700 --> 57:26.660 So that gets captured in here. 57:26.660 --> 57:28.600 All of this is kind of cool material. 57:28.620 --> 57:32.200 There's a lot of great work that's been done out there on NAND chips. 57:32.300 --> 57:47.000 Like I said, I'm constantly reviewing and reading and consuming this material on a regular basis because every time I encounter NAND chip that wants to fight with me, I have to go out and start rethinking, okay, what am I missing? 57:47.000 --> 57:49.120 What do I need to learn to expand my knowledge? 57:49.320 --> 57:51.600 How do we kind of move forward from here? 57:51.600 --> 57:54.400 And there's been a lot of work done out there. 57:54.480 --> 58:04.380 And of course, we shared much of the information that these people provided in this demonstration today. 58:04.380 --> 58:17.600 Also, if you're interested in connecting up with me on Twitter, my handles are %X, so it would be P-E-R-C-E-N-T underscore X. 58:17.600 --> 58:24.500 And feel free to connect with me out there or reach out to me and ask questions out there. 58:29.980 --> 58:30.780 Awesome, Daryl. 58:30.780 --> 58:31.980 Thank you so much. 58:32.220 --> 58:32.940 Oh, man. 58:32.940 --> 58:34.020 Sure enough, Sam. 58:34.020 --> 58:35.320 Had a blast. 58:35.780 --> 58:36.180 Yeah. 58:36.180 --> 58:40.740 And I mean, honestly, if there are any last questions, we still have a little bit of time. 58:40.740 --> 58:43.440 So was there anything else that you saw there, Jonathan? 58:46.000 --> 58:48.820 Yeah, another quick question just popped up. 58:49.100 --> 58:54.300 It says, this is some very epic stuff, extracting flash from NAND chips. 58:54.300 --> 59:00.860 I have to ask, did you find anything epic while performing post-exploitation on the extracted firmware? 59:03.520 --> 59:04.860 Yeah, yeah. 59:04.860 --> 59:09.800 I mean, any remote shells? 59:09.800 --> 59:11.480 No, unfortunately. 59:11.840 --> 59:20.100 But again, it's information that we could feed back to manufacturers to help improve processes. 59:20.260 --> 59:22.520 You know, they get into hard-coded stuff. 59:23.560 --> 59:31.960 Mainly, a lot of this lately was purely an effort to go, hey, what happens when IoT is thrown in the trash type thing? 59:32.020 --> 59:39.060 You know, is there data is like the guy with the Nest thermostat? 59:39.060 --> 59:41.860 What happens when something dies if you throw it away? 59:41.860 --> 59:44.900 Does it contain PII information? 59:44.900 --> 59:51.040 So this was kind of a lot of it was driven out of that and some paid assessments. 59:51.600 --> 01:00:01.440 And also, you know, we tested a lot of devices from, hey, if you go ahead and flush the thing back to factory, does data stay on it? 01:00:01.440 --> 01:00:04.060 We kind of started checking for things like that. 01:00:04.060 --> 01:00:13.080 It's all about a privacy risk and issue, because we've gotten really big on the whole idea that, hey, we don't want to throw our hard drives away. 01:00:13.080 --> 01:00:16.900 But in this modern world, these chips are our hard drives. 01:00:16.920 --> 01:00:21.760 And we need to probably start thinking about the potential risks there. 01:00:22.400 --> 01:00:32.160 So that was kind of the main purpose of a lot of this, and the fact that I had been working on some engagements that involved band chips that were problematic. 01:00:32.320 --> 01:00:38.480 In those cases, we were working on things like reconstructing keys to gain access. 01:00:38.480 --> 01:00:46.060 A lot of this stuff ties into engagements also, where we're looking for data since we're testing an entire ecosystem. 01:00:46.060 --> 01:00:51.560 That isn't just the hardware, but it may be mobile apps, cloud services, and other devices. 01:00:51.560 --> 01:01:05.300 What can we get off this chip that could allow us to attack everything else or attack somebody else using the same product because of key reuse and key session keys and password reuse and things like that. 01:01:05.300 --> 01:01:08.120 So a lot of it's within that direction. 01:01:10.540 --> 01:01:12.400 Another question here. 01:01:12.720 --> 01:01:25.920 This individual asks, does it seem like it is possible to be able to create a module so that a chip reader would automatically be able to detect those out of the OOB areas of the flash memory? 01:01:26.040 --> 01:01:29.440 Oh, a number of chip readers will let you identify. 01:01:29.440 --> 01:01:34.200 Some of them will allow you to extract the chip, the data, without the OOB. 01:01:34.360 --> 01:01:36.000 So yeah, it's not uncommon. 01:01:36.000 --> 01:01:46.360 You can often manipulate data from the chip reader and include or not include OOB on some chip readers. 01:01:46.360 --> 01:01:50.360 Obviously, you get more control as the price of the chip reader goes up. 01:01:50.780 --> 01:01:55.600 And I don't necessarily have a $2,500 chip reader in here. 01:01:55.600 --> 01:02:00.180 But I think one of these actually had some OOB settings. 01:02:00.180 --> 01:02:07.100 But for some reason, I didn't necessarily trust it because I told it not to include OOB. 01:02:07.100 --> 01:02:10.740 And then when I visually looked at the data, the OOB data was there. 01:02:10.740 --> 01:02:13.120 So I kind of lost trust. 01:02:13.120 --> 01:02:23.400 I have a tendency to also like to map it out, not let the chip reader make the determination for me that this data is structured a certain way. 01:02:23.400 --> 01:02:34.280 Because like I said, it's not uncommon with vendors to futz with it, so that it may be incorrect, or that the OOB shifted or is not used in the same place. 01:02:34.280 --> 01:02:39.300 And they've just written a new controller to read the data the way they made it. 01:02:39.820 --> 01:02:43.840 This way allows me to go, hey, this is correct or isn't correct. 01:02:43.860 --> 01:02:51.720 And then I can easily manually remove it or modify code or scripts that are out there to take it out the way I wanted it removed. 01:02:51.720 --> 01:02:53.580 So that's kind of my take on it. 01:02:55.740 --> 01:02:56.880 That makes sense. 01:02:56.880 --> 01:03:00.220 I think that we're at the bottom of the list here for the questions. 01:03:00.740 --> 01:03:04.460 Okay, well, I think that kind of concludes it. 01:03:04.460 --> 01:03:11.920 If anyone has any questions, catch up with me later, or reach out through me on Twitter. 01:03:11.920 --> 01:03:17.060 And again, I'll be on Discord probably tomorrow and maybe later tonight. 01:03:17.060 --> 01:03:19.040 So I'll catch everyone later. 01:03:19.040 --> 01:03:20.440 Have a good night.