[00:00.640 --> 00:06.460]  NAND flash chips, a little different than standard flash chips,
[00:06.460 --> 00:11.120]  be a little more problematic. So we're going to talk about that and kind of step through with
[00:11.740 --> 00:18.320]  a number of demos. So great, let's go ahead and get started. So thanks everyone this evening or
[00:18.320 --> 00:23.380]  this afternoon based on where you're at in the United States. Presentation is NAND chips,
[00:23.380 --> 00:29.760]  introduction to NAND chip extraction and data reconstruction. This is an intro primer.
[00:30.000 --> 00:34.800]  We're going to cover a number of different topics and we have like three different demos.
[00:35.020 --> 00:39.720]  I think you'll get a lot out of this. So my name is Daryl Hyland. I'm the Principal Security
[00:39.720 --> 00:48.500]  Researcher for IoT at Rapid7. So section one, we're going to just give a basic introduction
[00:49.080 --> 00:58.120]  and look at structure. So what is a NAND flash memory? Simple, you can read it. It's non-volatile
[00:58.120 --> 01:04.340]  storage technology that doesn't require power to retain the data. Pretty straightforward.
[01:04.700 --> 01:09.860]  See those references there? If you're interested in learning more in the nitty-gritty,
[01:09.860 --> 01:17.000]  take a screenshot of that real quick because that is great references. I go back to these
[01:17.000 --> 01:22.280]  quite regularly to kind of refresh myself because some of this stuff can get pretty deep
[01:22.920 --> 01:29.400]  as you're trying to reconstruct data in a number of cases. So go ahead and get a screenshot of
[01:29.400 --> 01:35.300]  that. I'm getting ready to jump off of it now. So a couple of things. When we start thinking
[01:35.300 --> 01:43.780]  about NAND flash memory chips, you need to go to the data sheet. You really need to understand how
[01:43.780 --> 01:50.780]  things are laid out. This will play a part in recovery a number of times. So this is the memory
[01:50.780 --> 01:59.980]  array structure that lays out the page, the blocks, and the planes. In this case, the page is
[02:00.740 --> 02:08.820]  2K in size or 2048 bytes. And then there's what's referred to as an OOB area, a 64-byte area at the
[02:08.820 --> 02:15.780]  end. And this is critical. So what is OOB? It's out of band. It's a spare area. It's used for
[02:15.780 --> 02:24.500]  containing error correction code information, bad block information, and it exists in every page.
[02:24.500 --> 02:31.100]  So what does that mean? Generally, what it means is you have all your data jammed full of
[02:31.100 --> 02:36.660]  potential garbage when you're trying to parse it out and rebuild the data. And this could be
[02:36.660 --> 02:43.100]  impactful and can cause problems in some cases. So we're going to talk about how we're going to
[02:43.100 --> 02:49.220]  fix that, and we're going to show that in a couple examples. The OOB area, the out-of-band area,
[02:49.220 --> 02:57.040]  is also laid out in two formats. One is known as adjacent, and one is known as separate.
[02:57.160 --> 03:06.800]  If it's laid out, the OOB is laid out adjacent. So a 2048-byte page would be laid out with a 16K
[03:06.800 --> 03:14.200]  or 16-byte of OOB after every 512 bytes. This would be an adjacent layout.
[03:14.920 --> 03:22.720]  The most common I see is what's known as separate. A separate area is the 64 bytes of data exist at
[03:22.720 --> 03:30.640]  the end of the 2048-byte page. This is the most common. But often you need to verify that
[03:31.220 --> 03:36.080]  as you're working through this and trying to remove the data or correct the problem.
[03:36.080 --> 03:41.540]  Another thing that comes in handy, and this is in reference to a number of the tools that
[03:41.540 --> 03:50.340]  we're going to use today, your chip ID code. An example, anytime you use a flash memory reader,
[03:50.340 --> 03:57.360]  so you pull a chip out, drop it into a memory chip reader, often it will, when you set it up
[03:57.360 --> 04:04.720]  for that chip, it associates that with a ID, and it'll read the chip, compare the ID on the chip
[04:04.720 --> 04:10.820]  to make sure it matches before it tries to read the data. Now, a lot of applications out there have
[04:13.080 --> 04:19.660]  a use for this type of chip ID data. This is out of the datasheet also. And in this example here,
[04:19.660 --> 04:28.260]  it's showing the datasheet for a S34MS02G1 chip, which is a 512-megabyte NAND flash.
[04:28.260 --> 04:33.800]  And you can see at the bottom, it shows what that chip ID is for that chip. And you'll see that
[04:33.800 --> 04:38.760]  we'll use this later on, and we'll probably come back and look at it in the datasheet again when
[04:38.760 --> 04:46.860]  we get ready to do that. So, at this point, are there any questions? So, Jonathan, has anyone
[04:46.860 --> 04:52.340]  posted any questions or anyone not have any questions about some of these basic concepts?
[04:53.560 --> 05:00.500]  Let's see here. So far, it looks like no questions posted here. One quick question that is actually
[05:00.500 --> 05:08.340]  coming up here that I'm seeing. Whenever you're, I guess, pulling firmware from NAND flash,
[05:08.340 --> 05:14.080]  do you have like a consistent philosophy that you follow, such as like always first see if you can,
[05:14.080 --> 05:19.780]  you know, first pull, maybe if you can like log into it via like serial port through UART?
[05:19.780 --> 05:24.280]  Or do you typically go straight for attempting to dump the NAND flash content?
[05:24.280 --> 05:31.140]  Yeah, usually the first way if I want to get the data, the most effective way is get the data
[05:31.140 --> 05:37.700]  when it's in a constructed fashion. That would mean if I can get a console on the device with
[05:37.700 --> 05:44.720]  the entire operating system running, and the entire partitions for all the data laid out,
[05:44.720 --> 05:51.560]  that's easier because then I could easily DD the partitions off the chip in their constructed
[05:51.560 --> 05:56.680]  fashion. And that would avoid me almost always having to do what we're going to talk about here.
[05:56.780 --> 06:01.040]  What we're going to talk about here is in that case where you actually have to read the data
[06:01.040 --> 06:07.160]  raw, right off the flash chip, either chip off where you remove the chip and drop it into a
[06:07.160 --> 06:12.720]  reader, or some other methods where you may be able to clip on or hook onto the device and read
[06:12.720 --> 06:18.500]  the raw data out of the chip. And that's when you get this structure that we're going to be dealing
[06:18.500 --> 06:27.590]  with today, and how to clean it up and a few methods of recovering that. So I shout out to
[06:27.590 --> 06:35.390]  everyone, if you're interested in interacting or asking questions during this, join the Zoom chat,
[06:35.390 --> 06:40.570]  the Zoom channel that we have set up for the webcast. Once you're into there, you'll be able
[06:40.570 --> 06:45.670]  to post questions up during the presentation, and hopefully we'll have time and we'll be able
[06:45.670 --> 06:51.430]  to answer them. So please do that if you can. It makes this way much more interactive. Instead of
[06:51.430 --> 06:56.010]  just being a data dump, we have a chance to actually interact with the audience and the people
[06:56.550 --> 07:04.030]  and learn more in this process. So I'll go ahead and move on. So section two is a general
[07:04.510 --> 07:11.050]  chip readers. Since I talked about chip off from the question Jonathan gave us,
[07:11.050 --> 07:17.530]  we get into chip readers. So when you're interacting with a NAND flash chip, here are
[07:17.530 --> 07:23.550]  the signals you're typically going to need to interact with. And they go everything from the
[07:23.550 --> 07:29.530]  8-bit data channel, the bi-directional channel, all the way down to input signal, output signal
[07:29.530 --> 07:36.070]  structure, and some other various pieces. So if you end up buying an off-the-shelf chip reader,
[07:36.070 --> 07:40.070]  this kind of solves all these problems. But there was some work done out there
[07:40.070 --> 07:47.030]  by this gentleman here. I would recommend taking a look at this write-up. If you're interested in
[07:47.030 --> 07:54.490]  building your own chip reader, this one actually works against a TSOP48 NAND flash chip. And he
[07:54.490 --> 08:03.050]  uses a technique known as bit banging using this FTDI board that he has here. He's able to bit bang
[08:03.050 --> 08:10.510]  the data off of the actual chip. So I would look at looking at what bit banging is, because I'm
[08:10.510 --> 08:17.410]  not going to explain it here, and look at how he's set this thing up. If you're the hacker mentality
[08:17.410 --> 08:21.950]  and you like the idea of building things your own, I would check this out. It's kind of cool.
[08:22.950 --> 08:28.610]  If you're like me and you just like to throw money at it, these chip readers are reasonably
[08:28.610 --> 08:37.770]  inexpensive. The TI-866 Plus, you get all the sockets with it. It's like $130. This will read
[08:37.910 --> 08:46.410]  a number of NAND chips. Pretty good. And it uses a TSOP48. Right below it is a socket that doesn't
[08:46.410 --> 08:53.890]  come with the set that you need to buy. This socket is for from TSOP32s to 48s. It's known as
[08:53.890 --> 09:01.110]  NAND08. This is for larger memory chips. So anytime you encounter larger NAND flash memory
[09:01.110 --> 09:06.710]  chips in the TSOP48, the one that comes with the kit will not work and only work with smaller
[09:06.710 --> 09:13.110]  memory blocks. So you need to get the socket for that. The ProMan isn't made, this model
[09:13.110 --> 09:21.890]  wasn't made anymore, but they do make a similar version. So if you search for ProMan, you'll find
[09:22.130 --> 09:30.870]  a similar version out there. The socket below that is designed for 1.8 volts. The ProMan is
[09:30.870 --> 09:39.190]  typically a 3.3 volt device. And we're starting to see way more chips in the 1.8 volt arena. So
[09:39.190 --> 09:44.530]  it's good to have that add-on. Most of these add-ons are $8, $9, $10 on both of these chip
[09:44.530 --> 09:52.970]  readers. So they're fairly inexpensive. The thing I like about the ProMan is if I can't identify a
[09:52.970 --> 10:00.530]  TSOP48, I can't find the data sheets or not, well, let's say I do have the data sheets and I can't
[10:00.530 --> 10:05.110]  find any information on none of my chip readers. They don't identify it, they can't detect it.
[10:05.110 --> 10:12.290]  This will let you hard set page OB block sizes into the actual tool that comes with the application
[10:12.290 --> 10:18.330]  that comes with this, so that you can attempt to read it using that kind of raw information.
[10:18.430 --> 10:24.890]  And it's been successful several times with me, making it possible for me to read a NAND flash chip
[10:24.890 --> 10:29.650]  where I had the data sheet to identify that structure, but none of my chip readers would
[10:29.650 --> 10:37.070]  recognize the chip. And the last one's RT809. And that one's a little more pricey if you buy it
[10:37.070 --> 10:44.870]  Amazon, but I think you can track that one down on AliExpress cheaper. I like this one. It's not a
[10:44.870 --> 10:52.070]  bad chip reader, and it has, you can buy a lot of different sockets for it. Most of the sockets
[10:52.070 --> 11:00.830]  that come with the TL866+, the one on the left, will work on this, all but the TSOP48 sockets.
[11:00.830 --> 11:06.390]  Because the TSOP48 sockets have a circuitry built into them, so they won't work on this reader. You
[11:06.390 --> 11:14.010]  have to have a straight pin through. And the one I have below shows that TSOP48 socket that's a
[11:14.010 --> 11:21.590]  straight pin through. I also recently had to pull a NAND flash that was a BGA ball girder array,
[11:21.590 --> 11:30.690]  63 balls on the bottom of it. And I was able to buy this off AliExpress for the RT809 for about
[11:30.690 --> 11:38.410]  40 bucks, which isn't bad. If you get into the high-end chip readers that run, start off at like
[11:38.410 --> 11:46.410]  $1,600, $1,700 and go up for them, the sockets for them start off at $100, $200 and go up into the
[11:46.410 --> 11:53.570]  thousands for BGA sockets. So, these things are very cost effective. The sockets are not high
[11:53.570 --> 11:58.910]  quality, so you have to take care of them. But they're overall, they're pretty good products.
[11:58.910 --> 12:05.670]  And I use them quite regularly in my lab without issues working with NAND chips.
[12:06.070 --> 12:14.530]  So, any questions in reference to the chip readers and chip reading process?
[12:16.550 --> 12:23.070]  One question here. What are the most challenging NAND packages or pitches to work with that you've
[12:23.070 --> 12:28.770]  encountered? Is there any package or pitch you would not attempt messing with?
[12:28.850 --> 12:34.910]  I would never say there's nothing I wouldn't attempt messing with. I would literally mess
[12:34.910 --> 12:41.430]  with everything. Probably some of the biggest problem is, obviously, if you're using a chip
[12:41.430 --> 12:48.420]  reader, try to read this. The BGAs or ball grid array devices are the more problematic,
[12:48.980 --> 12:57.900]  because there may not be a socket available for some of these devices. In cases like that,
[12:58.380 --> 13:04.720]  if you have the data sheets, you can always go to the process of dead bugging. That is turning the
[13:04.720 --> 13:13.520]  ball grid array chip on its backside and soldering small 40-gauge wire to the actual balls, and then
[13:13.520 --> 13:22.180]  feed that into a connector or a chip reader or a device for doing bit banging to actually be able
[13:22.180 --> 13:27.420]  to do that. I have been successful doing that a couple times, but it's a pain in the butt. It's
[13:27.420 --> 13:34.260]  way much easier to pull the chip with hot air or IR if it's a BGA, clean it up, drop it in some
[13:34.260 --> 13:40.680]  device, close the lid, and have it give you all the data. But typically, if I need the data off
[13:40.680 --> 13:49.180]  it, and I can't get it any other way other than chip off, I will chip off from the board. And if
[13:49.180 --> 13:57.460]  necessarily, I will dead bug one. I mean, yeah, me and you, Jonathan, we worked together at one time
[13:57.460 --> 14:06.560]  where we dead bugged that one weird device. We had a 0.25 millimeter pitch BGA, and we had to
[14:06.560 --> 14:15.320]  solder 40 wires to it or 20 wires to it to get it hooked up. So yeah, we're going to try anything. I'm
[14:15.320 --> 14:20.160]  not going to pass up any potential target if it has data on it that I want.
[14:21.420 --> 14:25.760]  Makes sense. And you're definitely bringing back some crazy memories with that. And actually, you
[14:25.760 --> 14:30.760]  partially answered the next question as well. The next question was, what hot air tools do you
[14:30.760 --> 14:32.980]  recommend for BGA removal?
[14:34.120 --> 14:44.760]  Actually, I have both hot air and I have a IR oven. I like the IR oven. Once I got it, it's a small
[14:44.760 --> 14:51.580]  it's a small device, it'll take a board that's probably eight by nine, eight by 10 inches or
[14:51.580 --> 14:57.080]  something like that. To me, that's the sweetest thing in the world. I usually just stick it in
[14:57.080 --> 15:02.120]  the oven, run it through the cycle that I brings it up to the temperature. And I take a suction
[15:02.120 --> 15:08.500]  tool, I pull the chip off of it, and then I let the whole board cool down. Because, you know,
[15:08.500 --> 15:12.580]  it heats up the whole board, you have to watch out for things like plastic components that are
[15:12.580 --> 15:18.240]  on the board. So you may have to remove those prior to putting them in the oven. But you can
[15:18.240 --> 15:23.080]  usually pick up one of these IR ovens. If I remember what was the name of the one I got,
[15:23.080 --> 15:31.520]  it's spelled P-U-H-U-I. It's a Chinese made product or Asian made product oven that I have.
[15:31.520 --> 15:42.000]  I think it's a model 2962 is what it is. Small oven. I think I only paid like $130, $140 for it,
[15:42.000 --> 15:47.280]  somewhere around there. And there's a number of products out there like that. But an infrared is
[15:47.280 --> 15:56.240]  way much better, in my opinion, than hot air. So... Another question. Sorry, Daryl.
[15:56.780 --> 16:02.140]  Elvis Cousin asks, is it hard to reball BGA without killing it to re-solder?
[16:03.040 --> 16:11.920]  Yes. It's not that hard. It's just a process of learning how to do it.
[16:12.260 --> 16:18.540]  Not to drag this all out, but I had a project where I wanted to learn how to do this. Because
[16:18.540 --> 16:25.800]  I wanted to get root access on a device. It had 153 ball BGA embedded multimedia controller on
[16:25.800 --> 16:32.880]  device. So I used hot air. I pulled it off. And I had a couple of devices because I wanted to,
[16:32.880 --> 16:37.700]  in case I killed one of them. The first one, when I put the balls back on, there's a couple
[16:37.700 --> 16:44.060]  ways of doing it. You can use screens with paste, or you can use just the balls. In my case,
[16:44.060 --> 16:49.260]  I just used the balls. I floated on there correctly, but I overheated the chip and it
[16:49.260 --> 16:53.760]  looked like a chocolate brownie when I was done. That's when I learned that you don't want to go
[16:54.360 --> 17:01.380]  on these BGA chips. You don't want to go over about 210. Well, melting temperature is about
[17:01.380 --> 17:07.240]  183 degrees Celsius, if I remember right. And you don't want to go over more than 25,
[17:08.560 --> 17:14.540]  27 on the temperature in Celsius. If you hit 215 or higher, it starts causing the
[17:15.140 --> 17:20.260]  fiberglass substrate of the chips to start breaking down. But if you keep it within those
[17:20.260 --> 17:26.740]  range and you don't shock it by overheating it too quick or cooling it too fast, it'll work.
[17:26.740 --> 17:32.060]  And I was successful. The first one I burned up and I failed. The second one was perfect.
[17:32.060 --> 17:38.500]  Since then, I've never had a problem ever working with BGAs, either pulling them or putting them
[17:38.500 --> 17:47.970]  back on without damage. And once I learned those temperature parameters. Okay, so the next section
[17:47.970 --> 17:54.810]  we want to move into is going to be demo. So, we're going to get off into the demo area and see
[17:54.810 --> 18:03.250]  how that works out. Hopefully, we won't have any technical problems. So, what we're going to do here
[18:03.250 --> 18:12.450]  is we have a flash memory that's been pulled. This has been pulled from... let's go ahead and look at
[18:12.590 --> 18:19.950]  a couple of them here. It's pulled from an Arlo device, which is a doorbell, if I remember rightly.
[18:19.990 --> 18:28.730]  It's actually a kind of like a ring doorbell type product. Its memory chip was a W25N01GV.
[18:28.750 --> 18:33.550]  So, typically when you pull this thing, what's the first thing you want to do is binwalk.
[18:33.550 --> 18:40.670]  And again, binwalk works many times, but sometimes it does not. So, we'll give this a try
[18:41.370 --> 18:46.470]  and see what happens here. Hopefully, we won't run into any real technical problems.
[18:46.590 --> 18:52.890]  And this is kind of the common flow. We see we hit a squash file system. We should have at least
[18:52.890 --> 18:58.670]  two squash file systems on there if it's coming off a standard embedded device. And there's a
[18:58.670 --> 19:05.490]  second one. And then it kind of hangs right there. If we sit here, this thing would sit here in idle
[19:05.490 --> 19:12.390]  for hours and not produce anything. So, we're going to stop that. So, if we change over to
[19:12.390 --> 19:19.250]  the extraction from binwalk, we can see that it opened up a bunch of stuff, but then there's a
[19:19.250 --> 19:32.610]  squash file system. So, as we notice, there's nothing in there. And truthfully, because it
[19:32.610 --> 19:41.680]  went past the squash file system file, it should have put something in there. So, as we can see,
[19:41.680 --> 19:50.720]  literally nothing in there. So, for the most part, this didn't work. This is actually a fresh rebuild
[19:50.720 --> 20:02.400]  of binwalk. And also, I have a binwalk pro account. The results are pretty much the same
[20:02.400 --> 20:07.240]  in this particular case because there's some things that have to be dealt with here. So,
[20:07.240 --> 20:15.300]  let's go ahead and remove this structure just so I don't start running into space issues by
[20:15.300 --> 20:23.380]  blowing all this stuff out of here. So, what do we want to do? Well, as we talked earlier,
[20:23.380 --> 20:31.180]  we talked about the actual OOB area. So, we want to be able to remove that area. So, we have a tool
[20:31.180 --> 20:37.660]  in here called NandDump. And at the end of this presentation, I'm going to have links for
[20:37.660 --> 20:44.200]  everything, which will cover all this stuff. So, here we can actually do input files, output files.
[20:44.200 --> 20:50.540]  Remember that ID I mentioned to you, chip ID? We can use chip ID, or we can specify actual page
[20:50.540 --> 20:57.720]  size, OOBs, and the layout. Remember, adjacent, separate. So, in this particular case, we are
[20:57.720 --> 21:07.840]  going to... well, before we go ahead and separate the OOB out of here, let's go ahead and look at
[21:07.840 --> 21:18.000]  the data sheet. So, as we look at the data sheet, we see that it is 2048 bytes by 64. So, it is in
[21:18.260 --> 21:24.520]  a separate format. That means the 64 bytes is at the end of the page. That's a good sign. That
[21:24.520 --> 21:30.580]  tells us the structure. You always want to go to the data sheet. But typically, I always like to
[21:30.580 --> 21:37.760]  dig a little deeper and make sure that the chip hasn't been kind of messed with. And why I say
[21:37.760 --> 21:42.580]  that? Because just because the data sheet says this is how the chip's going to be laid out,
[21:42.580 --> 21:48.360]  the manufacturer that actually used the chip, I've seen them do some strange things, to say the least,
[21:48.360 --> 21:54.200]  to the data. Meaning that, you know, you go and remove the OOB area, thinking you know where it's
[21:54.200 --> 22:00.520]  at, and literally have it not be laid out the same way. So, we're going to look at this
[22:01.440 --> 22:07.600]  with HexEdit. So, we're going to look at the raw data of the actual device so we can see what it
[22:07.600 --> 22:33.970]  looks like. There we go. So, at this point, we should be able to go to 2048, which would
[22:33.970 --> 22:41.990]  be 800 in hex, and it should be the beginning of the second block. So, we can see that there's
[22:41.990 --> 22:50.150]  something here. It looks a little different, but it may not be. We're not quite sure. It doesn't
[22:50.150 --> 22:58.230]  really tell us a lot. So, we could go to the next block. So, for example, if hex is 800,
[23:00.330 --> 23:07.370]  and we want to, of course, we want to go, remember, is 20, gosh, we get it right here.
[23:07.470 --> 23:13.910]  So, if we go 2048, we'll get it cleared here in a second. Try it again. 2048,
[23:15.230 --> 23:23.950]  which would be 800, and we want to go up to the next beginning, which should be 2,112 bytes.
[23:23.950 --> 23:34.870]  Remember, 2048 plus the 64, and we look at this, it should be at 1040. So, if we go to 1040,
[23:40.570 --> 23:45.490]  we still see some interesting things, and this may tell us that this is correct,
[23:45.490 --> 23:52.430]  but I've seen a lot of devices where the actual OOB area was nothing but all Fs and wasn't
[23:52.430 --> 23:58.610]  actually configured. So, what we're going to do is let's look a little deeper so we can see some
[23:58.610 --> 24:06.730]  patterns, because I'm always a big fan of spotting patterns. So, we come over here,
[24:06.730 --> 24:13.110]  and get it going, and we're going to search for strings so we can get some data. So,
[24:13.110 --> 24:16.150]  here we are. So, we're in an area where there's a lot of strings,
[24:16.150 --> 24:19.050]  and let's scroll down until we see some patterns.
[24:21.370 --> 24:28.450]  And quickly, we see a pattern right here. So, got a pattern right there. So, if that is...
[24:31.670 --> 24:48.150]  So, we go, hold up, 2112, and go. So, it'd be 840. So, we go 840 plus E880. E880 equals F0C0. So,
[24:48.150 --> 24:56.590]  the next pattern should be an F0C0. And we can see that we're starting to see that pattern there
[24:56.590 --> 25:05.390]  also. So, as we can see, more than likely, we are witnessing the 64 bytes at the end of the 2048
[25:05.390 --> 25:12.230]  as the OOB area is showing up. And again, if we look over in the string area, there's no common
[25:12.230 --> 25:17.310]  words, because I would not expect that to be in the area. So, let's scroll on down to the next
[25:17.310 --> 25:23.330]  one. There's another thing I like to do sometimes, especially if I'm having problems positively
[25:23.330 --> 25:31.230]  identifying this location. So, this will involve a little bit of math. So, I like to use a
[25:31.230 --> 25:42.690]  calculator for that. So, if we come over here, and we put this in here, like that, and we turn it to
[25:42.690 --> 25:50.730]  decimal. And then, because the programming calculator does not round off or have decimal
[25:50.730 --> 25:57.830]  points, we want to do this basic. So, what do we know about the overall structure? The first OOB
[25:57.830 --> 26:06.290]  area or OOB area showed up after 2048. So, we had the first 2048. And then, to get to the next
[26:06.290 --> 26:13.230]  leading ones, we had 2112. So, we go ahead and subtract. And hopefully, my math works here and
[26:13.230 --> 26:23.890]  doesn't mess it up. 2048. And then, divide it by 2112. And hopefully, we'll get an even number. And
[26:23.890 --> 26:31.550]  we did. If you do not get an even number, then this is not mathematically correct for an OOB area.
[26:31.550 --> 26:44.630]  So, what we're seeing here is 0500F900 is actually the 39,749th page of data in this particular
[26:44.630 --> 26:56.460]  device. Okay. So, now that I punished you with some math, we're going to go ahead and strip that out
[26:56.460 --> 27:09.840]  of it. So, we have this command called NAND Dump PI. We're going to use .I Arlo for the input. The
[27:09.840 --> 27:18.420]  layout's separate. Page size happens to be 2048. The OOB size is 64. And then, the output will be
[27:18.420 --> 27:28.460]  Arlo No OOB. Hopefully, this runs. And it finished up. So, there, once we finished up, we have a file
[27:28.460 --> 27:38.810]  that's a little smaller, as you can see, where the OOB's been stripped out of it. And let's go ahead
[27:38.810 --> 27:55.850]  and run this and see if this works for us. The first squash system. You see, the pattern's
[27:55.850 --> 28:03.650]  different now. And then, we also see that there's an OOB structure also starting to show up.
[28:05.050 --> 28:10.950]  Often, you'll see inside an operating system on embedded device, you'll see squash file systems,
[28:10.950 --> 28:19.130]  which are typically read-only. OOB file systems allow read and write. So, it may be an area that's
[28:19.130 --> 28:25.870]  used dynamically that's being created that can use to read and write data into the device. Okay.
[28:25.870 --> 28:33.610]  So, that one actually finished. So, let's go ahead and jump over here and see if it worked for us.
[28:34.210 --> 28:38.590]  A little different there. We see we got three folders. We have an OOB root system.
[28:38.690 --> 28:48.210]  So, we go to squash root. And there you go. We were actually able to extract this data
[28:48.210 --> 28:57.690]  by effectively removing the out-of-band data within the NAND flash dump. So, do we have any questions?
[28:59.110 --> 29:03.650]  Yeah, we got a couple of questions here, T-Dub. So, Elvis Cousin asks,
[29:03.650 --> 29:08.370]  I have two BGAs on a daughterboard. Is it likely I can read each individually,
[29:08.370 --> 29:13.190]  or will I need to reassemble them somehow to try to get a squash FS?
[29:13.190 --> 29:15.990]  And they're also both Intel-based flash.
[29:16.990 --> 29:22.690]  Oh, you're saying you have two flash memory chips on the device.
[29:23.270 --> 29:26.670]  Really comes down to how they lay those out.
[29:27.610 --> 29:34.550]  I would not expect them to stripe across separate chips like that. I've never seen that.
[29:34.550 --> 29:39.710]  Often, when I've seen two chips done up like that, each one will contain a different partition,
[29:40.130 --> 29:45.770]  or they may be redundant of each other. But typically, they'll be partitioned
[29:46.370 --> 29:49.570]  separately on embedded. That's the way I've seen it.
[29:49.570 --> 29:57.550]  And I've seen everything from where one chip actually contained the boot,
[29:57.550 --> 30:00.790]  U-boot, and the kernel, and the other one contained the file system,
[30:02.150 --> 30:06.810]  as an example. So, the only way is to pull them off, examine the data,
[30:06.810 --> 30:10.630]  and make a determination from there to see if you can do it.
[30:10.630 --> 30:14.750]  Of course, the best solution is if you can get a root console on the device somehow,
[30:14.750 --> 30:19.550]  then obviously, you could figure that out more quickly and DD the partitions off that way.
[30:19.550 --> 30:25.830]  Which is also the sweetest way of doing it. But if you have to chip off,
[30:25.830 --> 30:28.410]  yeah, you still may have to do some analysis of them.
[30:29.610 --> 30:34.590]  Yeah, you kind of pivoted off that, said, both are connected to a common connector,
[30:34.590 --> 30:39.390]  but I don't have a pinout to try to recover a cross connector by soldering something.
[30:40.730 --> 30:46.570]  Yeah, it's kind of hard to say. If you have some kind of pictures or images of the data,
[30:46.570 --> 30:50.930]  or not the data, the chips and the layout and the board, shoot them to me sometime,
[30:50.930 --> 30:57.090]  and maybe I can take a look at it and help you come up with a plan of attack.
[30:57.550 --> 31:00.810]  Like I said, anyone's feel free to reach out to me anytime.
[31:00.810 --> 31:05.850]  I may or may not have an answer for you, but I'd be more than glad to give you a hand if I can.
[31:06.850 --> 31:10.950]  One final quick question here that I'm seeing. So, in this particular case,
[31:10.950 --> 31:15.110]  it looks like you're able to find the data sheet. Have you ever encountered a NAND chip
[31:15.110 --> 31:20.690]  that you wanted to dump but couldn't identify it or could not locate a data sheet for? If so,
[31:20.690 --> 31:26.070]  how did you approach that? Yes, guessing.
[31:27.830 --> 31:36.350]  Often, if I can't find a data sheet actually for that, I've actually gone out and looked at
[31:36.350 --> 31:42.110]  other chips that may have data sheets that are in that same series or produced by that manufacturer
[31:42.110 --> 31:48.030]  and figure out how they do their layout. Even though I may not be able to
[31:49.830 --> 31:55.410]  quickly identify the chip ID, there are ways to actually enumerate those, and it
[31:55.410 --> 32:04.310]  usually typically need a data sheet to do that. But guessing does work, and crazy as can be,
[32:04.310 --> 32:11.050]  I've actually dumped a lot of chips by going, hey, here's a manufacturer, and it seems like
[32:12.110 --> 32:18.370]  most of their chips, or a large portion of them within this structure range, are laid out this
[32:18.370 --> 32:25.630]  way. They have this page structure, this size. Often the naming conventions, like here on the
[32:25.630 --> 32:35.370]  screen, W25N01GV, that's one gigabit chip. And often in the name, you can figure out what the
[32:35.370 --> 32:46.130]  chip is a lot of times. So, that would be eight bits, 128 kbytes or whatever. So,
[32:46.790 --> 32:52.470]  it's doable just looking at the name sometimes to pull some typical information out of it,
[32:52.470 --> 32:57.030]  and then guessing on some of the other stuff. But again, I've also smoked chips
[32:57.670 --> 33:02.650]  by doing weird things to them that obviously didn't work. So, got to take some caution
[33:02.650 --> 33:10.950]  and take some risk. But amazingly enough, it can be successful. One of the things to do also is
[33:10.950 --> 33:17.030]  kind of look at, if you can get a UART console onto the main processor, let's say the main
[33:17.030 --> 33:23.870]  processor has a console, even though you can't get root access to it, capture all the debug data
[33:23.870 --> 33:34.030]  from the boot up. That can also be very revealing sometimes. Okay, time to move on.
[33:34.350 --> 33:39.870]  So, we've kind of done that. We're able to get the data off that. I'm going to go ahead and
[33:39.870 --> 33:47.130]  clean this up. Because we have a couple exercises to go to and I don't want to run out of time.
[33:47.130 --> 34:04.890]  So, okay. So, the next one we want to look at... I'm also going to RM the article. So, the next
[34:04.890 --> 34:11.830]  one we want to look at is the Nest. This actually came from a friend of mine who had his Nest
[34:12.310 --> 34:19.110]  thermostat crash out. It just died. He threw it my way, said, hey, can you get any interesting
[34:19.110 --> 34:23.850]  data off of it? So, and I struggled with getting data off of this, but it was eventually
[34:23.850 --> 34:32.790]  successful. We're going to show some of this. There was no confidential data actually extracted
[34:32.790 --> 34:38.650]  from the device. So, there's no problem with us using it here. But so, what we want to do
[34:38.650 --> 34:44.150]  is what we're going to use is a process where we're going to create a NAND sim.
[34:44.570 --> 34:51.910]  So, what does that mean? We're actually going to simulate a NAND chip in memory. We're going to
[34:51.910 --> 35:00.990]  use like ModProbe. If you haven't used ModProbe, ModProbe will let you load kernel modules onto
[35:00.990 --> 35:09.730]  the system. So, we're going to build a block device using ModProbe kernel modules. We're
[35:09.730 --> 35:14.770]  going to load those up and then we're going to write the data out to the chip. We're actually
[35:14.770 --> 35:24.390]  going to tell the data with NAND write to use the OOB data to reconstruct any error corrections
[35:24.390 --> 35:32.630]  and that type of stuff. And then we'll mount it up. So, this particular one, I actually ran
[35:32.630 --> 35:41.770]  Benwalk on this and it never recovered anything. I actually shot it out to good friends at
[35:45.970 --> 35:52.310]  that do the Benwalk Pro. And unfortunately, I didn't get back to the system for a couple
[35:52.310 --> 35:57.750]  weeks and it churned out there. So, I apologize to them again for eating up several terabytes
[35:57.750 --> 36:04.250]  of data. So, this thing was constructually a pain in the butt and running Benwalk was
[36:04.250 --> 36:10.430]  potentially catastrophic. It would eat up all your memory and produce nothing. And I'm not
[36:10.430 --> 36:19.390]  going to run it here, but if you run it, every page or every header for a GIFs to file system
[36:19.390 --> 36:25.410]  or every header for one of those, it would create a folder for it, put nothing in it,
[36:26.170 --> 36:30.430]  and literally eat up terabytes of data in a couple days quickly.
[36:30.470 --> 36:36.190]  So, let's go ahead and get going here. So, we're going to load ModProbe MTD,
[36:36.190 --> 36:43.630]  ModProbe Block. This will help us set up a block device. And then we're going to go ahead and
[36:48.700 --> 36:54.900]  we're going to load this up here. So, the way this works, ModProbe NANDSim, BCH is
[36:55.460 --> 37:03.820]  error correction functionality. This was the chip, it uses a 8-bit structure. There's a BCH16
[37:03.820 --> 37:09.480]  for a 16-bit structure, if that's actually being used. And this here happens to be the
[37:09.480 --> 37:21.160]  chip ID. So, if we go over to this particular chip, that is not the one I want. So, this is the
[37:21.160 --> 37:33.220]  actual chip. And I believe we go to page 25. I have this memorized. Okay. And this would be the
[37:33.220 --> 37:40.620]  read ID, the ID of the actual chip. We know this chip is a 2-gig chip. It's organized in 8-bit
[37:40.620 --> 37:52.080]  structure. It is a 1.8-volt bit chip. So, 01 is the first one, AA, 90, 15. 44 isn't used in this case,
[37:52.080 --> 37:58.760]  but it deals with error corrections and multi-plane information. Multi-plane information,
[37:58.760 --> 38:05.040]  just to make note here, if you have a chip that has multi-planes on it, and you'll see it in the
[38:05.040 --> 38:12.920]  datasheet, it may or may not stripe across the two planes, meaning it puts one block on one plane
[38:12.920 --> 38:18.500]  and one block on the next plane versus putting them contiguously on the same plane. I have not
[38:18.500 --> 38:23.560]  encountered this, but I have read about it. So, it's something to be noted. So, at this point,
[38:23.560 --> 38:31.520]  we're going to go ahead and load this up. So, now we've loaded up that particular module, kernel
[38:31.520 --> 38:39.960]  module, and now let's go ahead and just look at some information, mtdinfo, and this will tell us
[38:40.080 --> 38:48.380]  a little bit about what we just created. So, we created a simulated. It's how many bytes?
[38:49.860 --> 38:58.340]  256-megabyte chip, 2048, subpage size, 64 bytes, pretty straightforward. And then we go ahead,
[38:58.340 --> 39:04.700]  and this one may show some errors. Sometimes it does, sometimes it doesn't. And we're going to
[39:04.700 --> 39:13.500]  look at this within DMessage. And if we come down here, we can see that it took the manufacturer ID,
[39:13.500 --> 39:19.440]  chip ID, and information, and actually have identified the actual chip. That is the chip ID
[39:20.100 --> 39:30.650]  that does that and helps lay this out. So, the next thing is we want to be able to write into it.
[39:30.670 --> 39:36.350]  So, we're going to use NanWrite, which is another tool.
[39:38.370 --> 39:43.890]  Hope you don't mind me not typing. It saves you guys a lot of headaches. So,
[39:44.830 --> 39:52.090]  let's cancel out of that. NanWrite. I want to see the command. So, here's all the functions. So,
[39:52.090 --> 39:57.550]  what we're going to do is we're going to input contains OOB data. So, we're telling it to
[39:57.550 --> 40:04.250]  utilize the OOB data. And we're going to go ahead and let this run. And hopefully,
[40:04.250 --> 40:11.070]  it'll run through clean. Boom! There we wrote all of the blocks out onto the device.
[40:11.110 --> 40:16.370]  So, the next step is... we'll come over here so you see this. We go ls.
[40:22.670 --> 40:28.890]  And we can see there's nothing there. And then what we're going to do is cut and paste. So,
[40:28.890 --> 40:41.880]  I don't have to type. We're going to mount this up on block zero. Change directory to gifs.
[40:43.200 --> 40:50.500]  And voila, we have data. So, if we look through here, it starts looking pretty good. Hey,
[40:50.500 --> 40:58.720]  we've recovered all this data. The sad thing is most of this stuff is system logs.
[41:01.700 --> 41:08.480]  That we recovered. But if we get up here toward the top, we find some weird things.
[41:08.540 --> 41:18.720]  Here's bin. It's a file. etc is a file. Well, truthfully, this is bullcrap.
[41:19.560 --> 41:27.200]  We know that's not a file. So, if we cat etc, wow, look what we get. Now, I wouldn't expect
[41:27.200 --> 41:34.400]  to see that in etc. So, we recovered a lot of it. But some of it's corrupted in a weird fashion.
[41:34.740 --> 41:40.020]  So, how did I recover from this? I experimented. And we're going to go over that a minute. But
[41:40.020 --> 41:51.250]  before we go forward, I actually want to see if there are any questions. Jonathan, any questions?
[41:51.910 --> 41:57.430]  Yep, taking a look now. No updates here. I'm checking Twitch real quick.
[41:59.050 --> 42:03.650]  Yes, here's a question. In IoT devices you've examined, do you ever come across
[42:03.650 --> 42:09.170]  NAND storing non-file system data? If so, do you just dump and then use binwalk
[42:09.170 --> 42:12.810]  and other usual tools and methods for poking around?
[42:13.530 --> 42:24.830]  Yeah, I've had them storing non-file system data on NAND flash. So, yeah, typically I get into that
[42:24.830 --> 42:30.910]  and when I get those and there's no structure to them, I start off with throwing things like
[42:30.910 --> 42:41.370]  strings to the thing. With strings, can we identify any known structure in there?
[42:41.930 --> 42:47.610]  I will run binwalk just to see what it'll do. Sometimes it'll point out some structure headers
[42:48.330 --> 42:54.470]  that I can more quickly find. So, I think that plays a big role. But yeah, sometimes you'll have
[42:54.470 --> 43:02.910]  them storing things like blocks of configuration data. And I've seen them just blocks of encrypted
[43:02.910 --> 43:10.250]  data, which may be a file system structure, but they're not framed as such. So, it's just
[43:10.250 --> 43:17.710]  encrypted blocks of useless information. So, yeah, that's not all that in common. Well,
[43:20.290 --> 43:25.070]  most of the time when I get a NAND chip, it's a file system, but you will encounter those,
[43:25.070 --> 43:32.650]  to be honest. So, right now, before we move on to the next section, I want to remove the
[43:34.830 --> 43:46.840]  kernel modules so they don't conflict with what we're doing. Okay, and we are down to...
[43:48.160 --> 43:53.960]  we still got some time, doing pretty good. So, the next section I want to get into,
[43:53.960 --> 44:03.060]  we are going to use MTD RAM to actually do this. So, we're going to take that same system,
[44:03.060 --> 44:11.900]  that same dump that we just did, where we mounted up the file system,
[44:11.900 --> 44:17.400]  it did not come out exactly the way we want it. And we're going to mess with that. So,
[44:17.400 --> 44:23.640]  we're going to make a make directory, and we're actually going to create our own block structure.
[44:29.350 --> 44:33.270]  And then we'll go, and then we're going to make a node.
[44:35.170 --> 44:44.150]  If I learn how to spell it, I should just copy this over, make node dev slash MTD block.
[44:44.710 --> 44:49.130]  And we're going to create zero. We're going to create it as a block device.
[44:49.130 --> 44:55.350]  We're going to give it a master number, a primary number, and a minor number of zero.
[44:55.350 --> 44:59.850]  So, we're going to create that device in there. And then what we're going to do is we're going
[44:59.850 --> 45:08.570]  use modprobe, and we're going to bring in a kernel module called MTDRAM. So, this kernel
[45:08.570 --> 45:20.970]  module's MTDRAM. We're setting the total size to 274432. That is bigger than a 256 megabyte RAM area.
[45:22.090 --> 45:29.370]  256 megabyte RAM area would be 262144, and that would not contain OOB data.
[45:29.370 --> 45:36.910]  In this case here, we're bringing in the OOB data also. Because if I bring in without the OOB data,
[45:36.910 --> 45:43.090]  I noticed that we get the same results as the other one. So, experimenting, and I encourage
[45:43.090 --> 45:49.070]  you to experiment when you're dealing with these NAND chips. It can be quite fascinating what you
[45:49.070 --> 45:56.370]  can find and figure out and make work to get you data out of this system. I'm currently working
[45:56.370 --> 46:04.270]  on one. It has me slightly befuddled, but I will master it eventually, where the actual entire data
[46:04.270 --> 46:11.750]  on the chip is shifted 10 bytes. They even shifted all the OOB data 10 bytes, and it took
[46:13.610 --> 46:19.390]  looking at it manually to figure this all out and doing that math and looking at the data
[46:19.390 --> 46:25.410]  in the structure to determine that they had also shifted that. Because I was getting block errors
[46:25.410 --> 46:31.130]  saying that block is 10 bytes longer than it's supposed to be type thing. That's when we found
[46:31.130 --> 46:36.450]  out that everything was shifted and was able to confirm that the OOB was shifted too. But I digress.
[46:36.450 --> 46:41.310]  So, what we're going to do is we're going to load this up, and it's supposed to contain the entire
[46:41.310 --> 46:53.430]  thing. And then we want to load one more module. MTD block for kernel module. And then we're going
[46:53.430 --> 47:06.480]  to dd. Excuse me. And dd is a bit copy. I use this all the time. Matter of fact, if you get access
[47:06.480 --> 47:12.060]  to almost every embedded IoT device, you'll find dd is actually installed on it. Because most of
[47:12.060 --> 47:19.600]  the time, dd is actually used for doing firmware upgrades. You'll have duplicate root file structures,
[47:19.600 --> 47:25.480]  duplicate kernels on the device. And what happens is one is primary and running. So, what will happen
[47:25.480 --> 47:32.760]  is when the firmware comes down, it'll dd it over the secondary, and then it'll change the U-boot
[47:32.760 --> 47:39.780]  or secure boot arguments to make the new one primary and the old one secondary. That way, if
[47:39.780 --> 47:46.360]  you have a failed firmware update, it does not break the device. It just goes back to the original
[47:46.360 --> 47:55.580]  one, which was not overwritten. So, again, input file here is the nest file. And then output is to
[47:55.580 --> 48:02.080]  our block device that we created. And if I did everything right, this should start writing data.
[48:02.080 --> 48:11.160]  Should be fairly quick. And as you see, we're able to write out 277 megabytes of information
[48:11.160 --> 48:19.320]  fairly quick, because it's all internal to the system. So, from here, we're going to go ahead and
[48:19.320 --> 48:28.100]  mount this up. Same way we did on the other one. And we're going to mount MTD block zero up.
[48:32.780 --> 48:40.360]  Now, if someone wants to explain to me why this worked, because I really not sure why.
[48:40.360 --> 48:47.280]  As I said, often dealing with NAND chips, little quirky things have worked for me. So, I'm not going to
[48:47.280 --> 48:54.700]  speak bad of them. And now we can see etc and bin are actual files. We do have some
[48:54.700 --> 49:02.000]  corruptions in here. So, there's a few files actually corrupted within this. But for the
[49:02.000 --> 49:09.060]  most part, a large portion of them were actually able to be recovered properly.
[49:17.990 --> 49:25.410]  And so, these are methods that I use quite regularly for recovering. When you start
[49:25.410 --> 49:34.070]  working with UBI file systems, if I ever get some good methods, methodologies hammered out
[49:34.070 --> 49:39.150]  for working with some of the strange UBI file systems, like the one with the 10 bytes that's
[49:39.150 --> 49:46.870]  shifted. I did one on an engagement here earlier this year, where again, nothing are large portions
[49:46.870 --> 49:53.730]  of it were not laid out contiguously and trying to build good solid methodologies around UBI.
[49:53.730 --> 49:59.570]  I'll probably come back around and put together a training session presentation for that down
[49:59.570 --> 50:04.070]  the road. That's what I'm currently working on, trying to work out those kinks and bugs and build
[50:04.070 --> 50:10.430]  that knowledge level to build to work with us. Now, probably half the chips that are UBI file
[50:10.430 --> 50:17.530]  systems, I'm successful recovering them using various techniques and UBI tools that are available
[50:17.530 --> 50:26.210]  out there. But typically, GIFs, squash file systems, things like that can easily be quickly
[50:26.210 --> 50:33.570]  recovered from NAND flash chips using any one of these three methods that we covered today
[50:33.570 --> 50:39.250]  in this presentation. So, hopefully there's some questions out there.
[50:40.550 --> 50:48.610]  Yeah, I got a couple here. So, one question here asks, have you encountered an IoT device
[50:48.610 --> 50:52.850]  that was encrypting the data it was storing in the NAND flash?
[50:54.330 --> 51:02.010]  No, I haven't, which is kind of amazing to me. I mean, the only time I've ever seen IoT
[51:02.010 --> 51:08.670]  technology using any kind of encryption, if it was running an Android operating system.
[51:08.670 --> 51:16.890]  And by nature, Android can support the data section being completely encrypted.
[51:16.890 --> 51:28.030]  So, that's quite common. I do have some other tech that I consider IoT, but it's more enterprise
[51:28.030 --> 51:34.890]  IoT and not necessarily the normal consumer grade, which actually is very much encrypted
[51:34.890 --> 51:41.390]  and most of the, not all the file system, but most of the file system that contains user data,
[51:41.390 --> 51:49.670]  similar to the Android file system, is also encrypted. So, typically, if I see it's an
[51:49.670 --> 51:56.970]  Android, it's usually 50-50. If it is a standard phone-style Android operating system dumped on
[51:56.970 --> 52:03.830]  there, then most always it has encrypted. But yet, if it's an older Android style that's put on
[52:03.830 --> 52:09.390]  there, there was no requirement or typically that's not always encrypted. So, it's a mix and
[52:09.390 --> 52:15.530]  match, but it's pretty rare to see encrypted, at least from a lot of the IoT I've looked at,
[52:15.530 --> 52:21.350]  unless you get into more enterprise level technology.
[52:23.690 --> 52:27.170]  Anonymous attendee asks, Hi, I am new to this. Sorry,
[52:27.170 --> 52:32.190]  you already have the binary file. How did you extract it from the hardware device?
[52:33.830 --> 52:41.410]  In the cases of these, both of them were chip off. So, early in the presentation,
[52:42.110 --> 52:47.630]  and so you can rewatch this presentation, or in the presentation earlier today,
[52:47.630 --> 52:52.750]  where we talked about building an IoT lab and we showed a number of devices,
[52:53.110 --> 52:59.210]  I have a number of chip readers. So, I actually use those. Other ways are
[53:00.650 --> 53:07.190]  often if the chip that's being used has some kind of debug function, you can put it in debug
[53:08.030 --> 53:13.490]  and pull the data off of it. Also, you may be able to check, connect in with JTAG or
[53:13.490 --> 53:20.290]  SerWire debug and actually read it off the device. But if those fail, I often rely on just
[53:20.290 --> 53:25.990]  desoldering the chip and dropping it into a reader and extract the data from there.
[53:25.990 --> 53:31.410]  If I need to put the device back into functionality, then I'll go through either
[53:31.410 --> 53:38.430]  resoldering the chip back on or reballing the chip and reflowing it back on an IR oven,
[53:38.430 --> 53:44.450]  which are fairly cumbersome, at least the reballing BGAs, but it's still doable.
[53:44.450 --> 53:51.730]  And it's not all that complex. I did a presentation not last year, but the year before last
[53:52.370 --> 53:59.070]  at Nuit de Hacq. I did it there. I also did it, oh gosh, I did it in Perth, Australia,
[53:59.070 --> 54:08.450]  and I also did it at DerbyCon. And it was, I'm trying to remember the name of it. Basically,
[54:08.450 --> 54:14.830]  it was how to get root access the hard way. And it talks about slashing, smashing,
[54:14.830 --> 54:20.390]  cutting the device apart and rebuilding it to get root access or something to that effect.
[54:20.390 --> 54:23.310]  So you can look at that, and it goes through some of the concepts
[54:23.750 --> 54:29.270]  in my learning process on learning how to manually reball BGA chips.
[54:31.050 --> 54:35.990]  Elvis Cousin asks, these are all variants of Linux. On a Windows CE device,
[54:35.990 --> 54:40.150]  might it still use SquashFS or some other Windows thing?
[54:40.310 --> 54:49.390]  Ooh, probably some other Windows thing. I have not looked at a Windows CE device.
[54:51.230 --> 55:01.090]  I have been looking at some Chrome devices. In those particular cases,
[55:03.290 --> 55:09.890]  this has an SD drive, solid state drive, where you're able to get the data off of it and just
[55:09.890 --> 55:15.410]  mount it up on your system. I'm not sure if they have NAND chips. Is it going to be
[55:17.570 --> 55:22.890]  a Linux file system? Probably not. It's probably going to be some variation of Windows. But I have
[55:22.890 --> 55:29.130]  not tore into any Windows or Windows CE type devices to be able to give you a solid answer
[55:29.130 --> 55:39.570]  on that. Let me know if you find out. Any other questions? Nope, all caught up.
[55:40.410 --> 55:45.650]  All right. Well, I think we're... what time we start? Six o'clock. So we're probably right on
[55:45.650 --> 55:58.010]  time. So let me jump over here. Let's do a final stop to share. So let's go ahead and
[55:59.270 --> 56:05.250]  ask question one more time. Anyone have any questions, any requests for questions or
[56:05.250 --> 56:12.550]  information? If not, you'll probably catch me on Discord either here in a little bit after I've
[56:12.550 --> 56:21.750]  eaten or late tonight or tomorrow. Feel free to hit me up for any questions. Also, oh gosh,
[56:21.750 --> 56:25.950]  yeah, I was promising that. Hold on before you all drop off there. Let's get this up.
[56:32.020 --> 56:49.120]  So I had promised... slideshow. So I want to be able to get some of these things captured in the
[56:49.120 --> 56:55.720]  video. So here's the NAND sim example that I use. The mod probe commands, the NAND write commands,
[56:55.720 --> 57:00.220]  and the mount to be able to mount this particular GIF file system up.
[57:00.840 --> 57:12.440]  And here is the commands that I use for mtd-dram example. Quick note on some of the commands we
[57:12.440 --> 57:22.000]  use. There is the link for NAND dump tool, concept of mod probe, make node, NAND write, DD. And then
[57:22.000 --> 57:28.600]  here is the reference. So that gets captured in here. All of this is kind of cool material.
[57:28.620 --> 57:33.500]  There's a lot of great work that's been done out there on NAND chips. Like I said, I'm constantly
[57:33.500 --> 57:41.000]  reviewing and reading and consuming this material on a regular basis because every time I encounter
[57:41.720 --> 57:45.660]  NAND chip that wants to fight with me, I have to go out and start rethinking,
[57:45.660 --> 57:50.700]  okay, what am I missing? What do I need to learn to expand my knowledge? How do we kind of move
[57:50.700 --> 57:56.420]  forward from here? And there's been a lot of work done out there. And of course, we shared
[57:57.300 --> 58:04.980]  much of the information that these people provided in this demonstration today. Also,
[58:04.980 --> 58:13.140]  if you're interested in connecting up with me on Twitter, my handles are %X, so it would be
[58:13.140 --> 58:22.180]  P-E-R-C-E-N-T underscore X. And feel free to connect with me out there or reach out to me
[58:22.180 --> 58:31.980]  and ask questions out there. Awesome, Daryl. Thank you so much.
[58:32.220 --> 58:38.720]  Oh, man. Sure enough, Sam. Had a blast. Yeah. And I mean, honestly, if there are any last
[58:38.720 --> 58:42.780]  questions, we still have a little bit of time. So was there anything else that you saw there,
[58:42.780 --> 58:51.880]  Jonathan? Yeah, another quick question just popped up. It says, this is some very epic stuff,
[58:51.880 --> 58:57.820]  extracting flash from NAND chips. I have to ask, did you find anything epic while performing
[58:57.820 --> 59:10.300]  post-exploitation on the extracted firmware? Yeah, yeah. I mean, any remote shells? No,
[59:10.300 --> 59:18.420]  unfortunately. But again, it's information that we could feed back to manufacturers to help
[59:18.420 --> 59:25.440]  improve processes. You know, they get into hard-coded stuff. Mainly, a lot of this lately
[59:25.440 --> 59:31.960]  was purely an effort to go, hey, what happens when IoT is thrown in the trash type thing?
[59:32.020 --> 59:40.180]  You know, is there data is like the guy with the Nest thermostat? What happens when something
[59:40.180 --> 59:46.860]  dies if you throw it away? Does it contain PII information? So this was kind of a lot of it was
[59:46.860 --> 59:54.940]  driven out of that and some paid assessments. And also, you know, we tested a lot of devices from,
[59:54.940 --> 01:00:01.840]  hey, if you go ahead and flush the thing back to factory, does data stay on it? We kind of
[01:00:01.840 --> 01:00:07.540]  started checking for things like that. It's all about a privacy risk and issue, because we've
[01:00:07.540 --> 01:00:13.080]  gotten really big on the whole idea that, hey, we don't want to throw our hard drives away.
[01:00:13.080 --> 01:00:19.100]  But in this modern world, these chips are our hard drives. And we need to probably start thinking
[01:00:19.100 --> 01:00:25.580]  about the potential risks there. So that was kind of the main purpose of a lot of this,
[01:00:25.580 --> 01:00:31.040]  and the fact that I had been working on some engagements that involved band chips that were
[01:00:31.040 --> 01:00:38.480]  problematic. In those cases, we were working on things like reconstructing keys to gain access.
[01:00:38.480 --> 01:00:44.460]  A lot of this stuff ties into engagements also, where we're looking for data since we're testing
[01:00:44.460 --> 01:00:50.180]  an entire ecosystem. That isn't just the hardware, but it may be mobile apps, cloud services,
[01:00:50.180 --> 01:00:56.640]  and other devices. What can we get off this chip that could allow us to attack everything else
[01:00:56.640 --> 01:01:03.100]  or attack somebody else using the same product because of key reuse and key session keys and
[01:01:03.100 --> 01:01:08.120]  password reuse and things like that. So a lot of it's within that direction.
[01:01:10.540 --> 01:01:16.980]  Another question here. This individual asks, does it seem like it is possible to be able to create
[01:01:17.180 --> 01:01:24.420]  a module so that a chip reader would automatically be able to detect those out of the OOB areas of
[01:01:24.420 --> 01:01:30.140]  the flash memory? Oh, a number of chip readers will let you identify. Some of them will allow
[01:01:30.140 --> 01:01:36.440]  you to extract the chip, the data, without the OOB. So yeah, it's not uncommon. You can
[01:01:36.440 --> 01:01:46.360]  often manipulate data from the chip reader and include or not include OOB on some chip readers.
[01:01:46.360 --> 01:01:50.360]  Obviously, you get more control as the price of the chip reader goes up.
[01:01:50.780 --> 01:01:58.460]  And I don't necessarily have a $2,500 chip reader in here. But I think one of these actually had
[01:01:58.460 --> 01:02:06.160]  some OOB settings. But for some reason, I didn't necessarily trust it because I told it not to
[01:02:06.160 --> 01:02:12.120]  include OOB. And then when I visually looked at the data, the OOB data was there. So I kind of lost
[01:02:12.120 --> 01:02:19.360]  trust. I have a tendency to also like to map it out, not let the chip reader make the determination
[01:02:19.360 --> 01:02:25.680]  for me that this data is structured a certain way. Because like I said, it's not uncommon with
[01:02:25.680 --> 01:02:32.660]  vendors to futz with it, so that it may be incorrect, or that the OOB shifted or is not
[01:02:32.660 --> 01:02:38.440]  used in the same place. And they've just written a new controller to read the data the way they
[01:02:38.440 --> 01:02:44.920]  made it. This way allows me to go, hey, this is correct or isn't correct. And then I can easily
[01:02:44.920 --> 01:02:50.740]  manually remove it or modify code or scripts that are out there to take it out the way I
[01:02:50.740 --> 01:02:58.320]  wanted it removed. So that's kind of my take on it. That makes sense. I think that we're at the bottom
[01:02:58.320 --> 01:03:06.040]  of the list here for the questions. Okay, well, I think that kind of concludes it. If anyone has
[01:03:06.040 --> 01:03:13.020]  any questions, catch up with me later, or reach out through me on Twitter. And again, I'll be on
[01:03:13.020 --> 01:03:20.440]  Discord probably tomorrow and maybe later tonight. So I'll catch everyone later. Have a good night.