WEBVTT 00:01.080 --> 00:03.400 Okay, I guess we'll go ahead and get started. 00:04.400 --> 00:10.500 I am Julian Grizzard and I'm a graduate student at Georgia Tech and studying computer security. 00:10.780 --> 00:17.100 And today I'm gonna give you a talk on Surgical Recovery from Current-Level Rootkit Installations. 00:17.740 --> 00:20.360 And this is on Linux-based systems. 00:22.890 --> 00:29.010 So first off, the latest slides, tools, and our spine architecture work is available online. 00:29.110 --> 00:31.110 The links to this is on your CD. 00:31.110 --> 00:35.090 And so definitely download the latest, spent a lot of changes. 00:36.850 --> 00:40.630 All right, so first off, some background on Rootkits. 00:40.630 --> 00:42.270 What does a Rootkit do? 00:42.710 --> 00:52.290 So a Rootkit is a toolkit that an attacker will use to retain access to your system and hide activity that he's doing on your system. 00:52.790 --> 00:57.070 A Rootkit doesn't give you access to a system if you don't have prior access. 00:57.070 --> 01:02.990 You have to exploit the system by some known vulnerability or something you figured out, or social engineering or whatever. 01:03.630 --> 01:14.350 And then the attacker will usually install a Rootkit in order to come back into the system at a later point in case the password changes or his vulnerability's patched. 01:18.630 --> 01:25.230 So how many people have personally had experience with a Rootkit installed on a system they administered? 01:25.790 --> 01:28.430 We got some people, oh wow, we got lots of people out there. 01:28.430 --> 01:35.090 Interesting, and so what is the recommended thing to do when you get a Rootkit installed on your system? 01:37.070 --> 01:39.490 That's right, format and reinstall. 01:39.790 --> 01:44.050 So I'm actually going against that grain, you know what I'm saying? 01:44.050 --> 01:48.130 Okay, maybe there's more efficient ways to recover from a Rootkit. 01:48.130 --> 01:53.650 I mean, what if you got a server that every minute of downtime costs millions of dollars? 01:53.690 --> 01:56.310 I mean, how can you format and reinstall that system? 01:58.210 --> 02:02.110 So we're going to look at different ways to recover from a Rootkit. 02:02.910 --> 02:11.850 So first off, I want to give you guys some background on operating systems because what a Rootkit does is it replaces parts of your operating system. 02:11.850 --> 02:18.490 So it's important to understand how an operating system works in order to understand the design of Rootkits. 02:19.030 --> 02:22.210 So this is a monolithic operating system. 02:22.490 --> 02:25.850 That's just a fancy term for this particular architecture. 02:25.850 --> 02:28.050 Linux looks like this. 02:28.070 --> 02:29.830 Windows, all the windows look like this. 02:29.830 --> 02:31.430 Most systems look like this. 02:32.270 --> 02:34.830 So you've got two parts to an operating system. 02:34.830 --> 02:47.950 You've got the kernel space and this is the code that runs right on top of your hardware, controls all your devices, schedules your processes, does your file access, network stack, these kinds of things. 02:50.580 --> 02:52.800 And you have user space. 02:52.800 --> 02:55.740 User space is where the programs that you really care about run. 02:55.740 --> 03:02.880 So P1 could be your word processor, P2 could be a LS, bash command, and so on and so forth. 03:04.420 --> 03:09.680 Now I want to give you a contrasted operating system design that I'll come back to later. 03:09.840 --> 03:14.200 But I just want to give you a preview and compare and contrast to current OSs. 03:14.200 --> 03:17.540 This is the microkernel operating system design. 03:18.260 --> 03:22.280 The difference here is you have a very thin layer for the kernel space. 03:22.700 --> 03:33.420 You have minimal, I mean, only code you have in the kernel is the minimum amount of code needed to guarantee isolation among processes and that kind of thing. 03:33.420 --> 03:43.120 All the drivers, video, disk, hardware, network, all this stuff is pushed into user space sitting alongside your user space processes. 03:43.220 --> 03:47.980 And each bubble here is guaranteed isolated by the microkernel. 03:50.430 --> 03:53.450 So what guarantees this isolation? 03:53.850 --> 04:01.650 Well, on the Intel x86 architecture, a lot of you have probably heard of this ring architecture where you have level 0, 1, 2, 3. 04:01.930 --> 04:06.190 And it turns out modern operating systems only use two levels. 04:06.190 --> 04:11.370 They use level 0 for the kernel, which gives you unlimited access to the hardware. 04:11.490 --> 04:18.970 If you have a program running at level 0, you can do anything to any bit anywhere in the computer. 04:19.630 --> 04:24.170 And the user space processes run at level 3, which is shown in yellow here. 04:24.730 --> 04:28.890 And if you want access, they don't have direct access to the hardware. 04:28.890 --> 04:33.430 If you want access to the hardware, you have to make a system call, ask the kernel to do it for you. 04:36.090 --> 04:42.970 So I want to show you just real quick how this is implemented in the hardware. 04:42.990 --> 04:49.970 There's a 16-bit code segment register on Intel Pentiums and x86 processors. 04:50.570 --> 04:57.970 And the lower two bits of that register specify what privilege level you're running at, which is which ring you're in. 04:58.710 --> 05:02.890 And this is some test code to test and see which ring you're in. 05:05.110 --> 05:08.950 So I'm going to demo this code running real quick. 05:08.950 --> 05:11.490 Just a quick demo here. 05:13.840 --> 05:17.640 So I have a Linux box running over here to my right. 05:18.320 --> 05:19.940 I'm going to log into that. 05:30.190 --> 05:36.290 All right, so I have this getcpl program that gets your current privilege level. 05:36.450 --> 05:41.870 And it returns three, because I'm running a user space process forking off the command line. 05:43.350 --> 05:45.090 Now let's check out the kernel. 05:53.880 --> 05:57.820 All right, so I inserted a kernel module called... 06:00.360 --> 06:02.440 Well, it's way too big on there. 06:03.780 --> 06:08.420 I inserted a kernel module called getcpl, which all it does is... 06:10.200 --> 06:19.460 Right here, all it does is pops the CS register from into a variable, and ends it with a three to get the ring level. 06:19.960 --> 06:28.060 And if you look at the bar log sys log, you can see that, in fact, the kernel is running at ring zero. 06:37.490 --> 06:41.430 All right, I'm going to get my next demo set up, and we'll go back to the presentation. 06:50.220 --> 06:51.900 And so these slides here are just showing that. 06:51.900 --> 07:02.800 All right, so if you're looking at the operating system, and you want to make a user level rootkit, how do you do it? 07:02.800 --> 07:06.500 Well, you attack files on the hard disk, shown in red here. 07:06.960 --> 07:11.080 And so you replace ps, netstat, ls, top, password. 07:11.120 --> 07:16.280 These kinds of files, you replace those with malicious files that hide your activities. 07:16.280 --> 07:19.480 Hide files, hide network connections, these kinds of things. 07:19.840 --> 07:26.060 And so when you replace those files, they get loaded into your processes, the green bubbles up there. 07:27.020 --> 07:29.300 So that is pretty much the user level rootkit. 07:29.320 --> 07:30.680 It's pretty easy to detect. 07:30.680 --> 07:36.660 You just run tripwire, check your system against your known good state, and it's not too difficult. 07:36.660 --> 07:37.900 We know how to do that. 07:38.480 --> 07:43.900 Definitely tripwire has its issues, but there's a way to deal with those. 07:44.480 --> 07:52.800 Now kernel level rootkit attacks, they modify the running kernel code and the running data structures, which is this block shown here in red. 07:53.280 --> 07:58.180 So anything in there is suspect to being attacked by a kernel level rootkit. 07:58.520 --> 08:02.820 And all a kernel is, is a running program on your computer. 08:02.860 --> 08:11.320 So in essence, these kernel level rootkits are modifying code on the fly and redirecting that code to malicious code. 08:11.820 --> 08:20.200 In particular, the system calls is the gateway from user space to kernel, and that's a good target. 08:20.200 --> 08:22.500 And I'll go over those two examples. 08:23.600 --> 08:33.240 So first I want to go over a system call and why that is the most popular attack target for kernel level rootkits. 08:33.380 --> 08:43.020 So it's important to understand how a system call works in order to A, detect that the system call has been tampered with, and B, recover from it. 08:43.600 --> 08:47.120 So what happens, what do you use a system call for? 08:47.120 --> 08:51.300 Okay, say you're a user process and you want to read files from a hard disk. 08:51.440 --> 08:55.260 Then you can't do that directly because you don't have access to the hardware. 08:55.260 --> 09:02.660 So you issue a system call to the kernel to open, read the contents of those files, and so on and so forth. 09:03.240 --> 09:06.120 So for instance, let's say we have a sys read system call. 09:06.120 --> 09:07.080 What happens? 09:07.260 --> 09:22.800 Well, the first thing that happens is your user space process issues an int 0x80 instruction, which says a software interrupt, which indexes into 0x80th entry of the interrupt descriptor table. 09:23.380 --> 09:24.860 And that's shown right here. 09:25.100 --> 09:37.320 So as soon as you issue that interrupt, you go over to the kernel, the kernel looks up where the IDT is in memory, looks up that 0x80th entry, and executes the code pointed to by that entry. 09:37.760 --> 09:41.660 The code pointed to by that entry is the system call handler. 09:42.680 --> 09:53.380 And the system call handler is responsible for setting up parameters of your system call, generic parameters, it works, that's the first thing that's called for any system call. 09:53.680 --> 09:57.520 And then it calls the system call you specifically requested. 09:57.960 --> 10:04.660 So you see this pointer coming out of the code in the system call handler, which points to the system call table. 10:05.020 --> 10:13.720 So it references, let's say for sys read, you look up this entry number three in the system call table over here. 10:14.380 --> 10:17.900 And that redirects you down here to the sys read system call. 10:18.480 --> 10:24.960 And the sys read then takes the parameters given to it by a system call handler and executes your request. 10:28.890 --> 10:31.070 So what's the attack points here? 10:31.110 --> 10:34.310 Anything in this chain is an attack point. 10:34.690 --> 10:48.850 You could redirect the IDT entry, you could redirect the system call handler reference to the system call table, you could redirect individual system calls, you could overwrite the system call with a jump instruction to go somewhere else. 10:48.850 --> 10:50.890 All these are susceptible. 10:59.310 --> 11:03.290 So, how do you recover from this system call attack? 11:03.470 --> 11:11.290 Well here's the algorithm, it's basically installing a new system call table into the running kernel. 11:11.290 --> 11:13.910 This is exactly what a rootkit will do, in fact. 11:13.910 --> 11:18.450 But we're gonna use a good system call table that has known good state. 11:19.070 --> 11:31.650 So the best place to get that is to keep a copy of your kernel on another disk on safe storage and you can rip out these system calls using what I did was modify GDB to get that code out. 11:32.250 --> 11:51.570 And so you, it's pretty simple, you just, well, it's somewhat simple, it is kernel stuff, but basically you copy each system call into kernel memory and then you create a new system call table pointing to all those system calls and then you can set the handler to point to that system call table. 11:52.330 --> 12:11.310 And there's, you can go full blown, you can, if we go back to this picture, you can set this pointer right here which is on the left for you guys and you can point that to a new system call handler if you want and that way you'll know the whole chain is correct. 12:15.360 --> 12:21.980 Okay, so it's a little bit, just want to give you guys some details on, it's not quite that simple, you can't just copy stuff over. 12:22.080 --> 12:32.640 When you copy code around in memory, x86 code has these call instructions with relative offset parameters, so you have to adjust them, basically relink them into the kernel. 12:33.180 --> 12:44.820 So you could recompile the code if you know where it's gonna be loaded, but I chose just to recompute those relative offsets as I was inserting each system call into memory. 12:45.300 --> 12:57.940 And you adjust the relative offset based on where it's placed in memory because you allocate dynamic memory for it and you compare that to where it was originally in memory and you just subtract the difference. 12:58.880 --> 13:05.140 Of course, it's a little bit more confusing with the way the Intel architecture works, but that's the gist of it. 13:06.800 --> 13:11.080 So another thing is there's multiple ways into the kernel. 13:11.440 --> 13:12.620 I'll just give you some background. 13:12.620 --> 13:21.060 This is another tidbit of kernel-level rootkit stuff is you can insert a kernel module into the kernel and muck with the system call table. 13:21.300 --> 13:23.760 And that's one way to penetrate the kernel. 13:24.480 --> 13:29.260 Another way you can get into the kernel is through the dev.kmem.device file. 13:29.580 --> 13:36.900 This device file gives you access to the kernel memory as a file. 13:37.400 --> 13:41.540 So if you're a user space process, you can open that dev.kmem like it's a regular file. 13:41.660 --> 13:49.440 You can seek to it and you pass it a kernel memory address and it seeks that memory and then you can read that data or write to it. 13:51.140 --> 13:58.220 Is anybody familiar with the SuckIt rootkit from those people that have had installations on their computers? 13:59.040 --> 14:01.120 We got some people, all right. 14:02.460 --> 14:03.760 So there's a few more. 14:03.760 --> 14:11.260 Okay, so if you want to install your rootkit from dev.kmem without doing a kernel module, there's a couple of things that are tricky. 14:12.300 --> 14:14.500 First, how do you find the system call table? 14:14.640 --> 14:15.960 You don't have the symbols. 14:16.400 --> 14:22.160 And two, how do you allocate memory to insert new code into the kernel? 14:26.470 --> 14:30.830 All right, so first I'll just go over how you find the system call handler right quick. 14:31.490 --> 14:35.090 This is some code pulled out of SuckIt. 14:35.090 --> 14:41.190 First thing you do is you pop the IDT register into memory. 14:41.890 --> 14:45.530 And what that is, is that tells you the address. 14:45.530 --> 14:47.610 Let's go back a few slides real quick. 14:47.610 --> 14:50.670 That tells you where this table is in memory. 14:54.210 --> 14:56.730 All right, so now you know where that table is. 14:56.790 --> 15:08.870 So then you read the 0x80th entry of that table and you can compute where the system call handler is in memory. 15:08.870 --> 15:12.870 And then you can parse the system call handler to find out where the system call table is. 15:15.980 --> 15:24.460 All right, so how do you get dynamically allocated memory from the kernel from user space? 15:24.460 --> 15:31.040 If you write a kernel module, you just use kmalloc, but you don't have access to that code from user space. 15:31.460 --> 15:37.820 Well, the way SuckIt does it is sets up kmalloc as a system call. 15:38.480 --> 15:42.120 So there's a number of unused system calls in the system call tabler. 15:42.980 --> 15:47.420 SuckIt takes over one of those system calls and points it to kmalloc. 15:47.420 --> 15:57.400 So now you can issue system call 180 or something like that and pass it a size, memory allocation size and you'll get kernel memory returns. 16:01.140 --> 16:04.020 Okay, so now I wanna do my next demo. 16:24.770 --> 16:36.390 Okay, so the scenario here is I've already got root access to this machine, so I'm wanting to install my rootkit to retain access and to hide some files. 16:36.650 --> 16:47.170 In this folder right here, I have some bank accounts I wanna store on this computer and don't worry, these are not legit. 16:49.990 --> 16:51.270 At least I hope not. 16:53.230 --> 17:02.710 Okay, so we've got this data that I don't wanna store on my own computer, but I wanna store it on this machine that I've compromised so I can trade it with somebody else, something like that. 17:06.140 --> 17:17.360 So then I've already downloaded my SuckIt rootkit onto the system and I'm going to install it and it's done. 17:17.800 --> 17:26.320 So now that directory that I was just showing you up here, this hidden info.sk12 is now hidden. 17:26.700 --> 17:32.440 The way this works is anything that ends in .sk12 will be hidden. 17:32.440 --> 17:40.580 For instance, if I touch a file called hide.me, you can see that it's there. 17:40.640 --> 17:46.720 If I move hide.me to hide.me.sk12, it's gone. 17:51.840 --> 17:58.720 So now I'm the administrator of this computer and I believe I may have a rootkit on my system. 17:59.360 --> 18:07.280 So what I'm going to do is try and install a new system call table to see if there's any differences in the file system. 18:10.120 --> 18:13.860 Okay, so this takes a little bit of time because it's not optimized. 18:14.100 --> 18:17.160 It's good for demo though because I can explain a few things. 18:17.520 --> 18:24.100 So what you see here is each little block of code, it's copying a system call into kernel memory. 18:25.500 --> 18:33.780 It issues a kmalloc for each one of these, copies it in, and then you see these asterisks, asterisks, asterisks, found, dot, dot, dot. 18:33.780 --> 18:41.880 It's finding these call instructions and it's recomputing where they need to be relinked to and setting that. 18:43.960 --> 18:48.740 So right now we're at about 130 system calls that have been copied into memory. 18:48.740 --> 18:53.440 Turns out Linux has 255-ish system calls. 18:54.400 --> 18:57.640 And so we'll be through this here in a second. 19:12.400 --> 19:14.500 Does anybody have any questions so far? 19:14.500 --> 19:15.900 We've got a few more minutes. 19:29.980 --> 19:36.680 From what I've found, I mean, that's a possibility that they have absolute calls, but you could relink those in as well. 19:36.780 --> 19:40.060 From what I've found, they don't have absolute calls. 19:50.470 --> 19:51.870 Excellent, excellent question. 19:51.870 --> 19:53.730 I will get to that very shortly. 19:56.050 --> 20:02.050 All right, so the new system call table has been installed. 20:02.050 --> 20:04.890 Now notice these hidden files come back up. 20:05.810 --> 20:09.050 So as you can see, all Sucket was doing was redirecting some system calls. 20:09.050 --> 20:11.630 So we install a known good system call table into memory. 20:11.710 --> 20:13.990 All of a sudden, these files come back up. 20:24.140 --> 20:28.320 Does anybody have any other questions while we got a stopping point in the presentation? 20:43.830 --> 20:45.330 How does it catch it? 20:48.680 --> 20:50.480 Right, let's go back to that figure. 20:53.860 --> 21:01.840 All right, so you're saying this code right here is being overwritten or something like that. 21:02.440 --> 21:07.820 Okay, so first off, the tools I was just showing you do not do rootkit detection. 21:07.820 --> 21:11.780 Well, some of them do, but the recover rootkit itself does not. 21:12.240 --> 21:22.740 I do have the ability to read any system calls, or I could dump this code for sysread to a file and see if it's been overwritten. 21:22.740 --> 21:25.400 For instance, to answer your question, that would be the way to do it. 21:25.480 --> 21:26.860 It's changing memory. 21:29.740 --> 21:39.340 So the way the recovery tools work is, right now it actually just replaces this pointer to a new system call table. 21:40.100 --> 21:43.200 And so that's gonna shoot over here into memory somewhere. 21:43.520 --> 21:46.800 You got a new system call table over here, you got a new sysread over here. 21:46.940 --> 21:52.420 So even if this sysread right here was overwritten, that's not a problem because we aren't depending on that. 21:52.780 --> 21:53.800 Good question. 21:59.250 --> 22:03.830 Okay, so back to our gentleman's over here's question. 22:04.330 --> 22:25.370 Okay, so it turns out if you use devkmem to modify the kernel, as I do with my recovery tools, or even if you use a kernel module to do the recovery, either one of these approaches relies on system calls to do the recovery. 22:25.490 --> 22:27.410 So there's an obvious problem here. 22:27.410 --> 22:44.030 If you've installed a rootkit that's redirected system calls and you're depending on those system calls that's redirected, then you may be depending on code that's going to prevent your recovery tools from working. 22:45.110 --> 22:46.730 Does that make sense to everybody? 22:46.730 --> 22:48.030 It's a little bit confusing. 22:48.430 --> 22:55.910 Okay, so how do we deal with this? 22:56.710 --> 23:03.110 Well, I propose we need a new system architecture. 23:03.310 --> 23:13.290 And let me take a minute to explain the vision I have here, which goes much beyond rootkits and much beyond anything like that. 23:13.290 --> 23:18.010 I've got this vision of intrusion recovery system. 23:18.670 --> 23:28.790 So this is like, you know, you've got your intrusion detection system, you've got your intrusion prevention system, but bottom line is we still have compromises every day. 23:28.790 --> 23:30.950 It's still a problem for the foreseeable future. 23:31.190 --> 23:39.090 So we need to think about ways to more quickly recover from compromises given that they're going to be a problem. 23:39.090 --> 23:40.790 Systems are so complex. 23:41.510 --> 23:52.690 So that's why I've come up with this notion of an intrusion recovery system, which has a new system architecture that's not currently used in widely available OSs. 23:52.690 --> 24:02.030 And it's capable of verifying the integrity of the running system and repairing that state if it has some problems. 24:02.690 --> 24:16.570 And it has, you know, obviously there's got to be some kind of, this has got to be the top most secure piece of your system because you rely on it for, you know, your system running good. 24:16.750 --> 24:21.930 So you have to have some kind of way to store known good state for your system. 24:22.430 --> 24:28.250 Because if you store the known good state in your system that is accessible by root, then that can be changed. 24:28.310 --> 24:32.370 So this new architecture supports these kinds of things. 24:32.450 --> 24:37.770 It supports what I call a state hold, which is kind of like a strong hold, but it stores your state instead. 24:38.710 --> 24:40.490 And it does these other things. 24:40.490 --> 24:43.890 And I'll go over, this is my research right now. 24:43.890 --> 24:45.350 So it's certainly in development. 24:45.470 --> 24:47.970 And I'm looking for any kind of feedback you guys have. 24:47.970 --> 24:51.110 It's probably got some problems, but I'll show you what I've got. 24:51.110 --> 24:53.710 And we'll see what you say. 24:54.630 --> 25:03.370 So back to the beginning, I showed you, you've got the monolithic operating systems and then you've got these microkernel operating systems. 25:04.230 --> 25:07.070 Well, I'm gonna go back to this microkernel operating system. 25:07.490 --> 25:10.990 Turns out this looks really good for security applications. 25:11.630 --> 25:19.050 If you look at the microkernel, it has about 20,000 lines of code that run the kernel. 25:19.670 --> 25:23.530 Compare that to the Linux kernel, which has on the order of 4 million. 25:23.950 --> 25:30.370 So we can hopefully better guess or better rely on the microkernel for correctness. 25:30.470 --> 25:38.050 Matter of fact, there's some research microkernels out there that try and formally prove that the microkernel is correct. 25:40.980 --> 25:47.240 The system calls to the microkernel, there's only 14 or so, just depends on the implementation. 25:47.440 --> 25:49.460 Maybe on the order of 10. 25:49.460 --> 25:52.980 So you've got a small number of system calls that do basic stuff. 25:52.980 --> 25:59.540 You do IPCs, you do inter-process communication, you do thread management, that kind of thing. 25:59.540 --> 26:00.800 That's it, actually. 26:00.880 --> 26:04.760 All the hardware access, these kinds of things are pushed to user level. 26:09.140 --> 26:18.580 So based on this microkernel design, I've laid out this spine architecture for the intrusion recovery system. 26:20.340 --> 26:31.360 At the base of the architecture you have the microkernel, which I'm using L4 Fiasco implementation, which is developed by a university in Dresden, Germany. 26:33.060 --> 26:40.040 And on top of that runs the guest kernel, the L4 Linux implementation runs on top of that. 26:40.040 --> 26:43.920 So you can run a full-blown version of Linux on top of the microkernel. 26:44.660 --> 26:49.980 So you're probably more familiar with the term virtual machine than you are microkernel. 26:49.980 --> 26:53.480 Turns out these are very highly related topics. 26:53.900 --> 26:56.440 Essentially the microkernel does the same thing for us. 26:56.660 --> 27:01.900 And so we have this guest kernel, which is L4 Linux running on top of it. 27:02.580 --> 27:08.760 At the next level you've got your user processes, and those run just like they do on Linux. 27:08.760 --> 27:10.180 You don't even have to modify them. 27:11.480 --> 27:17.440 And then over here on the first, only Fiasco runs in kernel mode. 27:17.440 --> 27:24.020 So only Fiasco runs in that ring zero, and everything else runs in ring three. 27:24.460 --> 27:35.060 That means even if you have root access on the kernel machine, you're not guaranteed access to all bits in the computer system. 27:39.310 --> 27:46.470 So on the right here I have at each level a component of the intrusion recovery system. 27:46.470 --> 27:49.470 I call them L0, L1, L2, L3. 27:49.930 --> 27:51.550 And L0 is just the hardware. 27:51.550 --> 27:55.850 There's really no support for that right now, just except the ring stuff. 27:56.390 --> 28:04.010 Turns out Intel and IBM, these kinds of companies, are very interested in hardware support for virtual machines. 28:04.010 --> 28:06.710 So in the future this will definitely be huge. 28:08.050 --> 28:11.530 L1 is a component I have running at the microkernel. 28:11.890 --> 28:20.810 The importance of this component is it's able to verify that the guest kernel and the L2 component are operating correctly. 28:21.550 --> 28:24.190 And similarly for L2 to L3. 28:24.270 --> 28:30.190 And why do I have these various levels in the system? 28:30.190 --> 28:43.930 Well it turns out if you're looking at the entire Linux operating system from L1, you're gonna have a hard time interpreting all the data structures, interpreting what's good, what's bad. 28:43.930 --> 28:53.670 So it's much easier to run components in each level in order to understand the semantics of that level. 28:54.770 --> 29:04.410 And the vertical hierarchy is a way of guaranteeing that those components are not trojaned or stopped or bypassed, these kinds of things. 29:07.250 --> 29:09.150 So there's a lot more details here. 29:09.150 --> 29:10.470 I just wanna give you a few. 29:10.470 --> 29:12.310 I'll just show you the memory real quick. 29:12.310 --> 29:14.530 How does the memory work for instance? 29:14.890 --> 29:22.190 So down here at the bottom, you have the microkernel L1 which owns all the memory. 29:24.510 --> 29:30.050 And it can allocate some of the physical, by memory I mean the RAM. 29:30.050 --> 29:34.710 It can allocate some of the RAM to the guest kernel which is shown in red here. 29:35.030 --> 29:42.090 And you can see that portions of the memory are still owned by the microkernel and the Linux guest kernel cannot touch that memory. 29:42.750 --> 29:50.710 And then in turn, the guest kernel, the Linux kernel can map some of its memory to user space processes. 29:53.050 --> 29:56.050 All right, so this is the system. 29:56.050 --> 29:57.290 How do we attack it? 29:57.290 --> 30:00.090 So you can attack any of these things in user space. 30:00.090 --> 30:05.370 If you have root access, you have access to all these components highlighted in red here. 30:06.070 --> 30:13.950 So again, the L2 and L3 components of the intrusion recovery system are suspect to attack. 30:14.070 --> 30:17.630 And that's why we need L1 to verify that they're operating correctly. 30:18.950 --> 30:26.350 So right now, I'm still working on algorithms to determine when L2 and L3 are operating correctly. 30:26.350 --> 30:40.230 So far, I only have monitoring the PC instruction counter and verifying the hash of the code at L2 to verify and check that the program counter runs in that code periodically. 30:41.130 --> 30:45.970 If anybody's got some ideas on other ways to verify, I'd definitely like to talk to you. 30:47.630 --> 30:52.650 Okay, so now I wanna look at another kernel-level rootkit target. 30:53.490 --> 31:01.110 So it turns out these same kernel-level rootkits will work on this new spine architecture. 31:01.310 --> 31:04.790 They will attack the guest kernel instead of the host kernel. 31:05.410 --> 31:11.890 So I wanna go through another target to give you guys some more background on how kernel-level rootkits work. 31:12.510 --> 31:15.870 This target is the virtual file system. 31:16.350 --> 31:26.090 Now, the virtual file system in Linux is a wrapper that gives you access to all file systems in Linux. 31:26.090 --> 31:28.790 And so you can insert your own virtual file systems. 31:28.790 --> 31:39.270 If you have an ext3 formatted partition, then you insert that in the VFS and you don't have to readjust your system calls to point to that. 31:39.270 --> 31:41.570 Everything goes through the virtual file system. 31:42.130 --> 31:43.670 Let me show you how it works. 31:45.070 --> 31:49.210 So first off, a system call is issued. 31:49.210 --> 31:53.450 Let's say I'll go over the proc file system in this example. 31:53.770 --> 31:55.610 Let me back up a minute. 31:56.110 --> 31:59.070 So you have lots of different file systems in Linux. 31:59.070 --> 32:02.550 You have your file system for your hard disk. 32:02.550 --> 32:07.990 You have remote disk storage stored on another network drive. 32:07.990 --> 32:20.590 You have the proc file system, which is a file system built by the kernel and gives you information from the kernel, such as task IDs, and so on and so forth. 32:21.510 --> 32:28.230 So if you want to, remember, if you wanna get access to the file system, any file system, you have to issue a system call. 32:29.210 --> 32:37.110 So in this example, I want to read the contents of the proc directory to see what processes are running on my system. 32:37.790 --> 32:41.670 So first, my program will issue a sys read dir. 32:42.750 --> 32:51.990 That sys read dir indexes into the virtual file system, which then, and you've told it to open up this proc, some directory in proc. 32:52.570 --> 32:59.430 So the virtual file system knows, based on the path you gave it, where it needs to go to service your request. 32:59.590 --> 33:01.810 For proc, that's a special file system. 33:01.810 --> 33:04.730 It indexes into its root structure. 33:05.570 --> 33:09.650 And there's a lot of pointers in that root structure. 33:09.810 --> 33:14.170 The F underscore op stands for file operations, I believe. 33:14.710 --> 33:22.390 And so for the proc, you look up what are the file operations for that file system. 33:22.850 --> 33:25.790 And so there's a F op structure, which is standard. 33:25.790 --> 33:30.630 And for each file system, you have to implement a number of function pointers. 33:30.630 --> 33:34.430 In this case, read dir is the one I'm going to highlight. 33:34.930 --> 33:37.950 So you've indexed all the way in here to read dir. 33:38.050 --> 33:42.310 And finally, you get to the proc read dir code. 33:43.150 --> 33:49.830 And so that'll then return information back to your system call, and then return that information back to your user program. 33:53.530 --> 33:56.370 Okay, so what's the attack points here? 33:56.370 --> 33:59.870 Again, it's the same basic idea. 33:59.950 --> 34:06.230 Anytime you've got a function pointer, anytime you've got a reference, any code along the path can be redirected. 34:11.760 --> 34:15.840 All right, so what's the recovery algorithm for this? 34:15.840 --> 34:19.620 Well, now's when we sort of step back and make it more general. 34:19.860 --> 34:28.700 So generally, the recovery methods for kernel-level rootkits is consistency checking on the data structures in the kernel and the code running. 34:28.700 --> 34:34.720 And you can store a copy of this, what it should look like in your protected spine architecture. 34:35.500 --> 34:44.280 And you can use that to check and see if it's been modified in a manner that's against your policy. 34:44.600 --> 34:49.320 And if so, then you repair it back to its known good state. 34:50.840 --> 34:53.800 And the IRS can do this automatically. 34:53.800 --> 35:05.800 And that's sort of the vision, is you have this system that's constantly monitoring your computer for malicious rootkit installations. 35:05.800 --> 35:09.960 If that happens, it undoes the damage from it. 35:09.960 --> 35:16.200 And furthermore, have longer-term goals of, okay , so yeah, you've got a rootkit installed. 35:16.200 --> 35:17.620 You undo this damage. 35:17.620 --> 35:19.740 The attacker still got in your system somehow. 35:20.300 --> 35:24.320 You also have to figure out how he got out, kick him out, change passwords. 35:24.320 --> 35:26.620 It's more than just repairing the system. 35:26.620 --> 35:27.860 It's a lot more complicated. 35:29.420 --> 35:31.560 All right, so let's go for another demo. 35:52.660 --> 36:02.920 Now, for the purposes of this demo, I'm not using the automated repair of the system because you wouldn't be able to see that anything happened. 36:05.490 --> 36:13.450 But one thing I wanna show you real quick is I am, in fact, running the spine architecture now. 36:16.380 --> 36:35.480 And you can see that I'm running the Linux 2.6.11-L4 kernel, which is, the way it works for the Linux kernel is you port the Linux kernel to the L4 architecture as opposed to the I386, the alpha, so on and so forth. 36:37.100 --> 36:44.980 So, in fact, we can go in here and check out our getcpl again to see if our privilege levels have changed. 36:45.960 --> 36:50.520 And so, the user processes still run at ring three, which is expected. 36:51.680 --> 36:55.220 And let's see what the kernel is now running at. 37:00.100 --> 37:02.860 And so, can everybody see this in the back? 37:02.860 --> 37:04.420 Or should I call it out for you guys? 37:05.640 --> 37:06.740 Move it up? 37:09.600 --> 37:10.920 Sorry about that. 37:13.320 --> 37:14.520 Is that better? 37:15.980 --> 37:16.860 Thanks. 37:17.400 --> 37:20.620 Okay, so what I've just done, let me scroll back up for you guys then. 37:22.000 --> 37:30.580 Um, so right here I just did the getcpl and it returned ring three, running in ring three. 37:31.620 --> 37:36.400 And I've just inserted the getcpl into the running kernel. 37:36.660 --> 37:44.500 And you can tell the var log, sys log, and see that it was printed out as running in ring three now. 37:44.500 --> 37:47.100 So now the kernel is running in ring three. 37:47.100 --> 37:53.600 If anybody out there is familiar with user mode Linux, it's a very similar concept to that, but it's much, much faster. 37:55.360 --> 37:58.440 In terms of performance, I'll take a slight venture. 37:58.560 --> 38:05.760 Just in case you guys are wondering, you figure about a 10% performance penalty for running in this architecture. 38:06.360 --> 38:11.900 So it runs about 90% the speed of Linux running directly on your hardware. 38:18.780 --> 38:28.000 Okay, so for the second demo, I'm going to install the Adore Next Generation rootkit. 38:28.000 --> 38:31.760 And this rootkit works on the Linux 2.6 kernel. 38:34.620 --> 38:40.860 And what this rootkit does is exactly what I was just showing you with the virtual file system. 38:40.860 --> 38:51.020 It redirects the virtual file system to malicious structures so it can hide files and processes. 38:57.710 --> 39:02.950 Okay, so I've just loaded the rootkit. 39:04.170 --> 39:10.990 Notice first off that my Adore NG directory that I was just in is now hidden. 39:10.990 --> 39:12.450 That's part of the configuration. 39:12.450 --> 39:15.550 It hides whatever your rootkit directory is. 39:16.090 --> 39:23.990 And then I'm going to launch this rootme binary, which doesn't really do anything, but it runs. 39:24.310 --> 39:28.330 And you can see right here in the process list that rootme is running. 39:29.430 --> 39:32.750 Now I've got to scroll back up here and see what my directory was. 39:34.150 --> 39:37.910 So I'm going to go back into the Adore NG directory. 39:37.910 --> 39:41.530 I'm tabbing over right now, and that's another system call. 39:41.530 --> 39:47.310 It doesn't find that there's an Adore NG directory, so I have to type the full path. 39:48.470 --> 39:56.010 And there's a user-level utility to go with this Adore NG rootkit, which can hide processes for you. 39:56.590 --> 40:00.210 So I want to hide this process 644. 40:09.250 --> 40:10.970 So now rootme's gone. 40:12.490 --> 40:22.690 All right, so now I'm going to put on my system administrator hat again, and I suspect that a rootkit's been installed. 40:23.330 --> 40:30.270 This could be done automatically with the intrusion recovery system, but again, for the purposes of this demo, I will do it manually. 40:32.830 --> 40:40.250 And so what I'm going to do is a similar algorithm to the system call table repair. 40:40.590 --> 40:44.170 I am going to, but this time I made it a little bit simpler. 40:44.170 --> 40:47.070 I just fixed the pointers back to where they're supposed to be. 40:52.560 --> 41:00.660 Okay, so I've just repaired the proc file system, and now you can see the rootme file, our process is back. 41:02.680 --> 41:06.360 And you can see the adore-ng directory is back. 41:06.820 --> 41:10.380 So in fact, we've been able to recover from these kernel-level rootkits. 41:15.210 --> 41:18.610 Okay, let's go back to wrap up things here. 41:31.600 --> 41:44.300 All right, so I've shown you the ability to recover from some kernel-level rootkits, and presented you with an architecture that's perhaps more robust, and you can somewhat rely on for this recovery. 41:45.160 --> 41:53.100 So, I mean, definitely there's some limitations to this system, I'm still developing it now, so any feedback you guys have would be great. 41:53.680 --> 41:58.120 Some of the limitations, you know, first off, how can you trust this microkernel? 41:58.120 --> 42:02.340 What if the attacker installs a rootkit into the microkernel? 42:02.340 --> 42:12.360 Well, you know, that's a hard question, but it's relying on the smallness of the microkernel to be correct to prevent that capability. 42:13.020 --> 42:23.920 You know, one of the things that some of you guys are probably thinking about already is, what about direct memory access, where you program a controller to overwrite the microkernel? 42:23.920 --> 42:28.880 Well, that's definitely a problem right now, that's a limitation. 42:29.020 --> 42:38.760 In future, Intel architectures that support better isolation, that hopefully will not be a problem. 42:39.260 --> 42:46.120 So, and finally, you know, just like everything else with security, there's no be-all, end-all solution. 42:46.320 --> 42:54.720 However, I believe this IRS can make your systems more reliable and save us a lot more time and trouble. 42:55.960 --> 43:01.800 So thanks to a lot of people that have helped me out, you know, there's lots more people on here than I have listed. 43:02.380 --> 43:05.000 These people in particular have been helpful. 43:06.140 --> 43:10.280 I got some links for you guys, if you wanna find out some more information about what I've been talking about. 43:10.600 --> 43:15.900 Various architectures, system projects here, you know, check rootkit, our research. 43:17.200 --> 43:23.920 And at this time, we got a few starter questions, if anybody wants to comment on, and if not, if you got your own questions, that's fine. 43:23.920 --> 43:26.280 We got about five minutes or so. 43:26.480 --> 43:30.500 And also after the talk, I will be heading out that door over there. 43:30.500 --> 43:34.720 So if you guys wanna catch me, definitely link up with me outside. 43:35.560 --> 43:43.880 So first off, let me just ask these three questions, and I'll take either answer these comments to these questions or your own questions. 43:44.240 --> 43:48.280 So I've already asked this question, how many people have personally dealt with the recovery of rootkit? 43:48.280 --> 43:51.880 If anybody has a testimony, we might can share that in a minute. 43:51.960 --> 43:54.660 Let me read all three of these and then I'll start calling on people. 43:55.260 --> 44:02.900 Secondly is, I would definitely be interested to see if anybody's seen any rootkits that use DMA, direct memory access. 44:02.900 --> 44:04.320 I've seen a lot of talk about it. 44:04.320 --> 44:10.480 Maybe I just have not seen the public talk about it or public implementation. 44:12.020 --> 44:22.260 And finally, if anybody has not followed the conventional wisdom of wiping their system clean and reinstalling, I'd be interested to hear that. 44:22.260 --> 44:25.260 So this time I'll open it up for questions and comments. 44:28.320 --> 44:29.540 Over there. 44:36.430 --> 44:41.170 Okay, so the question was with the L4 architecture, can you run multiple instances of your kernel? 44:41.170 --> 44:42.770 And the answer is yes. 44:42.770 --> 44:47.610 The L4 architecture is in fact pretty much virtual machine architecture. 44:47.610 --> 44:50.210 So you can run multiple instances of Linux. 44:54.510 --> 44:56.130 System pipes and log files? 44:56.130 --> 44:57.910 I have not addressed that yet. 44:58.650 --> 45:00.070 Good question. 45:07.100 --> 45:09.080 Certainly, but I mean, 45:12.320 --> 45:18.180 so it's suspicious, but that's, it's kind of like running a virtual machine, you know. 45:18.180 --> 45:20.240 You can tell when you're running on VMware. 45:20.240 --> 45:22.540 But, you know, legitimate systems run on there. 45:22.540 --> 45:26.420 So that definitely would be able to tell something that's not a normal system. 45:26.420 --> 45:27.240 Absolutely. 45:43.210 --> 45:43.850 Right. 45:43.850 --> 45:51.490 Actually, a lot of the tools that I've written for the system call stuff, it gives you all kinds of information on the system call table. 45:51.490 --> 45:57.250 So for instance, you can check and, you can check out all the, each individual system call with these tools. 45:57.250 --> 45:59.330 And that is much more surgical. 45:59.730 --> 46:00.770 Definitely. 46:01.350 --> 46:02.850 Good, good comment. 46:03.570 --> 46:04.730 Question here. 46:27.920 --> 46:28.940 Hmm. 46:28.940 --> 46:35.040 So the question was, one, this system would be useful just for detection, correct? 46:35.220 --> 46:36.600 And absolutely. 46:36.880 --> 46:39.920 Why am I doing intrusion recovery system as well? 46:39.920 --> 46:43.820 Because I'm a research student and we gotta do new and exciting things. 46:45.420 --> 46:46.540 So absolutely. 46:46.540 --> 46:50.020 And lots of people have talked about using similar architectures for intrusion detection. 46:50.020 --> 46:51.120 That's an excellent idea. 46:51.120 --> 46:53.620 That'll be the way it works 10 years from now. 46:53.920 --> 46:55.420 Got guys in the back? 46:55.520 --> 46:56.540 People over here? 47:01.920 --> 47:03.420 Oh yeah, absolutely. 47:03.660 --> 47:08.800 Matter of fact, I was gonna do a demo on that for this presentation, but didn't have enough time. 47:09.160 --> 47:12.200 Recovery from user level rootkits, I've done some work on that. 47:12.300 --> 47:23.040 It's pretty much tying a tripwire or aid kind of thing to a recovery program, which just checks your file system periodically and replaces the binaries back to normal. 47:23.040 --> 47:24.680 That's absolutely a part of the system. 47:24.800 --> 47:26.600 Cause you need the full system. 47:26.860 --> 47:29.560 We got a question way back there in the back and then I'll come back up here. 47:35.700 --> 47:37.240 That's an excellent question. 47:37.240 --> 47:40.860 So Xen is another very similar architecture. 47:40.860 --> 47:42.920 In fact, I got a link to that in the slides. 47:43.040 --> 47:44.840 Xen is very hot right now. 47:44.840 --> 47:49.340 It's a virtual machine architecture, open source. 47:49.380 --> 47:51.800 I believe it's England people are doing that. 47:51.820 --> 47:53.460 And it's very fast. 47:53.460 --> 47:56.320 Matter of fact, so Xen and L4 compete. 47:56.580 --> 48:01.760 I am going to compare Xen to L4 in my thesis to see which one would be better. 48:02.560 --> 48:04.140 That's just an implementation difference. 48:04.140 --> 48:05.760 Either one is certainly suitable. 48:05.760 --> 48:07.420 I just chose L4. 48:07.760 --> 48:11.500 We had a question up here and then I'll quit after this guy. 48:19.820 --> 48:23.280 Okay, so the question is, what's more background on this microkernel? 48:23.500 --> 48:35.240 And the reason I chose L4 fiasco implementation is basically it's one of the more mature L4 implementations developed by a university at Dresden. 48:35.520 --> 48:39.080 And the team there has been really good at helping me out. 48:39.080 --> 48:45.240 You know, any kind of questions I've got, big team there, lots of questions, you know, they've got the mailing list and everything. 48:46.680 --> 48:51.560 Comparing it to other, there's also two or three other implementations I could have used. 48:51.680 --> 48:55.860 I happen to know the guys a little bit at Dresden, so that's really the only reason. 48:55.940 --> 48:56.720 It's a good question. 48:56.860 --> 48:59.880 And okay, so I'm going to wrap up and let the next speaker get in. 48:59.880 --> 49:02.560 I'll be out here in the hall if you guys got any more questions. 49:02.980 --> 49:03.240 Thank you.