WEBVTT 00:01.550 --> 00:02.510 Welcome. 00:02.670 --> 00:10.090 Today I'm going to tell you a story about things that I learned about incident response by being a firefighter. 00:10.150 --> 00:11.710 Let's get to it. 00:12.230 --> 00:14.890 Here's the agenda, what we're going to cover. 00:15.070 --> 00:25.250 I'm going to start with a little introduction about myself, tell you a little bit about my background in IR, and I'm going to tell you a little bit about myself as a firefighter. 00:25.350 --> 00:27.810 Then we'll delve into what IR is. 00:27.810 --> 00:40.810 For those of you who don't know what IR is, and then we'll talk a little bit about firefighting, because even though you might know what IR is, you might not be familiar with firefighting beyond, obviously, putting out the fire. 00:40.830 --> 00:44.730 Then we'll do some comparisons between IR and firefighting. 00:45.110 --> 00:49.370 Definitely some similarities and some differences, of course. 00:49.590 --> 00:57.490 Then I'll describe some challenges and opportunities, the things that I learned from being a firefighter that I then brought back to IR. 00:58.210 --> 01:02.170 I'll summarize and then end with some final thoughts. 01:02.890 --> 01:04.710 Let's get going. 01:05.590 --> 01:06.950 All righty. 01:07.230 --> 01:11.720 So, for those of you who are not familiar with me, my name is Catherine Allman. 01:12.070 --> 01:21.090 I go by Investigator Chih on the internet, in the Twitter-verse, and I've been with the University of Buffalo for over 20 years. 01:21.090 --> 01:25.670 I'm on staff with B-Sides Rochester and a cyber camp that we hold. 01:25.670 --> 01:27.190 I volunteer with some staff. 01:27.190 --> 01:28.810 I've spoken at a bunch of conferences. 01:28.810 --> 01:31.610 I have some certifications and a couple degrees. 01:32.390 --> 01:39.470 Even those of you who know me from the IT realm may not be familiar with this information about me as a firefighter. 01:39.830 --> 01:47.810 I've been in the fire service since about 1997, and since then I've become a state fire instructor with New York State. 01:47.810 --> 01:51.330 I've done some credits, college credits, on firefighting. 01:51.330 --> 01:57.790 I was a certified fire investigator at both levels in New York State, and I was also a member of the J-Fire team. 01:58.190 --> 02:01.610 So, the fire thing is something I've been doing a long time. 02:02.870 --> 02:10.810 Just in case you're still not convinced, this is me at the scene of a fire, and this is part of the story that I'm going to tell you. 02:11.230 --> 02:13.570 I'm holding what's called a thermal camera. 02:14.050 --> 02:20.990 When you look into a thermal camera and you point it at something, what it's going to show is a difference in temperature. 02:21.230 --> 02:30.490 So, things that are darker are usually cooler in temperature, and things that are warmer are going to be are going to show up as lighter in color. 02:31.050 --> 02:33.490 We'll see why that matters later. 02:33.570 --> 02:39.310 And just in case you weren't sure, I'm the one with the camera, and the person to my right is our captain. 02:40.390 --> 02:44.110 So, now let's delve into what incident response is. 02:44.610 --> 02:54.090 I like this definition of incident response because it focuses on the technical components that you're going to need to analyze and contain an incident. 02:54.170 --> 02:58.730 We'll see in a bit why that's a little different from incident handling. 02:58.730 --> 03:00.950 So, you're going to be the boots on the floor person. 03:00.950 --> 03:05.250 The person who's actually focused on dealing with this incident directly. 03:05.470 --> 03:07.570 You're going to address network events. 03:07.730 --> 03:12.790 You might deal with them in a reactive or a proactive way, depending on your organization. 03:12.790 --> 03:18.330 And an event can be anything that's going to affect the CIA triangle. 03:18.330 --> 03:22.430 Your confidentiality, integrity, and availability of information. 03:23.650 --> 03:33.410 The goals of incident response are going to be to make sure you know when there is, in fact, an incident or, you know, some kind of thing going on. 03:33.410 --> 03:35.650 You're going to stop that attacker. 03:35.690 --> 03:39.530 You're going to minimize the damage that is caused. 03:39.570 --> 03:45.290 And ultimately, you want to prevent it from ever happening again or something similar from happening. 03:45.970 --> 03:49.370 These are your basic phases of incident response. 03:49.370 --> 03:55.070 Preparation, identification, containment, eradication, recovery, and lessons learned. 03:55.310 --> 03:57.830 Let's break these down a little bit. 03:58.810 --> 04:00.190 Preparation. 04:00.190 --> 04:04.730 So, during the preparation phase, we're going to document, document, document. 04:04.730 --> 04:08.430 You want to document both your network itself. 04:08.450 --> 04:12.270 You might want to document the particular machines in question. 04:12.450 --> 04:16.950 One of the most important things you'll want to document is where your important data is, right? 04:16.950 --> 04:18.790 Where are the keys to the kingdom? 04:18.830 --> 04:20.090 That's critical. 04:20.250 --> 04:22.070 You might do some tool building. 04:22.070 --> 04:25.130 Perhaps to help you respond better to an incident. 04:25.130 --> 04:29.910 Or perhaps to help you know when something is happening that shouldn't be. 04:30.370 --> 04:34.570 Then the next step is going to be awareness of the attacks. 04:34.570 --> 04:41.410 And ultimately, this could be something as simple as an automated ticket system that tells you that there's a problem . 04:41.410 --> 04:43.070 Could be from a phone call. 04:43.670 --> 04:48.290 Your containment phase, you want to stop whatever is happening. 04:48.410 --> 04:56.970 So, that might be patching a system, blocking some sort of C2 connection, maybe pulling the power out of the back of a machine. 04:57.010 --> 04:58.050 Depends. 04:58.530 --> 05:02.570 Then during the eradication phase, you're ultimately going to remediate. 05:02.570 --> 05:05.430 You're going to remove compromised hosts from your network. 05:05.450 --> 05:09.750 Whatever it is you have to do to get them to get the bad guys gone. 05:10.490 --> 05:14.850 During the recovery phase, you need to restore those business functions. 05:14.850 --> 05:21.070 Because, of course, that is the whole point of incident response initially, is to get up and running again. 05:21.070 --> 05:26.130 Then you want to learn from exactly what happened so that the same thing doesn't happen again. 05:27.650 --> 05:29.570 Here we have incident handling. 05:29.570 --> 05:31.190 You'll notice it's a little different. 05:31.190 --> 05:35.010 It focuses on the other things that are important in an incident. 05:35.090 --> 05:41.230 The logistics, communication, the coordination, planning, all of these other functions. 05:41.230 --> 05:43.950 And ideally, you want these two things to be separate. 05:44.570 --> 05:50.070 Now, I understand in smaller organizations, you might be the only person doing this job. 05:50.070 --> 05:51.550 You might have to do both. 05:51.810 --> 06:06.210 But if you're the person whose boots on the ground and you're working on containing an incident or eradicating the attacker, you don't also want to have to be the person who's communicating out every five minutes what's going on. 06:06.210 --> 06:11.590 So, if you have somebody else doing that, it's a whole lot easier to focus on what's at hand. 06:12.690 --> 06:14.570 Now let's move to firefighting. 06:14.570 --> 06:16.090 What is firefighting? 06:16.450 --> 06:18.050 Well, it's the obvious. 06:18.050 --> 06:21.410 We're going to put the wet stuff on the red stuff, as they say. 06:21.410 --> 06:26.090 We're going to prevent the spread and extinguish significant unwanted fires. 06:26.130 --> 06:29.830 So, you know, not your backyard barbecue unless it's out of control. 06:30.210 --> 06:34.590 And we'll do that for buildings and vehicles, woodlands, that sort of thing. 06:35.650 --> 06:42.690 Our goals, as you might expect, are to protect the health and safety of ourselves and of the public. 06:42.730 --> 06:46.170 We're also going to protect property and the environment. 06:46.170 --> 06:53.030 Just because we're in the process of putting a fire out doesn't necessarily mean we want to make things worse, right? 06:53.110 --> 07:03.150 So, for example, if we have a situation that might involve hazardous materials, we need to make sure that we're doing our best to prevent that from making the situation worse. 07:03.150 --> 07:08.210 And we want to minimize, excuse me, the disruption of community activities. 07:08.930 --> 07:12.870 Great example of that would be a car fire or a car accident. 07:13.270 --> 07:17.550 You have a lot of people who are, say, trying to get to work in the morning or trying to get home. 07:17.550 --> 07:27.850 And if one of those things occurs, we need to set up traffic support to help you still get from place to place and minimize any disruption while we're taking care of the particular incident. 07:29.110 --> 07:31.450 We, too, have a preparedness cycle. 07:31.450 --> 07:33.030 It looks kind of like this. 07:33.110 --> 07:34.130 We'll do some preparing. 07:34.130 --> 07:35.830 Then an emergency will happen. 07:35.830 --> 07:37.190 We'll respond to it. 07:37.190 --> 07:38.850 We're going to recover from it. 07:38.850 --> 07:41.290 We'll mitigate where we can. 07:41.290 --> 07:43.030 And we're back to preparation. 07:43.390 --> 07:49.210 Now, in this preparedness cycle in the fire service, we usually start more on the mitigation side. 07:49.210 --> 07:51.950 And let me show you how and why. 07:52.370 --> 07:59.390 So, mitigation for the fire service has to do with preventing future emergencies or minimizing their effects. 07:59.390 --> 08:09.010 So, if you've ever been to any kind of, say, fire department open house where they're doing fire safety demonstrations, they're handing out smoke detectors, right? 08:09.050 --> 08:10.450 That's a mitigation. 08:10.570 --> 08:16.370 We want to see if we can help people understand how to prevent fires and accidents in the first place. 08:16.470 --> 08:19.550 Sprinklers in a building are another great mitigation method. 08:20.950 --> 08:21.550 Preparedness. 08:21.550 --> 08:23.890 We need to be prepared to handle that emergency. 08:23.890 --> 08:31.510 So, we're going to make sure our gear is maintained, that we're drilling so we know how to use the equipment on the truck, that we have pre-plans of the building. 08:31.510 --> 08:38.370 Perhaps there's something unique about accessing certain parts of the building or getting people out if necessary. 08:38.490 --> 08:40.090 And then recovering. 08:40.230 --> 08:42.990 When we recover from an emergency, we're going to clean up. 08:42.990 --> 08:46.970 We're going to restore things to their functional purpose again. 08:46.970 --> 08:49.650 We're going to repair equipment that might have been damaged. 08:49.770 --> 08:52.210 This is pretty standard for us. 08:53.410 --> 08:55.670 So, here's the next part of the story. 08:55.670 --> 08:57.330 You saw me looking at the camera, right? 08:57.330 --> 08:59.550 This is the same fire. 08:59.550 --> 09:01.550 Two more pictures from this fire. 09:01.910 --> 09:10.450 The one on the left has me, if you probably can't tell because my name's not on my coat, but that is me. 09:10.490 --> 09:15.230 And what I'm attempting to do in that picture is change out an air bottle. 09:15.230 --> 09:23.470 The way firefighters breathe inside a building that is on fire is we wear something called self-contained breathing apparatus. 09:23.930 --> 09:33.390 And that's made out of a harness and a bottle and some other pieces that help redirect airflow. 09:33.690 --> 09:36.530 Needless to say, there are several components. 09:36.530 --> 09:40.970 My job on the left-hand side is to change the air bottle. 09:40.970 --> 09:48.550 Your average air bottle is about 30 minutes, 45, which can go very quickly if you're working hard in a scene. 09:48.890 --> 09:55.990 And when a firefighter runs out of air or starts to run out of air, they come out and they ask somebody to change that bottle. 09:56.310 --> 09:58.090 So, that was my job at this fire. 09:58.090 --> 10:01.230 I was changing bottles for the people who needed them. 10:01.410 --> 10:10.590 If you look in the picture on the right-hand side, you'll get at least a small idea of what that bottle and harness setup looks like, because you can kind of see the top of the bottle. 10:11.370 --> 10:20.650 What I want you to focus on in that right-hand picture is the fact that the firefighter's shoulder has a lot of damage. 10:20.650 --> 10:27.690 If you look carefully, you will see there's actually been flame and heat impingement to that shoulder area. 10:28.350 --> 10:43.790 What I didn't realize when I was changing that bottle is that it was not just his coat that had been damaged, but in fact the harness that holds the bottle and the air hoses that connect to it. 10:44.470 --> 10:53.370 So, I go to change that bottle, and unfortunately when I put the new bottle back in and connect it all up, I get hit with a blast of air in the face. 10:54.030 --> 11:05.730 Fortunately for me, I was wearing my gear correctly, there was no damage, and ultimately I instructed the firefighter to take the pack and make sure that it went out of service. 11:06.130 --> 11:08.330 Now, why is this important? 11:08.330 --> 11:16.030 Nobody got hurt, it wasn't a really huge deal, nothing bad ultimately happened. 11:16.210 --> 11:17.610 We'll get back to that. 11:17.610 --> 11:24.890 So, pay close attention to that picture, think about that story, and remember here, I was just doing my job. 11:26.170 --> 11:26.810 All right. 11:26.810 --> 11:33.750 So, one other piece of terminology in the fire system that's really important to understand is the incident command system. 11:33.750 --> 11:37.690 And it's this idea of an organized response to a problem. 11:37.710 --> 11:48.210 You're going to have a coordinated response with potentially multiple agencies, you need some way to organize that chaos, right? 11:48.210 --> 11:52.710 Such that everybody knows what they're doing and what their job's supposed to be. 11:52.710 --> 11:58.050 If you've been in the military before, you're probably familiar with chain of command. 11:58.170 --> 12:02.230 Obviously, if you're in law enforcement, that's something else you're familiar with. 12:02.430 --> 12:04.470 Not dissimilar idea. 12:04.470 --> 12:07.310 This is what forms our effective chain of command. 12:08.790 --> 12:11.370 So, here's what the incident command system looks like. 12:11.370 --> 12:14.050 And we're not going to spend time looking at all of this. 12:14.050 --> 12:19.230 But you'll see the incident commanders at the top, typically going to be one of your chief officers. 12:19.430 --> 12:24.130 And at the bottom, you're going to have operations and planning, logistics, finance. 12:24.510 --> 12:29.010 And in between, you'll have some other folks who report directly to the commander. 12:29.290 --> 12:35.850 What's important to note here is that the incident commander doesn't have a ton of people reporting directly to them. 12:35.850 --> 12:40.770 And the reason is, it makes this whole situation way more manageable. 12:41.150 --> 12:46.130 So, let's look at the pieces that are most often in smaller incidents. 12:46.810 --> 12:49.490 So, you have your incident commander. 12:49.590 --> 12:53.990 They're going to define what the goals and the objectives are for that particular incident. 12:53.990 --> 12:58.210 Again, usually your chief, or it might be a senior officer if no chief is available. 12:58.690 --> 13:01.010 And then you'll have operations. 13:01.170 --> 13:13.150 Operations is often the person who is in charge of making sure that those goals and objectives get met by figuring out what the strategy and the tactics are going to be. 13:13.310 --> 13:16.590 We sometimes have external operations and internal operations. 13:16.590 --> 13:20.170 If it's a large building, we might have operations on multiple floors. 13:20.190 --> 13:25.490 So, we might have operations for the first floor. 13:25.850 --> 13:30.750 We have a high-rise facility and sometimes we need multiple levels of operations. 13:31.050 --> 13:33.230 So, that's not uncommon. 13:33.310 --> 13:40.470 The last three here, logistics planning and admin finance, are more common in very, very large incidents. 13:40.470 --> 13:44.710 When you need to have lots of people and extra supplies and equipment brought in. 13:44.710 --> 13:54.730 If there's going to be a long-range plan, something like a hurricane hits and we have to mobilize troops or demobilize them quickly, that's going to be on the part of planning. 13:54.730 --> 14:03.490 And then if we need some kind of licenses or deal with compliance issues, get some money together, whatever, that's where admin and finance is going to come in play. 14:04.870 --> 14:05.670 All right. 14:05.670 --> 14:13.970 So, let's do a comparison of the two and kind of see, I think you'll find it interesting to see how they do compare. 14:14.690 --> 14:19.950 So, methodology-wise, I think you've seen they're very, very similar, right? 14:20.210 --> 14:26.850 The only thing that was really missing from the firefighting circle were the lessons learned. 14:26.850 --> 14:35.450 And I'll tell you, even though it's not officially part of the methodology, if you are good at what you do as a firefighter, you absolutely pay attention to lesson learned. 14:35.450 --> 14:41.310 So, I would say that these really, in terms of their functionality, are really very similar. 14:42.370 --> 14:44.970 But let's talk about some misconceptions. 14:45.830 --> 14:49.470 In incident response, there's this idea that every event is an incident. 14:49.530 --> 14:51.430 Well, that's certainly not the case. 14:51.430 --> 15:00.970 You can have an event where something is, you know, changed, but it's not necessarily an incident on the part of the particular organization you're with. 15:01.170 --> 15:05.110 And not every incident is handled the same way, right? 15:05.110 --> 15:15.710 I mean, certainly a small incident involving maybe one social security number is going to be different than an incident involving a hundred social security numbers or a thousand. 15:16.770 --> 15:18.730 Every incident is quickly solved. 15:18.730 --> 15:20.350 Well, we know that's not true. 15:20.350 --> 15:27.890 We can have an incident that's really fast, maybe gets resolved in 10 minutes, and it could take one that takes days, weeks, months. 15:27.930 --> 15:31.110 It can be a really long time, depending on how big the incident is. 15:31.110 --> 15:39.730 If law enforcement is involved, it can be even longer than that, depending on what court cases and all the other details play into it. 15:40.310 --> 15:43.130 This is, I think, one of the most important ones. 15:43.370 --> 15:47.910 There's this idea that every person on an incident response team needs to be a rock star. 15:48.050 --> 15:49.210 Well, that's not true. 15:49.210 --> 15:50.370 I'm not a rock star. 15:50.370 --> 15:51.910 I'll tell you that right now. 15:52.270 --> 15:57.190 And as new people come into this field, we shouldn't expect them to be rock stars either. 15:57.190 --> 15:59.210 We can all learn from each other. 16:00.530 --> 16:09.770 Another thing that's a misconception in IR is this idea that we can accurately attribute whatever's happened to a particular entity. 16:10.150 --> 16:16.030 It's hard enough to, you know, figure out attribution at all, but to do it accurately is even harder. 16:16.770 --> 16:19.530 Likewise, in firefighting, we have some misconceptions. 16:19.970 --> 16:22.810 How about this idea that firefighters are always paid? 16:23.050 --> 16:29.670 How many of you know that there are tons of firefighters around this country that aren't paid? 16:29.930 --> 16:31.390 I'm one of them. 16:31.390 --> 16:34.270 I've been a volunteer firefighter for many years. 16:34.270 --> 16:38.230 I've never, ever earned a dime from firefighting. 16:38.230 --> 16:40.150 And yet, I still love it. 16:40.150 --> 16:43.530 It's a fantastic hobby and one that I wouldn't trade for anything. 16:44.530 --> 16:49.970 There's also, from television especially, this idea that firefighters are big tough dudes. 16:50.250 --> 16:53.190 Well, I know you can only see my floating head, but guess what? 16:53.190 --> 16:54.630 I'm not a big tough dude. 16:54.630 --> 16:56.590 Never was, never will be. 16:57.150 --> 17:00.950 We also don't fight fires the same way every time. 17:00.950 --> 17:09.890 Again, yes, we put the wet stuff on the red stuff, as one of my fire instructors has said, but we don't do it the same way. 17:10.150 --> 17:13.110 Maybe we need to do a search and rescue first. 17:13.110 --> 17:15.830 Maybe the building has partially underground. 17:15.830 --> 17:22.310 There are all sorts of challenges with the situation that don't allow us to do the same exact thing every time. 17:22.830 --> 17:31.770 Certainly, there's this idea that, you know, fires get extinguished quickly, especially, again, on television shows, but it can take days. 17:31.770 --> 17:36.630 And in large, really large situations, it can take longer than that. 17:36.770 --> 17:38.230 Just depends. 17:38.270 --> 17:42.570 Maybe, maybe all you need is a water can and it's a little bit of burning mulch. 17:42.570 --> 17:46.050 That's extinguished quickly, but not all of them. 17:46.330 --> 17:51.310 And definitely not a full-time job, because that's not my full-time job. 17:51.310 --> 17:54.070 My full-time job is incident response. 17:54.130 --> 17:58.430 But we'll see, they're pretty similar things in some ways. 17:58.570 --> 18:01.030 First, we'll talk about the differences. 18:02.070 --> 18:08.790 So, in terms of incident response, it rarely involves life safety, except in the universe of healthcare. 18:08.850 --> 18:17.030 If you work in healthcare, you absolutely can be the difference between life and death in some of the incident response situations you might wind up in. 18:17.030 --> 18:18.630 Patient records are critical. 18:19.210 --> 18:24.210 Ideally, there are things in place to prevent that from being, you know, catastrophic. 18:24.430 --> 18:32.810 But in the rest of the world, despite what everyone would have you believe, life safety isn't usually the issue. 18:33.210 --> 18:35.230 That's important to remember. 18:36.090 --> 18:41.310 Certainly, IR teams typically get paid, and as I already mentioned, firefighters, not so much. 18:41.790 --> 18:46.490 We use computers and software for our day-to-day job. 18:46.490 --> 18:57.430 And while firefighters also sometimes use computers in certain capacity, they use mostly water, foam, chemicals, hoses, and show up in special vehicles, like fire trucks. 18:57.430 --> 19:00.890 Most of us doing IR, drive whatever we want, doesn't really matter. 19:00.910 --> 19:05.830 Obviously, if you're in an organization like law enforcement, well, you might have different experience. 19:06.030 --> 19:08.630 And we wear special gear as firefighters. 19:08.630 --> 19:12.970 I'm not going to show up at a fire in just a t-shirt and shorts. 19:13.050 --> 19:15.730 I'm certainly going to put on my gear, right? 19:15.730 --> 19:17.670 So those are some differences. 19:19.110 --> 19:21.290 Now, here's where they are similar. 19:21.710 --> 19:27.250 In both cases, we're going to focus on that immediate issue first and find that cause later. 19:27.250 --> 19:29.190 We need to put that fire out. 19:29.190 --> 19:30.990 We need to stop that attacker. 19:31.050 --> 19:34.150 We'll deal with attribution and the cause later. 19:35.190 --> 19:39.110 We're also going to use triage to determine the best course of action. 19:39.110 --> 19:48.630 If you're not familiar with triage, it's this idea that when you have more than one thing to consider, you're going to decide which of those things is most critical. 19:49.150 --> 20:00.830 So what most people are familiar with in terms of triage is a situation where there's been a motor vehicle accident and you have multiple people who are injured. 20:00.930 --> 20:05.470 You might have somebody who just has a bump on the head. 20:05.470 --> 20:09.130 You might have somebody who maybe is beyond saving. 20:09.130 --> 20:15.010 And you might have somebody who's serious, but with the right interventions could easily be saved. 20:15.090 --> 20:20.290 And looking at all those details, you have to decide which one to treat first. 20:20.290 --> 20:23.130 Fire situations are exactly the same thing. 20:23.130 --> 20:29.470 You're going to want to take a look at the facts and make some decisions about what to do first. 20:29.770 --> 20:33.650 Ideally, you've pre-planned that, but we'll come back to that. 20:33.870 --> 20:35.430 They're cyclic in nature, both of them. 20:35.430 --> 20:37.130 We've seen that already, right? 20:37.270 --> 20:42.150 And they both require some pretty interesting thinking outside the box. 20:42.330 --> 20:51.850 In incident response, sometimes we use tools that we might use for other things to try to get information for the response. 20:51.850 --> 21:12.970 And in firefighting, we do the same kinds of things in the sense that, for example, I have a friend who's a firefighter in my hometown who had a patient on a multi-story building who was very large and they didn't have equipment that would easily facilitate getting him out of the building where he was. 21:12.970 --> 21:22.570 So he went to the local hardware store, got some equipment, and built something to get this individual out of where he was stuck. 21:22.650 --> 21:30.030 Thought outside the box, didn't just use the equipment he had, worked with other things that he was able to get a hold of. 21:30.290 --> 21:33.650 In both cases, we often bring in outside entities. 21:33.650 --> 21:40.310 Firefighters rely heavily on mutual aid, bringing in other fire departments as necessary or perhaps specialty teams. 21:40.310 --> 21:42.630 We do the same thing in incident response. 21:42.630 --> 21:50.950 While I do incident response for the university, we might need another team if something were significantly serious enough to come in and help us. 21:50.950 --> 21:55.210 Maybe if it's a large enough incident, maybe if it's a certain kind of incident. 21:55.410 --> 21:59.010 And sometimes in both cases, there are inside teams. 21:59.030 --> 22:08.870 So again, I'm inside at the university and there are companies that have inside firefighting teams because they're a manufacturing organization that's large enough. 22:10.010 --> 22:16.390 So now let's talk about the things I've learned from being a firefighter that I think are relevant to IR specifically. 22:16.890 --> 22:20.550 I'll start with this first challenge, tunnel vision. 22:20.550 --> 22:27.230 This is an idea where we focus exclusively on one particular thing instead of seeing that big picture, right? 22:27.230 --> 22:30.210 And we make bad decisions when we do that. 22:30.350 --> 22:39.330 Think back to that image, the two images of me changing out that air bottle for the firefighter and his burn coat. 22:39.750 --> 22:42.430 This is an example of tunnel vision. 22:42.430 --> 22:44.310 I'm focused on doing my job. 22:44.310 --> 22:46.130 That's all I'm doing. 22:46.470 --> 22:51.970 And frankly, I could have gotten really hurt here because of tunnel vision. 22:51.970 --> 22:57.450 I wasn't taking a step back in my head and looking at the bigger picture. 22:57.450 --> 23:00.730 I was just focused on the task at hand. 23:00.950 --> 23:03.510 Here's another example from firefighting. 23:03.510 --> 23:05.770 Look carefully at those cones. 23:06.170 --> 23:10.150 Pull up to the fire, get off the fire truck, and this is what you see. 23:10.650 --> 23:12.230 What are those cones protecting? 23:12.230 --> 23:14.770 At first glance, you see a fire hose. 23:14.770 --> 23:15.890 No big deal. 23:16.050 --> 23:17.910 So maybe you're going to use that hose. 23:17.910 --> 23:19.350 You're going to help with the fire out. 23:19.690 --> 23:25.870 Maybe you're going to bring a piece of gear to the other person if you look carefully who's standing at the door. 23:26.990 --> 23:39.310 If you look more carefully, you may or may not see there is actually a live power line that those cones are actually trying to protect you from. 23:39.370 --> 23:48.690 And if you go rushing into the scene because you're so focused on putting the fire out or getting the gear that somebody needs, you could get really hurt. 23:48.930 --> 23:51.550 We have the same kind of thing in IR. 23:51.550 --> 23:54.850 We need to consider whether or not a scene is safe. 23:54.850 --> 23:58.530 In many cases, sure, it might be, but it depends. 23:58.530 --> 24:00.130 It depends on the situation. 24:00.330 --> 24:06.290 The picture that I'm showing you below is of a server room that has asbestos. 24:06.310 --> 24:10.190 If there's an issue in a server room with asbestos, you don't want to go running in there. 24:11.130 --> 24:20.750 Consider if somebody tells you that you have a malware situation and you get focused on that, but it turns out it's just a misconfiguration. 24:21.230 --> 24:38.110 Or maybe you do blind hardware acquisition, so you go running in and you grab drives, but that machine you find out later was powered on and was running encryption and now you don't have the keys anymore because all you've done is grabbed hardware. 24:38.110 --> 24:39.770 You haven't thought about it. 24:40.470 --> 24:51.010 Maybe you were told that a situation involved ransomware and you're worried now and you're focused on how you're going to handle the ransomware, but it turns out it's just a phishing email. 24:52.170 --> 24:56.030 So hand-in-hand with tunnel vision is reactionary behavior. 24:56.450 --> 25:04.150 Not only are we focused on one thing, but we react to it without thinking ahead, right? 25:04.150 --> 25:13.690 We're allowing those outside forces to make that decision instead of relying on the data we have at hand and thinking about that bigger picture. 25:14.230 --> 25:19.110 The fire service, it can lead to really deadly situations. 25:19.930 --> 25:27.510 So the photograph you see here is from a horrible fire that happened not all that long ago in Worcester, Massachusetts. 25:28.210 --> 25:34.380 It was a cold storage fire in an old meat packing plant that was built in 1906. 25:35.290 --> 25:39.150 People hadn't been in that building in a very, very long time. 25:39.150 --> 25:40.790 It had been vacant. 25:40.850 --> 25:42.950 No pre-plans had been made. 25:43.110 --> 25:47.450 It turns out it had a maze of meat lockers in it. 25:47.610 --> 25:51.670 When this fire broke out, there was supposedly somebody trapped inside. 25:51.670 --> 25:54.030 So they sent in two firefighters. 25:54.070 --> 25:59.170 The two firefighters went in and attempted to find the individual they thought was trapped. 25:59.390 --> 26:03.550 They got hopelessly lost and radioed for help. 26:03.830 --> 26:08.270 So the chief sent in two more firefighters to help them. 26:09.070 --> 26:18.190 Unfortunately, those two firefighters also got hopelessly lost and could not get out of the building, nor could they find the two original firefighters that had been sent in. 26:19.130 --> 26:23.210 So the fire chief sends in two more firefighters. 26:23.910 --> 26:25.230 Guess what? 26:25.230 --> 26:28.310 They too get hopelessly lost. 26:29.030 --> 26:32.530 And ultimately, the fire chief says, no more. 26:32.670 --> 26:34.630 I'm not sending any more in. 26:35.330 --> 26:45.070 Now, at first he was harshly criticized by his team for this, because, of course, their own folks were inside that building. 26:45.070 --> 26:52.150 But the reality is, any further reactionary behavior to try to save these individuals would just have led to more death. 26:52.810 --> 27:02.470 These folks died because pre-plans weren't made, and they didn't know what was in there, and it would have been a reaction to just keep sending firefighters in. 27:03.090 --> 27:09.470 We see the same thing within IR, although usually not as deadly, right? 27:09.470 --> 27:13.510 If we're doing forensics, we need clear, explicit goals. 27:13.510 --> 27:18.870 If the reactionary behavior is just figure out the bad stuff, that's not helpful. 27:18.870 --> 27:22.690 You need to know more specifically what it is they want you looking for. 27:23.710 --> 27:26.690 Another example is just pulling a network connection. 27:26.770 --> 27:31.970 What if you need more information about the attacker because of what they're doing on other systems? 27:32.270 --> 27:34.630 Or what happens if you just turn off a machine? 27:34.630 --> 27:42.810 We've already talked about the situation where, you know, you have a key in memory and you turn the machine off and it's gone. 27:43.790 --> 27:51.650 Suspending all accounts is another example where it could be you have one account that's compromised, now you suspend them all, and that causes more headaches. 27:53.710 --> 27:56.070 Another challenge is freelancing. 27:56.230 --> 28:08.610 Now, in our field in IT, a freelancer we often think of as somebody who's more of, perhaps, somebody who's just not working directly for the company, right? 28:08.610 --> 28:09.990 They're a contractor. 28:09.990 --> 28:24.170 But in the fire service, this notion of freelancing is this idea that somebody is not following chain of command, they go off on their own and kind of do what they want in an attempt to get the job done, but they're not following what's been laid out for them. 28:24.730 --> 28:31.050 And that can lead to dangerous and reckless situations and bad feelings and all kinds of problems. 28:32.270 --> 28:38.610 In the fire department, we see this typically where we have what's called the crossing of the streams. 28:38.610 --> 28:46.010 And for those of you who don't know, the picture in the bottom right is Ghostbusters, and they always talk about never cross the streams. 28:46.610 --> 29:01.470 Well, in firefighting, the idea is if I'm streaming water through the front of a building, and then a team comes to the rear of a building, and that building is a small house, now I'm fighting the team that's putting the water in, in both directions. 29:01.610 --> 29:03.610 And that could be really dangerous. 29:04.450 --> 29:16.610 Another way we see this freelancing in the fire department that can be really dangerous is something where doors are opened or windows are smashed without coordinating it with other people. 29:16.610 --> 29:25.170 We've seen situations where a window could get smashed and a firefighter could be standing right on the other side of it because they don't realize somebody's there. 29:25.170 --> 29:31.270 It can also ultimately feed the fire because where there's more oxygen, that fire is going to go. 29:31.270 --> 29:36.930 So if it hasn't been planned out and it's not coordinated, it can actually make things significantly worse. 29:38.330 --> 29:42.170 In incident response, we see the same kinds of things. 29:42.170 --> 29:47.150 A team goes in to do a response, and you get somebody who thinks they're just going to collect everything. 29:47.270 --> 29:50.650 Well, there could be legal ramifications for that. 29:50.790 --> 29:54.410 What happens if they don't collect what's absolutely required? 29:54.430 --> 29:56.290 Maybe they overlook something. 29:56.290 --> 29:59.770 Maybe there's a duplication of data collection which wastes time. 29:59.770 --> 30:09.590 Or maybe data gets altered or misrepresented because people are freelancing and not everybody's not following the same set of rules or the same guidance. 30:12.010 --> 30:12.970 Patience. 30:13.190 --> 30:15.910 This is a fantastic opportunity. 30:17.010 --> 30:23.290 Ultimately, you want to take that moment to determine what the best course of action is. 30:23.290 --> 30:27.470 As one of my fire instructors would say, slow is fast, right? 30:27.470 --> 30:39.190 You want to be deliberate about what you do because ultimately it will take less time if you take a moment, think things through, and then proceed. 30:40.610 --> 30:46.050 So in terms of firefighting, you may remember the same house here. 30:46.050 --> 30:49.550 It's the same story on the left is what we pulled up to. 30:49.550 --> 31:00.810 You'll notice you can see flame in the picture on the right hand side, the sort of orangey bit that's on the right part of that underneath the eaves next to the downspout. 31:00.810 --> 31:05.810 And you'll see a little bit of a glow toward the front of that third window on the right. 31:05.810 --> 31:08.610 But we don't see flame blowing through the roof. 31:09.050 --> 31:20.350 Now what I was seeing in that thermal camera image was that all of the fire, all of the significant flame and heat for this particular fire, that was up in that roof area. 31:21.290 --> 31:36.690 So what would have been ideal is for somebody to go in on the inside, break through from the second floor up into that attic area, and ultimately put the fire out. 31:36.810 --> 31:42.410 Unfortunately, we had people who focused solely on where they saw flame. 31:42.410 --> 31:54.390 And as a result, we see what we see on the right hand side where flame blows through the roof and there's a lot more of it. 31:54.390 --> 31:56.750 And it took us a lot longer to get this fire out. 31:56.750 --> 32:04.690 Which isn't to say that what everybody did was wrong, but taking a beat and thinking this through might have saved a lot of time with this fire. 32:06.270 --> 32:09.570 Okay, patience in terms of incident response. 32:09.710 --> 32:16.170 For those of you not familiar with Chegg, Chegg is a company that rents textbooks. 32:16.170 --> 32:31.510 Now what do you think the odds are of an entity that rents textbooks to students in higher ed, of them using, oh I don't know, same credentials at the university that they use at Chegg or vice versa? 32:31.750 --> 32:34.450 Yeah, pretty darn good. 32:34.450 --> 32:51.290 So Chegg had a massive breach that involved millions of accounts and because they provide services to multiple universities, as you might expect, many institutions wound up with this exact problem. 32:51.290 --> 32:53.750 Tons of compromised accounts. 32:53.850 --> 33:01.790 But is it really the university's problem in the sense that is it their breach? 33:01.790 --> 33:03.450 No. 33:03.450 --> 33:06.750 What it is, is password reuse. 33:06.750 --> 33:08.150 Chegg had the breach. 33:08.150 --> 33:13.870 Yes, the universities and institutions have to do something about making sure those passwords get reset, right? 33:13.870 --> 33:14.830 That's important. 33:15.150 --> 33:22.170 But it's not the same as having those passwords having been breached on the systems that are actually at the university. 33:22.390 --> 33:24.450 That is an incident. 33:24.450 --> 33:25.990 That's pretty serious. 33:26.450 --> 33:34.110 So before you panic when you hear about a huge breach, even if it could impact your institution, think for a moment. 33:34.110 --> 33:38.410 Take that beat about exactly what the implications are. 33:38.470 --> 33:40.570 What is the real risk here? 33:41.030 --> 33:49.550 And also consider that time is important, but as we saw with the picture of the asbestos room, life safety is more important. 33:49.630 --> 33:54.570 So taking that beat can also be life-saving in incident response depending on what you're walking into. 33:55.730 --> 33:57.790 All right, accountability. 33:57.950 --> 34:03.210 It's this idea, at least in the fire service, of knowing where all your people are. 34:03.430 --> 34:05.490 You want to know if they're in the trucks. 34:05.630 --> 34:08.090 You want to know who came with one truck. 34:08.090 --> 34:09.870 You want to know who's gone inside a building . 34:09.870 --> 34:12.690 You want to know who's getting rehabbed. 34:12.750 --> 34:16.110 But really, in incident response, we do the same kinds of things. 34:16.290 --> 34:20.550 We want to make sure that we prevent hazards and duplication of efforts. 34:20.550 --> 34:24.170 We don't want to step on each other's toes or have any sort of direct interference. 34:26.070 --> 34:31.190 So on the left are accountability tags that we use in firefighting. 34:31.410 --> 34:39.210 And for example, we'll use one of those tags we'll place on the truck so that everybody knows what truck you came in on. 34:39.330 --> 34:46.110 That way, the officer on that truck knows where their people are at all time. 34:46.690 --> 34:56.450 They might put another, if they go inside the building, they'll use a second tag and put it with the accountability person at the door so that they know where they are inside the building. 34:56.690 --> 35:00.450 For incident response, we can do the same kinds of things. 35:00.810 --> 35:02.650 Perhaps wipe card access. 35:02.650 --> 35:09.570 If you're going inside a facility where you work, we'll show you what room you're in. 35:09.570 --> 35:14.190 And maybe you need something more detailed, like an incident access log, depending on the situation. 35:14.350 --> 35:18.230 So we can do accountability with IR, and it's important. 35:19.730 --> 35:21.730 Okay, pre-planning. 35:21.730 --> 35:24.590 Pre-planning is absolutely critical. 35:24.590 --> 35:26.850 You want to know your environment. 35:26.850 --> 35:38.110 You want to know through your own documentation and planning, third-party documentation, and anything else that you can think of before it becomes a problem. 35:38.130 --> 35:43.930 And in the fire service, we're going to do that with what are called pre-plans. 35:43.930 --> 35:56.890 And in incident response, we're going to do it with things like tabletop and pre-mortem exercises, so that we have this idea of what we might run into before we run into it, and prevent those gotchas. 35:58.390 --> 36:01.390 So this is the kind of thing we do in firefighting. 36:01.390 --> 36:06.110 On the left is a map of a building, basically. 36:06.290 --> 36:20.410 I know it's a little difficult to see, but the idea is that it maps out for us where the hydrants are, where the electrical shutoff is, and it might tell us if there's any hazardous materials in the area. 36:20.410 --> 36:24.470 The thing on the right might tell us a little more about the building construction. 36:24.470 --> 36:35.210 This is more of a general pre-plan that's going to tell us about other hazards, other information, but it's the same concept, right? 36:35.210 --> 36:47.410 It tells us what we need to know about the facility we're going into to keep us safe, and those that we're trying to rescue, and perhaps the general public, depending on the particular situation. 36:47.790 --> 36:50.970 We can do the same thing in incident response. 36:50.970 --> 36:53.050 We can do tabletop exercises. 36:53.290 --> 36:57.890 If you're not already familiar with backdoors and breaches, you should be. 36:58.050 --> 37:17.150 The Black Hills folks, John Strand and his crew, have built this amazing card game, which uses a 20-sided die and this set of cards, and it can help you build an entire event from the ground up, and then you can do the investigation using those cards. 37:17.150 --> 37:22.330 It's really interesting, and I highly recommend it if you haven't already tried them. 37:22.590 --> 37:38.490 But it can walk you through scenarios, everything from something happening in a natural disaster type emergency to, you know, the intern who's done the wrong kind of thing and made an accidental mess. 37:39.250 --> 37:41.630 Obviously, documentation we've talked about, right? 37:41.630 --> 37:50.910 Document your servers, document your workstations, document where your data is and what your network is like, and the more information you have, the better off you'll be. 37:51.290 --> 37:58.870 And hopefully it goes without saying, to a certain degree, that this would also include your incident response plan and whatever team you have. 37:59.750 --> 38:04.830 So to summarize, we want you to start thinking like a firefighter. 38:04.830 --> 38:13.750 I think that applying these firefighter ideas to incident response has really made me a better incident responder, ultimately. 38:13.970 --> 38:16.070 You want to avoid tunnel vision. 38:16.070 --> 38:30.990 You want to do what's called a 360 of the problem, which in firefighting might literally mean a 360 around the building, which is what we try to do, or at least somebody does when we get there, whether it might be the operations person, it might be the incident commander. 38:31.110 --> 38:40.090 But at a minimum, we always pull past the building that's on fire, or potentially on fire, so that we can see at least three sides of the building. 38:40.430 --> 38:43.090 That gives us that bigger scope. 38:43.490 --> 38:47.450 We want to make sure we're acting, we're deliberate, we don't just react. 38:47.570 --> 38:50.090 We don't want to be misled by emotion. 38:50.830 --> 38:54.550 Follow whatever plan that you and your colleagues have come up with. 38:54.550 --> 38:55.830 Don't just freelance. 38:56.330 --> 39:04.810 It can be, you know, physically dangerous in certain circumstances, and it can just be plain frustrating in others. 39:04.950 --> 39:07.930 Have patience and take that beat. 39:07.930 --> 39:10.350 You never know what you might have missed if you didn't. 39:10.350 --> 39:15.770 Those 30 or 60 seconds could save tons of time. 39:15.770 --> 39:24.870 And even in the fire service, taking those few seconds ultimately could save a life, even if you think going faster might have done it. 39:25.950 --> 39:26.790 Location. 39:26.790 --> 39:29.330 Make sure you know where your people are. 39:29.330 --> 39:33.210 Document, document, document your location. 39:33.450 --> 39:41.630 Both of your equipment, your people, where your data is, and pre-plans they rock, right? 39:41.630 --> 39:45.310 Less chaos, way easier to respond if you already have a plan. 39:45.310 --> 39:50.430 And consider something like the incident command system, so that you're organizing your response. 39:50.970 --> 39:57.870 The thing about the incident command system that's so cool is that ultimately it is fantastic for scaling to larger incidents. 39:57.870 --> 40:04.370 As I mentioned, you can have only one person in charge of just a few people. 40:04.370 --> 40:08.690 And if you scale that out, it works really well in larger situations. 40:08.790 --> 40:14.350 So bringing in outside entities, this is so much easier because everybody's on the same page. 40:16.050 --> 40:19.390 So, some final thoughts. 40:19.490 --> 40:25.950 I know that many of you have never thought about incident response in this way. 40:25.950 --> 40:27.450 And that's okay. 40:27.450 --> 40:29.310 That's the idea. 40:29.430 --> 40:31.750 Think outside that box. 40:32.250 --> 40:35.230 Try some things you've never tried before. 40:35.270 --> 40:51.550 You may find that ultimately it makes your job maybe more challenging in the And the reality is great things never come from comfort zones. 40:51.650 --> 40:52.510 So, do it. 40:52.510 --> 40:53.810 Push that envelope. 40:54.670 --> 40:57.310 Hopefully you will find these tactics helpful. 40:57.830 --> 41:00.490 I want to thank you for coming to hear me speak. 41:00.490 --> 41:03.110 And I want to thank Wall of Sheep so much for having me. 41:03.190 --> 41:09.330 And I hope you have an absolutely wonderful DEF CON and a wonderful rest of your day.