[00:08.600 --> 00:14.280]  Welcome everyone to the Blue Team Village incident response panel. We have an awesome group here for
[00:14.280 --> 00:19.940]  you today with unique experiences and perspectives on security incidents and incident response.
[00:19.940 --> 00:24.640]  I'm really excited for today's discussion. But before we get started, take a look at the poll
[00:24.640 --> 00:30.540]  we posted on the Blue Team Village Twitter earlier today about ransomware and whether or not victims
[00:30.540 --> 00:35.460]  of ransomware should pay the ransom. We are going to get on that topic in a bit and we would like
[00:35.460 --> 00:41.240]  your input in the poll. Also, please post questions for the panel in the Blue Team Village Discord
[00:41.700 --> 00:48.140]  in the Talks Track 1 channel, which we'll be monitoring. My name is Russell Mosley. I go by
[00:48.140 --> 00:52.800]  SmokeEm on Twitter and I'm a lead with the Blue Team Village. I've been a Blue Teamer for my
[00:52.800 --> 00:57.740]  entire career, building and managing security operations at small and mid-sized companies.
[00:58.000 --> 01:03.700]  So I wear a lot of hats. I've done everything from sysadmin. I've been an IT director,
[01:03.800 --> 01:08.780]  a security ops lead, a compliance auditor, and today I'm a CISO at a rapidly growing government
[01:08.780 --> 01:15.200]  contractor. All my experiences involve developing and managing small teams. And as a result, I've
[01:15.200 --> 01:19.500]  had the opportunity to do a lot of different things in the security field and build decent
[01:19.500 --> 01:26.540]  security programs without a huge budget. I'd like to introduce my co-host, Xavier Ashe. Xavier?
[01:26.760 --> 01:32.120]  Good afternoon, everyone, or good morning to where you're at. So my name is Xavier Ashe. I go by
[01:32.120 --> 01:41.440]  Rubix1138. My IR experience is a little varied. I was an IR team lead at bit nine slash carbon
[01:41.440 --> 01:50.980]  black. And also I was able to start the first IR team at SunTrust, now Truist. So 28-year veteran,
[01:50.980 --> 01:57.900]  been around for a while. I can remember when my IR plan was you have to scan every floppy by hand.
[01:57.900 --> 02:06.440]  That's me. Cool. Next I'd like to introduce Lip Moose. Moose is frequently posting to Twitter,
[02:06.440 --> 02:11.400]  comments on exciting things that she encounters doing IR. And this week it was jump boxes that
[02:11.400 --> 02:18.420]  look like flaming toilets, I think was one of the highlights. She refers to herself as many things,
[02:18.420 --> 02:25.040]  including Florida woman, a DFIR violinist, and recently completed a cross-country hacker road
[02:25.040 --> 02:32.620]  trip with cats and snails on board. Isn't that right, Lip Moose? That's fairly accurate. I did
[02:32.620 --> 02:41.500]  remote DEF CON. We got to see a few goons. It was nice. So I go by Lip Moose on Twitter, also known
[02:41.500 --> 02:48.220]  as Moose. My name is also Heather. I'm currently working at CrowdStrike. I'll dox myself today,
[02:48.220 --> 02:54.420]  which is not a thing I usually do at DEF CON. But I'm very proud of where I work and what I do.
[02:54.420 --> 02:59.980]  I work with a lot of companies of all sizes. I come from a background
[03:01.320 --> 03:09.440]  for financial giants, as well as manufacturing giants, and have been doing this long enough to
[03:09.440 --> 03:15.200]  where I think I've been awake and can engage myself in the 60s if I add all of my cumulative
[03:15.200 --> 03:22.740]  IR experiences together without sleep. So I'm looking forward to retirement. I don't know when
[03:22.740 --> 03:28.680]  that will be, but I do love what I do. Outside of work, I herd cats and I play violin. And so
[03:28.680 --> 03:34.440]  I'd actually like to pivot and introduce somebody that I've worked with a little bit. 1099,
[03:35.580 --> 03:43.120]  one of my favorite people in the world. Look, I'm sober right now, which is a problem. But,
[03:43.120 --> 03:48.860]  Brian, I'll go ahead and pass it over to you. Well, hello, all you cool cats and kittens.
[03:48.860 --> 03:54.080]  My name is Brian Moran, and I go by the incredibly clever Brian J. Moran on Twitter.
[03:55.360 --> 03:59.880]  I've worked primarily in the Air Force. That's where I started my career, worked for a three-letter
[03:59.880 --> 04:04.500]  agency, did that for a bit, got out of the military, bounced around a couple of private
[04:04.500 --> 04:10.940]  sector companies, and eventually, back in 2014, started working completely for myself. So I
[04:10.940 --> 04:17.620]  started my own company, and I am an IR consultant. That's what I do probably about 60% of the time.
[04:17.620 --> 04:23.580]  The other 20, 30% is probably primarily focused on training and development plans,
[04:23.580 --> 04:29.220]  and the other 10% of my time is primarily spent trying to keep the DFIR FIT going, where
[04:29.220 --> 04:32.840]  it's all about just trying to get a little bit more active,
[04:32.840 --> 04:36.180]  and hopefully we can all be on this little blue dot a little bit longer.
[04:37.940 --> 04:42.200]  Cool. Yeah, I certainly was going to mention Brian's Twitter seems to generally be filled
[04:42.200 --> 04:50.260]  with DIR FIT and comments on rum. You left the rum out. Well, I have some right here.
[04:50.580 --> 04:52.540]  Oh, funny you should mention that.
[04:54.900 --> 04:56.900]  I thought you were going to be better than me, Brian.
[04:58.800 --> 05:04.220]  Let's hear from Virus before we have our toast. Virus, so he says he's only been a blue teamer
[05:04.220 --> 05:09.860]  for about a year and a half with a background in red team, and he's currently doing what I
[05:09.860 --> 05:15.340]  call kind of red-blue and innovative network defense for a large financial organization.
[05:15.340 --> 05:22.540]  If you want to elaborate, welcome, Virus. Yeah, so I work at Stripe currently. I'm the
[05:22.540 --> 05:34.360]  threat detection team. I guess the word for it is adversary direct engagement and threat hunting.
[05:34.360 --> 05:40.900]  Background-wise, started in the dark gray end of the pool as a young kid,
[05:40.900 --> 05:48.080]  like my first DEF CON, I was 17. Fell into penetration testing and then red teaming,
[05:48.080 --> 05:52.980]  like full-spectrum threat stuff, and passed around the world doing that for a while.
[05:53.420 --> 05:59.220]  Kind of drifted back towards pen testing, and then, surprise, had a kid, and, you know,
[05:59.220 --> 06:02.700]  turns out that gallivanting around the world, breaking into buildings in random countries,
[06:02.700 --> 06:07.040]  and getting arrested is not a good way to be a dad, so had to pivot for a while.
[06:07.760 --> 06:12.120]  Did a stint doing mergers and acquisitions security at Salesforce and found out I'm not
[06:12.120 --> 06:21.360]  bureaucrat. Now I'm doing blue team, so, yeah. Cool, that's awesome. Really looking forward to
[06:21.360 --> 06:26.080]  sharing some of your perspectives that we heard in our planning meeting yesterday.
[06:26.800 --> 06:32.680]  So before we get going, we wanted to carry out the DEF CON tradition of first-time speaker drink.
[06:35.780 --> 06:40.900]  Everybody have their glass ready? Awesome. Cheers to IR.
[06:49.530 --> 06:54.790]  All right, so to kick it off, you know, we're in a unique situation here in 2020, right?
[06:55.370 --> 07:02.150]  With COVID forcing work from home for so many people. So I wanted to start with a question for
[07:02.150 --> 07:08.390]  full-time incident responders, Moose and Brian. What types of incidents are you seeing the most
[07:08.390 --> 07:13.910]  of right now? What are the trends in attack methods, persistence, and anything else you've
[07:13.910 --> 07:20.250]  observed lately in your work? Yes, I can go ahead and start.
[07:21.710 --> 07:29.830]  It hasn't changed too, too much. The volume has been a huge uptick. I'm still seeing a ton of
[07:29.830 --> 07:37.930]  ransomware attacks. A curious thing that I'm seeing more of now is actually, I would say,
[07:37.930 --> 07:44.790]  more intrusion via remote code execution has had an uptick recently. With a lot of the bots out
[07:44.790 --> 07:49.810]  there, vulnerability scanning, I would say that those are getting hit before people can catch
[07:49.810 --> 07:58.390]  just a little bit faster than I used to see. But a lot of the end game for what I'm seeing now
[07:58.390 --> 08:05.370]  is either encryption and then extortion or exfil and then extortion. And I would
[08:05.370 --> 08:10.290]  say that the volume has just gone up versus what it was like a year ago.
[08:11.430 --> 08:17.450]  Yeah, I would 100% agree. The exact same stuff that's happening now has been happening for
[08:17.450 --> 08:23.330]  the past probably two to three years has been the general trend. But the volume of it has just
[08:23.330 --> 08:29.510]  immensely increased. And probably part of that is the bad actors have nothing else to do except
[08:29.510 --> 08:35.890]  stay at home, just like we have nothing else to do. So the idle hands will play, I believe,
[08:35.890 --> 08:43.090]  is the phrase that's used there. So the one thing that I have seen more recently an uptick of than
[08:43.090 --> 08:50.990]  I have over the past year or so is actually O365 and not necessarily intrusions. It's users
[08:50.990 --> 08:57.850]  clicking on links they shouldn't click on, and then the attackers getting access to stuff that
[08:57.850 --> 09:03.050]  they shouldn't necessarily be able to get. So it's not necessarily a 365 intrusion. It's just
[09:03.050 --> 09:08.310]  phishing, which is kind of directed more towards with the overall goal of getting some O365 access.
[09:12.300 --> 09:18.580]  Yeah, that's interesting. I wanted to ask, a lot of companies have focused a lot of their security
[09:18.580 --> 09:25.760]  on their networks, and perhaps not as much on their endpoints and cloud services.
[09:26.280 --> 09:32.980]  And now that many of their workers are remote, they don't have the option to do monitoring
[09:34.500 --> 09:40.060]  on their network, and maybe aren't as well prepared to monitor from their endpoints and
[09:40.060 --> 09:46.780]  their cloud services. Would you agree that that's the case? And how is that impacting
[09:46.780 --> 09:55.280]  what you're seeing out there? So I would say it definitely does. But the caveat to that,
[09:55.280 --> 10:00.800]  and I already kind of doxxed where I work, is I'm getting calls because they're not necessarily
[10:00.800 --> 10:07.120]  using an EDR to begin with, or maybe there's not as much focus on the endpoint. And I think that
[10:07.120 --> 10:14.680]  what's neglected in cloud is that there's this perception of security that's not really there.
[10:14.680 --> 10:19.520]  You're just working on a server in somebody else's data farm. And so you still need to put the same
[10:19.520 --> 10:27.220]  protections on it, and monitor your VPC flow logs, and look at what VPC you're putting stuff in,
[10:27.220 --> 10:33.260]  or what segment, rather. And you would monitor it just like you would something on-prem in your
[10:33.260 --> 10:39.520]  environment. And I'm seeing that being so quick to set up that sometimes those steps are missed,
[10:39.520 --> 10:46.220]  and then I'll get a call. Yeah, I think as we talked about a little bit last night,
[10:46.220 --> 10:51.500]  one of the things is the data that you need to actually respond to a breach of some sort,
[10:51.500 --> 10:56.920]  it's always been there. It's just never really been used or even thought of being looked at,
[10:56.920 --> 11:00.320]  because before it was all, well, we're just going to block everything at the network layer because
[11:00.320 --> 11:04.280]  everything is controlled. But now that everything's spidered out, all the logs are still
[11:04.280 --> 11:08.520]  there. All of that's still there. It just never was bothered to actually be configured properly
[11:08.520 --> 11:16.060]  in the first place. So that's why we're seeing what we're seeing. So with the more distributed,
[11:16.060 --> 11:24.380]  I'm hearing more endpoints and more cloud, has that really changed the way that we're doing IR now?
[11:28.630 --> 11:36.630]  I would say yes and no. So we're still asking for logs that are representative of the same data.
[11:36.630 --> 11:40.550]  Instead of asking for firewall logs in this case, you're looking for NetFlow,
[11:40.550 --> 11:48.130]  which might be DPC flow. We're still asking to run scripts to an endpoint, which whether or not
[11:48.130 --> 11:52.890]  it's in a cloud or on-prem, it's going to be capturing the same artifacts. We're looking at
[11:52.890 --> 11:58.270]  the same things on Linux that we're looking at on an AWS Linux server that we would look at
[11:58.270 --> 12:05.010]  in a server in your environment that's physically there. So I would say that while things have
[12:05.010 --> 12:10.990]  changed in slight ways in how we capture the data, so converting an EBS snapshot versus
[12:12.490 --> 12:19.030]  a VMPK versus I'm going to run FTK to it, those little minor changes are there. But I think just
[12:19.030 --> 12:23.930]  getting creative about, I'm still caring about the same things. I'm still caring about how the
[12:23.930 --> 12:29.350]  thread actor got there. So your original access point, I'm going to look at all logs. I'm going
[12:29.350 --> 12:34.790]  to look at, has there been any exfil, file creation, file deletion. So the same questions
[12:34.790 --> 12:39.850]  we've been asking forever, we're still going to keep asking. It's just that creativity around,
[12:39.850 --> 12:44.590]  we're in a different environment now. Unfortunately, those logs are generally
[12:44.590 --> 12:50.450]  there for companies that rely on Office 365. A lot of times you just have to have the right licenses
[12:50.450 --> 12:55.350]  to go look at them in cloud app security and other places. So what happens when an organization
[12:55.350 --> 13:01.570]  hasn't really invested in logging on their endpoints or anything beyond the defaults
[13:01.570 --> 13:07.570]  that come with Windows? You have a situation where you have people getting phished or
[13:07.570 --> 13:12.790]  getting malware, and there's really not much to go on. What do you do in that case?
[13:13.210 --> 13:18.570]  Generally, you're not going to have the easy access to get a forensic image and conduct
[13:19.290 --> 13:24.970]  analysis on the device because it's somewhere far away from you, maybe on a slow link.
[13:24.970 --> 13:26.430]  How do you deal with that?
[13:28.650 --> 13:35.090]  Well, I think the old way of, well, we would just show up and collect data from somebody's
[13:35.090 --> 13:39.770]  computer in the event that something happened. Obviously, that is no longer going to be
[13:39.770 --> 13:46.850]  existing. So you have to get creative with essentially trying to get the same people
[13:46.850 --> 13:51.970]  together on a call like this and figure out what you actually have access to. Not necessarily what
[13:51.970 --> 13:57.430]  you have, because more than likely in the event that someone like myself or Moose gets called in,
[13:57.430 --> 14:01.810]  stuff has already happened that's really, really bad. We're pretty much the last people you want
[14:01.810 --> 14:06.770]  to see in the event of an incident. So you have to get creative with essentially figuring out
[14:06.770 --> 14:11.950]  what you have access to and then how far back that goes, if it goes back at all,
[14:11.950 --> 14:16.850]  and then try to build out from there. Because a lot of times when we show up somewhere,
[14:16.850 --> 14:20.970]  the first question is, well, okay, do you have NetFlow? Do you have logs? Do you have
[14:20.970 --> 14:25.950]  X, Y, and Z? And a lot of times, the initial part of an incident isn't even actually responding
[14:25.950 --> 14:29.510]  to something. It's figuring out what we actually have access to and then trying to get creative
[14:29.510 --> 14:33.790]  with ways to get access to that data, get it as quickly as possible, and then turn that into
[14:33.790 --> 14:41.570]  usable information that we can use as quickly as possible. So what are some things...
[14:43.730 --> 14:47.890]  Go ahead, Moose. Sorry. I was just going to say, I think one of the boons right now is that so
[14:47.890 --> 14:54.110]  many people have invested in cloud security, so it almost negates the problem. A lot of times,
[14:54.210 --> 15:00.390]  a threat actor's pivot point of getting into a lot of devices, we can think about, and I almost
[15:00.390 --> 15:04.830]  want to turn it over to Byers for this one, but if you think like an attacker, you have a better
[15:04.830 --> 15:12.490]  chance of getting the data you need. And I worked a case recently where, you know, I couldn't run
[15:12.490 --> 15:16.690]  the scripts I wanted to, but I looked at what they did, and I said, well, let's just replicate
[15:16.690 --> 15:21.230]  what the red team did, or what the threat actor did in this case, and let's red team it,
[15:21.230 --> 15:27.550]  and I got the data I needed. So that's... sometimes you just got to look at the problem a little bit.
[15:28.830 --> 15:35.010]  I think another curveball that I started to see is, you know, for companies that use Wwise terminals,
[15:35.010 --> 15:40.690]  you really can't take those thin clients home, and so we're seeing an increased use of other
[15:40.690 --> 15:47.210]  devices like Chromebooks, and so I wanted to see any advice out there if somebody that's
[15:47.210 --> 15:53.830]  listening is going to be doing any type of instant response involving Chromebooks used as,
[15:54.590 --> 16:00.010]  you know, thin clients. Is there any advice that you can give to those folks out there?
[16:02.630 --> 16:08.590]  Looks like we lost Moose for a minute. Well, so I know that there's been some research into
[16:09.050 --> 16:13.150]  essentially the data that you can collect from Chromebooks. I know Jessica Hyde from Magnet
[16:13.150 --> 16:18.110]  Forensics has done quite a bit on that, and I believe this past year she's given a few talks
[16:18.110 --> 16:22.110]  on that. So there is data you can get from Chromebooks. You have to get clever with it,
[16:22.110 --> 16:26.450]  but one of the things that's interesting is if you use Chromebooks, more than likely you're using
[16:26.450 --> 16:31.550]  some sort of Google instance, and Google has a ton of data about you. Whether you use Google or not,
[16:31.550 --> 16:36.190]  Google has a ton of data about you, and because you're able to get all of that data back from
[16:36.190 --> 16:41.830]  them. And admittedly, there's a lot of stuff to go through. But it's not necessarily... like
[16:41.830 --> 16:46.290]  Chromebooks are incredibly secure. I love Chromebooks. I recommend Chromebooks to pretty
[16:46.290 --> 16:50.790]  much everybody who gets click-happy with any kind of links. Just use a Chromebook for a while,
[16:50.790 --> 16:55.090]  and hopefully that's going to eliminate a big portion of your attack surface. It's not going
[16:55.090 --> 16:59.950]  to completely eliminate it, but it's going to take a big portion of that away. But essentially,
[16:59.950 --> 17:04.270]  at the end of the day, you know, we're still trying to look for the oddity outlier information
[17:04.270 --> 17:09.510]  of data access, you know, data exfil, stuff like that. And with the stuff that you can get from
[17:09.510 --> 17:13.750]  Google specifically, like all of that's there. It's just a matter of being able to get it,
[17:13.750 --> 17:16.150]  being able to process it, and being able to digest it.
[17:17.250 --> 17:23.550]  What are your thoughts, Virus, on doing IR with cloud and distributed endpoints without the
[17:24.610 --> 17:27.150]  hardened network of tradition?
[17:29.570 --> 17:34.130]  In some ways, I guess I haven't been around in the Blue Team ecosystem enough to know the
[17:34.130 --> 17:42.410]  difference. Like, the few roles that I've had that are hard in Blue Team, I've always had the
[17:42.410 --> 17:48.110]  power to at least have a say in kind of what our telemetry looks like. And I mean, coming from the
[17:48.110 --> 17:53.830]  red side, I've frankly never really bothered wanting network stuff. It's always, you know,
[17:53.830 --> 17:58.310]  anything that's a network artifact, I only care about it in the context of what the endpoint did
[17:58.310 --> 18:05.150]  in terms of interaction. And so it's just usually easier to filter on down and kind of pinhole what
[18:05.150 --> 18:13.730]  I want. I have definitely, within the past year, I've dealt with some vendor breaches, which are
[18:13.730 --> 18:18.450]  all public, so I can say that, but I guess I won't say exactly what they are. So, you know, I won't
[18:18.450 --> 18:25.670]  get fired immediately. And it's kind of interesting. One of the nice things about, I shouldn't say
[18:25.670 --> 18:30.670]  nice, but one of the interesting things about kind of being tangentially related to a breach
[18:30.670 --> 18:36.390]  that gets released public is suddenly you have like a legal stick to beat them with to actually
[18:36.390 --> 18:41.970]  get IOCs back from the breach, as opposed to just reading the news and having a Twitter opinion like
[18:41.970 --> 18:50.130]  the rest of the world. And from what I've seen, it definitely, like, the game for the vectors
[18:50.130 --> 18:55.430]  in terms of the game for getting in doesn't seem to have changed. It's usually not the server.
[18:55.670 --> 19:02.310]  It's usually it's way easier to get on a laptop. And then because no one knows how to network in
[19:02.310 --> 19:07.410]  the cloud, like everything is just grabbing the AWS token or the Azure token off disk and then
[19:07.410 --> 19:13.030]  going to town. And then usually from there, it's a, oh, by the way, either there's no logs or there's
[19:13.150 --> 19:17.350]  a ton of logs, but nobody's ever actually triaged them to find out which logs are useful and which
[19:17.350 --> 19:22.770]  ones aren't. So there's no logs. And then it's usually going back and being like, oh, well,
[19:22.770 --> 19:27.030]  after they took this token from your machine, then they directly logged in with this super
[19:27.030 --> 19:30.970]  weird user agent that no one's ever heard of and stole all your data and you did nothing.
[19:32.490 --> 19:38.810]  At least that's been the trend from what I've seen. The Chromebook thing, there actually is,
[19:38.810 --> 19:44.190]  I don't have a lot of specific details that I can share, but I do know that there has been
[19:44.190 --> 19:52.970]  some publicly released stuff on Google implementing a kind of AuditD type thing
[19:54.290 --> 19:59.510]  that is going to be very similar to like OS X's security center in terms of accessible data. So
[19:59.510 --> 20:03.030]  I think that's going to be a game changer with all of the EDR tools that are out there that are
[20:03.030 --> 20:09.050]  hooking into AuditD as kind of a funnel for data. I think that'll be really cool. And then also
[20:09.050 --> 20:13.930]  anybody who's looking to do DFIR on Chrome locked devices in general, I would actually
[20:13.930 --> 20:20.830]  route you to some research from about eight or nine years ago from Mike Osmond and Kyle Osborne.
[20:20.830 --> 20:26.990]  They did some offensive research on basically how everything in Chrome is a browser and the
[20:26.990 --> 20:30.770]  browser doesn't know the difference between a file system link and a remote link. And while
[20:30.770 --> 20:34.630]  Google has obviously tried to address the security concerns that they found with some of that
[20:34.630 --> 20:38.950]  research, you can actually use a lot of those vectors to gather debug information about the
[20:38.950 --> 20:42.830]  Chrome browser environment. And that ends up being almost equivalent to system logs on a
[20:42.830 --> 20:49.450]  Chrome locked device. And you started your comments there with, you know, first getting
[20:49.450 --> 20:54.450]  on the laptop and then pivoting out from there. What's the most common method for getting on the
[20:54.450 --> 21:02.890]  laptop? It's the usual. It's either phishing or I've seen some supply chain stuff recently. I
[21:03.320 --> 21:09.650]  came out relatively recently that I know affected a bunch of shops. I've seen some slightly less
[21:09.650 --> 21:15.890]  talked about in the media because the Go ecosystem is so blurred in terms of the concept of standard
[21:15.890 --> 21:21.710]  lib versus some person's stuff on GitHub. But I have seen some bugs there that have caused some
[21:21.710 --> 21:29.990]  supply chain issues. There's little known fact, I have seen, well, it's a more known fact that OSX
[21:29.990 --> 21:34.990]  attacks are on the rise, but it's a lesser known fact that Perl is still on the OSX boxes. So,
[21:34.990 --> 21:40.610]  I have seen kind of some like, you know, oh, I haven't seen that attack since like Red Hat 5,
[21:40.610 --> 21:45.310]  but, you know, that Perl subsystem is still on OSX and way more people look at Python on OSX
[21:45.310 --> 21:51.930]  than Perl. So, there's a little more of that. That's great. I'm curious, Brian, your observations,
[21:51.930 --> 21:55.650]  you know, how they compare to viruses feedback, you know, and what you're seeing.
[21:56.990 --> 22:04.630]  Yeah, I think it's... so, I don't think the methods have changed. The methods haven't changed. And
[22:04.630 --> 22:09.170]  honestly, the methods don't really need to change because they still work. So, if they're still
[22:09.170 --> 22:15.610]  effective, there's no reason to burn some, you know, some zero day exploit or even go the
[22:15.610 --> 22:19.410]  route of looking for like RCE or anything like that, when if you can get somebody to click on
[22:19.510 --> 22:23.090]  a link and get the information you need, and that still works, like, that's what the attackers are
[22:23.090 --> 22:27.570]  going to do. Because, you know, again, at the end of the day, like, the attackers are human just
[22:27.570 --> 22:32.450]  like us. Like, they will do just as much efforts as needed to get them the goal that they need.
[22:32.450 --> 22:39.570]  And, you know, you have to be able to account for that. One of the tools I haven't heard yet so far
[22:39.570 --> 22:47.110]  for telemetry is, you know, cloud-based DNS. So, a couple providers where you can get that from a
[22:47.110 --> 22:51.410]  centralized source. Are we seeing more people, you know, take that on as we get to a more
[22:51.410 --> 22:59.250]  distributed environment? Or is that still an afterthought of like, oh, yeah, we can't get good
[22:59.250 --> 23:06.810]  DNS telemetry because everybody's at home using their home DNS? Has anybody got any experience
[23:06.810 --> 23:15.830]  with that? I do. I've had some that have it, but not very many. And with the caveat of saying
[23:17.090 --> 23:24.110]  that if they're calling me, something's gone terribly wrong. Or, or counter, we do do security
[23:24.110 --> 23:27.790]  assessments in environments that might be something to come back with because they've set up a cloud
[23:27.790 --> 23:33.090]  environment and they want advice on how to make it better. So, I will say that I have a little bit of
[23:33.250 --> 23:39.670]  a bias at answering this because even though I don't see it as much, it's usually because they're
[23:39.670 --> 23:47.750]  calling for help. Yeah, I mean, control is kind of a crapshoot. But in terms of telemetry, like,
[23:47.750 --> 23:54.630]  man, the combination of a low-level passive DNS service and OS Query seems amazing for DNS to me.
[24:00.120 --> 24:04.280]  Yeah, so I'm going to say pretty much the exact same thing that Moose did. Like,
[24:04.280 --> 24:09.020]  I would like to see it more. However, a lot of places don't see it. And the places that call
[24:09.020 --> 24:14.500]  us, more than likely, there have been several shortcomings that led to calling people like us
[24:14.500 --> 24:19.080]  in the first place. So, typically, we wouldn't see something like that because we're not the
[24:20.740 --> 24:24.860]  security architects or anything like that. Like, something horrible has happened,
[24:24.860 --> 24:28.540]  everything's on fire. I'll call them in and they're hopefully going to calm us down.
[24:28.720 --> 24:33.680]  So, before we pivot to another topic, if you were called in to give advice to a company today,
[24:33.680 --> 24:38.620]  and they wanted to spend some additional budget for security with people working from home,
[24:39.020 --> 24:46.740]  where would you advise them to focus? I mean, I can tell you where we have an
[24:46.740 --> 24:54.560]  entirely remote workforce right now. And there is some scatter in terms of operating the system
[24:54.560 --> 25:00.880]  layout. And pretty much 100% of our EDR effort at this point is done using OS Query. That's
[25:00.880 --> 25:04.900]  pretty public information. I don't really feel like I'm giving away secret sauce saying that.
[25:04.900 --> 25:09.460]  And at least from a telemetry perspective, even with specific regard to DNS, I can tell you,
[25:09.460 --> 25:14.280]  it's pretty amazing. Like, we have other issues, but
[25:14.820 --> 25:18.160]  DNS telemetry is certainly not one of them from my perspective.
[25:20.340 --> 25:26.660]  I think the one piece of advice I could give is don't ignore any front. I see a lot of people
[25:26.660 --> 25:32.360]  spending very heavily in one arm rather than maybe ignoring the other. So, they'll focus
[25:32.360 --> 25:37.900]  just on end point or they'll focus predominantly on network side. And I think that the balance
[25:37.900 --> 25:44.480]  is still very, very important. And I would say to borrow something that we all as a team talked
[25:44.480 --> 25:50.720]  about yesterday, too many people are trying to eat the elephant hole. So, don't do that. Don't
[25:50.720 --> 25:57.080]  keep your network flat either. So, my top advice to people is make a list of priorities in your
[25:57.080 --> 26:03.320]  environment. What would hurt the worst if that went public? And start segmenting there. Start
[26:03.320 --> 26:09.180]  segmenting. Know your group permissions. Run some of these Red Team tools. Run Blood Pound,
[26:09.180 --> 26:13.300]  Sharp Pound in your environment. Get a better picture of what it looks like and then make a
[26:13.300 --> 26:19.380]  decision. But make an educated decision, not just a flat blanket decision because not all
[26:19.380 --> 26:27.080]  businesses are the same and not all businesses have the same needs. I think before spending any
[26:27.080 --> 26:31.960]  money, the first thing that a company truly has to do is take a look at what they already have,
[26:31.960 --> 26:37.400]  the people they already have, and then try to build a solution from that. Just like Moose said,
[26:38.020 --> 26:42.380]  a lot of times when I go to places, they don't even have something basic like a network map.
[26:42.380 --> 26:48.300]  So, then I look for the attackers when essentially they dump IP.txt to some system and then I have
[26:48.300 --> 26:52.300]  an actual network map that I can go on because the attackers have left their forest, fortunately. So,
[26:52.300 --> 26:55.940]  that's literally the only insight that we have to what's actually on the network is what the
[26:55.940 --> 27:01.100]  attackers have done. So, you can't be just spending money while you have all of these
[27:01.100 --> 27:06.700]  gaping holes that just have never been addressed. You have to do a serious and honest and truthful
[27:06.700 --> 27:11.320]  evaluation of what you currently have already before you make a decision on what you're actually
[27:11.320 --> 27:17.120]  going to do to try to improve. And related to that, skills versus tools, right? Should you be
[27:17.120 --> 27:22.640]  investing in your staff and in their skills, training, sending them to conferences like
[27:22.640 --> 27:27.820]  DEF CON where they can learn from the community what others are doing and increase their
[27:27.820 --> 27:31.320]  capabilities instead of buying tools? Thoughts on that, anyone?
[27:32.440 --> 27:43.080]  I have strong feelings. So, I think it's both. So, you really need to invest in your people but
[27:43.080 --> 27:47.960]  don't overtax them so much that you're not giving them the tools to work with so that they're
[27:47.960 --> 27:54.820]  getting alert. You don't want to have them working with the bare nothing and then getting mad
[27:54.820 --> 28:00.380]  at them when they miss something. So, you know, take the team you have and really build them up
[28:00.380 --> 28:06.040]  but then also listen to them on their level of what they'd like to use and what they'd like to
[28:06.040 --> 28:10.100]  have and use that training of sending them to conferences and coming back and gathering that
[28:10.100 --> 28:15.940]  information to inform your decisions. And I think that it really needs to be that whole cycle of
[28:15.940 --> 28:20.660]  you're investing in their education and then going to conferences and networking and letting them
[28:20.660 --> 28:26.060]  come back to you with a solution. And that is going to sometimes be investing in tools for them.
[28:27.080 --> 28:32.700]  Yeah, I agree 100%. You know, you really want to listen to your team, you know, listen to
[28:32.700 --> 28:37.980]  their experience, right? You know, get them to get out there and participate in the community
[28:37.980 --> 28:44.040]  and learn what works and, you know, take their advice on those areas, you know, and figuring
[28:44.040 --> 28:52.220]  out where to invest in your team. So I wanted to talk a little bit about IR teams and
[28:52.220 --> 28:57.280]  folks who want to get into IR. So, you know, I think they're two related topics, but a lot of
[28:57.280 --> 29:01.660]  people have asked if we could give, you know, what's your best advice for people who are interested
[29:02.200 --> 29:10.300]  in getting into the incident response field? And that's for anyone who wants to go first.
[29:10.300 --> 29:19.000]  I'll go first. So my personal opinion on how to get into IR or really kind of any info sec,
[29:19.000 --> 29:24.240]  like specific sort of doctrine at all is just to simply be curious, like ask questions,
[29:24.240 --> 29:29.260]  try to research stuff on your own, figure things out. Like you may not be the best person at
[29:29.260 --> 29:35.900]  assembly or firmware architecture or IR or malware reversing or anything like that. But
[29:35.900 --> 29:40.020]  as you find questions and, you know, find things that you're curious about and want to research
[29:40.020 --> 29:44.280]  and learn more about, like you'll generally kind of find an area that you want to go down. And
[29:44.280 --> 29:49.860]  at the end of the day, like you have to kind of decide if you want it to be a typical job
[29:49.860 --> 29:53.520]  where you work eight to five every day and show up and you get paycheck and that's it.
[29:53.520 --> 29:58.440]  Or if you want it to be a career and you actually want to be good and want to actually, you know,
[29:58.440 --> 30:02.200]  be better at what you do and be more effective and be more efficient and network with people
[30:02.200 --> 30:09.620]  and all of that. And it's not really a lifestyle decision so much as it has to be an internal
[30:09.620 --> 30:20.460]  passion and desire that you actually want to do something. Anyone else want to comment on that one?
[30:22.080 --> 30:28.420]  I want to say that you're safe, but I will, I'll chime in and say everything that Brian said
[30:28.420 --> 30:35.780]  absolutely settled down on that. But just know that there's no wrong path and that to you can
[30:35.780 --> 30:40.960]  do anything in your spare time that you're passionate about and turn it into IR. The
[30:40.960 --> 30:47.760]  difference is what I typically see in good IR is you don't stop at the first answer.
[30:47.980 --> 30:53.300]  So, and you don't stop at the second answer either. It's this stubbornness and that drive
[30:53.300 --> 30:59.620]  of being stubborn to where, okay, make yourself a home lab, image your own devices, break them. Like
[30:59.620 --> 31:06.040]  see what you can do and see what you're seeing and use that to train yourself. And it
[31:06.040 --> 31:11.960]  really is something that it takes a lot of training and that there's not just you're
[31:11.960 --> 31:17.960]  there one day. So, I'll admit myself, I get imposter syndrome a lot, but that's because
[31:17.960 --> 31:22.200]  every day I have to Google. There will be something every day that I don't know the
[31:22.200 --> 31:36.020]  answer to or I haven't seen before and I am researching the hell out of it. I'm saying
[31:36.020 --> 31:40.100]  sometimes it feels like I'm banging my head against the wall, but that doesn't go away.
[31:40.100 --> 31:44.120]  And so, if you're okay with that and you have the personality that has that drive,
[31:44.120 --> 31:49.080]  you're going to do great. And being, like listening to this right now, it shows enough
[31:49.080 --> 31:53.420]  that you're willing to listen to the five of us ramble for an hour. You're probably
[31:53.420 --> 31:57.720]  in good company and you should probably just keep pursuing this.
[31:59.920 --> 32:06.980]  One of the, on that line, you know, talking about getting into IR, talk to the, you know,
[32:06.980 --> 32:15.240]  more along the lines of the type of work and particularly about the stress. You know,
[32:16.000 --> 32:22.540]  typically IRs, you're more like a fireman. You're dealing with sometimes some high stress
[32:23.340 --> 32:30.800]  situations. And so, any comments or suggestions for the listeners around, you know, kind of
[32:30.800 --> 32:38.140]  dealing with the excitement and the stress that can come along with being part of an IR engagement?
[32:39.180 --> 32:45.160]  Well, I think one of the things that you have to take into account is, so first of all, like from
[32:45.160 --> 32:51.520]  an IR perspective, like you have to try your best to be calm. Like you can't overreact like
[32:51.520 --> 32:56.440]  everybody else, like let everybody else overreact, let everybody else run around and scream and yell
[32:56.440 --> 33:00.440]  and all of that. Like you have to be the calming voice. However, that doesn't mean internally,
[33:00.440 --> 33:04.820]  you're not screaming and yelling too, because you absolutely are. You just have to portray yourself
[33:04.820 --> 33:11.040]  as having that calming presence. And you also have to essentially, not necessarily,
[33:11.040 --> 33:16.220]  not necessarily have the right answer, but you have to have a answer because the path to recovery
[33:16.220 --> 33:19.800]  always starts with one step forward. And it doesn't matter if that step is even down the
[33:19.800 --> 33:25.540]  wrong path initially, like you just have to go forward. Because the takeaway that you can always
[33:25.540 --> 33:30.660]  have, no matter what, is somebody out there has it much worse than you currently do. And you always
[33:30.660 --> 33:35.540]  have to remind yourself, like as bad as everything is, like there's some company out there that got
[33:35.540 --> 33:41.280]  owned and they are in, unbelievably, an even worse situation right now than you are.
[33:43.400 --> 33:48.020]  I think that everybody in IR is a little bit of an adrenaline junkie,
[33:48.020 --> 33:52.000]  and that you're kind of lying to yourself if you're not. Brian, didn't you go like
[33:52.000 --> 33:55.700]  space jumping or something at one of the cons that we've been to recently?
[33:55.920 --> 34:02.100]  Yeah, I jumped off the stratosphere in Vegas a couple of years ago, twice, because I wanted to.
[34:04.540 --> 34:10.200]  I was going to take the question and ask each of you individually,
[34:10.200 --> 34:13.700]  you know, how do you get away from it? How do you deal with the stress?
[34:14.660 --> 34:18.700]  So Brian goes jumping off the stratosphere in Las Vegas.
[34:19.580 --> 34:23.480]  How about you, Moose? How do you get away from this? What are your breaks?
[34:25.760 --> 34:31.780]  Well, it's twofold, right? Like, everything I see on a daily basis,
[34:31.780 --> 34:39.420]  it's somebody else's worst day. I don't work for their company. And I had to realize this
[34:39.420 --> 34:44.540]  about myself and every job I've had. So I've not always been IT. I used to be vet tech,
[34:44.540 --> 34:48.880]  I used to be medical, I was a lifeguard. All of these things have a similar, like,
[34:48.880 --> 34:56.620]  vibe in that it's really high stress, but it involves helping. So I think, to me, my release
[34:57.080 --> 35:01.760]  is in seeing that step forward and that progression. And even if it's not happening right
[35:01.760 --> 35:06.820]  away, being able to reassure the people that I'm working with that it is truly happening,
[35:06.820 --> 35:11.540]  at the end of this, their environment will be better than we left, than it started at.
[35:11.760 --> 35:18.520]  And then, if that doesn't do it, I have cats and a violin, and on the worst day,
[35:18.520 --> 35:24.220]  I'll just shut everything down, everything gets unplugged, and I open up the violin case,
[35:24.220 --> 35:29.200]  and I walk away. And I think that's really important, is that you don't have to eat,
[35:29.200 --> 35:35.180]  sleep, and breathe this. You really should walk away sometimes. And that's what's kept me alive.
[35:38.380 --> 35:42.540]  And how about you, Virus? I mean, it sounds like you certainly have some of the adrenaline junkie
[35:42.540 --> 35:50.100]  in your experiences, in your past, breaking into buildings, you know? So what do you do for breaks?
[35:51.960 --> 35:59.220]  Shenanigans. I mean, I guess the first thing is, like, don't get stressed. Like,
[35:59.220 --> 36:03.200]  that's a big part of it. I mean, I've made a lot of... I've definitely steered a fair
[36:03.200 --> 36:09.080]  amount of my career decisionary of, like, you know, when I finish an arc, I go,
[36:09.080 --> 36:15.300]  did I like that? Or was it just stress in the bad kind of way? And then I use those takeaways to just
[36:15.300 --> 36:19.780]  not do that anymore. You know, kind of quantify how much my time is worth. Because if I spend
[36:19.780 --> 36:26.420]  the whole time when I'm not at work, stressed out about work, then I'm essentially working for free.
[36:26.420 --> 36:30.240]  So if I factor that into my paycheck, I'm like, oh, this isn't that worth it. So that's definitely
[36:30.240 --> 36:34.060]  the first step, is just don't do stuff that stresses you out. You know, know what your zeal
[36:34.060 --> 36:40.340]  is. But on a practical level, like, I played music for, like, 25, 30 years, semi-professionally. So,
[36:40.340 --> 36:45.660]  you know, I do that. I kind of zen out. I cook. I dry-age meat. I make my own gin. You know,
[36:45.660 --> 36:52.960]  I like scotch. I do a lot of drugs. And then I play with my boy. You know, I'll stand up
[36:52.960 --> 36:56.080]  from my desk when I'm working on a hard problem and just say, hey, let's go for a rock. And he's
[36:56.080 --> 37:03.060]  great. So that's amazing for him. It's great. He just runs around. But yeah, it's, I mean,
[37:03.060 --> 37:08.560]  you know, I think a lot of it is just you got to be holistic about your approach. And if you love
[37:08.560 --> 37:12.280]  the chase, you got to love the chase. And you got to love your chase. You got to love the chase
[37:12.280 --> 37:17.580]  that you're in and know where it is and just not lie to yourself. You can't, you know, do it. And
[37:17.580 --> 37:21.940]  I mean, there's definitely an undercurrent if I'm being extremely frank and extremely personal
[37:21.940 --> 37:25.860]  about it. I do think people that, I think there's a fair amount of people on the blue side,
[37:25.860 --> 37:32.620]  specifically, who have a very, I don't want to say ethical, but they have a very empathetic
[37:32.620 --> 37:39.380]  attachment to the world, slightly larger, maybe than most red teamers do, where at some level,
[37:39.380 --> 37:42.320]  they're kind of in it because they want to, they want to fix something. They want to make something
[37:42.500 --> 37:47.600]  a little better. And I don't know, I mean, I wouldn't call myself a complete prick, but
[37:48.460 --> 37:51.700]  I'm definitely on the side of like, yeah, I like money.
[37:51.700 --> 37:52.580]  Yeah.
[37:53.860 --> 37:58.880]  Long as I moved the needle a little bit and I got paid, I'm not too worried. Maybe that's a
[37:58.880 --> 38:03.880]  whole other conversation. If someone knows you from the outside, you're one of the nicest people
[38:03.880 --> 38:13.160]  I know, so that's a lie. If we could get back on the topic of, you know, of the IR field and IR
[38:13.160 --> 38:20.260]  teams, we had a question in the BTV discord about, you know, what are your top recommendations,
[38:20.260 --> 38:27.880]  and maybe some don'ts, you know, for growing your SOC and your IR team? You know,
[38:27.880 --> 38:33.240]  if you're at a company and, you know, it's growing and, you know, you get budget to
[38:33.240 --> 38:39.600]  have some full-time folks doing security operations and, you know, skills and tools
[38:39.600 --> 38:44.260]  ready to do incident response, you know, where do you want to focus? You know, what skills do
[38:44.260 --> 38:49.560]  the folks need to have? What are some of your preferred tools? And I know it depends a lot,
[38:49.560 --> 38:53.920]  but just in general, you know, where would you focus? Brian, why don't you go first?
[38:54.980 --> 39:02.500]  Well, I think it goes back to... it very much does depend, but it also depends on what you
[39:02.500 --> 39:07.460]  currently have already and essentially what you have the ability... what you want to essentially
[39:07.460 --> 39:13.300]  try to strive for. Like, if you want to go a very, very, you know, cheap route as much as possible,
[39:13.300 --> 39:17.880]  you would use something like OS query or use something like Velociraptor and, you know,
[39:17.880 --> 39:22.140]  try to essentially keep it on the cheaper side. But, you know, at the end of the day,
[39:22.140 --> 39:26.100]  it's still going to be able to work. Of course, if you have unlimited budget, like, you just line
[39:26.100 --> 39:30.060]  up boxes, you know, like crazy, and then you have all the tools and everything. But, you know,
[39:30.060 --> 39:35.560]  to be completely honest, that's not feasible for anybody. Like, companies will not just blindly
[39:35.560 --> 39:40.920]  give you ridiculous amounts of money to spend on people and tools and resources. So, you have to
[39:40.920 --> 39:45.860]  try to make whatever you have work with what you can. And that's one of the things that I stress.
[39:45.860 --> 39:50.780]  It doesn't necessarily matter what the dollar amount is. It's trying to make every dollar
[39:50.780 --> 39:55.380]  count as much as possible. So, that way, at the end of the day, you end up with a truly better
[39:55.380 --> 40:00.240]  result. If you have a very high budget, great. Like, you can afford expensive tools. But do you
[40:00.240 --> 40:04.480]  really need those expensive tools? Do you really need an expensive person to come in maybe for,
[40:04.480 --> 40:09.340]  like, you know, three, six months or something like that and help train and make the team better
[40:09.340 --> 40:13.640]  to already use the stuff that they already have? You know, in that case, you could argue very much
[40:13.640 --> 40:18.660]  that bringing somebody in external for a little bit and helping them not really as a staff log,
[40:18.660 --> 40:23.760]  more of kind of like an overseeing mentorship almost. Like, that is a much better investment
[40:23.760 --> 40:29.460]  than just blindly buying, you know, a million dollars worth of random tools. So, it truly does
[40:29.460 --> 40:34.340]  depend. But, you know, it's like everything else. Like, you have to make a business decision about
[40:34.340 --> 40:40.120]  it and essentially try to maximize every dollar and get the best value for that as much as possible.
[40:43.040 --> 40:51.320]  You know, and we've used the word a few times in this panel, telemetry. You know, so,
[40:51.320 --> 40:56.600]  from my perspective, one of the first things that I would want to do is, you know, is set all that
[40:56.600 --> 41:02.440]  up, is set up, you know, a central logging system, whether it's SIEM, whether it's, you know,
[41:02.440 --> 41:09.840]  something much simpler. But, you know, for me, doing incident response, you know, those are the
[41:09.840 --> 41:15.400]  keys, right? I mean, I've heard it said, you know, logs are everything. And that that would be one
[41:15.400 --> 41:20.820]  of your first steps would be to make sure that you, you know, you're gathering enough information
[41:20.820 --> 41:26.420]  so that, you know, when an incident happens and just in general, you know, having logs has so
[41:26.420 --> 41:33.260]  much value for IT support, for everything that we do in security. My approach to this is really,
[41:33.260 --> 41:41.320]  it's like a three-step. It's like a three-layer moniker. So, I think the first thing that it's
[41:41.320 --> 41:47.660]  important to grasp is that a program, we talk about a program, whether it's an IR program or
[41:47.800 --> 41:52.720]  a security program, large or blue team program, a program is business process. That's what a
[41:52.720 --> 41:59.320]  program is. It's not blinky boxes. It's not even the talent in your bullpen. It's documented process.
[41:59.320 --> 42:04.880]  That is the definition of your program. So, the first rule is if you're going to build a program,
[42:04.880 --> 42:10.580]  build a program. That means encourage your people to, and this kind of leads into the second thing,
[42:10.580 --> 42:16.560]  which is tribal knowledge is the enemy, right? So, like the first job of everybody on the team
[42:16.560 --> 42:21.600]  is to take everything in their head and inject it into business process. That is your job. It
[42:21.600 --> 42:26.460]  doesn't matter if you are, you know, the self-proclaimed bottom rung of the team or if
[42:26.460 --> 42:31.300]  you're the epic gangster rock star at the top. Your job is to take whatever is in your head that
[42:31.300 --> 42:36.220]  applies to the program and to build the program. And the more that you encourage people to do that
[42:36.220 --> 42:41.720]  and the more that you reward them for doing that, it's like a self-incentivizing loop, right? That's
[42:41.720 --> 42:46.980]  how you balance burnout and all this other stuff because it's not focused on one person or one
[42:46.980 --> 42:52.380]  tool. And then, like I said, the second thing is, and I feel like this is way more common on red
[42:52.380 --> 42:56.260]  teams than it is on blue teams, and I kind of don't know why. It is extremely common on the
[42:56.260 --> 43:01.040]  red side to not use tools that are touted as being the latest and greatest thing just because you
[43:01.040 --> 43:06.100]  don't know them, right? I mean, as a red teamer, I feel like there's this moniker that is
[43:06.520 --> 43:10.320]  accepted more and I feel like it's true on both sides. So, I kind of don't understand why it's a
[43:10.320 --> 43:14.740]  lopsided. We're like, you know, it's the old ninja moniker, right? We're like, not every ninja uses
[43:14.740 --> 43:19.120]  every weapon, right? You have your weapon and it's got your epic moves in it and you know it like the
[43:19.120 --> 43:23.000]  back of your hand. It's almost an extension of yourself and that's the point. I don't really
[43:23.000 --> 43:28.040]  think that's different on blue team. I mean, case in point, the team I'm part of at Stripe that kind
[43:28.040 --> 43:31.880]  of got built about a year and a half ago, we were all very new to the company and was, you know, granted
[43:31.880 --> 43:36.260]  we were very fortunate to have leadership with a lot of experience and so they reached out and
[43:36.260 --> 43:40.940]  you know, there's a lot, there's an extremely high amount of experience and talent on my team that's
[43:41.080 --> 43:46.300]  a little bit askew for most. I'm not saying that to brag, I'm just saying, you know, a lot of
[43:46.300 --> 43:52.200]  teams don't have that in their tool bin. But the first thing we did was take what EDR we had
[43:52.200 --> 43:56.480]  and rip it out. We ripped out almost everything and then built everything off of bare bones telemetry
[43:56.480 --> 44:02.040]  because those were the systems we could use better. That would, that yielded better to our kung fu.
[44:02.040 --> 44:09.640]  And then from there, the third thing is build, like tool your kung fu. You know, take your
[44:09.640 --> 44:14.860]  brain dump, put it into your program and then hypothetically start from scratch and go
[44:14.860 --> 44:21.040]  what tools do we need to do this, what's written better? Frankly, every deviation from that is a
[44:21.040 --> 44:27.080]  waste of time and money. So you said that you ripped out your EDR and you built up bare bones telemetry
[44:27.080 --> 44:32.380]  are the things that are most useful to you. Can you elaborate on what those are? I mean, it's literally
[44:32.380 --> 44:37.660]  just stuff that's built into the OS. I mean, we use OS query at scale so that because we use the
[44:37.660 --> 44:42.580]  fleet manager concept in order to kind of achieve orchestration goals. But our detection system
[44:42.580 --> 44:48.960]  is entirely homegrown. Our response system is entirely homegrown. Our orchestration elements
[44:48.960 --> 44:53.320]  and the way that we do log tracking is entirely homegrown. Even our pace reporting. I mean,
[44:53.320 --> 44:57.320]  we're kind of sort of bubbling around the hive right now, but it's honestly, it's all Jupyter
[44:57.320 --> 45:02.960]  notebooks and Python. It's all just kind of manual tooling. And at the end of the day,
[45:02.960 --> 45:09.580]  if I take that and I put that in the ROI bin, we spent way less on human time writing some Python
[45:09.580 --> 45:15.500]  than we would ever spend on third-party utilities. And to be honest, most of the third-party
[45:15.500 --> 45:19.880]  utilities are trash. I mean, I won't pull any out just because I don't want anybody to
[45:20.500 --> 45:25.340]  kind of, you know, have to be under the boot because I said something, but like,
[45:25.340 --> 45:29.740]  man, there's just so much money spent on these tools that just don't even do what they say on
[45:29.740 --> 45:34.480]  the side of the box. I just, I don't understand why that flies. And then that's just, you know,
[45:34.480 --> 45:38.280]  I don't know. Say what you will about red team or blue team infra, at least the red team stuff does
[45:38.280 --> 45:41.840]  what it says. It just doesn't say a lot. Blue team stuff, man, this seems to be a problem.
[45:41.840 --> 45:44.340]  Cereal box, nutrient banks are just wrong.
[45:48.420 --> 45:55.300]  So, Rubix, you've built an IR team, right? I'm curious, you know, your thoughts on all this.
[45:56.220 --> 46:03.240]  Yeah, you know, it was, I always go back to the, you know, kind of build it using the fireman
[46:03.240 --> 46:11.340]  analogy. So IR for, you know, a, you know, we're in financial services, there's a whole lot of
[46:11.340 --> 46:19.700]  downtime. And so figuring out how to practice and be ready for the big event and giving them
[46:19.700 --> 46:26.320]  enough work of a real, you know, I guess, you know, smaller level incidents to be able to stay busy
[46:26.320 --> 46:32.920]  and relevant, but then also, you know, plenty of time to prepare for, you know, bigger stuff. And
[46:32.920 --> 46:41.780]  so, you know, lots and lots of practices, you know, tabletops, drills, and, you know,
[46:41.780 --> 46:50.600]  encouraging our red team to really give us some good scenarios. And so I think that if you are,
[46:51.360 --> 46:59.720]  you know, not balancing that, you know, if your guys are all, or folks are always spending time,
[46:59.720 --> 47:06.920]  you know, doing, you know, we talk about being really busy, that they might not be honing their
[47:06.920 --> 47:11.180]  skills enough. And so, you know, finding that balance between, you know, doing and training
[47:11.180 --> 47:25.280]  and practicing was very key. I think the one thing I have to add to that, and everybody else
[47:25.280 --> 47:31.460]  has given really good advice here. And this is a part that I'm really passionate about, because
[47:31.460 --> 47:36.780]  it's one of the reasons that we lead companies. Don't silo your people and listen to what they
[47:36.780 --> 47:42.720]  have to say. If you have somebody on your team, it doesn't matter what notch in the chain they are.
[47:42.800 --> 47:48.280]  If you're calling on them for some kind of instant response, whether it be a SOC person,
[47:48.280 --> 47:53.580]  somebody who's doing sys admin work, someone who's a network admin, you're trusting them
[47:53.580 --> 48:01.060]  with that role. Trust them if they say they have a passion in response as well. Let them
[48:01.060 --> 48:07.580]  play in other fields, because there is not a silo in incident response for our attackers,
[48:07.580 --> 48:12.820]  so there really shouldn't be a silo for our responders. And one of the reasons that I work
[48:12.820 --> 48:19.320]  where I work is because there are some days where I'm not incident response. I have the freedom to
[48:19.320 --> 48:25.780]  do red team stuff. I have the freedom to help build our tools if I want to, or scripts. Some
[48:25.780 --> 48:31.500]  of the most brilliant scripts I've seen to date were built by my teammates in response to the
[48:31.500 --> 48:38.600]  Citrix vulnerability, for instance. So, you know, knowing where your strengths are on your team
[48:38.600 --> 48:44.100]  and building those, and then listening to them and what they say they need. Because, you know,
[48:44.100 --> 48:55.720]  if you're the one building, that assumes that you're at a higher level position,
[48:55.720 --> 49:02.200]  and I think that's really key, is to... my cats are wrecking the apartment right now,
[49:02.200 --> 49:06.780]  so if you hear anything bad, that's what I keep looking over for, is anytime I'm on one of these,
[49:06.780 --> 49:12.880]  they wreck something. So anyways, but listening to those people that you've hired, and obviously
[49:12.880 --> 49:18.140]  you trusted them enough in the first place to hire them. They're smart people, and we're smarter
[49:18.140 --> 49:26.560]  not because we know something, but because we know who to go to when we need certain knowledge.
[49:26.560 --> 49:32.800]  And I think leaning on your teammates, no matter what level they are in your company,
[49:32.800 --> 49:38.080]  having that kind of group decision is still important.
[49:38.720 --> 49:45.620]  Yeah, I think that's really great advice, you know, that more companies should heed,
[49:45.620 --> 49:53.300]  to try to cross-train their folks and give folks time for training and for research, right,
[49:53.300 --> 50:01.100]  and have budget for tools and devices, right, and lab time. I think it's really important.
[50:02.420 --> 50:13.540]  So I wanted to pivot and talk about ransomware. So we had a passionate discussion about this
[50:13.540 --> 50:19.580]  yesterday, and I'd like to try to rehash some of that for our attendees. And to kick it off,
[50:19.580 --> 50:27.340]  earlier today, the Blue Team Village Twitter, we posted a poll, and we were looking for people
[50:27.340 --> 50:33.800]  to give us their opinions. The poll question is, your organization is the victim of a ransomware
[50:33.800 --> 50:43.280]  attack. Should you pay the ransom? And about 89% of the responders said no, and about 11% said yes.
[50:44.080 --> 50:50.740]  I think, you know, in summary of our discussion yesterday is, it's not that simple. But,
[50:50.740 --> 50:55.060]  you know, for the benefit of everyone who wasn't in our private discussion yesterday,
[50:55.440 --> 51:01.380]  you know, what goes into the decision of paying the ransom or not? And I think
[51:01.380 --> 51:04.820]  Moose started first yesterday, if you want to go first again today.
[51:06.220 --> 51:13.560]  Sure. This is the most, like, let me tell you right now, this question made my blood pressure,
[51:13.560 --> 51:18.780]  like, go up a little bit. Because for me, when somebody says, do I pay the ransom?
[51:19.100 --> 51:23.800]  On any call I have, or even if I've been in person, I look at them and say,
[51:23.800 --> 51:27.520]  well, what does your legal team say? And that's still kind of my answer,
[51:27.520 --> 51:33.340]  is that I don't think you should make that decision for your business in a vacuum. And
[51:33.340 --> 51:40.500]  I usually recommend that people talk to either their internal counsel or have external counsel
[51:40.500 --> 51:47.680]  involved in that decision making process. And not necessarily make a decision for themselves.
[51:48.540 --> 51:53.080]  Because these are, if you look at it this way, the way I like to frame it,
[51:53.080 --> 51:56.460]  because I'm not going to say yes or no, spoiler alert, I'm not going to say yes or no,
[51:56.460 --> 52:01.380]  because it really does depend. And that's not my decision as a responder to make.
[52:01.380 --> 52:06.420]  But I think that the most intelligent things I've seen done was calling on experts who do
[52:06.420 --> 52:11.660]  this every day. Because if you think about who hit you, you're not their first hit.
[52:12.000 --> 52:17.840]  You're not the first people, they've probably ransomed, they've hit other companies. These
[52:17.840 --> 52:22.420]  are professional criminals that are going after you almost all the time, if they're exporting you
[52:22.420 --> 52:28.780]  for something. So get a professional to represent you. That's kind of my opinion. And I will preface
[52:28.780 --> 52:40.640]  that with that as well. Okay. Brian, you kind of immediately had a, you know, from the heart
[52:40.640 --> 52:47.600]  response that you then elaborated on. Could you share that with all of us? So as Brian Moran,
[52:47.600 --> 52:54.900]  human being, my first response is always no. However, that is simply me as a human being,
[52:54.900 --> 53:02.040]  because I don't feel that it's right to pay criminals for doing a crime. However,
[53:02.040 --> 53:08.180]  we'll preface all of that with put on the consultant hat, the business process hat and
[53:08.180 --> 53:14.420]  all of that. And honestly, the answer is more than likely, it's usually cheaper to pay the ransom
[53:14.420 --> 53:21.940]  than it is to deal with the cost of everything else involved. Like, it should never, just like
[53:21.940 --> 53:28.400]  it should never be anybody on the InfoSec team ever. It should never, ever be their call. It
[53:28.400 --> 53:32.700]  should be legal's call because legal absolutely needs to be involved. Not just from the perspective
[53:32.700 --> 53:37.400]  of, is this okay with the company? Also from this perspective of, you know, you're potentially
[53:37.400 --> 53:42.200]  looking at paying somebody in another country, like, are there sanctions against that country?
[53:42.200 --> 53:48.300]  Like right there, you could violate even more laws just by paying the ransom. So now you're in
[53:48.300 --> 53:53.800]  double because all your debt is gone and now you're getting sanctions and fines and investigations
[53:53.800 --> 53:58.180]  levied on you because you are literally, you know, potentially financing something that you shouldn't
[53:58.180 --> 54:04.560]  be financing. So it's not our call to make at the end of the day. Like, you know, we can respond and
[54:04.560 --> 54:09.660]  say, here's X, Y, and Z, here's our thought process. But generally from the overall business
[54:09.660 --> 54:13.960]  perspective, it's usually much cheaper just to pay the ransom and be done with it than to deal
[54:13.960 --> 54:19.120]  with any of the public scrutiny, deal with any of that. But, you know, we're not lawyers.
[54:19.120 --> 54:23.500]  I don't think any of us are even going to pretend ever to be a lawyer. Like, even if we say at a
[54:23.500 --> 54:28.720]  Holiday Inn Express, we are definitely not lawyers by any stretch of the bean. So we can have our
[54:28.720 --> 54:33.240]  opinion and we can give you thoughtful, evaluated opinions, but like our actual thoughts at the end
[54:33.240 --> 54:38.460]  of the day truly don't matter because it 100% decides on legal and what the course forward
[54:38.460 --> 54:46.040]  is from them. Yeah, thank you. You know, I think a lot of folks have that reaction of, no,
[54:46.040 --> 54:50.520]  you know, I don't want to pay criminals, but there is so much to the decision to be made.
[54:50.920 --> 54:56.400]  And Virus, you came at this from a different angle than I've ever really thought about it.
[54:56.400 --> 55:03.760]  And that angle was about ransoms and kidnappings and kind of relating ransomware to that type of
[55:03.760 --> 55:09.460]  crime. Could you, could you rehash that for everyone? So I basically distill everything
[55:10.000 --> 55:18.000]  that I tried to kind of get across in our joint meeting as ransomware is a crisis. The breach
[55:18.000 --> 55:23.580]  that put together is an incident. So if you have a flat out answer on whether or not you should pay
[55:23.580 --> 55:28.180]  the ransom when it comes to ransomware, you're doing it wrong because that's not how crisis
[55:28.180 --> 55:37.880]  management works. And I don't understand why as even an industry InfoSec would try to solve
[55:37.880 --> 55:45.860]  crisis management when every major Western power, every branch of every military and military
[55:45.860 --> 55:51.620]  contractor that's worth their salt, every political, you know, wing contractor, I mean,
[55:51.620 --> 55:58.780]  crisis management is a well understood science that has been researched by every tactical
[55:58.780 --> 56:06.120]  engagement and business risk school in the world forever. And the fact that the thing that's being
[56:06.120 --> 56:11.480]  ransomed is a piece of electronic capital versus human capital does not change the game. I just
[56:11.480 --> 56:16.880]  don't understand why people would insist on coming up with a pat answer for that. I think it's stupid.
[56:16.880 --> 56:22.920]  And while I think that, you know, the response to a ransomware crisis, because it is a crisis,
[56:22.920 --> 56:29.020]  will likely involve legal, I don't necessarily think that legal is the end all be all shop on
[56:29.020 --> 56:32.980]  that. It probably is at most shops. But you know what? The world is bigger than the law.
[56:33.780 --> 56:39.480]  You know, I could tell you all day long about, and all of you could too, about every major
[56:39.480 --> 56:44.580]  insert fortune 500 company here, where they have a crisis management team that has nothing to do
[56:44.580 --> 56:49.080]  with computers. And I guarantee you, the first stop they call when the CEO gets kidnapped by,
[56:49.080 --> 56:53.040]  you know, random activists in South America on a business trip is not the lawyer. They're like
[56:53.040 --> 56:58.040]  the third. So I don't know why it's different because it's a computer and not them. And then
[56:58.040 --> 57:02.700]  after you handle the crisis, however you handle it, maybe it's paying, maybe it's not, maybe it's
[57:02.700 --> 57:07.620]  using shadow currency. I don't know. Right? Then you handle the incident, which is if you have
[57:07.620 --> 57:12.300]  ransomware, you were breached. Don't handle that. That's your incident. The ransomware, that's not
[57:12.300 --> 57:21.460]  an incident. That's a crisis. With the shift of a lot of ransomware actors, for example, Maze,
[57:21.460 --> 57:28.920]  to not only just destroy anymore, but also, you know, steal the data, threaten to, you know,
[57:28.920 --> 57:36.860]  do more stuff with it than just destroy it. Has that changed your way of dealing with ransomware?
[57:40.370 --> 57:47.610]  It has in my advice. So I'll go ahead and speak to that in that I have
[57:49.010 --> 57:54.810]  given a little bit of advice, you know, working with a legal team, of course, if it makes sense
[57:54.810 --> 58:01.110]  for the company and with their backing. So all of that included. So assume that to from the get-go.
[58:01.110 --> 58:05.870]  But, you know, a lot of these groups that do extortion, they're saying that they have something
[58:05.870 --> 58:12.950]  and usually they'll serve up what they have. If there is at all 24 hours
[58:12.950 --> 58:19.610]  that you can buy your IR team to investigate if that claim is true or not,
[58:19.610 --> 58:27.970]  negotiate and buy the team time. If you can figure out if what they say was exposed is true,
[58:27.970 --> 58:33.910]  and if there's any more to that, so if they've gotten more than they say they have,
[58:33.910 --> 58:41.170]  take that time and lean on your investigations team to see that extra piece and see if you can
[58:41.170 --> 58:46.510]  see that creation and that within your system. And that's a very hard thing to do in a short
[58:46.510 --> 58:53.430]  amount of time, especially because it assumes that you have full visibility at every endpoint
[58:53.990 --> 58:59.270]  on the network of your company. And to have that kind of visibility to begin with
[59:00.070 --> 59:07.470]  is incredibly difficult for your team, let alone the outside team. So I would say that
[59:07.470 --> 59:12.470]  the good things that come out can come out of that, not that I'm for negotiating with jurists,
[59:12.470 --> 59:20.050]  so to speak, that are withholding your data and extorting you for money. But, you know,
[59:20.050 --> 59:27.210]  sometimes I've seen outside counsel get a lower price so that if you are going to pay,
[59:27.910 --> 59:33.810]  but time is valuable too, and having that on your side is important.
[59:37.410 --> 59:45.370]  Yeah, I think one of the things that is hugely important, but also at the same time incredibly
[59:45.370 --> 59:51.970]  underrated, is the way that ransomware works. Ransomware is a business. Just like any other
[59:51.970 --> 59:58.690]  business, ransomware at the end of the day is just a business. And I don't recall seeing
[59:58.690 --> 01:00:03.310]  personally, and I'm sure it's happened, but I don't recall a case personally where the actors
[01:00:03.310 --> 01:00:09.450]  have claimed to have something and they truly did not, that they were bluffing. Because if you're
[01:00:09.450 --> 01:00:13.330]  running a successful business, you're not going to lie about what you have and everything. Well,
[01:00:13.330 --> 01:00:17.670]  unless you're Enron and stuff like that. But for the most part, you're not going to actually
[01:00:17.670 --> 01:00:24.430]  not tell the truth about certain things. So if they claim that they exfil data and you as a
[01:00:24.430 --> 01:00:28.450]  company have no way to prove that, like number one, you're already in way worse problems than
[01:00:28.450 --> 01:00:34.330]  having ransomware. Like you have so many other problems that you need to deal with. But if they
[01:00:34.330 --> 01:00:38.470]  claim that they have something and they more than likely do, and then you have to evaluate the cost
[01:00:38.470 --> 01:00:42.310]  of all that. Well, what if that data goes public? What if that data goes even more publicly
[01:00:42.310 --> 01:00:46.250]  widespread? What if that data is then sold to other places and everything like that? And you
[01:00:46.250 --> 01:00:52.330]  have to take all of that into account with the entire process. And we often try to look at it
[01:00:52.330 --> 01:00:58.230]  much as a pure like, essentially, is there a correct answer? Or is there not a correct answer?
[01:00:58.230 --> 01:01:03.970]  Is it a yes? Or is it a no? And it's never just a yes or no. There are so many factors that you
[01:01:03.970 --> 01:01:09.490]  have to take into account all of that through it. And we in the information securities field,
[01:01:09.490 --> 01:01:14.850]  we only see a very small portion of all of that. So you have to involve other people,
[01:01:14.850 --> 01:01:21.650]  you have to involve other people in the process. But just like it was said, if you have ransomware,
[01:01:21.650 --> 01:01:28.390]  I guarantee you have at least five other problems besides just how ransomware got there. You have a
[01:01:28.390 --> 01:01:33.290]  lot of problems that need to be addressed. Ransomware is the big public facing thing on
[01:01:33.290 --> 01:01:38.850]  all of that. But there are so many problems that need to be addressed internally. And then you as
[01:01:39.490 --> 01:01:43.930]  the practitioner have to take into account, all right, here's my little take on the ransomware,
[01:01:43.930 --> 01:01:47.770]  here's all that. And then essentially, that decision goes to somebody else. But
[01:01:49.010 --> 01:01:53.250]  ransomware, I've never seen anywhere where like, oh, well, we got infected with ransomware. And
[01:01:53.250 --> 01:01:57.050]  that's the only problem that they have. There are many, many other gaping holes that they absolutely
[01:01:57.050 --> 01:02:05.170]  have. I can guarantee that. And on that note, unless somebody else wanted to say something
[01:02:05.170 --> 01:02:11.770]  else on ransomware virus, were you about to speak up? I was just going to answer Rubik's question.
[01:02:11.770 --> 01:02:16.910]  Yeah, go for it. If I understand the question correctly, it was in a nutshell does introducing
[01:02:16.910 --> 01:02:22.430]  extortion into the ransomware topic change the game? So my answer to that would be no,
[01:02:22.430 --> 01:02:27.710]  at least not for me, because I've already given my two cents on how I think that the orchestration of
[01:02:27.710 --> 01:02:34.590]  the conversation on ransomware is wrong. In terms of have I seen it change the game?
[01:02:35.290 --> 01:02:40.390]  I would say yes. And I would say the way that I think it is changing the game is that
[01:02:41.650 --> 01:02:45.930]  counterintelligence is a thing. Counterterrorism is a thing. Black on black kill squads are a thing.
[01:02:45.970 --> 01:02:50.570]  And they have always been a thing in blue team. They just haven't been very talked about.
[01:02:50.950 --> 01:02:56.150]  Because until words like hacking backs that are being introduced into the info set conversation,
[01:02:56.150 --> 01:03:01.090]  those were mostly gray areas to a lot of professionals. My experience is that the
[01:03:01.710 --> 01:03:05.250]  reintroduction, because I have seen it before, especially dealing with like adult fields,
[01:03:05.250 --> 01:03:09.410]  it's way more common than what I've seen in corporate space. And now I'm seeing it a lot
[01:03:09.410 --> 01:03:16.630]  more in the corporate space. I think those areas are getting a lot less gray. And they're showing
[01:03:16.630 --> 01:03:20.810]  up on the radar of a lot of risk teams, because they're starting to realize that that's how you
[01:03:20.810 --> 01:03:28.390]  fight this. But again, is my prescriptive advice that there should be more of that? No, that
[01:03:28.390 --> 01:03:35.730]  doesn't answer the question. Awesome. Thank you. Any other comments on ransomware?
[01:03:37.510 --> 01:03:45.570]  I will say that the other thing that I've seen is ransomware used to be kind of a singular
[01:03:45.570 --> 01:03:50.850]  program that executes and they've got this programmatic way of doing destruction,
[01:03:50.850 --> 01:03:59.210]  is that now when you hear of ransomware, it's a full-on incident. They have a number of backdoors,
[01:03:59.210 --> 01:04:04.690]  they have a number of accounts, right? You can't just go and contain it and say,
[01:04:05.510 --> 01:04:10.550]  we found ransomware on five machines, we've re-imaged those five machines, we're good.
[01:04:10.710 --> 01:04:16.490]  Now they're going to have spent some time in your environment, stolen accounts,
[01:04:17.050 --> 01:04:22.990]  have other ways of getting back in. And so you really have to do a full incident response,
[01:04:22.990 --> 01:04:30.450]  find out all the things that the attacker got. It's no longer a simple case like we used to deal
[01:04:30.450 --> 01:04:39.190]  with. The only other thing that I would add, this is the hill I die on, because I'm one of our,
[01:04:40.250 --> 01:04:47.450]  say, Linux enthusiasts, where I work, is most people associate ransomware with Windows.
[01:04:48.010 --> 01:04:52.470]  And while that's the case, that doesn't mean you should ignore your Linux environment and
[01:04:52.470 --> 01:05:00.530]  never assume. So make sure that you're building that up and you're looking at it, and don't ever
[01:05:00.530 --> 01:05:09.650]  assume that it's just, you know, if I have access to your environment and I'm a bad guy,
[01:05:09.650 --> 01:05:16.410]  if you give me an easy pivot point, why shouldn't I do good? So kind of have that in the back of
[01:05:16.410 --> 01:05:24.890]  your mind. And don't, like, go tunnel vision only on your Windows systems. And don't make it easier.
[01:05:27.440 --> 01:05:29.440]  Sorry, Virus, I didn't mean to.
[01:05:30.800 --> 01:05:35.280]  I just never heard anyone say that ransomware is a Windows thing. I'm like,
[01:05:35.280 --> 01:05:41.980]  there's been ransomware on mobile phone boot roms since, like, 2001. Who are these people?
[01:05:41.980 --> 01:05:50.920]  I hear it. I do. I hear it. I hear that this usually only hits Windows and look at the
[01:05:50.920 --> 01:05:57.880]  Windows environment and all that. And sometimes that's absolutely true. But you should be scanning
[01:05:57.880 --> 01:06:02.640]  for lateral movement to everything. Don't assume anything. What about Max? I thought Max were immune
[01:06:02.640 --> 01:06:12.060]  from viruses. Mainframe. Move everything to the mainframe. There's no mainframe viruses.
[01:06:12.180 --> 01:06:17.040]  Yeah. So we've already gone over time a little bit, but I think we have about 10 minutes left.
[01:06:17.060 --> 01:06:21.340]  And so I wanted to kind of reward the folks who have stuck around through all of this very,
[01:06:21.340 --> 01:06:27.820]  very serious conversation on incident response and try to end on a lighter note. And so I was
[01:06:27.820 --> 01:06:33.400]  going to ask each of you if you could share one good story about, like, one of the craziest things
[01:06:33.400 --> 01:06:39.240]  that you've seen, whether it was on a response or, you know, in some of your own actions,
[01:06:39.240 --> 01:06:47.900]  Varys. And I will remind you that this is being recorded. So who wants to go first?
[01:06:51.640 --> 01:06:58.940]  Anybody got one queued up? I have one kind of queued up a little bit. So this is actually a
[01:06:58.940 --> 01:07:04.440]  combination of two. So one is the worst thing that I've ever seen in an environment.
[01:07:04.540 --> 01:07:10.740]  The worst thing that I've ever seen in an environment was there's a firewall company,
[01:07:10.740 --> 01:07:16.860]  who is a very, very popular, very, very known firewall company. And for whatever reason,
[01:07:16.860 --> 01:07:22.340]  they somehow implemented all of these firewall rules that were okay. They weren't great. They
[01:07:22.340 --> 01:07:27.680]  were just okay. And then somebody put in a triple anti-firewall rule. Not a double,
[01:07:27.680 --> 01:07:32.940]  an actual triple any. So it was any, any, any. Come to find out from speaking with someone who
[01:07:32.940 --> 01:07:37.480]  will rename nameless, but is on this panel, that company actually allows you to have as many as
[01:07:37.480 --> 01:07:44.160]  five anys. So not only is there a problem with five anys, like even the option for you to be
[01:07:44.160 --> 01:07:48.540]  able to put that many anys in something, like, that doesn't make any sense. Like, what are they
[01:07:48.540 --> 01:07:53.400]  possibly thinking? Like, we're just going to make sure no matter what, that you can actually find
[01:07:53.400 --> 01:08:01.800]  some way through our firewall. So the hybrid of that is it was with the same client. It was kind
[01:08:01.800 --> 01:08:06.820]  of a cool story, because we had a chance to talk about it, I believe, last year in New Orleans,
[01:08:06.820 --> 01:08:13.220]  because the actual indictment came down. So they were a part of a breach that was essentially
[01:08:13.220 --> 01:08:18.780]  targeting the aerospace industry. And the attackers did all kinds of stuff. So we were able to actually
[01:08:18.780 --> 01:08:23.780]  go through essentially, like, their Gen 1 through Gen 5 of their malware, including, like, the last
[01:08:23.780 --> 01:08:28.740]  ditch absolute effort where it was incredibly sophisticated, because it was worth burning
[01:08:28.740 --> 01:08:34.120]  something that was amazing on it. And when I found out, like, that was just incredible. It was amazing.
[01:08:34.120 --> 01:08:37.840]  I actually did a little dance, because like, all right, now we found this. Now we have to find out
[01:08:37.840 --> 01:08:42.400]  how we're actually going to get rid of it. But the way that we're able to track back to exactly
[01:08:42.400 --> 01:08:48.040]  where it was is, again, because attackers are human beings, and they make mistakes. So for the most
[01:08:48.040 --> 01:08:53.280]  part, they bounce their IP through other countries, and everything was just fine.
[01:08:53.280 --> 01:08:59.180]  Except we had a total of 11 instances over the course of, I believe, it was a three-month period
[01:08:59.180 --> 01:09:05.240]  where they forgot, or they just made the connection before it actually made all the hops.
[01:09:05.240 --> 01:09:09.280]  And we had 11 times where we were actually able to pin down pretty much to the exact building
[01:09:09.280 --> 01:09:15.340]  in a certain company that rhymes with Rhina of exactly where the attacker was, exactly what
[01:09:15.340 --> 01:09:20.960]  they were doing, and exactly what they were after. And that was amazing, because we literally
[01:09:20.960 --> 01:09:26.960]  had gigabytes of logs of all this access in, and in, and in, and in. And then we happened to find,
[01:09:26.960 --> 01:09:31.620]  like, oh, well, here's this web shell that they had put in. The only person to access it was this
[01:09:31.620 --> 01:09:36.780]  person. We know that for sure. And 11 times, they just made a mistake, and we were able to pin down
[01:09:36.780 --> 01:09:42.280]  where they were. So it literally takes just one time of messing up, and like, okay, now we can
[01:09:42.280 --> 01:09:47.660]  actually build an investigation. And we turned all that over to law enforcement. And unfortunately,
[01:09:47.660 --> 01:09:51.540]  it took them about five years to actually go through the process, because they had so much
[01:09:51.540 --> 01:09:56.200]  stuff to investigate, primarily for our case. But it was also kind of disheartening to see,
[01:09:56.200 --> 01:10:00.680]  like, flash reports get put out that is essentially the summary of our report that we had written,
[01:10:00.680 --> 01:10:06.640]  and just has an agency logo at the top of it. Like, it's like, that's my stuff. Like, don't do that.
[01:10:11.780 --> 01:10:14.040]  All right. So who can go next?
[01:10:15.520 --> 01:10:22.100]  Oh, man. I guess I should follow, just because, Brian, at the time, I had told you, I saw five
[01:10:22.100 --> 01:10:27.820]  enemies on a firewall, and I was like, what in the hell? And it was like a horror for me, because I
[01:10:27.820 --> 01:10:32.740]  was just like, it was at the top, too. So if you think of how firewalls work, if you've ever
[01:10:32.740 --> 01:10:38.000]  done any kind of architecture on them, it's top-down. And it was five, and it's
[01:10:39.300 --> 01:10:46.960]  any source, any destination, any application, any user. And I don't know which one I left out.
[01:10:49.060 --> 01:10:53.640]  Location and service. There are actually six that can be set, and they set one as
[01:10:53.640 --> 01:10:56.820]  deny. And I didn't even know why they set one as deny, because I was just like,
[01:10:56.820 --> 01:11:00.740]  it basically just works. That's the answer I got, by the way, was it just works.
[01:11:01.940 --> 01:11:09.160]  But my worst story is actually recent. I have a lot of empathy. I see a lot of stuff in
[01:11:09.160 --> 01:11:15.040]  ransomware, but one of the coolest things that I've worked a couple of was not too long ago,
[01:11:15.040 --> 01:11:22.880]  there was a salt stack vulnerability that came out that allowed for, basically, the long story
[01:11:22.880 --> 01:11:29.040]  short is it allowed you to get root to anything that was exposed that was a salt master. So
[01:11:30.600 --> 01:11:36.020]  a salt is used, salt stack is used to manage and administer a Linux environment.
[01:11:36.760 --> 01:11:42.100]  And it's a really cool thing. I actually really like it for administration purposes.
[01:11:42.620 --> 01:11:46.920]  But there was a vulnerability, and if it wasn't patched, you could get root into anything that
[01:11:46.920 --> 01:11:53.200]  was internet exposed. And I just happened to see it in a couple of cases. I thought going into this,
[01:11:53.200 --> 01:12:00.820]  man, if I had root on something automatically that was internet exposed, what could I do?
[01:12:00.820 --> 01:12:05.440]  And I spent a long time looking at this environment, thinking I'm going to find something
[01:12:05.440 --> 01:12:13.900]  cool. We got in and they used it to spam mining bots everywhere using the minion service.
[01:12:13.920 --> 01:12:20.920]  And that was literally it. And so I want to tell that boring horror story because I live
[01:12:20.920 --> 01:12:25.680]  too for the cool, like, when am I going to find an Oday? And I've seen them. Don't get me wrong.
[01:12:25.680 --> 01:12:31.580]  I've seen some really cool stuff from APTs and some really cool tactics. I've seen people
[01:12:31.580 --> 01:12:37.900]  absolutely stomp out servers to where I have to go into unallocated space and get really creative
[01:12:37.900 --> 01:12:43.900]  about carving things out. But this one was one where I was going in going, I'm going to see
[01:12:43.900 --> 01:12:49.120]  like something absolutely crazy because you automatically had root access and you could do
[01:12:49.120 --> 01:12:55.080]  all of these things. And that was not the case. And I've seen so much of that over the course of
[01:12:55.080 --> 01:13:01.980]  doing IR where I'm just like, you know, you have this level and you worked at this level. And I
[01:13:01.980 --> 01:13:07.180]  think that's something to keep in mind just when you're doing these things is, you know, I was
[01:13:07.180 --> 01:13:13.080]  super happy to go back to the client and say, I have great news. All they did is run up your cloud
[01:13:13.080 --> 01:13:19.700]  build. But at the same time, like never assume anything because I was thinking like I'm going
[01:13:19.700 --> 01:13:28.260]  to boil the ocean and find something and I did and I didn't. In 2005, I was on a plane coming back from
[01:13:28.400 --> 01:13:36.320]  a job and a co-worker of mine is sitting next to me on the plane and we're talking about war stories
[01:13:36.320 --> 01:13:40.300]  and he tells me a war story from the week before, which is why he was late to the job that we were
[01:13:40.300 --> 01:13:45.520]  on. And I go, what did you do? And he goes, oh, I was doing IR on an old Windows XP network that
[01:13:45.520 --> 01:13:50.200]  was locked down by this government thing because it was a defense contractor. I'm like, oh, well,
[01:13:50.200 --> 01:13:58.920]  how'd the bug get in? It's like old Java bug, super like 265 day, right? Like old Java on Windows XP.
[01:13:59.040 --> 01:14:04.300]  What was the vector? Facebook. Yeah. Ops were on there, you know, running Facebook and click
[01:14:04.300 --> 01:14:08.640]  something got infected. Oh, that's interesting. Why did it take you a week to clean up? Well,
[01:14:08.640 --> 01:14:12.440]  didn't have a lot of ambient room on the operating system to install tools. Why?
[01:14:12.440 --> 01:14:15.300]  Well, it was a drone control station for a Predator 3.
[01:14:20.000 --> 01:14:29.300]  Yeah, I had a similar story to Moose about, you know, tracking down a, what we realized was a
[01:14:29.300 --> 01:14:36.680]  O-Day for a printer. And, you know, we thought for sure that this was a very advanced threat actor
[01:14:37.540 --> 01:14:44.560]  burning an O-Day for this situation and found out it was just to run a ware server
[01:14:44.560 --> 01:14:51.700]  off of our printers because they have so much storage. And so that, you know, again,
[01:14:51.700 --> 01:14:57.060]  yeah, really excited. All right. Yeah. This is going to be like going to make me famous, right?
[01:14:57.060 --> 01:15:03.980]  No, they're just running ware. So, but they somehow managed to get an O-Day and, you know,
[01:15:03.980 --> 01:15:12.900]  that was what they spent it on. Cool. Well, I certainly don't have any stories as exciting as
[01:15:12.900 --> 01:15:20.280]  those that the rest of you have shared in my somewhat boring career in InfoSec, not getting
[01:15:20.280 --> 01:15:25.480]  to go around and do incident response all over the place. But, you know, I just want to take
[01:15:25.480 --> 01:15:31.580]  the opportunity to thank all of you for accepting our invitations and coming here and sharing your
[01:15:31.580 --> 01:15:36.700]  experience and, you know, helping all the attendees, you know, who are here at the Blue
[01:15:36.700 --> 01:15:42.500]  Team Village to learn from you. I've learned a lot from you. I think this is, you know, this is such
[01:15:42.500 --> 01:15:49.380]  great, valuable information and getting, you know, a variety of people's perspectives, you know, to
[01:15:49.380 --> 01:15:54.740]  help us all, you know, learn new things and different ways to approach problems. It's
[01:15:54.740 --> 01:16:00.940]  awesome. I want to thank you so much for participating. And for the attendees who are
[01:16:00.940 --> 01:16:09.580]  here in the BTV Discord, we're going to move over to the voice talks chat. Anyone who's available,
[01:16:09.580 --> 01:16:14.200]  panelists who are available, pop over there and take some questions like hallway con.
[01:16:16.240 --> 01:16:19.400]  So I guess that's a wrap. Thanks, everyone.
[01:16:21.320 --> 01:16:22.100]  Thank you.
[01:16:22.100 --> 01:16:22.520]  Transcribed by https://otter.ai