WEBVTT 00:01.740 --> 00:07.080 And welcome back, everybody, to DEF CON 28 Safe Mode, Blue Team Village. 00:07.100 --> 00:09.280 We have our next talk for the day. 00:09.420 --> 00:11.960 We're going to have Bill talking to us today. 00:11.960 --> 00:15.580 No question, team viewer, police, and consequence. 00:15.880 --> 00:18.680 So take it from there, Bill. 00:21.190 --> 00:24.170 Hello, everybody, and happy end of the world to you. 00:24.210 --> 00:33.410 The purpose of this talk is to explain a situation I'd found myself in last year, to detail a breach from the perspective of a small business. 00:33.590 --> 00:42.830 After the story itself, I've got a couple ideas relating to bringing the bigger ideas about security to some of our smaller clients. 00:43.650 --> 00:52.030 Alongside that, I wanted to make it clear to anybody who feels the familiar sting of the imposter syndrome that you're capable. 00:52.650 --> 00:58.150 I'm presenting this as a narrative, more of a story than a formal presentation. 00:58.150 --> 01:04.710 The overall scale of this breach is probably something much smaller than most of you are used to dealing with. 01:04.890 --> 01:12.070 But I do know that there are some people here from small businesses that tune into the BTV, because I'm one of them. 01:18.070 --> 01:25.550 Online, my name is Corvus Actual, and I'm the co-owner of a small four-person IT shop in Canada. 01:25.990 --> 01:31.830 This is the story of what a breach looks like for a small business, start to finish. 01:31.830 --> 01:37.670 So, we start about a month before the breach itself with my very first trip to DEF CON. 01:37.970 --> 01:40.130 I was overwhelmed with the opportunity. 01:40.170 --> 01:44.770 Everybody says when you get there, there's so much to do, but there really is so much to do. 01:44.970 --> 01:47.550 I've been watching DEF CON videos for years. 01:48.450 --> 01:53.750 I spent quite a bit of my time at the Blue Team Village, since that's at least a part of my day job. 01:53.750 --> 01:59.510 I wanted to see if I could pick anything up from between the talks, the CTF, and the people milling about. 01:59.510 --> 02:05.610 I was given access to the CTF VPN at the same table as everybody else. 02:06.130 --> 02:09.730 But here's the thing, I flew to DEF CON completely solo. 02:09.970 --> 02:18.730 So, even though I later hooked up with the Lonely Hackers Club, I stumbled through the first bit of the CTF without anybody to bounce ideas off of. 02:18.730 --> 02:27.630 I was absolutely thrilled to have captured a good handful of flags on my own, both sitting in the hall outside the BTV and later on in the weekend. 02:28.350 --> 02:45.330 The reason I mention this, though it falls outside the actual breach itself, is because that experience, maybe fueled by the mana of so many like-minded people, gave me the confidence to jump in there and fight it out, solo or not. 02:45.330 --> 02:46.950 I really believed that. 02:46.950 --> 02:57.910 I felt like I had some kind of drive or motivation that wouldn't have ordinarily been so handily available, thanks to the community behind the Blue Team Village at DEF CON. 02:59.170 --> 03:04.050 So fast forward a few weeks and I'm sitting in front of my workstation at about 10.30am. 03:04.390 --> 03:11.010 I get a call from a co-worker who has seen a weird log file that's been left on his workstation. 03:11.290 --> 03:15.990 He noticed this text file left open with a mishmash of characters, some sort of encryption. 03:16.610 --> 03:23.250 The file referenced a location on his system that threw a trigger in his head, enough to call somebody and talk it through. 03:24.830 --> 03:27.890 C Program Data FKL. 03:28.070 --> 03:35.370 After a short chat, we both set to work on Google to look for information on this folder, who we had just discovered was set to hidden. 03:35.670 --> 03:37.050 Hmm, interesting. 03:37.510 --> 03:43.590 After a few keyword searches, we found a link to a number of sites selling software called Family Keylogger. 03:44.050 --> 03:44.830 Uh-oh. 03:45.290 --> 03:49.710 A little deeper now, we started to look into the hidden folders. 03:49.710 --> 03:58.130 We saw logview.exe, the program for viewing the logged keystrokes as advertised by the site selling the software. 03:58.350 --> 04:01.970 When we clicked to open the program, it was password protected. 04:02.590 --> 04:05.570 All the usual default passwords didn't work. 04:05.570 --> 04:09.210 A quick Google search didn't provide any results for default credentials. 04:09.770 --> 04:10.570 Uh-oh. 04:10.850 --> 04:14.390 So finally, I logged into a number of other computers via TeamViewer. 04:14.490 --> 04:18.050 On all the major systems I had access to, we found the folder. 04:18.050 --> 04:25.410 I could feel that deep, gut-turning anxiety start to bubble up as the situation started to spin. 04:28.920 --> 04:33.280 I immediately tasked our other techs to meet at the infected client. 04:33.600 --> 04:40.900 They're an organization with size and scale, but unfortunately, not a lot of budget to be thrown at traditional IT security stuff. 04:40.900 --> 04:45.560 So, no IDS, no IPS, nothing to track logs. 04:45.640 --> 04:54.200 Add to that, no 2FA, no ongoing employee training for cybersecurity stuff, and minimal formal policies in place. 04:54.900 --> 04:59.600 So, we have these client machines, and they're online. 04:59.600 --> 05:03.380 There's an unknown number of them that are infected with this keylogger. 05:03.940 --> 05:10.500 During this time, in the course of about 40 minutes, my coworker hit a stroke of good luck. 05:10.780 --> 05:18.980 After copying out the FKL folder for backup, he started deleting one file at a time, just to see if one of the files was locking the program. 05:19.380 --> 05:20.300 He found it. 05:20.360 --> 05:34.860 Again, just by dumb luck, by deleting one of the files in the client subdirectory, he managed to unlock that database, showing the logged key presses, and most importantly, the vector for uninstalling the application cleanly. 05:35.380 --> 05:37.740 At the same time, I emailed the vendor. 05:37.980 --> 05:45.720 They noted, of course, that if we bought a license, they'd be able to unlock the application and provide uninstallation instructions. 05:46.180 --> 05:56.220 So our technicians got to work backing up found copies of the program and uninstalling it from infected machines, while I handled the servers separately. 05:57.680 --> 06:01.060 At this time, it was about 5 p.m. 06:01.340 --> 06:06.080 I was, in all sincerity, overwhelmed. 06:06.500 --> 06:15.140 Beyond the scope of this problem, our business was expected to maintain operations and fluidity to the rest of our support issues. 06:15.840 --> 06:20.740 The rest of our clients needed to be supported and regular business was to continue as normal. 06:21.100 --> 06:22.880 I was dejected. 06:22.880 --> 06:24.100 I was defeated. 06:24.960 --> 06:31.760 Especially after freshly returning home from Hacker Summer Camp, I just felt like I'd been had. 06:35.770 --> 06:45.210 When my wife arrived home, she told me to take a 20-minute break to clear my head before giving me the best instruction anybody in crisis could get. 06:45.350 --> 06:46.710 Get after it. 06:46.710 --> 06:47.950 Fix this. 06:48.070 --> 06:49.430 I know you can. 06:50.130 --> 06:52.970 But this is where the human side kicks in. 06:53.470 --> 06:56.210 This is where I feel like I've already lost. 06:56.430 --> 07:05.110 I know I'm going to have to provide something, anything, in an email to the people involved, and I don't know really where to start. 07:05.110 --> 07:13.470 I've worked in IT for more than a decade, but I don't have any formal certifications in security, or I didn't at that point. 07:13.890 --> 07:21.330 I felt like this was getting way past my skill set and into territory that I definitely wasn't comfortable with. 07:21.510 --> 07:28.370 I knew, though, that somebody had to start, so I just sort of logged in and got to work. 07:28.370 --> 07:31.770 Without any real direction, I just started. 07:32.070 --> 07:39.530 I backed up log files from TeamViewer before looking through any of them to a separate system for safekeeping. 07:39.730 --> 07:49.310 With no formal training in forensics or chain of custody, I figured it was a good idea in any case just to get copies of those log files just in case. 07:54.030 --> 08:03.050 In some instances, I found the TeamViewer date underscore log file dot log had been deleted. 08:03.550 --> 08:10.290 Luckily, we had offsite backups of that folder where we could retrieve the log files from before they'd been deleted. 08:10.910 --> 08:13.650 Secondarily, we had shadow copies as well. 08:14.090 --> 08:21.550 I found a couple servers where the attacker had deleted the log files, but strangely, some of them were left untouched on other servers. 08:22.790 --> 08:26.450 At this time, the local municipal police had become involved. 08:26.950 --> 08:34.510 A detective from their cybercrime unit had come to the main headquarters of our client to chat with the co-owner of our company. 08:36.480 --> 08:43.150 I asked my partner to let the detective know that I was digging to stand by for any information that I'd found. 08:43.290 --> 08:52.510 I recognize now that just getting started and just getting to work and diving in there without formal training could have cost the investigation. 08:53.170 --> 09:00.350 But in a real-life situation, as it was happening with real-life consequences, that's the decision I made. 09:00.350 --> 09:05.530 I just decided at that moment to dive in there and get to work. 09:06.550 --> 09:09.390 I started looking through the TeamViewer connection logs. 09:09.390 --> 09:16.630 I figured I would start with the easiest infiltration vector, or the easiest vector I could think of. 09:16.870 --> 09:21.890 The remote access application installed is unattended on the servers themselves. 09:22.110 --> 09:23.710 I considered other options. 09:23.710 --> 09:35.370 I was at least aware of USB rubber duckies that could have been preloaded with a script and dropped in the parking lot, or an installer dropped in from an exploit or something like that. 09:35.750 --> 09:39.370 I figured I would just start at the very bottom and work my way up. 09:39.370 --> 09:41.850 I'd start with what I knew how to handle. 09:42.370 --> 09:49.870 I scrolled through a handful of logs, stopping abruptly on seeing a peculiar username having accessed one of the servers. 09:55.360 --> 09:57.520 Okay, that's something. 09:57.600 --> 10:00.120 I made a note of the username and kept digging. 10:00.620 --> 10:03.340 I started noticing this username everywhere. 10:03.340 --> 10:08.340 And when I say everywhere, I mean in multiple TeamViewer logs on multiple different servers and workstations. 10:08.740 --> 10:14.820 Generally speaking, this user was accessing systems well outside of our standard operating hours. 10:14.820 --> 10:19.340 Most of the connections were happening around midnight, some of them just after. 10:20.620 --> 10:26.300 Many TeamViewer installs had another user added as unattended, named the number zero. 10:27.200 --> 10:32.960 We had to access every system in the agency to ensure this user was disallowed any further access. 10:33.640 --> 10:39.520 Nonetheless, I documented multiple file transfers and remote control sessions from this user. 10:39.740 --> 10:44.000 In the TeamViewer logs, I had dates and times with a connected username. 10:44.000 --> 10:48.760 I figured TeamViewer would be able to lend a hand in getting more information together. 10:49.500 --> 11:04.160 Since the client systems were all installed as unattended, I suspect the attacker waited to find one of our technician's TeamViewer credentials to log in and access that list of unattended systems in the agency. 11:04.200 --> 11:07.360 Without evidence, though, I can only assume. 11:09.180 --> 11:14.500 Alongside the detective from the cybercrime unit, we asked TeamViewer to help. 11:14.860 --> 11:31.380 We were swiftly transferred to a legal department who told us that since TeamViewer's servers were located in Germany, we would require international warrants to have any data about these connections compiled and sent for use in a criminal investigation. 11:31.980 --> 11:37.420 At the time, I thought that was just absolute bullshit, but I do get it. 11:37.420 --> 11:55.200 It was frustrating because we were in the middle of that scenario as it was developing, so I felt all this pressure on top of my shoulders to just summon these fixes with people who were not unwilling, but unable to help. 11:57.630 --> 12:02.530 I had an old contact at TeamViewer that I sent an urgent message to. 12:03.150 --> 12:09.750 He called back and lent some advice as to where we could look through our own log files for any more information. 12:10.590 --> 12:23.590 You have to figure we're looking at maybe 20 or so log files that span several years, sometimes two or three years, a real good chunk of data. 12:23.590 --> 12:33.070 At the time, the best I could muster was literally mouse-wheeling through those things, but I had this little indicator, that username, to chase after. 12:34.370 --> 12:42.510 During this time, over the period of about 4 to 5 hours, I started up my laptop and began scouring the Internet for OSINT. 12:42.670 --> 12:44.490 I'm not really sure why I did that. 12:44.750 --> 12:50.690 Maybe just to bounce the username off of the Internet to see if anything useful came back. 12:50.690 --> 13:00.010 Well, I found a user on Twitter with, believe it or not, the same username that had logged in over TeamViewer. 13:00.530 --> 13:09.050 Scrolling through their post history, I found, believe it or not, a post about the very key logger that was installed on our systems. 13:09.570 --> 13:17.510 So obviously, I doubled down on my efforts in OSINT for about an hour and pulled all kinds of relevant information. 13:17.510 --> 13:25.330 For example, we found an email address in the configuration for the key logger, where the logs were being pushed to. 13:25.510 --> 13:32.770 The Twitter user shared a Facebook account with several email addresses published with the same username. 13:33.710 --> 13:39.190 I found multiple references to nefarious stuff on the social media profiles linked to this person. 13:39.310 --> 13:43.810 I even found the same username dumping hashes on a cracking forum. 13:43.810 --> 13:46.310 And then, the breakthrough. 13:47.090 --> 13:54.530 An IP address was found in a TeamViewer log file that coincided with the time the external user logged in. 13:54.870 --> 14:02.670 I ran it through a quick IP geolocation search on Google and found it traced back to a small town in Ontario, Canada. 14:02.810 --> 14:09.290 The same small town both the Twitter and Facebook account were declared as being from. 14:09.850 --> 14:13.210 Now we're going to refer to the attacker as bad guy. 14:17.550 --> 14:21.890 I brought this to the attention of both the detective and the client. 14:21.990 --> 14:26.030 I put together a dossier of all the information I found and handed it over to both parties. 14:26.030 --> 14:29.210 I figured it'd at least be a good starting point. 14:29.670 --> 14:31.410 Here's where things get tricky. 14:31.590 --> 14:39.390 This person, referenced on social media, was actually known to the organization as being a person who was enrolled in their services. 14:39.390 --> 14:45.990 The story takes a brief pause here as the justice system lags behind the real world. 14:46.250 --> 14:52.150 Fortunately, in time, the municipal police were able to secure a search warrant for bad guy's house. 14:52.510 --> 14:58.170 On the arrival of police services, bad guy's devices were on and doing bad guy stuff. 14:58.190 --> 15:04.850 They were taken for analysis and bad guy confessed in full, on site. 15:05.030 --> 15:08.450 After a few weeks, the analysis came back. 15:08.450 --> 15:11.590 Bad guy's computers were full of all the usual stuff. 15:11.690 --> 15:16.930 VMs, multiple pirated keyloggers, PDFs with instructions, videos, all that stuff. 15:16.930 --> 15:18.970 Signed, sealed, and delivered. 15:18.970 --> 15:19.890 Right? 15:20.070 --> 15:22.370 Well, something happened. 15:22.790 --> 15:29.790 During the five months this process took to finally wash out, something must have happened. 15:29.990 --> 15:31.590 No question. 15:31.630 --> 15:41.970 I was told there was no question that bad guy was responsible, had conducted these unauthorized connections to the network, and that he had the means to do so at his ready. 15:42.750 --> 15:52.750 I wasn't told specifically why the charges were dropped, only that it would be difficult to prove his intent was malicious in nature. 15:53.310 --> 15:53.870 K? 15:54.650 --> 15:59.390 At the end of the day, I think this is how bad guy got his initial access. 15:59.730 --> 16:02.550 I don't think this was a particularly technical attack. 16:02.550 --> 16:07.770 I'm not nearly as skilled at detection and forensic stuff as some folks around here. 16:07.770 --> 16:09.650 But this is what I think happened. 16:10.210 --> 16:18.450 The organization didn't have hard policies in place about clientele using the organization's systems. 16:18.970 --> 16:22.730 I'd heard through the grapevine that this was happening, but you all know how that goes. 16:22.730 --> 16:25.250 An email goes out to leadership saying this shouldn't happen. 16:25.250 --> 16:31.230 The email goes down the chain, but at the end of the day, no policy equals no action. 16:31.850 --> 16:43.330 I believe bad guy had an opportunity at some point to either access the system directly or, which is the more likely scenario, just snapped a picture of the team viewer splash screen with a cell phone. 16:43.630 --> 16:50.030 They could have accessed the system, likely not locked or turned off, later on that night and started to infiltrate from there. 16:50.030 --> 16:51.750 But here's the kicker. 16:52.150 --> 16:59.890 Bad guy got his devices back and is back on site accessing services from the organization. 17:00.110 --> 17:01.650 So that's awesome. 17:02.030 --> 17:05.650 All told, we learned a pile of valuable lessons. 17:05.770 --> 17:10.370 We're a small shop, and yep, I've read about situations like this. 17:10.490 --> 17:16.670 I'd say we made it out lucky from a scenario that could have been much, much worse. 17:17.010 --> 17:25.350 Most of you will be able to read between the lines here and identify points of failure on our part that led to this type of breach being a possibility. 17:26.350 --> 17:35.730 I think the largest problem that small IT shops face is the juggling act required to keep all the elements of business equally sorted. 17:35.730 --> 17:43.870 I remember from the Blue Team Village last year, a panel on small business cybersecurity that touched on that. 17:43.870 --> 17:53.950 The concept that cybersecurity is sometimes left as an afterthought with some of our smaller clients from budget limits or lack of concern or understanding. 17:53.950 --> 17:57.230 But the same threats and outcomes still exist. 17:57.930 --> 18:03.210 At last year's DEF CON, I played a game of D&D with a guy who gave me a great little piece of advice. 18:03.950 --> 18:10.430 Present controls to mitigate risk and defer responsibility where those controls are not used. 18:10.750 --> 18:14.650 Right or wrong, I think it's a great starting point for guys like me. 18:15.090 --> 18:19.690 I think there's some use in talking about the after-action side of things, too. 18:19.690 --> 18:33.470 For the people that work in smaller environments and hail from smaller companies, I've seen a subtle gap that exists right before a certain sized organization where cybersecurity starts to matter more to the people in leadership roles. 18:37.130 --> 18:45.750 These are a couple things I've been doing recently that seem to help bring more hesitant people around to the idea of tightening up security around the network. 18:45.970 --> 18:50.330 If you're selling security services, this stuff might help you talk about your stack. 18:50.710 --> 18:57.570 But I think for anybody in a position where you're talking to end users about security, these three things might be a decent idea. 18:58.130 --> 18:59.390 Define the why. 18:59.390 --> 19:03.850 Really, I'm trying not to use LinkedIn like business speak here. 19:03.850 --> 19:06.470 I've used this specific idea. 19:06.750 --> 19:08.590 It works for me. 19:09.190 --> 19:16.310 Chip away at the organization's goals until you can directly attach data security to the primary function of the business. 19:16.310 --> 19:21.150 Forests Ontario is a local nonprofit that I pulled off of Google. 19:21.150 --> 19:23.330 They help rebuild forests around here. 19:23.330 --> 19:25.190 They have the following goal. 19:26.150 --> 19:30.390 Forests Ontario is dedicated to making Ontario's forests greener. 19:30.490 --> 19:40.390 Our ambitious tree planting initiatives, extensive education programs, and decades of community outreach have resulted in millions of trees being planted each year. 19:40.390 --> 19:50.430 You could just as easily write a few edits into that statement when talking to shop callers about security to start emphasizing the importance of security. 19:50.430 --> 19:56.110 For example, Forests Ontario is dedicated to making Ontario's forests greener. 19:56.150 --> 20:11.250 Our ambitious tree planting initiatives, extensive education programs, and decades of community outreach are powered by scheduling software, email, and a company fleet of laptops, and have resulted in millions of trees being planted each year. 20:11.250 --> 20:28.770 I've used this approach to help frame cybersecurity not as an element of IT infrastructure or something that happens in the server room, but as the starting point to building a culture within the organization that values security as part of how the whole thing runs. 20:29.090 --> 20:33.790 Without cybersecurity, we can't do the things we're trying to do. 20:35.050 --> 20:36.950 Find a framework. 20:37.250 --> 20:44.390 Most of the smaller organizations I see out there have very, very little security infrastructure. 20:44.390 --> 20:47.870 Most have next to zero staff training. 20:47.890 --> 20:54.850 Most rely on the IT guy to just do the security magic and do it quietly and do it cheaply. 20:54.850 --> 21:05.470 Find a published framework that you can use to better explain why you're introducing security controls that cost cash and convenience. 21:05.470 --> 21:21.190 For instance, we found the baseline cybersecurity controls for small and medium organizations from the Canadian federal government to be of great help with both translating some of the technical side and actually implementing controls. 21:21.430 --> 21:40.870 I found even if the client calls you to help set up and secure a network, a written and industry accepted framework can help, as Jocko Willink might say, prioritize and execute a short list of must-haves from a long list of would-be controls. 21:41.950 --> 21:45.110 Lastly, start talking about it. 21:45.310 --> 21:47.090 This seems really obvious. 21:47.090 --> 21:53.070 Out there in the offices, I see a common theme between most of our clients. 21:53.250 --> 21:56.350 Some just give us money to fix the blinky lights. 21:56.550 --> 22:01.810 Others want us to verify that we're using every minute of each build hour. 22:01.950 --> 22:04.470 The thread that runs between them is this. 22:04.530 --> 22:08.190 When I talk to them about security, they tend to listen. 22:08.690 --> 22:27.450 Now, that doesn't always translate into putting better practices in place, but I can usually at least get a window to speak at a staff meeting, to start building that foundation at the very base of the organization, and at least start the conversation about cybersecurity. 22:28.170 --> 22:31.950 This seems like a given, to just talk about security. 22:32.130 --> 22:39.670 Too often, though, our clients are caught up in the grind of their day job, and they don't have time to worry about security. 22:40.010 --> 22:48.290 More often still, users assume the responsibility of security falls onto the guy in the server room, not the person at the reception desk. 22:49.290 --> 22:57.590 Purposefully bringing a quick briefing on cybersecurity to monthly staff meetings or as part of a smaller team meeting helps keep that topic on the table. 22:58.870 --> 23:14.310 One of the benefits of security, cringy as it might be to some of the guys who have been around DEF CON forever, is that information security still has that cool factor to a lot of people who don't directly work in IT or around security. 23:14.370 --> 23:23.030 If you really put some investment into bringing security into the conversation, like any other part of your life, it pays you back. 23:24.310 --> 23:29.710 I have to say, speaking at DEF CON is an absolute moonshot for me. 23:29.930 --> 23:35.310 There are times I had felt like one of the only guys in my town that was interested in hacking. 23:35.470 --> 23:40.750 From a place of true gratitude, I thank you all for the opportunity. 23:41.990 --> 23:45.950 The hard lesson to learn is where your weaknesses are. 23:46.310 --> 23:52.490 Working through the breach last year served to detail exactly where our failures were. 23:52.630 --> 23:56.350 Since this event, I've started to swing for some certifications. 23:56.450 --> 23:59.090 I've fundamentally shifted my focus. 23:59.860 --> 24:02.910 At the end of the day, it's all up to you. 24:02.910 --> 24:05.010 And nobody's coming to save you. 24:05.530 --> 24:12.370 Please feel free to DM me on the BTV Discord or drop your questions in the BTV Discord so I can respond. 24:12.410 --> 24:22.420 With collectives like the Blue Team Village, we can fortify ourselves to meet real world challenges, even if we're not fully ready to fight. 24:22.500 --> 24:24.420 Because we have to. 24:24.540 --> 24:25.940 Thank you.