start	end	text
1740	5720	And welcome back, everybody, to DEF CON 28 Safe Mode,
5720	7080	Blue Team Village.
7100	9280	We have our next talk for the day.
9420	11960	We're going to have Bill talking to us today.
11960	15580	No question, team viewer, police, and consequence.
15880	18680	So take it from there, Bill.
21190	24170	Hello, everybody, and happy end of the world to you.
24210	26830	The purpose of this talk is to explain a situation
26830	29850	I'd found myself in last year,
29850	33410	to detail a breach from the perspective of a small business.
33590	36550	After the story itself, I've got a couple ideas
36550	40410	relating to bringing the bigger ideas about security
40410	42830	to some of our smaller clients.
43650	46270	Alongside that, I wanted to make it clear to anybody
46270	49890	who feels the familiar sting of the imposter syndrome
50590	52030	that you're capable.
52650	54590	I'm presenting this as a narrative,
54590	58150	more of a story than a formal presentation.
58150	62150	The overall scale of this breach is probably something much smaller
62150	64710	than most of you are used to dealing with.
64890	68630	But I do know that there are some people here from small businesses
68630	72070	that tune into the BTV, because I'm one of them.
78070	80610	Online, my name is Corvus Actual,
80610	85550	and I'm the co-owner of a small four-person IT shop in Canada.
85990	88510	This is the story of what a breach looks like
88510	91830	for a small business, start to finish.
91830	95450	So, we start about a month before the breach itself
95450	97670	with my very first trip to DEF CON.
97970	100130	I was overwhelmed with the opportunity.
100170	102430	Everybody says when you get there, there's so much to do,
102430	104770	but there really is so much to do.
104970	107550	I've been watching DEF CON videos for years.
108450	111090	I spent quite a bit of my time at the Blue Team Village,
111090	113750	since that's at least a part of my day job.
113750	115490	I wanted to see if I could pick anything up
115490	119510	from between the talks, the CTF, and the people milling about.
119510	123070	I was given access to the CTF VPN
123070	125610	at the same table as everybody else.
126130	129730	But here's the thing, I flew to DEF CON completely solo.
129970	134070	So, even though I later hooked up with the Lonely Hackers Club,
134070	135990	I stumbled through the first bit of the CTF
135990	138730	without anybody to bounce ideas off of.
138730	141410	I was absolutely thrilled to have captured
141870	143930	a good handful of flags on my own,
143930	146130	both sitting in the hall outside the BTV
146130	147630	and later on in the weekend.
148350	150130	The reason I mention this,
150130	152750	though it falls outside the actual breach itself,
152750	155850	is because that experience,
155850	161010	maybe fueled by the mana of so many like-minded people,
161010	163970	gave me the confidence to jump in there and fight it out,
163970	165330	solo or not.
165330	166950	I really believed that.
166950	170890	I felt like I had some kind of drive or motivation
170890	174610	that wouldn't have ordinarily been so handily available,
174610	177910	thanks to the community behind the Blue Team Village at DEF CON.
179170	180590	So fast forward a few weeks
180590	182110	and I'm sitting in front of my workstation
182110	184050	at about 10.30am.
184390	185910	I get a call from a co-worker
185910	188450	who has seen a weird log file
188450	191010	that's been left on his workstation.
191290	192510	He noticed this text file
192510	194750	left open with a mishmash of characters,
194750	195990	some sort of encryption.
196610	198830	The file referenced a location on his system
198830	200630	that threw a trigger in his head,
200630	203250	enough to call somebody and talk it through.
204830	207890	C Program Data FKL.
208070	209030	After a short chat,
209030	210410	we both set to work on Google
210410	212970	to look for information on this folder,
212970	215370	who we had just discovered was set to hidden.
215670	217050	Hmm, interesting.
217510	219370	After a few keyword searches,
219370	220830	we found a link to a number of sites
220830	223590	selling software called Family Keylogger.
224050	224830	Uh-oh.
225290	226850	A little deeper now,
226850	229710	we started to look into the hidden folders.
229710	232350	We saw logview.exe,
232350	235290	the program for viewing the logged keystrokes
235290	238130	as advertised by the site selling the software.
238350	240430	When we clicked to open the program,
240430	241970	it was password protected.
242590	245570	All the usual default passwords didn't work.
245570	247650	A quick Google search didn't provide any results
247650	249210	for default credentials.
249770	250570	Uh-oh.
250850	252990	So finally, I logged into a number of other computers
252990	254390	via TeamViewer.
254490	256770	On all the major systems I had access to,
256770	258050	we found the folder.
258050	261170	I could feel that deep, gut-turning anxiety
261590	265410	start to bubble up as the situation started to spin.
268920	270900	I immediately tasked our other techs
270900	273280	to meet at the infected client.
273600	276380	They're an organization with size and scale,
276380	278080	but unfortunately, not a lot of budget
278080	280900	to be thrown at traditional IT security stuff.
280900	285560	So, no IDS, no IPS, nothing to track logs.
285640	287540	Add to that, no 2FA,
287540	291220	no ongoing employee training for cybersecurity stuff,
291220	294200	and minimal formal policies in place.
294900	298720	So, we have these client machines,
298720	299600	and they're online.
299600	301100	There's an unknown number of them
301100	303380	that are infected with this keylogger.
303940	308020	During this time, in the course of about 40 minutes,
308020	310500	my coworker hit a stroke of good luck.
310780	313380	After copying out the FKL folder for backup,
313380	315780	he started deleting one file at a time,
315780	318980	just to see if one of the files was locking the program.
319380	320300	He found it.
320360	322200	Again, just by dumb luck,
322200	326220	by deleting one of the files in the client subdirectory,
326220	328540	he managed to unlock that database,
328540	330340	showing the logged key presses,
330340	331680	and most importantly,
331680	334860	the vector for uninstalling the application cleanly.
335380	337740	At the same time, I emailed the vendor.
337980	340760	They noted, of course, that if we bought a license,
340760	342660	they'd be able to unlock the application
342660	345720	and provide uninstallation instructions.
346180	350540	So our technicians got to work backing up found copies of the program
350540	353400	and uninstalling it from infected machines,
353400	356220	while I handled the servers separately.
357680	361060	At this time, it was about 5 p.m.
361340	366080	I was, in all sincerity, overwhelmed.
366500	368720	Beyond the scope of this problem,
368720	371400	our business was expected to maintain operations
371400	375140	and fluidity to the rest of our support issues.
375840	377600	The rest of our clients needed to be supported
377600	380740	and regular business was to continue as normal.
381100	384100	I was dejected. I was defeated.
384960	389640	Especially after freshly returning home from Hacker Summer Camp,
389640	391760	I just felt like I'd been had.
395770	397650	When my wife arrived home,
397650	400230	she told me to take a 20-minute break to clear my head
400230	405210	before giving me the best instruction anybody in crisis could get.
405350	409430	Get after it. Fix this. I know you can.
410130	412970	But this is where the human side kicks in.
413470	416210	This is where I feel like I've already lost.
416430	420030	I know I'm going to have to provide something, anything,
420030	422470	in an email to the people involved,
422470	425110	and I don't know really where to start.
425110	427470	I've worked in IT for more than a decade,
427470	431850	but I don't have any formal certifications in security,
431850	433470	or I didn't at that point.
433890	436730	I felt like this was getting way past my skill set
436730	441330	and into territory that I definitely wasn't comfortable with.
441510	444930	I knew, though, that somebody had to start,
444930	448370	so I just sort of logged in and got to work.
448370	451770	Without any real direction, I just started.
452070	454350	I backed up log files from TeamViewer
454350	459530	before looking through any of them to a separate system for safekeeping.
459730	463250	With no formal training in forensics or chain of custody,
463250	465730	I figured it was a good idea in any case
465730	469310	just to get copies of those log files just in case.
474030	477790	In some instances, I found the TeamViewer
477790	483050	date underscore log file dot log had been deleted.
483550	485970	Luckily, we had offsite backups of that folder
485970	488310	where we could retrieve the log files from
488310	490290	before they'd been deleted.
490910	493650	Secondarily, we had shadow copies as well.
494090	496290	I found a couple servers where the attacker
496290	498930	had deleted the log files, but strangely,
498930	501550	some of them were left untouched on other servers.
502790	506450	At this time, the local municipal police had become involved.
506950	509070	A detective from their cybercrime unit
509070	511710	had come to the main headquarters of our client
511710	514510	to chat with the co-owner of our company.
516480	519390	I asked my partner to let the detective know that I was digging
519390	523150	to stand by for any information that I'd found.
523290	526870	I recognize now that just getting started
526870	528930	and just getting to work and diving in there
528930	532510	without formal training could have cost the investigation.
533170	535350	But in a real-life situation,
535350	538730	as it was happening with real-life consequences,
538730	540350	that's the decision I made.
540350	545530	I just decided at that moment to dive in there and get to work.
546550	549390	I started looking through the TeamViewer connection logs.
549390	553370	I figured I would start with the easiest infiltration vector,
553370	556630	or the easiest vector I could think of.
556870	559670	The remote access application installed is unattended
559670	561890	on the servers themselves.
562110	563710	I considered other options.
563710	567450	I was at least aware of USB rubber duckies
567450	569210	that could have been preloaded with a script
569210	570690	and dropped in the parking lot,
570690	575370	or an installer dropped in from an exploit or something like that.
575750	579370	I figured I would just start at the very bottom and work my way up.
579370	581850	I'd start with what I knew how to handle.
582370	584230	I scrolled through a handful of logs,
584230	587310	stopping abruptly on seeing a peculiar username
587310	589870	having accessed one of the servers.
595360	597520	Okay, that's something.
597600	600120	I made a note of the username and kept digging.
600620	603340	I started noticing this username everywhere.
603340	605560	And when I say everywhere, I mean in multiple TeamViewer logs
605560	608340	on multiple different servers and workstations.
608740	611480	Generally speaking, this user was accessing systems
611480	614820	well outside of our standard operating hours.
614820	617640	Most of the connections were happening around midnight,
617640	619340	some of them just after.
620620	624480	Many TeamViewer installs had another user added as unattended,
624480	626300	named the number zero.
627200	629240	We had to access every system in the agency
629240	632960	to ensure this user was disallowed any further access.
633640	636380	Nonetheless, I documented multiple file transfers
636380	639520	and remote control sessions from this user.
639740	642340	In the TeamViewer logs, I had dates and times
642340	644000	with a connected username.
644000	646220	I figured TeamViewer would be able to lend a hand
646220	648760	in getting more information together.
649500	653560	Since the client systems were all installed as unattended,
653560	656420	I suspect the attacker waited to find
656420	658860	one of our technician's TeamViewer credentials
658860	662660	to log in and access that list of unattended systems
662660	664160	in the agency.
664200	667360	Without evidence, though, I can only assume.
669180	672360	Alongside the detective from the cybercrime unit,
672360	674500	we asked TeamViewer to help.
674860	677080	We were swiftly transferred to a legal department
677080	680240	who told us that since TeamViewer's servers
680240	682420	were located in Germany,
682420	685060	we would require international warrants
685060	687920	to have any data about these connections compiled
687920	691380	and sent for use in a criminal investigation.
691980	695300	At the time, I thought that was just absolute bullshit,
695580	697420	but I do get it.
697420	700100	It was frustrating because we were in the middle
700100	703320	of that scenario as it was developing,
703320	706180	so I felt all this pressure on top of my shoulders
706180	710080	to just summon these fixes
710080	713140	with people who were not unwilling,
713140	715200	but unable to help.
717630	719890	I had an old contact at TeamViewer
719890	722530	that I sent an urgent message to.
723150	725050	He called back and lent some advice
725050	727990	as to where we could look through our own log files
727990	729750	for any more information.
730590	733230	You have to figure we're looking at
734050	738730	maybe 20 or so log files that span several years,
738730	740910	sometimes two or three years,
741450	743590	a real good chunk of data.
743590	746250	At the time, the best I could muster
746250	749270	was literally mouse-wheeling through those things,
749270	751610	but I had this little indicator, that username,
751610	753070	to chase after.
754370	757730	During this time, over the period of about 4 to 5 hours,
758190	760950	I started up my laptop and began scouring the Internet
760950	762510	for OSINT.
762670	764490	I'm not really sure why I did that.
764750	767890	Maybe just to bounce the username off of the Internet
767890	770690	to see if anything useful came back.
770690	774650	Well, I found a user on Twitter with,
774650	777090	believe it or not, the same username
777090	780010	that had logged in over TeamViewer.
780530	783150	Scrolling through their post history, I found,
783150	786890	believe it or not, a post about the very key logger
786890	789050	that was installed on our systems.
789570	792510	So obviously, I doubled down on my efforts in OSINT
792510	795630	for about an hour and pulled all kinds
795630	797510	of relevant information.
797510	800510	For example, we found an email address
800510	802930	in the configuration for the key logger,
802930	805330	where the logs were being pushed to.
805510	807770	The Twitter user shared a Facebook account
808250	810410	with several email addresses published
810410	812770	with the same username.
813710	816110	I found multiple references to nefarious stuff
816110	819190	on the social media profiles linked to this person.
819310	822090	I even found the same username dumping hashes
822090	823810	on a cracking forum.
823810	826310	And then, the breakthrough.
827090	830330	An IP address was found in a TeamViewer log file
830330	834530	that coincided with the time the external user logged in.
834870	837830	I ran it through a quick IP geolocation search
837830	840030	on Google and found it traced back
840030	842670	to a small town in Ontario, Canada.
842810	845670	The same small town both the Twitter
845670	849290	and Facebook account were declared as being from.
849850	853210	Now we're going to refer to the attacker as bad guy.
857550	859710	I brought this to the attention of both
859710	861890	the detective and the client.
861990	864410	I put together a dossier of all the information I found
864410	866030	and handed it over to both parties.
866030	869210	I figured it'd at least be a good starting point.
869670	871410	Here's where things get tricky.
871590	873870	This person, referenced on social media,
873870	875990	was actually known to the organization
876490	879390	as being a person who was enrolled in their services.
879390	882130	The story takes a brief pause here
882130	885990	as the justice system lags behind the real world.
886250	888510	Fortunately, in time, the municipal police
888510	892150	were able to secure a search warrant for bad guy's house.
892510	894570	On the arrival of police services,
894570	898170	bad guy's devices were on and doing bad guy stuff.
898190	899370	They were taken for analysis
899370	904850	and bad guy confessed in full, on site.
905030	908450	After a few weeks, the analysis came back.
908450	911590	Bad guy's computers were full of all the usual stuff.
911690	913830	VMs, multiple pirated keyloggers,
913830	916930	PDFs with instructions, videos, all that stuff.
916930	919890	Signed, sealed, and delivered. Right?
920070	922370	Well, something happened.
922790	926770	During the five months this process took to finally wash out,
927250	929790	something must have happened.
929990	931590	No question.
931630	935890	I was told there was no question that bad guy was responsible,
935890	939210	had conducted these unauthorized connections to the network,
939210	941970	and that he had the means to do so at his ready.
942750	947170	I wasn't told specifically why the charges were dropped,
947170	949650	only that it would be difficult to prove
949650	952750	his intent was malicious in nature.
953310	953870	K?
954650	956530	At the end of the day,
956530	959390	I think this is how bad guy got his initial access.
959730	962550	I don't think this was a particularly technical attack.
962550	965790	I'm not nearly as skilled at detection and forensic stuff
965790	967770	as some folks around here.
967770	969650	But this is what I think happened.
970210	973130	The organization didn't have hard policies in place
974590	978450	about clientele using the organization's systems.
978970	981110	I'd heard through the grapevine that this was happening,
981110	982730	but you all know how that goes.
982730	985250	An email goes out to leadership saying this shouldn't happen.
985250	988690	The email goes down the chain, but at the end of the day,
988690	991230	no policy equals no action.
991850	994330	I believe bad guy had an opportunity at some point
994330	997470	to either access the system directly or,
997470	1000310	which is the more likely scenario, just snapped a picture
1000310	1003330	of the team viewer splash screen with a cell phone.
1003630	1007030	They could have accessed the system, likely not locked or turned off,
1007030	1010030	later on that night and started to infiltrate from there.
1010030	1011750	But here's the kicker.
1012150	1016290	Bad guy got his devices back and is back on site
1016290	1019890	accessing services from the organization.
1020110	1021650	So that's awesome.
1022030	1025650	All told, we learned a pile of valuable lessons.
1025770	1030370	We're a small shop, and yep, I've read about situations like this.
1030490	1033890	I'd say we made it out lucky from a scenario
1033890	1036670	that could have been much, much worse.
1037010	1039510	Most of you will be able to read between the lines here
1039510	1042190	and identify points of failure on our part
1042190	1045350	that led to this type of breach being a possibility.
1046350	1049830	I think the largest problem that small IT shops face
1049830	1053230	is the juggling act required to keep all the elements
1053230	1055730	of business equally sorted.
1055730	1059570	I remember from the Blue Team Village last year,
1060030	1061550	a panel on small business cybersecurity
1061990	1063870	that touched on that.
1063870	1067650	The concept that cybersecurity is sometimes left as an afterthought
1067930	1070790	with some of our smaller clients from budget limits
1070790	1073950	or lack of concern or understanding.
1073950	1077230	But the same threats and outcomes still exist.
1077930	1080130	At last year's DEF CON, I played a game of D&D
1080130	1083210	with a guy who gave me a great little piece of advice.
1083950	1086170	Present controls to mitigate risk
1086170	1090430	and defer responsibility where those controls are not used.
1090750	1093270	Right or wrong, I think it's a great starting point
1093270	1094650	for guys like me.
1095090	1097210	I think there's some use in talking about
1097210	1099690	the after-action side of things, too.
1099690	1102230	For the people that work in smaller environments
1102230	1105510	and hail from smaller companies, I've seen a subtle gap
1105510	1108710	that exists right before a certain sized organization
1108710	1111290	where cybersecurity starts to matter more
1111290	1113470	to the people in leadership roles.
1117130	1119430	These are a couple things I've been doing recently
1119430	1123310	that seem to help bring more hesitant people around to the idea
1123310	1125750	of tightening up security around the network.
1125970	1128950	If you're selling security services, this stuff might help you talk
1128950	1132150	about your stack. But I think for anybody in a position
1132150	1134950	where you're talking to end users about security,
1134950	1137570	these three things might be a decent idea.
1138130	1141690	Define the why. Really, I'm trying not to use LinkedIn
1141690	1143850	like business speak here.
1143850	1146470	I've used this specific idea.
1146750	1150810	It works for me. Chip away at the organization's goals
1150810	1153210	until you can directly attach data security
1153210	1156310	to the primary function of the business.
1156310	1159570	Forests Ontario is a local nonprofit
1159570	1162430	that I pulled off of Google. They help rebuild forests
1162430	1165190	around here. They have the following goal.
1166150	1168710	Forests Ontario is dedicated to making Ontario's
1168710	1171830	forests greener. Our ambitious tree planting
1171830	1174470	initiatives, extensive education programs,
1174470	1177630	and decades of community outreach have resulted in millions
1177630	1180390	of trees being planted each year.
1180390	1183930	You could just as easily write a few edits into that statement
1183930	1186610	when talking to shop callers about security
1186610	1190430	to start emphasizing the importance of security.
1190430	1193070	For example, Forests Ontario is dedicated
1193070	1196110	to making Ontario's forests greener.
1196150	1199070	Our ambitious tree planting initiatives, extensive education
1199070	1202150	programs, and decades of community outreach are
1202150	1205050	powered by scheduling software, email, and
1205170	1207990	a company fleet of laptops, and have resulted
1207990	1211250	in millions of trees being planted each year.
1211250	1214410	I've used this approach to help frame cybersecurity
1214410	1217790	not as an element of IT infrastructure or something
1217790	1220630	that happens in the server room, but as the starting
1220630	1223570	point to building a culture within the
1223570	1226590	organization that values security as part
1226590	1229430	of how the whole thing runs. Without
1229430	1232610	cybersecurity, we can't do the things we're trying
1232610	1233790	to do.
1235050	1238370	Find a framework. Most of the smaller
1238370	1241410	organizations I see out there have very,
1241410	1244590	very little security infrastructure. Most
1244590	1247870	have next to zero staff training.
1247890	1250450	Most rely on the IT guy to just do
1250450	1253550	the security magic and do it quietly and do
1253550	1256270	it cheaply. Find a published
1256270	1259410	framework that you can use to better explain why
1259410	1262230	you're introducing security controls
1262230	1265470	that cost cash and convenience.
1265470	1268650	For instance, we found the baseline cybersecurity
1268990	1271870	controls for small and medium organizations
1271870	1274830	from the Canadian federal government to be of great
1274830	1277650	help with both translating some of the technical
1277650	1281190	side and actually implementing controls.
1281430	1283590	I found even if the
1283590	1286550	client calls you to help set up and
1286550	1289590	secure a network, a written and industry
1289590	1292710	accepted framework can help, as Jocko Willink might
1292710	1295850	say, prioritize and execute a short
1295850	1298550	list of must-haves from a long list
1298550	1300870	of would-be controls.
1301950	1305110	Lastly, start talking about it.
1305310	1307570	This seems really obvious. Out there
1307570	1310310	in the offices, I see a common theme
1310310	1313850	between most of our clients. Some just
1313850	1316810	give us money to fix the blinky lights. Others
1316810	1319710	want us to verify that we're using every minute
1319710	1322730	of each build hour. The thread that runs
1322730	1325650	between them is this. When I talk to them about
1325650	1328190	security, they tend to listen.
1328690	1331430	Now, that doesn't always translate into putting better
1331430	1334310	practices in place, but I can usually
1334310	1337690	at least get a window to speak at a staff
1337690	1340590	meeting, to start building that foundation at the
1340590	1343730	very base of the organization, and
1343730	1347450	at least start the conversation about cybersecurity.
1348170	1350650	This seems like a given, to just talk
1350650	1353910	about security. Too often, though, our clients
1353910	1356750	are caught up in the grind of their day job, and
1356750	1359670	they don't have time to worry about security.
1360010	1362690	More often still, users assume the responsibility
1362690	1365930	of security falls onto the guy in the server room,
1365930	1368290	not the person at the reception desk.
1369290	1371810	Purposefully bringing a quick briefing on cybersecurity
1371810	1374430	to monthly staff meetings or as part of a smaller
1374430	1377590	team meeting helps keep that topic on the table.
1378870	1381890	One of the benefits of security,
1381890	1384050	cringy as it might be to some of the guys
1384050	1386890	who have been around DEF CON forever, is that
1386890	1389710	information security still has that cool factor
1389710	1392570	to a lot of people who don't directly work in IT
1392570	1395550	or around security. If you really put some
1395550	1398210	investment into bringing security into the
1398210	1401330	conversation, like any other part of your life,
1401330	1403030	it pays you back.
1404310	1407490	I have to say, speaking at DEF CON is an absolute
1408150	1410790	moonshot for me. There are times
1410790	1413410	I had felt like one of the only guys in my town
1413410	1416450	that was interested in hacking. From a place
1416450	1419430	of true gratitude, I thank you all
1419430	1420750	for the opportunity.
1421990	1425950	The hard lesson to learn is where your weaknesses are.
1426310	1428110	Working through the breach last year
1428110	1432490	served to detail exactly where our failures were.
1432630	1434730	Since this event, I've started to swing
1434730	1437270	for some certifications. I've fundamentally
1437270	1439090	shifted my focus.
1439860	1442910	At the end of the day, it's all up to you.
1442910	1445010	And nobody's coming to save you.
1445530	1448150	Please feel free to DM me on the BTV Discord
1448150	1452370	or drop your questions in the BTV Discord so I can respond.
1452410	1455340	With collectives like the Blue Team Village,
1455340	1458560	we can fortify ourselves to meet real world
1458560	1462420	challenges, even if we're not fully ready to fight.
1462500	1464420	Because we have to.
1464540	1465940	Thank you.