And welcome back, everybody, to DEF CON 28 Safe Mode,
Blue Team Village.
We have our next talk for the day.
We're going to have Bill talking to us today.
No question, team viewer, police, and consequence.
So take it from there, Bill.
Hello, everybody, and happy end of the world to you.
The purpose of this talk is to explain a situation
I'd found myself in last year,
to detail a breach from the perspective of a small business.
After the story itself, I've got a couple ideas
relating to bringing the bigger ideas about security
to some of our smaller clients.
Alongside that, I wanted to make it clear to anybody
who feels the familiar sting of the imposter syndrome
that you're capable.
I'm presenting this as a narrative,
more of a story than a formal presentation.
The overall scale of this breach is probably something much smaller
than most of you are used to dealing with.
But I do know that there are some people here from small businesses
that tune into the BTV, because I'm one of them.
Online, my name is Corvus Actual,
and I'm the co-owner of a small four-person IT shop in Canada.
This is the story of what a breach looks like
for a small business, start to finish.
So, we start about a month before the breach itself
with my very first trip to DEF CON.
I was overwhelmed with the opportunity.
Everybody says when you get there, there's so much to do,
but there really is so much to do.
I've been watching DEF CON videos for years.
I spent quite a bit of my time at the Blue Team Village,
since that's at least a part of my day job.
I wanted to see if I could pick anything up
from between the talks, the CTF, and the people milling about.
I was given access to the CTF VPN
at the same table as everybody else.
But here's the thing, I flew to DEF CON completely solo.
So, even though I later hooked up with the Lonely Hackers Club,
I stumbled through the first bit of the CTF
without anybody to bounce ideas off of.
I was absolutely thrilled to have captured
a good handful of flags on my own,
both sitting in the hall outside the BTV
and later on in the weekend.
The reason I mention this,
though it falls outside the actual breach itself,
is because that experience,
maybe fueled by the mana of so many like-minded people,
gave me the confidence to jump in there and fight it out,
solo or not.
I really believed that.
I felt like I had some kind of drive or motivation
that wouldn't have ordinarily been so handily available,
thanks to the community behind the Blue Team Village at DEF CON.
So fast forward a few weeks
and I'm sitting in front of my workstation
at about 10.30am.
I get a call from a co-worker
who has seen a weird log file
that's been left on his workstation.
He noticed this text file
left open with a mishmash of characters,
some sort of encryption.
The file referenced a location on his system
that threw a trigger in his head,
enough to call somebody and talk it through.
C Program Data FKL.
After a short chat,
we both set to work on Google
to look for information on this folder,
who we had just discovered was set to hidden.
Hmm, interesting.
After a few keyword searches,
we found a link to a number of sites
selling software called Family Keylogger.
Uh-oh.
A little deeper now,
we started to look into the hidden folders.
We saw logview.exe,
the program for viewing the logged keystrokes
as advertised by the site selling the software.
When we clicked to open the program,
it was password protected.
All the usual default passwords didn't work.
A quick Google search didn't provide any results
for default credentials.
Uh-oh.
So finally, I logged into a number of other computers
via TeamViewer.
On all the major systems I had access to,
we found the folder.
I could feel that deep, gut-turning anxiety
start to bubble up as the situation started to spin.
I immediately tasked our other techs
to meet at the infected client.
They're an organization with size and scale,
but unfortunately, not a lot of budget
to be thrown at traditional IT security stuff.
So, no IDS, no IPS, nothing to track logs.
Add to that, no 2FA,
no ongoing employee training for cybersecurity stuff,
and minimal formal policies in place.
So, we have these client machines,
and they're online.
There's an unknown number of them
that are infected with this keylogger.
During this time, in the course of about 40 minutes,
my coworker hit a stroke of good luck.
After copying out the FKL folder for backup,
he started deleting one file at a time,
just to see if one of the files was locking the program.
He found it.
Again, just by dumb luck,
by deleting one of the files in the client subdirectory,
he managed to unlock that database,
showing the logged key presses,
and most importantly,
the vector for uninstalling the application cleanly.
At the same time, I emailed the vendor.
They noted, of course, that if we bought a license,
they'd be able to unlock the application
and provide uninstallation instructions.
So our technicians got to work backing up found copies of the program
and uninstalling it from infected machines,
while I handled the servers separately.
At this time, it was about 5 p.m.
I was, in all sincerity, overwhelmed.
Beyond the scope of this problem,
our business was expected to maintain operations
and fluidity to the rest of our support issues.
The rest of our clients needed to be supported
and regular business was to continue as normal.
I was dejected. I was defeated.
Especially after freshly returning home from Hacker Summer Camp,
I just felt like I'd been had.
When my wife arrived home,
she told me to take a 20-minute break to clear my head
before giving me the best instruction anybody in crisis could get.
Get after it. Fix this. I know you can.
But this is where the human side kicks in.
This is where I feel like I've already lost.
I know I'm going to have to provide something, anything,
in an email to the people involved,
and I don't know really where to start.
I've worked in IT for more than a decade,
but I don't have any formal certifications in security,
or I didn't at that point.
I felt like this was getting way past my skill set
and into territory that I definitely wasn't comfortable with.
I knew, though, that somebody had to start,
so I just sort of logged in and got to work.
Without any real direction, I just started.
I backed up log files from TeamViewer
before looking through any of them to a separate system for safekeeping.
With no formal training in forensics or chain of custody,
I figured it was a good idea in any case
just to get copies of those log files just in case.
In some instances, I found the TeamViewer
date underscore log file dot log had been deleted.
Luckily, we had offsite backups of that folder
where we could retrieve the log files from
before they'd been deleted.
Secondarily, we had shadow copies as well.
I found a couple servers where the attacker
had deleted the log files, but strangely,
some of them were left untouched on other servers.
At this time, the local municipal police had become involved.
A detective from their cybercrime unit
had come to the main headquarters of our client
to chat with the co-owner of our company.
I asked my partner to let the detective know that I was digging
to stand by for any information that I'd found.
I recognize now that just getting started
and just getting to work and diving in there
without formal training could have cost the investigation.
But in a real-life situation,
as it was happening with real-life consequences,
that's the decision I made.
I just decided at that moment to dive in there and get to work.
I started looking through the TeamViewer connection logs.
I figured I would start with the easiest infiltration vector,
or the easiest vector I could think of.
The remote access application installed is unattended
on the servers themselves.
I considered other options.
I was at least aware of USB rubber duckies
that could have been preloaded with a script
and dropped in the parking lot,
or an installer dropped in from an exploit or something like that.
I figured I would just start at the very bottom and work my way up.
I'd start with what I knew how to handle.
I scrolled through a handful of logs,
stopping abruptly on seeing a peculiar username
having accessed one of the servers.
Okay, that's something.
I made a note of the username and kept digging.
I started noticing this username everywhere.
And when I say everywhere, I mean in multiple TeamViewer logs
on multiple different servers and workstations.
Generally speaking, this user was accessing systems
well outside of our standard operating hours.
Most of the connections were happening around midnight,
some of them just after.
Many TeamViewer installs had another user added as unattended,
named the number zero.
We had to access every system in the agency
to ensure this user was disallowed any further access.
Nonetheless, I documented multiple file transfers
and remote control sessions from this user.
In the TeamViewer logs, I had dates and times
with a connected username.
I figured TeamViewer would be able to lend a hand
in getting more information together.
Since the client systems were all installed as unattended,
I suspect the attacker waited to find
one of our technician's TeamViewer credentials
to log in and access that list of unattended systems
in the agency.
Without evidence, though, I can only assume.
Alongside the detective from the cybercrime unit,
we asked TeamViewer to help.
We were swiftly transferred to a legal department
who told us that since TeamViewer's servers
were located in Germany,
we would require international warrants
to have any data about these connections compiled
and sent for use in a criminal investigation.
At the time, I thought that was just absolute bullshit,
but I do get it.
It was frustrating because we were in the middle
of that scenario as it was developing,
so I felt all this pressure on top of my shoulders
to just summon these fixes
with people who were not unwilling,
but unable to help.
I had an old contact at TeamViewer
that I sent an urgent message to.
He called back and lent some advice
as to where we could look through our own log files
for any more information.
You have to figure we're looking at
maybe 20 or so log files that span several years,
sometimes two or three years,
a real good chunk of data.
At the time, the best I could muster
was literally mouse-wheeling through those things,
but I had this little indicator, that username,
to chase after.
During this time, over the period of about 4 to 5 hours,
I started up my laptop and began scouring the Internet
for OSINT.
I'm not really sure why I did that.
Maybe just to bounce the username off of the Internet
to see if anything useful came back.
Well, I found a user on Twitter with,
believe it or not, the same username
that had logged in over TeamViewer.
Scrolling through their post history, I found,
believe it or not, a post about the very key logger
that was installed on our systems.
So obviously, I doubled down on my efforts in OSINT
for about an hour and pulled all kinds
of relevant information.
For example, we found an email address
in the configuration for the key logger,
where the logs were being pushed to.
The Twitter user shared a Facebook account
with several email addresses published
with the same username.
I found multiple references to nefarious stuff
on the social media profiles linked to this person.
I even found the same username dumping hashes
on a cracking forum.
And then, the breakthrough.
An IP address was found in a TeamViewer log file
that coincided with the time the external user logged in.
I ran it through a quick IP geolocation search
on Google and found it traced back
to a small town in Ontario, Canada.
The same small town both the Twitter
and Facebook account were declared as being from.
Now we're going to refer to the attacker as bad guy.
I brought this to the attention of both
the detective and the client.
I put together a dossier of all the information I found
and handed it over to both parties.
I figured it'd at least be a good starting point.
Here's where things get tricky.
This person, referenced on social media,
was actually known to the organization
as being a person who was enrolled in their services.
The story takes a brief pause here
as the justice system lags behind the real world.
Fortunately, in time, the municipal police
were able to secure a search warrant for bad guy's house.
On the arrival of police services,
bad guy's devices were on and doing bad guy stuff.
They were taken for analysis
and bad guy confessed in full, on site.
After a few weeks, the analysis came back.
Bad guy's computers were full of all the usual stuff.
VMs, multiple pirated keyloggers,
PDFs with instructions, videos, all that stuff.
Signed, sealed, and delivered. Right?
Well, something happened.
During the five months this process took to finally wash out,
something must have happened.
No question.
I was told there was no question that bad guy was responsible,
had conducted these unauthorized connections to the network,
and that he had the means to do so at his ready.
I wasn't told specifically why the charges were dropped,
only that it would be difficult to prove
his intent was malicious in nature.
K?
At the end of the day,
I think this is how bad guy got his initial access.
I don't think this was a particularly technical attack.
I'm not nearly as skilled at detection and forensic stuff
as some folks around here.
But this is what I think happened.
The organization didn't have hard policies in place
about clientele using the organization's systems.
I'd heard through the grapevine that this was happening,
but you all know how that goes.
An email goes out to leadership saying this shouldn't happen.
The email goes down the chain, but at the end of the day,
no policy equals no action.
I believe bad guy had an opportunity at some point
to either access the system directly or,
which is the more likely scenario, just snapped a picture
of the team viewer splash screen with a cell phone.
They could have accessed the system, likely not locked or turned off,
later on that night and started to infiltrate from there.
But here's the kicker.
Bad guy got his devices back and is back on site
accessing services from the organization.
So that's awesome.
All told, we learned a pile of valuable lessons.
We're a small shop, and yep, I've read about situations like this.
I'd say we made it out lucky from a scenario
that could have been much, much worse.
Most of you will be able to read between the lines here
and identify points of failure on our part
that led to this type of breach being a possibility.
I think the largest problem that small IT shops face
is the juggling act required to keep all the elements
of business equally sorted.
I remember from the Blue Team Village last year,
a panel on small business cybersecurity
that touched on that.
The concept that cybersecurity is sometimes left as an afterthought
with some of our smaller clients from budget limits
or lack of concern or understanding.
But the same threats and outcomes still exist.
At last year's DEF CON, I played a game of D&D
with a guy who gave me a great little piece of advice.
Present controls to mitigate risk
and defer responsibility where those controls are not used.
Right or wrong, I think it's a great starting point
for guys like me.
I think there's some use in talking about
the after-action side of things, too.
For the people that work in smaller environments
and hail from smaller companies, I've seen a subtle gap
that exists right before a certain sized organization
where cybersecurity starts to matter more
to the people in leadership roles.
These are a couple things I've been doing recently
that seem to help bring more hesitant people around to the idea
of tightening up security around the network.
If you're selling security services, this stuff might help you talk
about your stack. But I think for anybody in a position
where you're talking to end users about security,
these three things might be a decent idea.
Define the why. Really, I'm trying not to use LinkedIn
like business speak here.
I've used this specific idea.
It works for me. Chip away at the organization's goals
until you can directly attach data security
to the primary function of the business.
Forests Ontario is a local nonprofit
that I pulled off of Google. They help rebuild forests
around here. They have the following goal.
Forests Ontario is dedicated to making Ontario's
forests greener. Our ambitious tree planting
initiatives, extensive education programs,
and decades of community outreach have resulted in millions
of trees being planted each year.
You could just as easily write a few edits into that statement
when talking to shop callers about security
to start emphasizing the importance of security.
For example, Forests Ontario is dedicated
to making Ontario's forests greener.
Our ambitious tree planting initiatives, extensive education
programs, and decades of community outreach are
powered by scheduling software, email, and
a company fleet of laptops, and have resulted
in millions of trees being planted each year.
I've used this approach to help frame cybersecurity
not as an element of IT infrastructure or something
that happens in the server room, but as the starting
point to building a culture within the
organization that values security as part
of how the whole thing runs. Without
cybersecurity, we can't do the things we're trying
to do.
Find a framework. Most of the smaller
organizations I see out there have very,
very little security infrastructure. Most
have next to zero staff training.
Most rely on the IT guy to just do
the security magic and do it quietly and do
it cheaply. Find a published
framework that you can use to better explain why
you're introducing security controls
that cost cash and convenience.
For instance, we found the baseline cybersecurity
controls for small and medium organizations
from the Canadian federal government to be of great
help with both translating some of the technical
side and actually implementing controls.
I found even if the
client calls you to help set up and
secure a network, a written and industry
accepted framework can help, as Jocko Willink might
say, prioritize and execute a short
list of must-haves from a long list
of would-be controls.
Lastly, start talking about it.
This seems really obvious. Out there
in the offices, I see a common theme
between most of our clients. Some just
give us money to fix the blinky lights. Others
want us to verify that we're using every minute
of each build hour. The thread that runs
between them is this. When I talk to them about
security, they tend to listen.
Now, that doesn't always translate into putting better
practices in place, but I can usually
at least get a window to speak at a staff
meeting, to start building that foundation at the
very base of the organization, and
at least start the conversation about cybersecurity.
This seems like a given, to just talk
about security. Too often, though, our clients
are caught up in the grind of their day job, and
they don't have time to worry about security.
More often still, users assume the responsibility
of security falls onto the guy in the server room,
not the person at the reception desk.
Purposefully bringing a quick briefing on cybersecurity
to monthly staff meetings or as part of a smaller
team meeting helps keep that topic on the table.
One of the benefits of security,
cringy as it might be to some of the guys
who have been around DEF CON forever, is that
information security still has that cool factor
to a lot of people who don't directly work in IT
or around security. If you really put some
investment into bringing security into the
conversation, like any other part of your life,
it pays you back.
I have to say, speaking at DEF CON is an absolute
moonshot for me. There are times
I had felt like one of the only guys in my town
that was interested in hacking. From a place
of true gratitude, I thank you all
for the opportunity.
The hard lesson to learn is where your weaknesses are.
Working through the breach last year
served to detail exactly where our failures were.
Since this event, I've started to swing
for some certifications. I've fundamentally
shifted my focus.
At the end of the day, it's all up to you.
And nobody's coming to save you.
Please feel free to DM me on the BTV Discord
or drop your questions in the BTV Discord so I can respond.
With collectives like the Blue Team Village,
we can fortify ourselves to meet real world
challenges, even if we're not fully ready to fight.
Because we have to.
Thank you.