Lastly, now my pleasure to introduce you, Eric Capuano.
I appreciate that.
And also the line that you guys were in to get in here was super humbling for me.
Thank you so much for taking time out of your awesome DEF CON to come and listen to me drone on a little bit about security operations on a budget and some tips and tricks that I've learned in my time.
As Ming said, we go back a little ways because I'm actually a member of the Pack of Hacking Village and the Wallace Sheep here.
I run the Honeypot Project.
So if anybody here has seen that or if you haven't, please come take a look.
We've got a really cool deal going on over here.
So I'll dive right in.
Heart rate is 101 right now.
So let's see how that fluctuates throughout the talk here.
Here.
Oh, clicker.
Okay.
Okay.
All right.
So a quick introduction.
So I am the SOC manager for the Texas Department of Public Safety.
I'm also a cyber warfare operator for the Texas Air National Guard.
In my limited free time, I'm also a private consultant.
As I mentioned, I support the Pack of Hacking Village.
I'm also a member of the Pack of Hacking Village here and the Wallace Sheep at DEF CON each year,
specifically emerging threats, honeypots, and deception systems.
Cyber patriot instructor as well.
And that's something that I think everybody, if you're in the industry, you should be doing that.
Pay it forward.
Teach the kids.
We need more talented folks in this industry.
I'm a blog author.
Not often enough, though.
Security against obscurity.
Blog.ecapuano.com.
And I'm a hobbyist teacher student.
So if you notice, there's all these things over here, and it all revolves around coffee.
A few disclaimers, right?
These are just my opinions and mine alone.
There is no absolute truth in any situation.
Your mileage may vary.
And to cover my ass, all products, vendors, figures, and potentially anything else in this presentation are completely fictional.
And also, the only wisdom is knowing that you know nothing.
Guys, I don't have all the answers, okay?
I just know a few things that I figured out in my journey.
So here's the million-dollar question, right?
Why is enterprise security so hard?
And then why is it so damn expensive, right?
Quick history lesson.
114 years ago, this guy, right, Marconi, he wanted to go out and demonstrate to the public that he could securely transmit information, completely hacker-proof, right?
Anti-APT, zero-day proof, secure transmission of data.
Well, just like today, most of those promises were empty.
And this guy, Maskeline.
He was actually, Neville Maskeline was actually a magician, right?
This guy wasn't a hacker.
He wasn't any kind of a computer dude, but he was a magician.
And he said, I'm going to embarrass the hell out of you.
And so what happened was this guy, he's like, hey, look, I can broadcast things over radio, totally secure.
Trust me, because there's a padlock picture next to the radio tower, right?
And this guy over here is like, nope, what I'm going to do is I'm going to hijack your transmissions and I'm going to replace them with my own.
And I'm going to do this.
And I'm going to talk all kinds of shit about you in Morse code during your proof of concept.
So he totally rocked this guy during his POC just to demonstrate that security is hard.
So here's the question, okay?
114 years ago, this is happening.
Why are we still getting it wrong?
In my opinion, honestly, what it boils down to is a mindset.
It's not stuff you buy.
It's not, you know, magic boxes that you put in your data center.
It's a mindset that we've got to get.
It's a mindset that we've got to get right.
And slides, you've got to get those right, too.
Okay, what does it not come from, right?
Padlock.png on your website.
Magic box in the data center.
Next gen magic box in the data center.
Or spending millions, right?
There is not a direct correlation between how much you spend and how secure you are.
Checking yes.
Yes, on your compliance audit, we're good.
It's a mindset.
Okay, so some common problem statements, right?
These are things that you hear all too often.
I can't hire good people, right?
There's this major talent shortage, apparently.
If that's the case for you, please leave me a business card
because when I post a job in Austin, Texas,
I end up getting about 20 times the applicants
than I can afford to hire.
So I have too many.
I have too many qualified applicants.
So let me know if you need help there.
I can't train the people I have, right?
We're going to talk about some ways to do that.
I'm too busy responding to incidents
to improve incident response.
That's one of my favorites.
And my budget has a lot of zeros,
but they're on the wrong side of the decimal.
This is a really common one.
And this one drives me crazy.
My users are dumb.
And they keep clicking and opening and downloading stuff, right?
Next, next.
Gen APT Signature Avialess Firewall is too expensive.
It's true, it is.
It is too expensive.
But the good news is you don't need it.
If only there was an affordable way to solve the problem, right?
Open source solutions are not feasible because of reasons.
If anybody wants to debate that, please see me after the talk.
All right.
So going into what I think is the foundation for solving this problem,
it's people.
You absolutely must hire the right people.
And then when you have them, you need to take care of them
to keep that team.
So I'm going to show you how to do that.
Because if you can't get this right,
nothing else is going to matter.
This is the foundation for doing this.
So I know you guys have seen this before.
And if you haven't,
this is something to consider.
When you post your job rec,
you're hiring keywords.
You're hiring certifications.
Or at least that's what your HR department is looking for.
We've got to get away from that.
Because keywords do not fill SOC analyst seats
with the right people.
So your cybersecurity job requisition, right?
This Venn diagram illustrates
that the people that perform security jobs
often do not overlay with your job requisition.
Bridge that gap.
Stop hiring based on degrees or certifications.
Okay?
I do not have a degree in cybersecurity.
I do not have a degree in computer science.
But I do this stuff night and day.
And somehow stumbled upon this.
I'm pretty good at it.
Same story for every single analyst in my SOC.
Look for passion.
Look for drive.
Look for someone that has the ability to learn.
Right?
If you guys have seen CISSP Googling,
I mean it's a bit of a parody.
Not to pick on the CISPs, right?
But what it really is here is that this industry
is driven entirely by certifications.
When you're hiring just based on the certs
that someone has, you're doing it wrong.
Instead, take a chance.
Take a chance on the newcomer that has no certifications.
No experience.
But in the interview expresses a passion.
Says, hey, I go home at night and I build
a malware reverse engineering lab
just because I want to learn how this stuff works.
That's the person that you want on your team.
Once you get your rock star team,
you absolutely got to take care of them
and you've got to give them the tools that they need to succeed.
Right?
Especially for someone like me.
I'm state government.
I do not have infinite budget.
I've been in a lot of ways.
So first, enabling effective security operations
communication, you got to get it right.
If you're still using e-mail for security operations,
you're going to have a bad time.
Alright?
You've got to come up with something that can keep up
with the speed of operations.
Because your team's never gonna be any faster
than the method by which they communicate.
So when we migrated away from e-mail based communication,
we actually achieved a rate of about 90,000 emails
thousand useful messages per year between our analysts. It's a
lot of traffic. And that's not just analysts talking to one
another, but that's integrations. That's chat
operations. You know, my analysts are able to blacklist
IP addresses from Slack. You know, that's pretty cool. You
know, if you're on call and you're at the movies with your
significant other and you find out you're getting attacked by a
phishing campaign, it's kind of neat that you can just pull out
your phone and handle the situation. But at the same time,
it raises the situational awareness for everyone on the
team. Train your people. Because hacker typer and the
threat, the Norse threat map are only going to fool your
executives for so long. Training is a must. So how do you get it,
right? Training is so expensive. Well, guys, you know, it can be
done on a budget. Send your folks to DEF CON. Send your
folks to SANS if you have the budget for it. And I'll go over
a few more training options. Documentation. The least
favorite thing among developers, engineers, analysts,
but it's a must have. If you're not documenting your procedures,
if you're not building repeatable standardized
procedures for the things that your analysts do on a day-to-day
basis, you're not going to have a good time. So it doesn't
matter how you do this, but crowdsource it. Each one of the
folks on your team, put them in charge of a program or a tool
or a system, and they need to be the one that develops the
documentation for others to follow. Because everyone on the
team knows something that the others don't know. And that's
Do not capture that. Tribal knowledge will be your Achilles
tendon. So some ways you can train your team, right? CTFs.
Send the team. As a team, participate in CTFs. If you
haven't known, CyberArea.IT, fantastic resource, completely
free. And this was, this was, I wrote this a while back, they're
well over 400 now, free courses in IT and security. Send your
people to conferences, especially ones like DEF CON,
B-Sides. And if you've got the budget, again, SANS is pretty
unbeatable. You've got to train your people for worst-case
scenarios. Now this one, I could have a whole, actually I
do have a whole other talk on just this point alone. If you're
not training your team for the worst-case scenarios, they will
not be ready when that day comes. Because you cannot train
for something you've never, you cannot participate, or you
cannot effectively respond to something that you've never seen.
You have never encountered before. So tabletops are a good
place to start. If you don't have the resources to actually
simulate an incident, then do tabletop scenarios. What if? What
if we were breached today, and someone moved loudly through the
network, and now they're exfilling, you know, MySQL
database across the network. What would we do? And find out
if your team is prepared to work as a team effectively in the
middle of a shitstorm. If you do have the resources, then you
can use simulated incidents. So a DFIR simulation is as close as
you're going to get to actually performing as a team when the
heat's on. And like I said, tomorrow around this time, I'm
giving a talk on how you can do this yourself, to build an
actual incident response simulation platform. But one of
the best parts about this is it can be extremely, it can be
done very inexpensive if some open source concepts are applied
here. Okay, and then this is really more kind of your
administrative, your managerial, but lead. Don't manage, okay,
because micromanaging, it's a morale killer. You've got to
maintain the morale of your team. They're already in a very
high stress environment, working long hours, dealing with
incidents in the middle of the night, Sunday mornings. Morale's
a big deal, right? So what I do once a month, you know, the
first Friday of each month, I let my team go half day. We go
to the local pub for team building. I mean, it's things
like that that improve morale, that keep people happy, that
keep a good team with you.
Once you've figured all this out and you can keep a good team,
that's when you can move on to the more technical sides of
these things, how to do security operations just a little bit
better without the latest tool that leverages artificial
intelligence to take zero days in your environment. So be less
busy with this one simple trick, right? Love it. It's my
click bait, sorry. By shifting from a reactive to a proactive
posture, right? And that's one of those buzz words that we love
to see and hear that everyone uses, but no one can
explain. So how do you start? How do you achieve this, right?
Well, first thing we have to do is identify where are your actual
threats? Like, where are you most vulnerable? Not where the
latest headline or vendor tells you that you are. And here's
some extreme examples. Fansmitter. Who remembers this
one? Transmitting information across air graphs using the CPU
fan RPM. Who's seen this in their environment? Raise your
hand.
Exactly. So why as an industry do we care about things like
this? When this hits the news, we all go crazy, right? Everyone
starts tweeting, oh, my God, they're exfilling data with CPU
fans. In a laboratory at MIT. Not in the real world, okay? So
we need to stop getting all crazy about the theoretical
nonsense and focus on things that apply, like word macros.
Now, here's an extreme on the other side of the spectrum,
right? Is anyone familiar with the word macro? It's an
IP address with APT squirrel cyber squirrel one. Now this is
real. These numbers are real here. Somebody actually did, ran
the numbers to find out how many major outages were caused by
animals. 927 major information system outages caused by
squirrels. So you've had zero incidents caused by APT or
fansmitter, but we've had over 927 across the nation caused by
squirrels. So this is real. So we're going to try to figure out a
So should we invest in anti-fansmitter or anti-squirrel?
Again, extreme example but just to show you the differences on
both sides. You've got to be aware of where your actual
threats are. So think critically and apply this, apply this
mindset to all of the incidents that your team's handling. When
you encounter an incident whether it's phishing, we'll
call it phishing, how frequently do you see this type
of incident? Does it have similarities to other recent
incidents, right? This is how you're racking and stacking to
prioritize the things that you're encountering. Now let's
say you're just seeing a rash of word macros that are coming in,
infected word macros that are dropping ransomware in your
environment, okay? Well what type of in-house proactive
controls can you put in place to stop that? Doesn't mean rush
out and look for the latest anti-word macro product, there
are actually things you can do in-house. So let's apply it to
a real world scenario. So ransomware, right? You're
looking for a real world example. So last six months at
DPS, well, we ran some numbers. We saw over 140 ransomware
campaigns and we've got a user base of about 10,000. So if one
campaign went out to 400 people, on average, then you can do the
math on how much exposure we have to this problem. So it's
real. This is definitely something that we were concerned
about. So we did a little bit of research, right? And I just
want to say this threat research, I've never, it's not
been shared with the public till now. So where was the
ransomware coming from? It was coming from emails, right? Email
attachments. Really advanced delivery mechanism and
fortunately, we have some in-house controls that we could
use to mitigate this threat. But what else do we want to know?
What else do we know about the delivery? What else do we know
about the threat? That if you're using ransomware, you
know, if they're macro enabled, macro enabled documents, we can
actually control macro execution on our workstations. So we stood
up a CA and we required digitally signed macros now on
our environment. So now we've, we've, we've stopped, we've,
we've stopped malicious emails at our boundary, at our
perimeter. And we've also mitigated the execution of
macros on our workstations and it cost us nothing but time. So
we've, knock on wood, 100% of our workstations are
100% mitigated this threat without spending a dollar. If
anyone's curious about how you can go about doing that, actually
have a, a blog post. You can go and see step by step, even from
selling your ex, your executives on this to actually implementing
the plan and rolling it out. Yes. So we found that, that, that,
that eliminated about 91% of the ransomware that we were
encountering on our network. The remaining 9% were coming in the
form of, you know, the ransomware that we were
using, you know, Billy's resume dot PDF dot EXE dot JS dot WSF,
et cetera. Right? So it's, it's, it's fooling the, the average
user that's not paying really close attention to their file
extensions to running executable code thinking that it's
something else. So what we, we did was we looked at these rogue
file extensions that were coming in and we noticed, hey, wait a
minute, you know, Susie in HR does not need to run dot JS
files on her workstation or dot VBS files. So how about we
just change the default opener for all these weird files? So,
weird file types to notepad so that instead of double clicking
and running script, she double clicks and sees script and no
harm is done. Another completely free solution to a
very real problem. So, that was a good, that was a good win
because with all that money I saved, I was able to hire
another analyst. So let's look at one more that I'm sure
everyone here has probably dealt with, right? Phishing
campaigns. So this is, this is one that I think a lot of teams
are, are losing, are losing a lot of their time and they're
losing a lot of time over because we're, we're, we're
still playing IP address and URL whack-a-mole, right? Quick,
blacklist the URL that's never going to be used again. There's
probably a better way to do this and you don't have to buy a
special tool to do it. Any kind of IDS, IPS, including a free
one like Snort or Suricata can do this next, this next thing I'm
going to show you. So let's take, we got a phishing URL, it
takes us to this really cool looking Facebook site that's
obviously not really Facebook. How can we proactively stop a
this with next, next, next gen anti-APT technology? Well, let's
just look at the source code and that doesn't look good at all,
I'm sorry. But if you can, if you can make this out, the
source code of the page actually has an entry for the tool that
was used to clone this website. So HTT, HTTAC website copier. I
don't know if anyone analyzes phishing pages often, but this
one is very frequently used. And it's, it's, it's, it's a
great tool. And our red, our attackers are way too lazy to
pull it out. So let's capitalize on that, right? I can write a
really quick IDS signature now that's going to detect this
string and now proactively stop not 100% of phishing sites, but
a hell of a lot of them, and it costs me nothing. And it saves
my team a ton of time. So again, it's finding ways to
proactively prevent incidents, leveraging things you already
have.
Cool. So moving on, we're getting a little better at this
security thing, right? Thinking proactively, what can we do to
reduce our, our, our attack, our attack surface without spending a
ton of money? But at the end of the day, guys, security does
cost money. So how do we get money, right? You need executive
support. But there's a right and a wrong way to go about getting
it. And if you're not getting the executive support that you
need, there's a chance that you might be selling it wrong. So
how do you do it right?
Instead of asking for it, convince them why you should
have it and it will come. So example, stop using the fear
tactics, guys, okay? Don't go into your CISO or your CIO's
office, if you don't give me this much money, we're going to
get ransomware. He's heard it before and he doesn't want to
hear it anymore. Instead, what you need to do is you need to
put the stats in his face on how much awesome you're already
doing with the nothing that you have, right? That's not common
actually. Most IT folks go in and say, well, I could do that.
But I won't because I don't have budget or I don't have people.
Okay, well, that guy is going to continue to not have budget or
people. But if you just go ahead and kick ass with the little bit
that you have, generate some stats and go and put that in
front of the executives, all of a sudden, they're seeing value.
Okay, I'll throw a little bit more money your way. But get
away from the fear stuff because it's counterproductive. Also,
showing your executives what you're doing and showing your
users what you're doing. Publish a newsletter. Honestly, it
sounds like a waste of time, but I'm going to tell you that's
something we started doing that has paid dividends. Publishing a
newsletter to our 10,000 users about what our team actually
does to protect them, we get a flood of thank you emails every
month after we put that newsletter out. And we honestly,
we put it out just as like a trial, like let's see what
happens. People actually give a damn, like seriously. So put that
out because now you're winning friends. You're gaining
influence.
You're getting a lot of comments, which trickles up to
executives, which trickles back down to you in the form of
budget. And it turns out that your C-level folks do like
honey. So, securing all the things without spending all the
money, right? Because budget doesn't always come, but
security must continue. So how can we do this on a very small
budget? Well, first of all, we're going to have a lot of
people say we're going to have to invest on this. So here's the
little trick. We're going to talk about breaking bad habits,
building your own tools, ditching bad solutions starting with the
pew pew map in your sock. So here's my advice on how to do
this. When it comes to the stuff that you already have,
right? Stop suffering from contract renewals. Let's just
renew this because we've had it for years. We just, we need it.
Let's just renew it. But hold on a minute. You're spending a
And I've got a few examples for you. But stay away from these statements here. Well, we've always
used it, so just renew it, right? Or why have we been paying for X when we're only getting Y?
And then the big brand romances, right? If you're in love with big brand and that's why you're
hanging on to it, then you're living in a silo. When you're assessing a problem that you're
trying to buy a solution for, absolutely ask yourself the question, like, is this actually
a problem that exists or is this just something that you've manufactured that someone else has
manufactured? You know, my CISO comes to me all the time, hey, we should buy this new anti-APT.
And I say, okay, but we have these other four tools already in the environment that do what
you're asking. Do we really need to buy this? And then ask yourself, are we even using the
tools that we already have? Because that is a huge issue right now. How many tools are in
your environment and how many people on your team know how to use all of those?
That answer is always a frightening one, right? So instead of buying more stuff, use the stuff you
already have. But when you do need to buy something, do your research. Don't talk to vendors.
Gartner, it's a love-hate thing, but it's a better place to start. You know, NSS labs, read
the reports, find out how these tools actually perform in real-world environments. Resist the
marketing. And then always demand a hands-on demo in your environment, because you're going to be
getting paid for it. Because it's just like clothing. If you don't absolutely love it the first
time you try it on, it's not going to get any better after that. Also, buy what fits in your
environment. Nothing sucks more than buying a device or a component or software and then
finding out that it doesn't integrate with anything, but that manufactures other stuff. Big
firewall company sells a threat Intel platform that, guess what, only integrates with big
firewall company's firewall. Bad data. Bad data. Bad data. Bad data. Bad data. Bad data. Bad data.
Bad move. That's how they're trying to trap you into their ecosystem. Most importantly, why buy
what might be free? Ask yourself, is there an open source tool that can solve this problem? The
answer is probably yes. I'm going to ask you guys to consider something, right? Why is it that
security product, why, costs millions of dollars, but the security product, why, costs millions of
dollars? The social engineering tool kit that Boris just used to break into your company is
free. Slippery slope, right? What if I told you there was a free tool that could block Tor
exit notes, blacklist phishing URLs, automate analysis and incident response, pull hundreds of
open source threat feeds, deceive your adversaries and warn defenders in the early stages of
attack and almost anything else you can imagine. Has anyone used this tool? You probably
have. Python. Come see us in the honey pot project where we're actually doing a lot of those
things with Python. Constantly assess the tools that you already have. For every noteworthy
security incident that your team is handling, you need to be holding your solutions
accountable. Are the indicators of this attack new or old? Because if they're a year old but your
sensors did not detect them, you need to be making a phone call to attack. Now, let me tell you a bit about this. The first thing when you
Why the hell did this get through my million dollar IPS
when I found open source intel on Twitter from a year ago for
this attack. That's embarrassing and that shouldn't be, you
know, that's not something vendors should be getting away
with. So ask yourself what systems had visibility over this
threat. If it was an e-mail born threat, obviously your e-mail
security appliances had visibility. Maybe your IPS had
visibility and then obviously endpoint agent had visibility.
So those are three solutions that you should be holding
accountable to why did this incident happen. And hold your
vendors accountable for it. So I'll give you guys some really
quick examples, I'm going to glaze over these, on how we've
walked the walk at the Department of Public Safety in
Texas with said vendors. So scenario one, big brand e-mail
security appliance, right? Big brand usually means big
marketing. We were paying $234,000 annually for these
systems. Terribly difficult to use, configure. Stability sucks,
they don't cluster. Zero bells and whistles. I mean it was just
basic e-mail security. I mean it was just basic e-mail
security stuff, right? $234,000 a year. I replaced them with a
solution, also a big brand, but very little marketing. And that
saved me about $200,000 a year. Way easier to configure. Oh, and
it came with attachment sandboxing. Pretty cool feature
for e-mail. And actually our in-house testing showed better
security performance. Win-win. Scenario two, perimeter UTM next
gen firewall. Now this is a big brand that's actually a
marketing company. This is a big brand that's actually a
sales firewalls. $750,000 buy in. $274,000 a year with a 4%
increase annually. Terrible support. Absolutely terrible
support. Matter of fact, they once held support for ransom
because I was a month out from my renewal and they said we'll
help you as soon as you renew. False negatives on threats with
year old OSINT. Hey, I just used that example, right? And
then usually the response from support on said incident is, oh,
good catch. That's going to be in our next signature. That's going
to be a signature update. I'm glad that I pay a quarter of a
million a year for these. So migrate to almost as big brand
with no marketing. $36,000 buy in, $40,000 annually. Magic,
awesome greatness. We're so happy. I'm saving a fortune,
guys. What does that equate to? That equates to more SOC
analysts because if you recall from the beginning, people,
that's how you do this right. So then let's get to the fun
part. Where's all this free stuff, right? We're going to do
this. You're telling me to use free stuff. Where is it? It's on
pirate bay. I'm just kidding. But try GitHub, okay? But there's a
few of you out here. You don't have to identify yourself but
you're saying to yourself open source has no place in an
enterprise environment. Well, I've yet to hear an argument
that actually sticks because I have a single DevOps engineer on
my team that's able to maintain about 14 open source projects
that brings a ton of value to my team but I'd be glad to have
that conversation with anyone that feels strongly. I will say
though, if you're totally against open source, when's the
last time you looked under the hood of the commercial product
that you have? What do you think it's running on? Open source
stuff. So, you know, at the end of the day, yes, there's a lot
to be said about the support number you can call when things
break but when you're doing security on a budget, you're not
going to be able to avoid using open source, nor should you.
It's a fantastic resource to use. I'm going to go back to
this. This is a good one, right? You would not get clone a sim,
would you? Right? Why the hell not? Because, guys, think about
this. In the context of InfoSec, right? Hacking was built by
communities of smart, passionate people exchanging free and open
information. And the good news is that still exists today. You
just have to tap into it. So here's an interesting thought
for you, right? If you think back to the Cold War, right? One
school of thought about why Russia lost the Cold War is,
well, we outspent them, right? You know, they lost the Cold War.
They basically, they went broke. Well, consider this. What if
during the Cold War, our weapons cost money but Russia's weapons
were free? Do you think it would have ended the same way?
Think about how that applies to cyber security today, right?
We're spending millions of dollars on defense. Our
attackers don't have to spend a dime. We're not going to
outspend them. So it's just another way of looking at
leveraging open source tools that you're evening the playing
field a little bit.
So there are so many great open source projects, guys, that I
could have spent this entire talk just listing them off. So I
won't. But I will cover a few. And then I will publish these
slides later on. But one that's certainly noteworthy, that
everyone in this room, if you work in a SOC, if you manage a
team of security analysts, you should check out this one. The
Hive is an incident response collaboration platform. It is
phenomenal. It costs $3,000. That's free with an F. And it
is phenomenal. And it will, I'm not going to go into too much
detail, but it will allow your analysts to collaborate on
incidents, automate analysis, track observables, you know, all
the magic stuff that a $140,000 a year platform that I could
name but won't does. But it does it for free. And here's
just a little screenshot of the Hive. It's actually been updated
a few times, so it's even cooler and better than this. And
by the way, if anyone does use the Hive or is interested in
Hive stickers, they actually shipped me some to hand out
after this talk. So I have some swag from these guys. The Hive
is also highly integratable. Like I said before, how does a tool,
you know, fit into your environment? Well, this one does
very well. Here's some other things, right? Sim. So if anyone
here is familiar with AlienVault, AlienVault has a
freemium sim. It is $300,000. And it does all of the things
that your $2 million sim does. Asset discovery, vulnerability
scanning, hits, nids, behavior monitoring, and it integrates
with a free threat Intel platform. Pretty cool stuff,
right? If you're doing security on a budget, that's a lot of
capability for nothing. Log aggregation, right? We all know
Splunk. Splunk is awesome, but Splunk is not cheap. So if
you're on a budget, GreyLog, Elk Stack, which GreyLog actually
runs on the Elk Stack. Grafana, there are free tools that can
do all of those things. For HIDS, Antivirus, OSEC, Clam AV,
these are phenomenal tools that guess what? A ton of those
enterprise grade, super expensive platforms run these tools
underneath the hood. Vulnerability scanning, OpenVAS,
NIC2, WAPT, the list goes on. There's so many of these.
Intrusion detection, Suricata, and Snort. They're very similar
one is a fork of the other. Firewall, PFSense, Indien. Again,
I'm not going to say that you're going to want to replace all of
your systems with these tools. I'm just saying that in the
absence of the budget to buy the enterprise grade stuff, these
things will get the job done. IOC sharing, right? AlienVault
OTX is a personal favorite, but Anomaly Stacks, there's plenty of
these things. If you're into the really fun stuff, Honeypot.
That's right. Check out Modern HoneyNet, Cowrie Kippo, RDPi for
remote desktop stuff. Tons more in that area, threat research.
So, again, I know this is a wall of information. I'm kind of
blasting through it really quickly, but I will publish these
slides and I'll also, if anyone's interested in open
source tools in any other area, let me know. I've got a long list
of them. Forensics, if you're into that. Google's rapid
response platform, live collection, magnet ram capture.
These are all free. Every one of these tools is free. You can
download them. If you're paying a ton of money for some sort of
CBT to train your user base, PhishMe actually publishes free
CBTs that you can load right into your LMS and train your
entire company with. That would probably save a lot of people a
lot of money. I probably don't need to tell anyone in this room
about Cali or Metasploit or SCTK. And that's all I got for
you guys. So if you have any questions, if you want to share
intel, feedback, any of that good stuff, tell me that the talk
was terrible. Please, you can meet with me afterwards or find
me on Twitter and I'd love to hear from you. Like I said, come
see us at the Honeypot project. We're doing some really cool
stuff over there. And also if you're interested in building
your own incident response training simulations, I have
another talk at this time tomorrow about how to do that.
Way more technical. So thank you guys. We're going to spend what?
10 minutes for questions. We've got 10 minutes for questions.
All right. You have a question. I'm sure there are plenty of
questions. Please. Sir? So he's asking about what sort of
metrics do we report up to executives. So that's going to
vary based on obviously what you've got in your environment
now. So if you already have some sort of IDS component or
vulnerability scanning, you know, e-mail security, something
that's producing a lot of data, you're going to be able to get
logs that you can generate reports from. That's an
excellent place to start. Don't fall into the trap of, you know,
hey, our firewall stopped 400,000 threats today because
eventually they're going to ask you what that's based on and
then they're going to find out that that's a pretty empty
metric. But if your team, like my team actually, we write
custom IDS signatures every day. We'll probably average anywhere
between 5 to 15 custom IDS signatures a day. That's a real
metric. I report that up to my executives and they're like,
hey, that's a metric. And I report that up to my executives
because that's stuff that says, hey, look, the team that you pay
for, that I hired, that runs this SOC is producing high
quality things and we're sharing them with sister
agencies. That's a metric that hits home with my executives.
They're like, hold on a second. We're making this stuff? Yes,
you bet. So metrics like that have a lot of weight. But, you
know, until you get there, you know, do your best to report on,
you know, hey, because we recently fine-tuned our e-mail
security appliance, we had an increase of threats stopped. We
increased by 30 percent. You know, things like that. That's
what your executives want to hear about is what are you doing
with the stuff you already have? Then we'll talk about giving you
more. They want to see, you know, executives, they want to
see value before they invest. It's not the other way around,
which we want it to be, right? No, invest. Then I'll show you
value. No, it's got to happen the other way, which sucks for us
because you got to do a lot with a little to get there. Anybody
else? Sir?
One of the most common kind of pushbacks I get on open source
software is the support aspect. You bet.
I love it. So he said one of the arguments about leveraging open
source in the enterprise environment is the support,
right? There's no 1-800 number to call if this thing breaks. But
here's another cool thing to think about. So the Hive, that's
one of my favorite projects. So when we started using the Hive,
one of my analysts, she found a bug. And we put in an issue in
GitHub and it was fixed in 48 hours. Do you know any commercial
vendors that will fix a bug and they're going to fix it in 48
hours? No, because you are just one of millions that are sending
them money. Yes, we'll put that in our queue, thank you. Open
source projects are open source because anyone can contribute.
If you don't like the bug, you can fix it. You know what I
mean? So yes, there is a difference between having a 1-800
number to call at 4 o'clock in the morning on Sunday. That is
sometimes a saving grace. But I will say that open source can
compete. Because if it's open source, that means there's going
to be tons of, in some cases, there's tons of documentation
publicly available and there's a repository where you can submit
issues or submit a patch, submit a bug fix yourself. So, sir?
Okay, great question, great question. So very real problem
too because he got sick a couple of weeks ago and we were
feeling the pain. So his question was, so I mentioned I
have a single DevOps engineer maintaining 14 open source
platforms and he does a phenomenal job. What happens
when he is not here? If he gets hit by a bus, am I in trouble?
Yes. So luckily, if you go back to one of my recommendations
about documentation, right? So I do not allow anyone on my team
to maintain all of their TTPs in their head. If you have things
that you do on a daily basis, routine, maintenance items,
incident response items, that needs to get documented. So if
you are not here, anyone can pop open your procedure and do
what you do in your absence. So that guy, while he is really good
at what he does, he has documented all the things that
he does. All his systems, his diagrams, all that are publicly
accessible, publicly, but by my whole team. So if he's not here
and we need to restart a system or we need to go in and change
something, we've got the information we need to do that.
But the real solution to that problem is not to have a single
point of failure also, right? So if you only have one highly
specialized person on a team, you're not going to be able to
team, then you've got a choke point. Ideally, you'd have two.
But I, like the talk covers, I'm doing security on a budget. So
I've got to make do with a single engineer. So I compensate
with those procedures for continuity purposes. How much
time spent documenting? How much time spent documenting? I
would, this is going to be a stab in the dark. I'd say
probably about, you know, 10 to 15 percent of their time,
because I don't say, hey, stop work and go write documents. What
I say is, while you're working, keep your documents up to date.
Because, you know, the best way, the best time to be documenting
something is while you're doing it, right? So maybe you're just
adding, you know, a few extra minutes to the task, because
while you're doing it, you're jotting down notes over here
about what you're doing. So I wouldn't imagine it's a ton of
time. Yeah, no, but the return on investment is significant,
though. It's significant. Sir? I'm sorry. I'm sorry. I'm sorry.
He's asking, in my portfolio of open source solutions, is there
an area where a commercial product would be recommended? So
yes, I would say there's always going to be areas that, you
know, it's going to be better to put a commercial tool. And not
necessarily because it's better. It's a better tool to put a
better tool, or, you know, it's faster, or more next gen, or
whatever. But for instance, you know, for an agency my size, like
I said, I've got over 10,000 users, and we're spread across
460 remote locations. So I have a very big network that's
literally the size of Texas. So what it boils down to is I need
a very reliable infrastructure. So if I'm looking at, you know,
perimeter firewalls, IDS, IPS, and things like that, I'm
dealing with, you know, 40 gig throughput in the core. I'm
dealing with 10 gig throughput at the perimeter. I'm dealing with
that. I need to have things that can handle that sort of traffic
without issue, without hiccup. And then if there is an issue or
hiccup, I need to have a vendor that I can be hanging from the
ceiling until he gets it right. Because it's just too critical.
It's too critical for our infrastructure, right? Would I
be happy to put an open source tool on that perimeter?
Absolutely. But at 4 o'clock in the morning on a Sunday, I'd
much rather call brand X to come in and fix this. So yes, there's
always a time and place for that commercial grade product.
Credential management. Are you talking about, say, for my
security team or for my entire agency? For the security team,
we leverage LastPass. Phenomenal tool. They leverage perfect
forward secrecy, all the encryption's done, client side.
So you're not sharing your secrets with LastPass. They're
simply holding them for you. We had started off on a local, like,
key pass type thing where it's reading, writing from a flat
file sitting on a network share. It's okay from a security
perspective. It's terrible from a collaboration perspective
because what happens when two people are writing to it at the
same time? And so the best tool that I found, and there are
others, don't get me wrong. There are others. Dashlane's one
of them. But the best tool that I found for the job is
LastPass. They do a hell of a job, especially for teams. So
that teams can share highly sensitive information without
trusting your stuff to a third party. It's pretty good stuff.
Yes?
.
.
.
.
.
.
.
Absolutely. So, yeah, you bet. So my
DevOps engineer is not only maintaining the applications
that we use internally, but he's also maintaining the underlying
infrastructure. So, yes, the ESXI, the, you know, route
switch, everything. Everything in that environment is under
his. Now, luckily, though, being a cybersecurity team, we've got
in-house network engineers. We've got in-house, you know,
sysadmins or previous, former sysadmins. And so that's one of
the cool things about a security team that you guys absolutely
need to tap into is that almost all security teams are going to
have security people who are previously something else in IT.
They came from sysadmin. They came from network engineering.
You know, they came from development. So leverage those
tools. Don't just look at your people and be like, oh, you
used to be a developer. That's cool. Well, now you're this and
you're just this. So do that, you know. No, tap into that.
Hey, you used to be a developer, right? Can you help us automate
this thing that takes us all day long to do? Because that's what
you did. You're a developer. You're still a developer. So
leverage those tools. Leverage those experiences.
.
.
You bet. So he's asking about that newsletter and what sort of
information garners the most attention and appreciation from
our users. So what we found is obviously so our security team,
we see a wide, you know, breadth of things, you know, all kinds
of things. Things that may not really impact the everyday user,
right? Like really unique threats that no one's going to
see or notice. So obviously you want to write about things that
are applicable to everyone, you know. So write about things that
are not applicable to everyone. Write about the things like the
IRS tax scams, right? That's something that everyone needs to
worry about. Grandma needs to worry about it. You know, like
everyone needs to worry about that because it's indiscriminate.
It applies to all of us. And so things like that that are going
to have higher impact are the things that are going to affect
your audience. Don't write about, you know, the latest zero
day in Adobe because they don't care. They don't care. You know,
as far as they know, as long as they're updating and patching,
they're fine, right? Which to an extent is true. But, you know,
write about things that, look, no amount of updates or patches
are going to stop you from getting an IRS scam e-mail. You
need to know what that looks like. You need to know that
that's a real problem. So that's the kind of stuff that we
tend to write about. Ma'am?
Awesome. Awesome.
And we actually don't have any security staff. And I recently
learned about a library. And I want to start getting some of my
colleagues up there to do that. You bet.
Okay. Okay. Love it. Love it. Absolutely. So her question is
essentially how do you make time for your team to train? If training
is so important, right? Well, I stand by what I say. Training is
important. Therefore, it's a function of your job. Therefore, you
can do it at your job. You must be, you must enable your people
to spend time training without requiring them to do it on their
own time. Now, I will caveat that with you will not succeed in
this industry if you are not dedicated and motivated enough
to train in your own time. If you're a nine-to-fiver and
that's it and you go home and you just leave work at work and
you don't care, okay, well, you're just going to float and
kind of coast right where you are for the rest of your career.
That's fine if that's what you're into. But I will say that
I specifically, I make time, I let my people do training at
work.
I have a couple of guys doing OSCP right now. I bet if anyone
here has done OSCP, when did you do it? You did it nights and
weekends, you know, many of them, right? No. If I'm asking you
or if I'm enabling you to take OSCP, I'm going to give you time
during your day to do it within reason. So, enable your people
to get training. That's critical. Half my team is here at
DEF CON right now and has been. So, you know, taking time away
from work for training. Sure.
What kind of validation environment? Okay, great
question, right? Because always, always, always validate your
tools, especially in security, especially in forensics. You
know, don't just, you know, get cloned that repository without,
you know, giving it a once over, making sure that it's got a
reputation. So, how we validate our stuff, usually it boils
down to this. Between those of us on the team that have
experience with a particular tool or platform, we're
personally vouching for it. So, the Hive, for instance, I've
been working with it for some time before I even brought it up
to our team. Hey, guys, let's use the Hive. I'd already looked
at the code. I'd actually already pushed a commit to the
repository. So, I was pretty familiar with the code base. I
said, okay, this is a trustworthy, it's a stable
platform. You know, so there has to be validation. I'll be
honest with you. I don't have a structured validation plan aside
from just, hey, what's the reputation of the repo? You
know, who are the developers? You know, how frequent are the
commits? One thing to definitely look for is how frequently do
they push the code? How frequently do they push the code?
To this repository? If it hasn't been touched in years, you
need to be aware that it may have issues and that they're not
going to get fixed. So, how fresh is the project? You know,
how common, how frequently updated is a big one for sure?
Sir? So, if you had a small team, what would maybe the top
tools be to implement? Okay. I'm imagining you have a
security perspective and you're starting with nothing from a
security perspective. Okay. If you're starting with nothing,
well, you probably have some things like maybe a firewall and
you know, some other things. Okay. The very first thing, the
most critical thing that you need to have is a SIM of some
sort because you have all these systems already, right? Just in
a native Windows environment, you've got domain controllers
that are generating millions of logs. You've got firewalls
generating millions of logs. Maybe you have an email
appliance generating millions of logs. If you don't have a
central pane of glass to look at all that stuff, you're not
going to be able to find anything anywhere else. So, if
you're going to be able to find anything anywhere else, you're
not going to be able to find anything anywhere else. So, the
first thing, log aggregation. Have all of your things sending
logs to a central place. That's priority number one. Grey log.
Elk. Phenomenal tools. Extremely powerful. Do I need
to go? Okay. Okay. But, and then for SIM, OSIM. AlienVault
OSIM. It is a completely free SIM platform. They hold back
like two features that you would otherwise have to pay for and you
wouldn't even know they were gone. Those two features. I
mean, 147 capabilities, too, that you'd have to pay for that
you've already replaced with grey log effectively. So, that
would be my top two right there. Log aggregation and SIM.
Because you're also, with OSIM, you're getting vulnerability
scanning, passive asset detection to learn about all
those things in your network you don't know exist. All kinds of
cool tools. The list goes on. If you want to know more about
it, meet with me afterwards. I'll tell you some more. Wrapping
up? Okay. That's all I got, guys. Thank you.
Thank you.