Hello, everyone. Welcome back. My name is Rubix1138, and I welcome you to Serocata,
an introduction into OpenSoC Capture the Flag tools. I'd like to welcome Josh to the stage
here to talk about Serocata. A reminder, if you are not already in the Discord chat,
please go to BlueTeamVillage.org, click on the links for
DEF CON 28, and join our channel. So let's please give a warm welcome to Josh. Thank you.
All right. Thank you very much for the introduction, and thank you to the Blue
Team Village and all of DEF CON for all of the hard work that I know is going into this,
and the opportunity here to talk to you all today a little bit about Serocata.
So this will be really a brief introduction intended to give you some idea as to what
Serocata is capable of. Hopefully there's a few things that you learn about that you didn't know
it could do. I'm going to use a VM here to demonstrate that. So I have a lot of the text
and all the UIs here fairly large. It should be legible, so you can see that. A little bit about
Serocata then before we get too far into the technology is that Serocata is an open source
project. Everything that you see here that I'll be talking about today is also open source. So
there'll be a few tools I'll touch on beyond Serocata, but most of these are developed and
supported by the OISF, the Open Information Security Foundation. So this is the 501c3
nonprofit that manages and maintains and supports Serocata and all the activity around that. So
we have a very globally diverse team of executive development trainers and others
that support and really make this project happen. There's also a very active community,
and we have a number of different platforms that we have available to get engaged in the community.
And so I would highly encourage you, if this is a new technology to you, to reach out to those
communities and get involved. This is me. You can find me located just about anywhere on the
internet. I am available at this email address as well as on Twitter. But if you search for my
name, you'll probably be able to find me. And please feel free to direct any questions or
anything I can help you with after this presentation in the future to any way you
can get a hold of me. So to get started then, many know Serocata, or if you know Serocata,
you recognize it as an IDS, an intrusion detection system. And likely in the OpenCTF scenario that
you'll be engaging in, what Serocata will be doing then is generating IDS alerts,
intrusion detection alerts. So that is, of course, one of Serocata's primary capabilities and primary
roles. But it can do quite a bit more than that. It can do things like full packet capture. It can
do protocol-specific logging. It can do file identification and extraction. It can do offline
PCAP processing. So you can take a PCAP maybe from our sandbox and run that PCAP through
Serocata to get IDS alerts and other protocol logs. I'm only going to talk for 15-20 minutes
or so. So I want to touch on and give you a real brief demonstration about that capability.
Realize, though, that we could probably spend days going through any one of those aspects. So
again, there's a lot of resources that I can help point you to if that's something that would be of
interest to you. Now, as an IDS, Serocata, again, primarily is generating those IDS alerts. And
with those IDS alerts, then, they come in the form of rules. And rules, then, are simply put.
They're a syntax, a pattern, or a series of patterns that are then applied to your network
traffic. And if those patterns match, an alert is generated. Now, alerts can be for a large,
wide variety of categories. You can have alerts around malicious behavior, but then you can also
have alerts around policy violations or anomalies in your network. Let's say you have non-encrypted
HTTP traffic going over a standard HTTPS, a TLS port, like 443. Not necessarily malicious, but
certainly could be something that you'd want to know about in your environment. So the rules can
be very broad, and they can provide different looks into the traffic in your environment
beyond just the known malicious behaviors. Rules, if you are familiar with things like Yara,
Yara signatures, you write a pattern and you use that Yara rule to match on different things,
that's a lot how the rules work. From a practical perspective, most users are going to get their
rules from a rule source, and they're going to feed them into the engine, and then they're
going to monitor when those alerts are generated. You can, of course, create or construct your own
rules, but that's something that is certainly a capability of the engine. You can define custom
rule sets, but a lot of users are just going to consume those. It is important, of course,
as alerts are generated, as you'll see here in just a moment, to also understand the rule syntax a bit,
at least to the point where you can understand what that rule is telling you. So you have that
ability to look at the rule itself as well. But again, rules are really a topic in and of
themselves, so I want to stay a little bit more focused on the pragmatic side at this point.
Now, I mentioned that rules, the way that we can look at rules is that rules come from
sources. Those sources then can be combined to create rule sets, and those rule sets are
what we feed into the engine, into Suricata. A very popular rule set, an open rule set,
so an open source rule set, is the etopen. And Suricata, by default, will use that rule set.
There are other open source rules that you can go and consume, that you can configure your engine to
use. And then it really becomes up to you to determine the rules that you need or want,
depending on the environment that you're using Suricata in. For example, I use it for a lot of
malware analysis, so I grab pretty much any rule set I can find, because I'm okay with any
individual analysis generating a lot of noise. In a production environment, I'd be a lot more
cautious or careful. The VM that we're in, I maybe jumped ahead of myself just a tad,
the VM that we're in is called CELTS. This is, believe it or not, my system crashed just before
this presentation, so I had to restart. This is also an open source distribution, it's Debian-based,
it's designed to really highlight all of the capabilities of Suricata. So you can, if you're
not familiar with CELTS, or if you're not familiar with Suricata, you can go to Statements Networks,
you can download CELTS Suricata ELK Stack, and you essentially have the VM that you see here
in this demonstration. The only minor differences will be a script that I run, and then I have some
PCAPs in here. You're not going to have any PCAPs when you download this. Otherwise,
everything else should be the same. So that's what CELTS provides you. It's sort of like a
security onion. It doesn't have some of the host-based stuff, and there's a number of differences
in tools. With this, then, you have some interfaces, such as Sirius. And Sirius, then, is a graphical
interface that allows us to manage rule sets. This comes in CELTS. Now, if you were to install
Suricata, at least a more recent version of it, by default, Suricata comes with Suricata Update.
So it's a command line utility that has been developed to manage the rule sets and the rule
sources for your Suricata instance. I'm not going to run that here, because I already have CELTS
set up and installed, or I'm sorry, Sirius, and I don't want to confuse the rule manager. I'm just
going to use one or the other. But if you run the help, dash dash help, you'll see that it has all
of the basic commands that we're going to cover here. We look at updating our sources, we look at
listing sources, we look at enabling sources, and then applying those sources to build the rule set
and deploy it to the engine. Again, so that's what we'll be seeing with Sirius. Now, I mentioned we
have sources, and so we can, if we want to add any different type of public or custom source that is
available, you can do that just by simply selecting these actions. And adding a source is usually as
straightforward as adding the URL where that source is coming from. Now, there are some commercial
rule sets, rule sources, and those then, of course, you have to pay for. Typically, you get an API key
or some way to authenticate before you can use those. Some other examples of rule sets that are
out there, if you go to abuse.ch, which is an awesome project and a lot of great resources,
you can look at the blacklist project. And from there, we can look at the actual list.
And you'll see that there are, for example, in this particular project, a number of rule sets
that are available based off of things like blocking of C2 IPs or J3 fingerprints. Something
to keep in mind, then, as you're looking at the different rule sets that are out there,
you do need to, oftentimes, if you find there is a Suricata specific one,
I would, typically, I would opt over that. A lot of rule sets are going to be compatible,
not only with different versions of Suricata, and there's a fairly significant version change
between 4 and 5, where 5 is the latest, but then a lot of them are compatible between
Suricata and Snort. Some of the more recent rule sets, ETOpen or the Emerging Threats List
rule sets, they are starting to also create Suricata 4 and Suricata 5 rule sets. And the
changes in 5 are all based off of rule syntax. And so, you're getting, likely, more performance out
of the 5.0 rule set. You also have to be careful, because, typically, the 5.0 rule set won't work
with the 4.0 version or the 4.x version of Suricata. So, just some things to keep in mind.
And, again, in general, you'll find a lot of compatibility, but I would always opt for,
if I'm running Suricata, to take a set specifically designed for it. If we go back to Sirius, then,
you can see that we have our sources listed here. Again, it's just a click away, or if you're using
Suricata Update to add those sources. And then, once we have our sources, we can create and manage
our rule set. The rule set, then, is just a combination of rule sources. The default here
in this VM is to take the two sources that we have, add them to this rule set. And we can see
that we have those two sources, and we have just under 21,000 rules. Once we have that configured,
we have to deploy the rule set. We have to make sure that we've downloaded the most recent version,
and then deploy those to the engine. And in this interface, you need to go to the Suricata tab,
and then there's this rule set action. And what will happen here is that it will update,
it will build, and it will push. And what you see with a lot of rule set managers is that they will
combine all the rules that you've told it to use, and they will write it into a single file,
and then deploy that into the location in the file system that Suricata looks for rule files.
Suricata update also does something very similar. And in order to create the actual rule file,
you just run Suricata update. So that will run based off of the current configuration,
update the rule sets, and then write the file for Suricata to use. Now, I mentioned the rule location.
So in order to understand where the rules are written to, maybe we want to confirm that they
are in fact getting updated. So we want to look at the rule files or look at the timestamp on that
rule file. The default location for the Suricata configuration is in etc Suricata. And then in
particular, Suricata.yaml. So there is a yaml file that you can open and look at. And yes,
I did just nano in public. Everything Suricata related is in here. You probably don't need to
get too deep into the configuration file. That's something that will come with time. But definitely
there are some things that you might want to check. Probably one of the most important,
outside from we can look at the rule location here in just a moment, is this home net external net.
If you dig into the rules, the syntax of the rules, they use home net and external net.
And what they are using those variables for is to define the direction, the flow of traffic,
and then when the engine should apply the rule. Should it be on the response? Should it be on
the request? And so that's very important. It also, of course, defines the IP space. And so
the default is to use standard RFC 1918 addresses, internal IP addresses. And again, the default then
for your external network is to just negate your home network. But you want to make sure that these
are correct. So if the environment that you're in or defending is using addresses that fall outside
of this, that's just something that you want to definitely keep in mind. Now, one more quick thing
about the configuration is that the engine supports sub-configuration. And with CELKS,
we do have a sub-configuration, which means that in our primary configuration file, if we include
sub-configurations, these sub-configurations, if they overwrite any configuration options in
the primary, then including the sub-yaml, the sub-config will do that. It'll overwrite in
the primary. So this can become helpful if you have a base configuration that you want to use,
and then you deploy sensors into different areas of your network that have slightly different
requirements based off of the type of traffic that they're seeing. In CELKS, we do have in the
sub-yaml the change to the default rule file. So here you can see that's commented out. If we
go a little bit further, the property default rule path and those rules will be located in
etc-suricata-rules. And it's just going to be a single rules file, Sirius.rules, right? So that's
just a real quick about the configuration. And again, unless you have reason, you probably don't
need to get into it right off the bat, but definitely it's worth checking the whole net
external net and tweaking that if you need to. Going back to the interfaces here,
so we've now looked at different sources. You've got an understanding of where you can
maybe grab some different rule sources. We know the process for updating and pushing those.
Suricata can do a live rule set reload, so you don't necessarily have to restart the engine.
The next thing that we'll do is we'll talk about just some of the protocol parsing and
file identification capabilities. So as I mentioned here just a few minutes ago,
one of the differences with this VM versus the one you would download from the GitHub
is that we do have a couple of scripts that help us with the analysis and then the PCAPs.
So that script is Siri.sh, although it's a relatively straightforward script in that it's
running Suricata. Essentially, the main goal of it is to run Suricata in offline mode.
Let me see... I forgot what PCAP I want to use. There we go.
So one of the other... again, one of the capabilities I mentioned is that Suricata
can run in offline mode, which means that you can feed it a PCAP and it'll process that PCAP
as if it were, you know, analyzing the traffic live on the network. And then you get the same
results from Suricata generating its output. Now, one thing I haven't mentioned yet is where
Suricata generates all of its output. The default is to drop all of the data into a JSON file. It's
the eve.json. And so the alerts, the protocol logs, file stats, anything that the engine generates,
it will be in that eve.json. The JSON file then is very flexible. It's a very flexible file format
because you can do things like submit it, just the JSON file itself to Elastic and other tools
that allow you then to take that data and build, you know, visualizations and dashboards and then
parse it in different ways. Now, the PCAP capability then is, again, this is offline mode.
And when we run a PCAP, it's going to use the original timestamp from the PCAP. So we just need
to keep that in mind because as we look at some of these interfaces, we need to oftentimes adjust
the time and date. So if I look at a dashboard and it's looking at the last 24 hours, even though I
just ran the PCAP, if the PCAP was from last week, I need to go back and adjust the timestamps,
the filters, and that UI to go back to last week. We can run this script now. And again, main purpose
is to get the PCAP running through Cercata in offline mode. So Cercata is going to load,
the engine is going to load as it would normally, and it's going to load the rule set that it's
configured to utilize, and then it's going to process the PCAP. Now, as that is loading,
what we'll find is another open source project that is maintained and developed by one of the
Cercata core developers is called Evebox. And Evebox provides you with a graphical ability
then to look at basically alerts and some of the other data, the protocol data and the flow data
that Cercata is generating. So we'll look at the end of the script. The script does
one more thing before it finishes, and that it queries the eve.json file, looks for any alerts
that were generated, and prints them here to our terminal. Now, that's great. It's helpful to see
those alerts right away. But again, it's kind of hard to do all this work in a terminal, and it
probably doesn't scale that well. So that's why we would switch to something like Evebox. Now,
with Evebox, and it's going to be a little bit tight here in the UI, just because I have the
zoom up high. But what we're seeing now are the alerts that were generated. So not only do we see
specific alerts for events that happened, but we see other alerts. And in this case,
we see other alerts around the same time that maybe this first alert was generated.
So that helps us to possibly build a little bit better context around each individual alert.
We can see here that we have several. In fact, these four right here are all based around
an executable download. If we scroll up a little bit further, then we'll see that after that
executable appears to be downloaded, we have alerts around Fyodo Tracker and Win32 Emotet
command and control activity. As you start to understand what the alerts are telling you,
you can start to read into them a bit more. We can see that this particular alert here, it's an
info alert. There's color coding to help you visually gauge the severity of the alert. So this
is an info one. It's a different category that the alerts are categorized in. An executable is
retrieved with minimal HTTP headers, right? So this is an alert. And in one event, the download
of an executable can actually generate several alerts. So this is telling you that something
downloaded an executable file, and there were very minimal HTTP headers. And why this becomes
significant is because oftentimes, our scripts like PowerShell, for example, which is regularly
used from an office document to download executables, by default, they don't use very
many HTTP headers. Typically, it's the get request header, the first header that's needed with the
HTTP and the version, and then maybe the host. And so that is telling you that maybe where this
originated from was something like a malicious office document, which actually is the case here.
So those can definitely help to add additional context. And again, we can get many alerts around
a singular event. We can select any of the alert and get a little bit deeper into the analysis.
So if we select the ET policy, PE, or DLL download, this allows us then to get information
about the traffic that matched from this alert. So we see, of course, the timestamp, we see the
sensor that it came from. So if you're working in a multi-sensor environment, we have the protocol
TCP, but this was HTTP, we have the source and the destination ports, the flow ID, we'll take a look
at the flow ID here in just a moment, the signature, the category, as well as the signature ID. And
what's helpful about having the signature ID available is that not only could we use this
in this particular interface, we can filter off of that. So we could look back in time over
however much data our environment is collecting on to maybe filter off of a specific signature.
But then we can also take that signature ID and we can use... we'll actually use Sirius to do this.
We can pull up that rule itself and select the rule and then look at the rule syntax.
So we can now begin to understand exactly what it is in this rule that was matching on the traffic.
So if we have to dig a little bit deeper to provide a little bit better context and understanding.
Okay, so we'll go back. If you continue to scroll down in the interface,
you'll have different values from the HTTP request. So that's some of the protocol parsing
and logging capability that it has. You might have the response body. We can very clearly see
if you're familiar with PE files anyway, and we can certainly see the PE file that was returned.
And then, you know, scrolling down further, you'll eventually get to the raw JSON. So PE files are a
bit large, but here you have them, just the raw JSON data that correlates to this alert.
So if there's, for some reason, maybe some data that you know is in Suricata or it's being
generated, it's in the EVE.JSON, it's not there. In the UI, you can come down here and check.
It's also a good place to look just to get a better understanding of all of the data that
Suricata is generating. And maybe there are some fields that it's not producing for you
that you might need. And then there actually are ways to customize the engine to help correct that.
Now, I mentioned the flow ID. So we can select the flow ID. And what that will do is essentially
give us the ability to take a step back and to look at all of the other, you know, events that
are around this particular flow. So not only now do we see the alerts, but we also see the different
different type of events. We have the flow information, we have the HTTP information,
we even have file information. So we can look at the HTTP request, for example. And now we can get
the host, the URI, maybe the user agent, any of the data that is around the HTTP request itself.
Very similarly with the file info. So Suricata has the ability then to not only identify
files in the traffic, certain file types, it does have its limitations and everything is
available under the read the docs. But then also, it has the ability to extract those files if you
want it to. It doesn't do that by default, though. So here we have an executable. We know that it is
pretty, you know, we know it's bad because we saw alerts related to EMETET traffic after
it was requested and returned. And so now we can look at information about the file.
The magic will determine the type of file. So your lib magic telling you the file type.
We already knew it was a PE file, but here it confirms that. Of course, if we have PE files
that we don't have alerts associated with, then this is another sort of piece of data point that
then we know is data that's being generated that we could search on. We have the hash,
which is also potentially very helpful because now we could take the hash of the file,
and maybe we go and search for that on our favorite threat research platform, such as VirusTotal.
So I'll paste the hash in. Now, you'll notice here, this is a PCAP from a week or so ago.
And so likely, if it's EMETET, it would have been probably submitted to VirusTotal by now.
We don't get any matches. And the reason that that is, is the state is truncated.
And so what happens is, by default, Cercata only reads so far into the HTTP response.
And because PE files tend to be large and larger than that response, it stops reading at its
threshold and then hashes that content. So while it certainly read enough of the file to give us
the signature, the magic, the PE file identification, it wasn't able to read all of it.
This is done for performance reasons, and it's a very small change in the configuration
in order to tell it to read further into the HTTP response. But again, those are determinations that
you have to make based off of considerations for your own environment. But that is, again,
something that you'll want to recognize and understand. Because if you didn't, and you
thought this was in fact the hash of the file, results like this could maybe be a little bit
misleading. Let's see here. Okay, in addition then to looking at alerts, again, you have the
ability to comment, you have the ability to escalate or archive, you have under your events
tab in the UI here, the ability to view any of the protocol logs that have been generated. So
if you want to see all of the HTTP, you could select that. If you want to look at just DNS or
TLS or SMB, it's all available here, and it allows you to see it again in the interface.
This particular PCAP did not have any SMB, so we're certainly not going to see it there.
And then there's also some basic alerting or reporting, some dashboards that help with
that. And you can see that, again, you'll have a summary of the alerts, the top signatures,
the top categories, your top source IPs and desk IPs. And so it can provide you, again, an ability
to take a step back further and maybe monitor your environment from a bigger picture, broader
perspective. Of course, because the output that Cercata is generating is JSON, you can put it
into a pipeline. You can modify, enrich the data. You can submit it directly to Elastic. And from
there, you have the ability then to create any number of dashboards or visualizations that you
need. With CELKS, CELKS has a number of dashboards and visualizations already built. So if you
download CELKS, you can use these dashboards and visualizations for inspiration and ideas to
get an understanding or a sense of the type of visualizations that are quite helpful.
Kibana is another open source tool that's something that you can utilize. But getting
into all of the details of Elastic and Kibana are a bit beyond the scope of this particular
workshop. The other tool that I always like to point out is Molok. Molok is another open source
tool developed by AOL, supported by AOL. It's a great tool. It's like Wireshark on top of Elastic.
And since Cercata can create full packet capture, it can generate and capture your network
traffic. And then you could have Molok pick up the PCAP and ingest it so that now you can
use this interface to search for aspects of your traffic over time, however
you have your Molok configured to. Of course, if you're familiar with Molok, it also has its
own capture capabilities. But again, I like to bring it up because if you already have Cercata
deployed, these are capabilities that you might have a system that you've already set up
and configured to utilize. Okay, so I think that was my about 25-minute crash course on Cercata
and discussing some of the main primary features, what it's capable of, the kind of data that you can
get, and how you can utilize that, as well as a VM that you can go and grab and start experimenting
with it pretty quickly. At this point in time, though, that's all I wanted to discuss. So if
anyone has any questions, I believe the Text Workshops Track 1 is available for questions.
Of course, I have my contact information and I'll pull that up again. Please feel free to
direct any questions to me, any way you can get a hold of me or prefer to get a hold of me.
Thank you, Josh. Yeah, there's a lot of good discussion in the text chat. One question
that popped up that I thought would be good to answer is, is Cercata based on the engine
from Snort or is it developed completely different? It's completely different, yeah.
There is a lineage that goes back to the Snort days, but it is
not a fork, it is a rebuild. So there is a significant number of differences in the engines
capabilities and then things like that, rules, syntax, so that they are different. The best
thing I could probably tell you to do is under the Cercata Read the Docs. I'm not sure I'll be
able to find it just by searching. I think there is a comparison page. There's certainly compatibility
information, but I'm pretty sure somewhere in our Read the Docs there is a Snort versus Cercata.
So if you're looking for the high-level big differences, this is a good place to go.
Josh, thank you very much and thank you all for sticking through the Cercata workshop.
In case anybody does have any follow-up questions, Josh will be around in the Discord channel. Like
he said, it is the techs-workshops-track1 under the Flamingo Hotel group. Just scroll all the
way down to the bottom to Flamingo Hotel. Check us out in that channel and he'll be answering
questions. Now, Blue Team Village wants to say thank you to everyone that joined
and this concludes the Cercata workshop. Thank you.