Okay, good morning everybody. Welcome to DEF CON 28, Blue Team Village. We are going to start this
morning's workshop off with Lennart Koopman from Greylog. He's going to be doing an introduction
to the OpenStock CTF tools. Definitely worthwhile listening to. Hope everybody enjoys. I'm going to
turn it over to Lennart now to get to the meat and potatoes of it. Thank you. That sounds great.
Thank you so much. First time I'm doing a talk at a remote conference. I think we're all learning
new things in this year. I wanted to, before I start with anything else, I just wanted to say
thank you to the organizers not only of DEF CON, but also the organizers of Blue Team Village,
because I think that's just an amazing job they're all doing. They're doing that every year,
but now this year I think it's even harder. And I as a speaker, and I think I was the first in
the lineup here, so some things were new to them too. I feel like they did an absolutely
fantastic job. So thanks everyone, and I hope everyone has an absolutely fantastic time at
at DEF CON and at Blue Team Village specifically. So what I'm going to do today is I'm going to
walk you through a very, very high-level overview of what Greylog is. And we're doing that because
Greylog is also part of OpenSOC. And OpenSOC, the CTF of the Blue Team Village, if you're
competing in it, you're going to be using Greylog a lot. So I was asked if I could give an introduction
to it, especially I think for everyone, but even a little focused on the people that will be using
it during OpenSOC. So I think it's going to be a fantastic overview if you're new to Greylog,
or if you haven't looked at Greylog in a while, or if you're generally interested in open source
software, DFIR, block management, whatever. I'm going to walk you through it, like I said,
very high level. And very important, we're going to have a bunch of time for Q&A and questions at
the end. That means that if you have any questions, we're going to go through them
then, and I'm going to make sure that we leave enough time there. We will also be, when I say we,
we Greylog people are going to be around in the Blue Team Village
Discord, and there is a Greylog channel there under the Sponsors tab. And we'll be around
there. We're in multiple time zones. There's a good chance you're going to meet
someone from us over there. I believe we have a little flag at our username or something that
shows that we're part of Greylog, and we'll be happy to also answer any kind of questions that
you might have, maybe during the CTF or OpenSoc itself, or maybe in general about Greylog. So
we'll be around. We'll be happy to answer any questions you have. Next slide is the agenda.
I'm going to be very careful with not moving around too much and not skipping through slides
too fast. I hope there's not too large of a delay between what I'm saying and what you're seeing.
So that that is all in sync, but I'm going to be careful that that works. If there are any issues,
you can always post in the Discord channel. There are moderators there, and they can make me aware
if there's something going massively wrong in any way. The agenda for today, you'll see
this here on the left, I think. I only have like three real slides, because I want to really
show you Greylog as a product. I want to show you what it does. So it's going to be a big focus on a live
demo. But I'm going to go over one slide, just real quick, about me. Then one about Greylog,
give you a little bit of background. Then we can take a quick look at the Greylog architecture.
Not kind of in the level of detail for when you would get ready to deploy this yourself, but
just to give you an idea of how the parts of Greylog are kind of coupled together,
so that you can, when you use it later, maybe a few things might make a little more sense. I just
feel we should take a quick, just a minute, at the components of Greylog itself. Then, like I said,
live demo. We're going to focus on analysis, because I think that's, as a user and as
an analyst, that's probably the most important part for you, especially if you're part of OpenSoc
this year. But we're also going to look at some differences if you're used to other tools.
I think that could also be a really important kind of question that you might want to ask.
If you are used to another log management tool out there, maybe if you have questions about how
Greylog compares to that, from kind of how you use this, does Greylog do that, is there another way
of doing that, so you can apply the knowledge that you have already, so that you can
really apply this in a good way with a new product like Greylog. And then, like I said,
we're going to have a lot of time for Q&A. For the questions that you have, please use the channel
in the Blue Team Village Discord. If you're not in that yet, it's a good moment to just quickly
sign up for that. Go to the Blue Team Village Discord, and in Discord you see these categories
on the left in your sidebar. There's one that's called the Flamingo Hotel category. Made me miss
Las Vegas very much when I saw that this morning. But there's the Flamingo Hotel category, and
there's a channel in there that's called Text Workshop Track One. We are here right now,
we're Workshop Track One, and that's a text channel. So that's where you can post your
questions. The moment we're going into the Q&A phase, at the end of this, I'm going to go to
that channel, and we can start to look at your questions and start answering them. All right?
So, like I said, Text Workshop Track One. If you get lost somewhere in there after you join,
like I said, there are moderators swarming that place, and they'll be happy to help you
and guide you to the right channel. So just ask anywhere, and you should find it.
About me, it's real quick. Lennart Koopman. My Twitter handle is at underscore Lennart,
L-E-N-N-A-R-T. If you want to follow me on your own risk, you should definitely do that.
My background is actually software development and architecture. So I've started my career as
a software developer, and also started the Greylock project in 2009 already. I realized
with horror that it's been 11 years already. It's been 11 great years, but 11 years is a pretty
long time, and I felt like it was more like three years. But Greylock as an open source
log management product started in 2009, I believe, actually pretty much on the day in 2009,
like in early August. We had a nice little celebration at our last party
over in Vegas. That was fun. I think there are pictures. I then started a company behind Greylock
in 2013. That was after the open source project was showing some commercial interest
from people out there. So they asked me, hey, can you give me support? And I said, hey, I don't know
how to write an invoice. So that was a good moment to start a company behind it. That was 2013.
I'm currently the CTO over here at Greylock. That's really what I started as, too.
And I was born and raised in Germany. That's where my accent is coming from, if you can make it up.
But I am living in beautiful Houston, Texas now, which is also where one of our offices is.
The other offices are in Germany and then also straight out across the United States.
I guess I have to say, we are hiring. If you want to go to greylock.org careers,
there might be something that you could be interested in that's remote in Germany or the
United States. But enough about me. Real quick about Greylock. This would be the moment at a
conference or at a physical talk where I would ask who knows about Greylock, and then I would
see hands go up or not go up. And then I could adapt this a little to the audience. So I'm just
gonna tell you the whole story real quick. Open source log management. So you send Greylock or
instruct Greylock to pull logs from somewhere. It will ingest these logs, it will parse them,
and it will make them available for you for searching, analysis, threat hunting, for
alerting, and also maybe to filter some out and say I'm actually not interested in these.
Throw them away, route them somewhere else. Really a way to handle logs that are being generated in
your environment. The three main use cases for that are dev or ops or DevOps, depending on what
you're doing. Definitely very interesting for developers to do root cause analysis, to do
monitoring, to do performance monitoring. Maybe when they know that there's an issue on the platform
that they need to research or that they need to investigate, they use Greylock to go into
the logs to figure out what exactly happened and to make sure that either it doesn't happen again
or to put an alert on it or simply to resolve the issue that's causing it. Same for
ops. For ops that could be just classic IT system ops. This could be network operations. I see a lot
of people using it for that. The fastest growing use case has been for a while now security use
cases, which I guess you as the audience here is probably very aware of, which basically means
you're using it to keep a track record and a history of everything that's going on in the
environment. That means you need to have a system that is able to really ingest all of your logs
and to make sense of them and keep them for long enough so you can run your analysis against it.
And it will then allow you to set up pretty complex alerting, like you would know it from
some sim systems out there, for example. That's a feature that came out, I think, about a year
or a year and a half ago. A new alerting engine that lets you help be a little more proactive
about what's going on in your environment. But one of the other really classic use cases is you need
to investigate an IP address. You need to investigate some kind of indicator of compromise. Something
new came out and there's a new patch. You want to make sure that that specific file has not been
anywhere in your environment in the last six months. Let's go in and let's see if that happened
or not. And then, of course, classic DFIR incident response, figuring out if there was an attacker,
what that attacker did, all of that fun stuff. And, of course, compliance, just having the
logs somewhere. I think that is a valid use case. And for that, you need to centralize them.
And I think Greylock is a really good choice for that, too. Like I said, started in 2009,
really constantly grown since then. Very, very happy to have been a Blue Team Village sponsor
really from the beginning. Like I said, absolutely fantastic people. Make sure to watch as much of
their content as you can. And next year when we're hopefully all back in Vegas, make sure to stop by
in person. It's always a lot of fun. And Greylock is also part of OpenSOC, the CTF that runs during
Blue Team Village that has got so much positive feedback over the last years. Also, of course,
extremely happy about that. And like I said, we'll be around to help you with any questions
that comes up during your OpenSOC participation. Now, during OpenSOC or during the Blue Team
Village, no booth this year. I was always happy to spend a day over there and just
chat with people. Maybe give out some shirts and some Greylock swag. Because that is not
happening this year. Make sure to stop by the Greylock channel in the Blue Team Village Discord,
because I hear that we will be sending out swag from there. And then we'll be there to answer
any of your questions. Very important is that OpenSOC is not running the latest version of
Greylock yet. They're slightly behind it for some very good reasons. So the demo that I'm going to
show you is the latest version. If you see something that's slightly different, definitely
reach out to us in the Discord channel and we'll be happy to give you a maybe a workaround or
tell you where that is. Or if it's simply not there and what else you could do about that. So
just keep that in mind. But there's some structural differences about where
some elements are. But overall, it's pretty much the same. And especially the core
ideas and the core principles are of course the same, because they simply haven't changed in the
last months. And then most importantly, have fun. Stay in touch. It's open source. So if you
get interested in this, just start using it. Start running it yourself. It's designed for that.
And there are many ways to install and run Greylock, especially if you maybe just want to
try it out really quick. There's some ways to get something up and running really fast. There's
a big community around this that can always help you out. So I want to say have fun and open
SOG. I've only heard fantastic things about how much fun it is. So I'm sure you will have fun.
And even if you're not participating in it, just download it. Give it a try. It's open source.
That's what it's all about. So the next slide here promises a live demo. Before we do that
I want to send you straight to the docs real quick. So we can look at the...
I have to find where they are. First link here. Architectural considerations. And we will not be
detailed. No worries. But I want to show you just really quick the parts that Greylock
consists of. If you ever get stuck somewhere with Greylock, always go to docs.greylock.org
if you feel like just looking it up yourself. That is up here. Docs.greylock.org. That's where
all of our documentation lives. And if you can't figure it out there because your use case might
not be described in there, then, like I said, reach out to us. But that's always a good place
to get started. And I'm just using... you won't have to deal with the architecture because the
open SOG folks are already setting that up for you. But just so you understand it, Greylock
really consists of three big parts. Or three and a half big parts, I would say. There's Greylock
itself, which consists of Greylock server and Greylock web interface. But it's really one
system that you're running or one executable that you're running. And it has two dependencies.
It connects to MongoDB. It does that to store some state, to store some cluster communication,
to store some configuration that lives in MongoDB. And because there's really not much data, that
MongoDB is really, really small in the end. Actually, most people will just run it on the
same node as the Greylock server. And that way, you can almost forget that it's there and set it
up in a replica set to make it highly available. The Elasticsearch part, though, that's what we're
using to store the messages. And so that will, of course, you have to scale that properly because
that gets all the load from searches. And it gets, of course, all the load from actually writing the
messages to disk. And what happens is, if you connect to the Greylock web interface, it will
basically execute API calls against the Greylock server. And it will say, hey, my user of the web
interface is trying to search for these messages. And then the server says, yep, I'm going to search
for them, queries Elasticsearch, gets the data back, does all its funky Greylock stuff with it,
sends it back to the web interface, and the web interface displays it. I'm mentioning that because
all the communication between the web interface and the Greylock server is really API calls,
nothing else happening. All it is, it's just authenticated HTTP REST API calls. That means
that you can also automate everything. So if you are getting at a point where you want to
write a script to look at data, for example, you can always use the Greylock API. There's an API
browser in Greylock, and you can start to write your Python scripts and start to pull that data
out of Greylock and do something with it. That's kind of my secret sauce tip for if you're working
in OpenSoC for sure. If you're good at Python or another language, then definitely keep that in
mind that you can do that. But overall, Greylock server accepts the messages, you send the messages
to the Greylock server, use the web interface to analyze, display, configure, set up alerts,
all of that fun stuff. And then Greylock behind the scenes will do all the management of Elastic
Search. You can define your retention times, all of that. So if you're maybe used to the Elastic
Stack where you have to do a lot of stuff in Elastic Search itself, for Greylock you really
just have to tell Greylock where the Elastic Search is and then Greylock takes care of the
rest for you so you don't have to become an Elastic Search expert. I think that's a very
important part. If you want to, you can scroll down here and look at what complex looks like
with multiple nodes, clustering, load balancer in front. I think that is definitely out of scope
right now for what you are about to do here or if you just want to give it a try. But be aware
that this is all documented here and then there's also another setup guide that talks you through
that. So it should be a good place to get started. All right, so now we're going to the real life demo
part. And what I'm going to do is I'm going to walk you through... I made some
notes here. I'm going to work with one, two, three, four areas really. And I want to spend the most time
in the search and analysis. Like I said, I think that's where most of your time will be spent.
But I just want to show you a few other parts of the system, at least real quick, so you are aware
that they are there. So I always like to start in the system area. And that is simply
because I think that gives you an idea of kind of the philosophy behind Greylock itself.
Our idea is that you should have to set up a Greylock in a config file only initially
and only once. Basically tell Greylock, where's my MongoDB search? Give it some data it needs
to encrypt passwords, to set up TLS on the web interface, kind of this basic setup.
And then from there on, we believe that you should be able to do everything from within the web
interface. Like I said, those web interface calls are really just API calls. So if you want to use
automation, you can just use curl to set up stuff automatically. But during, I would say,
after a base setup, everything is configured in Greylock itself. And then Greylock keeps
the configuration basically in MongoDB for you. What that means is, I just want to give you a real
quick overview. We're going to go into that very deeply. But if we look at our pipelines, for
example, that is where all of the message processing of Greylock is happening. This is not some API
call where you have to post some cryptic language to. This is not a huge YAML file or JSON configuration
of that. This is all, no, here we go. Here comes my Linux syslog. And I'm going to say,
I want to parse some stuff out of it. And you just basically configure Greylock here with a
very efficient and small rule language that then says, OK, everything has Linux syslog. Apply this
regular expression on it. Extract some fields. You'll see here on the right, this is all the
functions you can execute on it. You can drop a message. You can check what a message looks
like and then define other actions on it. You can thread Intel lookups on it. You can run a
Whois lookup on it. DNS reverse lookup on it. Run it against your own lookup tables. You can
check if something is in a certain subnet. All of this processing happens. And this is very important.
This is a very important thing that you need to understand, especially if you come from other
systems. All the processing happens when the message arrives in Greylock, OK?
That means that once a message has been written,
that means that that message basically becomes immutable. That means that you cannot change it
afterwards. That has some positive side effects and some side effects that you might have to
work around. What that means, most importantly, for you, if you're coming from a system that
allows you to transform data while you search, Greylock simply won't do that for you. Especially
not in a way that you can then further search on. Everything that's in Elasticsearch is in
Elasticsearch as it is. You can't transform the messages during search time. Very important to
keep in mind. Of course, we've built our whole web interface and the processing in a way that
you can still do all the good stuff with it. And it also means that we're getting a huge
usability and performance hit from that in a good way, as in it's much easier to use. We hear a lot
of people who come from these schema-on-read systems who say that is useful for a bunch of
queries. But overall, just being sure what data you're already looking at and not having to mess
with super complex queries all the time is actually something that in the long term people
really, really start to like. So just be aware of that. You'll see in the inputs area, you'll see
where all your messages are coming in. We've got a whole section here for authentication, if you
want to connect that to LDAP or an AD. We have API tokens. There's all kinds of authentication
mechanisms happening here. And also, that's the last thing I really want to touch on real quick,
we have content packs. So if you want to export any kind of configuration, dashboards, reports,
alerts, anything like that, and insert it into a different setup, you can export and import
these things here using a feature called content packs. We've got outputs. There's a whole index
management here, how long you want to keep data. Then after that data has been deleted, you want
to archive, you want to close it, you want to delete it. All of that happens here. It's, I think, for you
for now, and especially if you're just getting started, not super relevant yet. Just be aware that
Greylock has that, Greylock does that for you. So always a good idea to maybe just click around in
the system area a little. The other thing I just also really want to tell you, so you know that it's
there. I am honestly not sure how much of that can be used during a CTF, but know that there
is an alerting engine. I have here, this is kind of my small demo setup. I have an alert setup
here that alerts me every time there's a high priority IDS alert, Greylock, and I have another
one, and we can actually look at those. I live in southeast Texas, so we're pretty good at storms,
and I live in the United States, so we're pretty good at not burying our power lines.
So that combination will definitely require for you to have some kind of an uninterrupted
power supply. So I'm reading the logs from one of my UPSs, sending those into Greylock, and I'm
triggering alerts to my iPhone when the power goes out, because I really want to know about that,
because maybe if it doesn't come back up within five minutes, I'm just going to start to shut
down some servers remotely real quick, or do something else. So I have an alert setup here,
let me look at this first. Under event definitions, I have a condition set up here called
UPS without external power. Click on that. Really what it does is, it says when you find a message
that has its status field set to OB discharge, which is the UPS reporting that it is now
discharging, which means it has no power, I say when that happens, I don't need aggregations,
I don't need groupings, I don't need any other further rules and statistics. Just the moment
that happens, I want to be sent a message. Set this up here on notifications, I'm using Ops Genie,
which is the system that Greylock can post these alerts to, and it will then send an alert on my
phone. I say when that happens, send me a notification, and you can actually see here,
if I go to my alerts and events, and I filter my events by UPS, you will see, because I'm looking
here at the last 360 days right now, usually you would probably look at the last day or so,
you'll see now a history of all the days when my UPS was out of power. So you see here, for
example, just about almost exactly a month ago, my UPS was without external power and it did
trigger a notification to my phone. So you can set up these things, you can set up
all kinds of conditions, I can say only send me this if certain events happen in a
certain order, all of that is there. Also, I don't think anything from this demo today, but
just something to be aware of and know that it's there if you want to use it.
Then we have the streams, and after that I'm going to go into search and analysis and give
you a quick overview of that. But we have the streams here. Streams are basically, think of
categorization of messages when they're going into Greylock. They are an important concept
for Greylock because, for example, our retention settings are sitting on top of that. So if I say
I want to keep my NetFlow logs for two days and my domain controller logs for two months,
I would route them into two different streams with a very simple rule. This one here, for example,
if the field Greylock type matches exactly the value NetFlow, then route this into the
stream NetFlow. Another one here would be this one. Yeah, if the message matches the regular
expression filter log at the beginning of the string, then route it into my firewall logs.
I think here's one for, yeah, this is anything that has a snort process ID in it. Route that into my
IDS alerts. So you're basically pre-configuring or pre-configuring messages and you put all kinds
of stuff on top of that. Like further processing, extractions, enrichments, you can put access
rights on top of that and say only team A is allowed to see my AD controller logs, but my
support team is allowed to see, for example, my database debug logs or something like that.
Same for retention times. A bunch of concepts in Greylock streams. I think for you, if you're
getting started with this or using it during OpenSoC, I think it's just a great way to make
certain types of messages available with one click. Okay, so we're now going into search and
analysis. If I wanted to see all my firewall logs, I just click on the firewall stream
and it opens a search page that has the stream selector here on top already pre-configured to
only look at my firewall logs. I can select multiple here. So if I wanted to see all my
firewall and all logs, you just do that. This thing's going to update. Here we go. And now
we also got my NetFlow logs in it. For this example, though, I'm only going to look at my
firewall logs. I'm going to take this out again. Every time I make a change to a query, you will
see that this is... you get this little yellow icon here, which means, hey, you haven't executed
your new search yet. I don't believe this is the case in the version you're going to be playing
with. But if you download it fresh, that's definitely going to be the case. So I have to manually execute
that one more time. What we're looking at here is our search page. So I think this is what you're
going to spend the most time. So let me walk through the main areas, and then we can run more.
Maybe execute a search query or two, and then I would say we go into Q&A. And I will ask a
moderator to let me know when we run out of time during the Q&A. I think we started a little
late. So just let me know what the best time for you as the organizers is. But you have three
main areas here. You've got the search query bar here on top. This is basically where you are
selecting what exactly you want to be searching for, in which stream, what query, and also in
which time range. That's always the first thing you've got to think about. Then you've got a site
bar here on the left. I think the important part of this is the fields area that shows you all the
fields that are in these messages. So pre-parsed fields in the message.
And then, of course, you've got the actual result set. So everything you're seeing here, everything
under the... is your result set. So that is actually everything that Draylog returns for your search
query. So let's say I don't want to search in the last five minutes. I want to search in the last
hour. Change that. Execute it. Here we go. So now we're looking... you saw that this message
count chart here changed. We're now looking at the whole hour. And I think that we can also...
I don't think that's this one yet. But you can also definitely generate a widget that shows
you how many messages were found in total. We can do that in just a second, actually.
So let's say we're looking at all my firewall logs here in an hour. You see that we have a little
bit of a spike here. And this is... firewall logs are always great because they have a lot of data
in it, how much value you actually get out of them. So I think it's up to you. But it's a great example.
And overall, the searches work the same for everything, no matter if it's a firewall log, a
Windows log, a Linux log, a Amazon API log, anything like that. So if I click on it, you see here this
is the original message that came into Greylock. Very hard to read. If I click on it, you get the details
and you get what Greylock has actually returned. So you see that Greylock here has parsed this out
into some fields. Like, for example, the destination address and destination port,
header length, which interface it came in, which IP version it was, the length of the whole
of the whole packet that was let through, which protocol it was. And you'll also see here that
in this case, the action was block. Everything that I'm looking at here on these logs,
I'm only sending me the block logs. So I'm not interested in what the firewall let through. I'm
only interested in what the firewall actually blocked. So with all of this, you also see source
address and source port. Let's say we only want to see the messages from a certain source,
where the source address is set to this value. What I could do is I could go into the search
query bar on top here and say source underscore address equals, and then 1356, whatever I want
to look for. Much faster is to just simply click on the value. This context menu opens,
and you say add to query. And it will just build that for you and immediately execute.
So what we're now looking at is we're looking at all blocked packets from the last hour where
this was the source address. So basically all the packets this source address sent us
locked. Okay, so let's try to get a little bit of a better overview here of what that looked like.
So I want to find out what's this all TCP or both. Do you see we have a field here called
protocol. Try to get the top values. I want to see they're all set. I see something that's a
protocol. Show top values. This opens, and you see that no, actually this source only sent
UDP messages. Okay, let's go a little deeper and say is this always the same kind of package?
Do they all have the same length? Like very, very uniform? Same thing. Data length. Show top values.
Here we go. We see that actually all these packages are UDP and exactly 278 bytes long.
That looks like we're just blocking the same package over and over and over again.
Now what would be interesting, this setup isn't doing it, but if we were to run a
lookup. That's a built-in lookup table onto the source address or our GeoIP to see where
this is coming from. Who owns this IP address? Look up the underlying ASN. Do all of that stuff.
I'm simply not doing that here in this setup, but that's all stuff that you could do.
And now let's maybe say I want to see... now let's actually go back. I'm going to execute
this query again. And what you're going to see is all the data in the result set, because the
result set change, has now updated. Now we see for the last one hour, for everything that was blocked,
UDP, TCP, and ICMP. Okay, I'm going to delete this widget here. I want to show you the aggregation
builder. That's really the last part I want to show you here. For the search query language,
go to docs.greylock.org. There's a whole page that explains the search query language for you.
I don't want to go into too much detail here, because you can simply read it up there.
It's pretty simple. It's in the end, if you've been using the Elastic Stack,
it's the same language. We're just passing it through Elastic. So if you're used to that,
same thing. So whenever Greylock creates a widget, that is under the hood. In fact,
an aggregation and an aggregation builder. So you will always see this little icon,
top right of the widget, that you can click on and say edit. And now edit is going to show you
the aggregation builder. And it has one very important drop down box, which is the visualization
type. Let's say you want to visualize this not as a data table, but as a bar chart. And say,
I really want to see three bars here, with UDP the largest, TCP on second, and then a tiny one
for ICMP based on the count. You switch it up. Art chart. Here we go. Save it. I could now make
a dashboard out of this. Here we see for the last hour, we mostly brought UDP stuff. Okay,
interesting. You could, of course, if you really wanted to, make it... I think they're much harder
to read, but sure, could do that. Greylock is not going to keep you from building weird charts
either. So you want to make this a long chart? Sure, not a problem. Can do that. So this is our
aggregation builder. You can do sorting. You can add multiple rows. We're going to do that in just
one second. And you can also add metrics. So let's say I want to get the total bytes that were blocked
over the last one hour. Okay, so you know we got the data length field here. Click on that,
press chart. That is going to create a new widget, and that widget is basically just the same
aggregation builder, but with other predefined values. Okay, you're going to see that in one second.
Here we go. So now we got the average data length by timestamp. Click on edit. See, okay, that's our
line chart. That's how much data was transferred per minute, in this case, on average. But you
know what? Let's make this a data table, so you see what this looks like under the hood.
It's like for each minute, it calculates the average data length. And with this kind of column
row structure, you can put that into a bar chart. You can put this into a line chart.
But I think what we're really interested in here, we don't want to look at the average data length.
We want to look at the sum of the data length.
Here we go. Data line chart. And now we see how many bytes are blocked in total. I just
had to know that this was called sum. You can do standard deviation, max, min, count, cardinality,
average, variance, percentile. You can do all of that stuff with it if you wanted to. Okay.
One more thing I want to show you, and then we're going to go into Q&A.
I really want to make sure that you start playing around with aggregation because that one
is extremely powerful. It will really help you solve these challenges that OpenSock is going
to give you. One more thing I want to show you is, let's say we want to build an aggregate.
We say grouped by source address which protocol was used. Okay. So let's go into the source
address. Get the top values of that as kind of a start. Add it. And we can add multiple rows here.
So let's set this to protocol. I want to add a second type of row which is protocol.
It's going to update. And you actually see here that these have all only communicated
with UDP or TCP. They're not really doing multiple things. So to explain this a little
better, let's build pairs of source address to destination. Take the protocol out. Add a new field,
destination address. Here we go. So now you see this is not really what I wanted to show.
It should be the other way around. Maybe destination address to source address. Here we go. That's what
I wanted. So you get this grouping where you say this destination address was reached by these
source addresses. And you can go lower and lower and lower as you wish and add more and more stuff.
So let's say as the metric, here's the third column, I actually don't want the count.
I do actually want the total bytes that were transferred. So the sum of the data length.
So without any kind of complex queries, you establish that this destination address
was hit by this source address with this many bytes over the last hour. And I can make
this a little larger, put this wherever I want. I could now save this as a dashboard,
share it with colleagues. There's a whole concept of parameters in there. It's all stuff
I don't want to go into today. But that's all stuff that you can do. And you can also really
start to combine this in very many different ways. What I can tell you is this might be,
this might look like it's a pretty good learning curve associated with all these fields in the data
table and the visualizations. I would encourage you just to play around with it. The worst thing
that can happen is that it shows you either a chart that doesn't make sense or that it shows
you an error when something really doesn't compute. Okay, you can't break anything. Just really start
to play around with this stuff. Maybe think about something you want to visualize and see how you
would approach that. If you get stuck, go into Discord and there might be someone who can help
you from our team. And also, whenever you feel like, okay, this stops making sense,
go back to the data table and look at what underlying data we're really looking at. Once
you have multiple dimensions, you can do stack, bar charts, you can do all kinds of stuff. So,
I would encourage you to play around with that. This is really an absolute core idea of
what CorelDoc does and what you can do with it. So, with that, I would say that's my demo.
And as many minutes as we have left, I would use for Q&A. And like I said, if a moderator could
let me know if we're kind of starting to run short on time or when it's time for me to stop.
I would say we're going to do five minutes. I really, I honestly hope we have a little more
time. I'm going to do five minutes of Q&A. I'm breaking my promise here a little of having
time, but I'm going to promise I'm personally going to be around in that Discord channel
for the next days. So, everything that's left after these five minutes, I'm just going to
answer there. And like I said, there are a bunch of other people too, and I hear that it's whack.
So, Linux on the desktop, I do not know how to hide it when you can't see it. So,
I'm going to go straight into that. I'm going to go straight onto this one here,
and I'm going to try to find some questions, okay?
We have about two minutes, if you don't mind.
Okay, understood. Thank you very much.
We'd just like to move the questions over to the text window, or the chat, I'm sorry,
if we run out of time. Thank you.
Understood. So, I see a few questions here. I'm going to skim through a lot of them,
but I see a question about if there is an OVA. There is, if you go to Greylock.org,
and you go to Get Greylock, big button here, these are all the ways that you can install it.
There's Docker images, OVA, virtual appliances. We're integrating with Chef, Puppet, Ansible,
and there are Debian and RPM packages to get started really quick, and the documentation
is going to guide you through doing that. Okay, let me scroll down a little more here.
This is available on Linux, and the best way to set up the environment is described in the
documentation. In fact, it only runs on Linux. It does not run on Windows.
So, I would definitely recommend to run this on Linux. All of the official
installation methods are going to be based on Linux, for sure.
I'm going to pick one more, and then I'm going to stick around in the channel, and also in the
Greylock channel, please. Let's see if I find one more. There we go. That's a good one. Is there
some kind of how-to for reading in a log file written by another application? So, you have
another server, and there's something running on it that writes a local log file, right? How do I
add these into my Greylock now? There are several ways of doing that. Greylock has a concept called
sidecar to even remotely manage of collectors. That is, however, not required. You can also
just use any kind of open source collector that's out there. I personally recommend FileBeats or
NXLog. FileBeats is coming from Elastic. They can send straight to our inputs, and you really just
tell it, hey, here's a log file, and please send me everything that goes into this Greylock system
with a buffer locally. So, look at FileBeat or NXLog. There's also WinLogBeat and a PacketBeat,
and NXLog is also great for reading properly. And so, with that, I will keep my promise and stick
around for more questions that are coming in. Follow me on Twitter if you have questions in
the long term. That's at underscore Leonard, L-E-N-N-A-R-T, and I hope you enjoyed that,
and I really, really hope that you had a fantastic time playing OpenSoC or trying out Greylock.
We're building this for you, for the community in the end, and so let us know
if anything doesn't work, if anything doesn't make sense, if you want to prove anything,
it's open source. Use it, play it, improve it. That's it. Thank you very much.
Thanks a lot, Leonard. I appreciate your time. A great presentation. We look forward
to hearing more from you guys.