Hi, everyone. Welcome to our talk, Collaborative Penetration Testing with Lair. I'm Tom Steele.
I'm from Seattle. I'm currently a senior consultant at PhishNet Security. There's my Twitter if
you want to follow me. And I'm Dan Coatman. I also work at PhishNet. I'm a consultant.
We do pen tests and social engineering engagements, sometimes physical security assessments as
well. So we were kind of ‑‑ we kind of built this tool, Lair, to fix some problems
that we were having on our engagements, on our penetration tests. And those include lots
of files and lots of data everywhere, lots of terminal windows open, just tons of files
everywhere and no real way to correlate those. The other problem we wanted to kind of fix
was duplication of work. A lot of times when you're doing an attack with multiple people,
you know, two or more, you tend to sometimes be looking at the same thing and it's not
very efficient.
The next thing we wanted to do is come up with a way to be thorough and make sure that
we didn't miss anything and that we were manually investigating everything on an engagement.
So we kind of looked at the current tool sets and we didn't really ‑‑ they didn't really
focus ‑‑ they all might have handled some of these issues, but they really ‑‑ none
of them fixed all of our problems. So what we did is we decided to create our own tool.
And that's what we're going to be releasing today. It's called Lair. And it's just ‑‑ at
its heart, it's a web application.
So it's all in browser. And we think it solves all the problems that I previously mentioned.
Here's an overview of the architecture. In the top right corner, you see this thing
called Meteor web server. So this web application is built with something called Meteor. That's
a web framework built on top of Node.js, which is a JavaScript run time. The reason
we chose Meteor was a few reasons. The first being data on the wire. It's very, very snappy,
very, very fast. Originally when you load up the application, all you get is a bunch
of JavaScript and a bunch of HTML templates. Everything else after that is all JSON back
and forth to and from the server. So it's very efficient, very quick. The next thing
was Meteor is all written in JavaScript, so it's one language from the client to the server.
Makes developing a lot better. Next is database everywhere. So with Meteor, you actually have
a database client in the browser. So when you're writing the application, it's very,
very, very cool to be able to write your queries actually in the client and kind of makes things
more efficient. You can develop a realtime application very, very quickly.
The next thing that's probably the best feature of Meteor is full stack reactivity. Everything
that you build in Meteor is realtime, meaning it doesn't do any sort of Ajax polling. It's
all using WebSockets on the back end. And when you have that query in the browser,
when those queries kind of invalidate, it's all back in the browser. So it's kind of
what we're looking for. Of course, the web application is only useful if we can get
data into it via, you know, all the automated tools that we use. And so Dan's going to talk
about how that happens. Yeah. So we ‑‑ the tools that actually
consume the data we've ‑‑ we call drones. They're command line tools written in Python
that use a common API. And they parse the data from some of the tools that we commonly
use for our pen tests. And that would be ‑‑ we have drones for Nessus, Nexpos, Nmap, and
Burp as well as a raw JSON one. And we wrote them in Python for a couple of reasons. We
really thought that maybe we would get some better community support if we wrote them
in Python rather than JavaScript because that can be a little bit difficult, I think, to
code in, especially if you don't have any experience with it before. And we also decoupled
it from the application rather than building it directly into the web application. So that's
why we wanted to do, you know, the Meteor node application because we didn't want to force
people to upload their files to the server just to import them. You know, you're just
sitting there watching your browser spin as it's importing. It's a little bit annoying.
And we also wanted it decoupled so that, you know, if you develop a tool to consume
data, maybe it's for, you know, something that you run in‑house that maybe the community
wouldn't want, it would be a little bit easier to kind of integrate with the framework if
it was decoupled.
So the majority of this talk is actually going to be a demo. So we're going to show off the
app now.
There we go. All right. So I'm just going to create a project real quick. So when you
first create a project, you're brought into this kind of centralized host view. So if
we had some data in here, it would be showing you a list of all the IP addresses, their
host name, their operating system, and who last modified them by.
Everything in layer you can do manually. We do a lot of manual testing, and so we really
need to be able to, you know, enter data randomly from many different data points. But, of course,
to kind of populate these initial things, you, of course, want to be using automated
tools such as Nmap. It's a great tool. So to do that, we're going to import into the
app. So to do that, you grab this unique identifier here and use the client side, Python drone
dash Nmap. That's kind of the naming convention that we chose. But you can name these things
anything you want. And then I'm going to import this first Nmap file. This is a vanilla scan
of my network. So no scripting engine, no version detection, no operating system detection.
All right. So it actually parses it on the client, connects to the database and inserts it.
So as you can see, this automatically got repopulated. There was no screen refresh or
anything like that. Let's take a look at this first ‑‑ this .1, .2 address. And, yeah, you
can see that it all got populated. This is the service level view. So it kind of drills down
into each service for each service. So you can see that this is the service level view. And
take a look at this telnet service here. And you can see that the product is set to unknown.
That's because we didn't have any version detection on Nmap. So I did a second scan
of just port 23 on this host with version detection. And you can see that it automatically
updated the product of that host. There was no AJAX polling or anything like that. It
was automatically just synced up with the client. So these drones are all additive,
and if there's something that's missing, they will actually add to them. And that's kind of how
all of it ‑‑ they all work. That's how the API works.
The next thing I want to show off is the Nmap scripting engine is great. But what happens
is when you run a ton of Nmap scripts, you end up with a bunch of different files, and
you kind of have to look at them all manually in something like Vim or parse them out yourself.
So what we did is we actually made the drones parse those. So this is ‑‑ this next scan was a full scan of my network.
With version detection, operating system detection, and script scanning enabled.
See that we get the FTP anonymous Nmap scripting output put in what we call a service level
note here. And this is just a service level view into each port. You can move back and
forth between them. You can update and modify all of these as well. The next thing ‑‑ so
that kind of takes care of ‑‑
I'm importing our data, right? Let me import a Nessus scan real quick.
All right.
So once ‑‑ we just imported a Nessus scan and it again added more information,
such as host names identified and operating systems detected. The next thing that we wanted
to work on was the collaboration effort and not duplicating work. So what we came up with
was kind of this color‑based system.
Oh, my God. Was that boring.
All right. Here we go.
You guys know the drill. These are new speakers. All right. Get up here.
I love that. We walk into rooms, now people have their hands up. It's awesome.
What do we call this, Paul?
What do we call this? Shot the noob. That's right. And what is your name?
Nassim.
Nassim. We have Nassim. We have our speakers whose names I don't know.
Tom.
Tom. It's Tom.
And Dan.
And Dan. Do we have shots?
We do have shots. I'm just trying to work.
I'm sorry.
Oh, my God. All right. So we've, you know, done a couple of these already today.
This hour.
Somebody pull that bottle away from him. Did you guys drink before your talk?
No.
I'm sorry. Have you not been to DEF CON before?
No.
No.
I don't drink.
I don't drink.
All right. You're doing two shots.
All right.
Thank you.
Thank you.
Welcome to DEF CON.
Good man.
I'm sorry. DEF CON's been canceled.
All right. Round of applause for Dan for taking the bullet for me.
Thank you.
And now look what you've done. You've locked my computer. Okay. So now I have to move really
fast. But the idea here is that we didn't want to duplicate work. So what we came up
with was a color coded based system. So first I need to add Dan to my project here. So now
Dan has full view into the application. And we came up with this color coded based system.
It has five different colors. So you see those at the top here. You see gray, blue, green,
orange and red. And those can mean whatever you want. They mean certain things for us.
In particular, blue means that it's in progress. So I want to know what Dan's doing at all
times. And I don't want to duplicate the things that he's doing. So all Dan needs to do is
click a host. And as soon as he clicks it, it turns blue. And I can know that Dan's working
on that. And then what might happen is, you know, Dan's
like, we turn them green, there's no serious issues. Maybe if we turn them orange, that
means we want to come back to it later. And if we turn it red, that may be a foothold
in the network or it's pwned or there's sensitive data leaking out of that thing or something
like that. And so that's kind of the color coded based system that we came up with so
that we can track exactly what we're doing and not duplicate work.
What's really neat is all of these kind of filter. So if I click that blue button, it
will filter out the blue ones. If I click the green one, it will filter out the green
ones.
And so we can really focus on, you know, see what hasn't been done and what has been
done. And this works at every level, the service level, the vulnerability level as well. We
imported that Nessa scan and you can see that a list of vulnerabilities got imported
in here. You can make these ‑‑ create these manually, et cetera. And like I said,
they all have statuses as well. The typical things that you'd expect from an application
like this, description, evidence, solution, a list of hosts, these are all linkable, everything
like that. And, you know, any notes that might be available as well.
The next thing that we kind of want to show ‑‑ when I put the project update, the client
side update ‑‑ since ‑‑ you know, which is kind of like the best thing about
Meteor is that it allows client side updates. For security purposes, we have those turned
off, meaning that any time you do a database query, it gets shipped down to the server.
But what's very useful is we do allow you to do that. You don't have to worry about,
like, turn that functionality on so that you can do anything on the client. And what
that lets you do is kind of create some very interesting JavaScript scripts that you can
run in your browser. So I have to allow these. And you're given a security warning telling
you that you're allowing them here. I come back to my project, load it up. Dan has graciously
provided me with this script. So what this script does, it's kind of lengthy,
but it just goes through each host, looks to see if any of those hosts have any available
services, and if they don't, it's going to turn them green, because there's nothing
to test, and when you're testing thousands of hosts, it can kind of be efficient. So
the idea is that you can write one‑off JavaScript scripts to kind of transform your data in
various ways. So if you just open up the JavaScript browser ‑‑ or console, I'm sorry ‑‑ and
you paste this in ‑‑ so you can see ‑‑ so I know .6 and .46 don't have any services.
So these queries are being run on the clients, and they're being shipped down to the server,
and the data has been updated. Another cool thing that's just built in is chat. So if
you're on an internal engagement or something and you don't want to feel like getting around
a firewall, you can do that.
You can just use the in‑built chat like that. Yeah, and that's kind of it. There's
other things such as ‑‑ okay. Next thing is this service tab. A very common thing when
you're on a penetration test is you want to get lists of hosts that have certain services
open. So you can do that here and search through these, or you can also just click.
So this is a unique list of port protocol service and product. If you click on any of
these, it will reduce the search.
And give you just a list of the host with that port open. Which is very, very cool.
It's just kind of a convenience thing. Another thing you might want to track during a test
is credentials. So those happen at the service view. So you can add those in here, or you
can maybe build a drone that adds these in. So they're shown here. And you can see that
they're shown at the top service level view here. And then they're also given to you in
kind of its own tab here as credentials.
So that's it. Let me bring up the slides again here.
Okay.
So the next step that we have in the project is that we need more parsers. Like I said,
we only have them for four tools. So we'd like to write more. So if you have any tools
in particular, like Qualys I know is a huge one, we will write those. We'd also like the
community to contribute writing them as well. One of the biggest things that we're looking
at doing is syncing with this Metasploit database so that you can basically use both applications
and have the data synced simultaneously. So we are working on that.
And another big thing is we know we need more documentation. If you go to the web site,
like when you see the GitHub site, you'll see that the documentation is pretty sparse.
We have been spending the past six months on this trying to make it very, very polished
and make sure everything works. So the code and getting it working has been our major
goal here. The documentation is next. So we'll have documentation up of how you can interact
with the API to build your own parsers, how you can install it from scratch, et cetera.
The source code is all available on GitHub. That's the link.
We do provide you with precompiled packages. So it has a database, a specific version of
Node, the application, all bundled up with some easy‑to‑use shell scripts. Those
are about 100 meg each. And they're for Linux, 32‑bit, 64‑bit, and OSX. I didn't have
the bandwidth to upload them here at DEF CON. So they will be uploaded tomorrow as soon
as I get home. But you can just follow that GitHub link.
Also, if you want to hit me up on Twitter, I can send you the link. That's my IRC name
on Freenode. And that's Dan's Twitter. So we have some extra time. Are there any questions?
Yes?
Sorry, just quickly on the ‑‑ why did you start 3D boarding when you were
managing ‑‑
Yeah. So the way Meteor works ‑‑ like,
I said, it has a database driver kind of written on the client. We deny all of those so that
the queries actually get shipped down to the server. You can kind of do it either way.
So that security warning is just basically saying, hey, you're going to allow all of
your users to modify the database without any sort of validation. So on the server,
we actually do pretty strict validation on what data you're putting in. So it will validate
that you're putting an actual IP address in. That doesn't do anything of that. So you kind
of have to trust what you're doing. And that's only available ‑‑ those settings are
only available to administrative users. So that's what we're trying to do. So if you're
going to do that, you're going to have to trust what you're doing. Any other questions?
Okay. Well, thanks for coming out.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.