So real quick about me. I have a bit of a reputation as kind of approaching this as an academic
frameset, simply because I got started in the whole password cracking thing
through my research when I was getting my PhD. But I really do strongly believe in learning by
doing. So I'm an active member in Team John Ripper, and I do participate in password cracking
contests like, you know, Crack Me If You Can, that's going on right now. Luckily, this talk is,
you know, being filmed, you know, before the contest starts. So no spoilers here,
unfortunately. But good luck to everyone else who's participating. So password cracking,
really, it's my hobby and a little bit of an obsession. But it's not my day job, unfortunately.
But my day job has been very exciting recently, though, because I really focus on medical device
security. And you can imagine with all the craziness around COVID-19, it's been an interesting
time. So one project that I really kind of want to highlight is the Open Ventilator Monitoring
and Alerting Project that I've been helping to contribute to. And there's actually a talk at the
Biohacking Village this Sunday that I really highly recommend people go ahead and listen to,
so that our team members are giving it. And it really is going to talk to you about, you know,
the lessons learned and how other people can help contribute as well. Because this has been a big
problem because, as I'm sure you're aware, there's been a huge demand for ventilators to be able to
help deal with COVID-19. So there's been a lot of different projects that have kind of stood up
to try to help produce, you know, low-cost ventilators to help fill that need pretty
quickly there. So rather than have every single do-it-yourself ventilator develop their kind of
own monitoring and alerting framework, we're trying to produce one common one that can be
applied to all these different projects across the board. So because when you have these ventilators
being able to treating, you know, patients, the patients are highly infectious, so you don't want
to have the nurses exposed to that. But if something goes wrong, you need, you know, seconds
count. So you need to be able to forward all that sensing information that these ventilators are
doing back to a centralized nursing workstation. And you need to do that securely because you're
not just on a real hospital network. So that's been a really fulfilling project that I've been
working on. So another thing I'm kind of helping out with here, as I move my head, is I'm helping
out run the Defcon Biohacking Village's Capture the Flag contest. So this was originally supposed
to be in Vegas. That changed, of course. So now a lot of medical equipment is actually sitting in
my house. So I have to be able to provide a way for hackers from all over the world to be able
to log in and hack these infusion pumps here without also hacking my smart thermostat.
As part of that, I actually had to repurpose one of my password cracking rigs, as you can see there,
in order to run all the VMs that are helping to keep people, you know, on those ventilators
and hacking those and not, you know, hacking my smart thermostat. So probably the first
questions I should start kind of addressing here is, you know, what does that PCFG stand for in,
so originally, and I guess still technically, it stands for Probabilistic Context-Free Grammar,
which is the kind of the modeling framework it uses in order to model how people create passwords.
So if you're into, you know, the Siri Autonoma or, you know, formal languages, this might actually
mean something to you. But for most people, you know, they hear that and they're like, oh god,
that's like mass and stuff like that. I mean, there's no way it's going to run on my computer.
And then he's kind of like slowly walk away. So I decided in order, I need it need to have a more
descriptive name. So I went ahead and rebranded it the Pretty Cool Fuzzy Guesser. So, and this
kind of explains it a little bit better about what it's actually doing underneath the hood,
because you train this on a list of passwords. And then it'll go ahead and create guesses that
are similar to those passwords, but different, which is really kind of important in order to
help, you know, expand your cracking session. So I don't this is my favorite slide I've ever made.
So it's all downhill from here. So really kind of what it's doing is it's using machine learning
in order to crack passwords. And when I say machine learning, I mean, in the traditional
sense of a whole bunch of if then statements. So it's not using neural networks or artificial
intelligence. But you are training on passwords that you expect to be somewhat similar to the
target passwords that you're cracking. And when it processes that training password set,
it extracts all sorts of probability information about the components of those passwords that
it finds there. So it figures out things like, you know, capitalization masks, whether numbers
go at the beginning of the password versus the end, the probability of individual letters and
numbers found in that password, keyboard walks, and so on. And so it goes ahead and creates a
model based upon all those different types of probability information there. And then it uses
those in order to generate very highly probable password guesses in probability order. So it'll
start with the most probable password guess and go to the second most probable password guess and
then go to the third one and so on until you crack the password that you're trying to find or
you give up. So let me just move my thing here a little bit here. So just to kind of tie us back
into probably what's going on right now, as I said, I don't know what the actual contest is going to
be like. Forward or crack me if you can. But CoreLogic helpfully provided a brief summary of, you know,
what the scenario is going to be at least here. And so we're going to be targeting 12 different
individuals. And those individuals change their passwords over time in order to be able to deal
with more complex password creation requirements. And that sounds a little bit something like
something that, you know, a PCFG might actually be useful for. So I'm really optimistic for this
contest. You'll see how, you know, optimistic I am on Saturday when I'm actually, you know,
we're giving this talk. But, you know, this is kind of the scenario that this was originally
developed for in the first place of, you know, you know how a subject creates passwords. So you
want to create passwords kind of similar to that. But you also want to go ahead and change them and
maybe, for example, you use more complex rules or, you know, complex password creation requirements
added on top of that there. So I'm available probably on Discord right now. And so I'll be
able to answer questions about how potentially you might be able to tweak this in order to
help in a scenario like this here. The fact that there's a lot of academic papers about this,
though, is when I give a talk like this, I don't actually have to create any of my own graphs. I
just can go to other papers there, you know, look at the research that other people have done,
and just pull out their graphs in order to be able to talk with it here.
So one thing that I really kind of want to highlight, though, and you need to kind of
look at this with a bit of a skeptical eye, is that you'll notice that all these cracking
sessions are really, you know, short. I know, you know, one, you know, trillion guesses here,
that might sound like a lot. But when you start talking about, you know, GPU password cracking,
you're talking about like under a second in order to generate all those. So that's just, you know,
no time whatsoever. And part of reason for this here is that the PCFG approach, it's very slow,
it doesn't, you know, scale very well with multi-threading currently. So when you start
talking about, you know, passwords that you want to be able to crack, it works very well
when you're going ahead targeting very slow password hashes, where you can only make, you
know, you know, thousands of guesses a second, because the hash is very slow. But we start
talking about things like, you know, unsalted MD5, other attacks that are going to be much
more effective, because you can just make so many more guesses in the same time frame there.
So when you start talking, looking at, you know, faster password hashes, you can certainly go
ahead and still use a PCFG to supplement your attack. And you can still go ahead and crack
some passwords that you might not normally get. But in general, for the faster password hashes,
you really are going to want to go ahead and use more traditional types of password cracking
attacks in order to really make use of the hardware that you have available to you.
So I want to talk about this graph kind of, though, and really focus on it, because this
was a really neat study done by Carnegie Mellon University. And one of the problems when you look
do academic research, especially when you start talking about offensive
tactics, is that the academics are, you know, running attacks themselves. So you're looking at
the, you know, the how effective students are at cracking passwords versus someone professional,
potentially. So CMU, you know, took the probably the, you know, the most straightforward approach
to be able to solve that problem was they went out and reached out to CoreLogic. You might have
heard of them. They're running this, you know, password village. They run the Crack Me If You
Can competition. So when you're trying to find an expert, you know, they're like, you know,
way up there. So they're a pretty good representation for that there. So what they
did was they gave, you know, one of the CoreLogic engineers a password list. They asked them to
crack it. And they recorded, you know, how many passwords they cracked over time. It was the
number of guesses they made. And then they compared it against other cracking sessions as well.
And, you know, one thing I'm really, you know, it makes me smile every time I see this here
is that the PCFG did really well compared to the pros, which was CoreLogic,
for that short cracking session. So when you start asking, like, you know, can this represent,
you know, how a real professional password cracker operates, the short answer is it
certainly, you know, appears to be able to be able to do that there. So, you know,
full disclaimer, when you gave, you know, CoreLogic more time, they definitely performed
way better. This is a logarithmic graph, so that's about 100 times more guesses.
And also, I'll admit this wasn't fair to CoreLogic either, because, you know, that's not typically
how, you know, people crack passwords in real life there. It was such a short, you know,
cracking session. And when it is that short, usually it's against a really strong password
and you have a lot of time in order to really manually tweak your attack that you're running
there. That being said, if any of you are listening, I would love to have a repeat or a
rematch of this, you know, attack just to see, you know, how this performs with all the new improvements
that have been made into PCFGs. And I'm sure that, you know, CoreLogic has really been upping
their game over the years as well. So that's why I'm somewhat hopeful that we'll be able to find it,
now. So enough about, you know, all the research side of that there. Let's talk about how to
actually make use of this PCFG password cracker. So the first thing is you just go ahead and
download it from the GitHub repo. And the requirements of it, I really have strived to
make it as simple as possible. So you need to have Python 3, and that's it. So there's an optional
Karadet Python module that can help during the training. And that's because it helps detect
what character encoding the training set is, because character encoding is the bane of my
password cracking existence. But even that's optional. And actually, it's now being installed
as part of PIP 3. So if you have PIP, you probably don't even need to install it yourself as well.
And this is really useful, though, because I find a lot of situations where, like, when I'm cracking
passwords, like, I don't have, like, internet access. So it's really nice to be able to go
ahead and quickly throw my tool on a box and get it to run. So if you can run Python 3 on a box,
you can probably run this here. So I've tried it on a bunch of different OSs. I've actually even
gotten it to run on NetBSD. And it was the only thing I've ever gotten to be able to run NetBSD.
So hopefully this is easier than your typical academics tool set in order to get it installed
and start cracking passwords with as quickly as you can.
So we start talking about hardware requirements, because that's always an important portion when
we start talking about password cracking. The PCFG tool set, it is single-threaded CPU bound,
which is why it's so awfully slow. But it will use an entire CPU thread. So you do really need
to dedicate one full CPU, you know, thread to the PCFG. The other thing is it has very high RAM
usage. It basically maintains a lot of different data structures and memory. And those data
structures become more complex over time. So it just grows. So I could have done some things,
tried to go and prune that or, you know, move some of it to disk. But RAM is cheap. So I haven't.
So it'll just keep on growing over time. So initially, it starts out pretty low usage.
But if you're talking about running this, you know, password cracking session for like a week
or two, you really need to have at least 16 gigabits of RAM to really kind of just fully
dedicate to the PCFG tool set itself. So the next step is to actually make use of it and run it.
So I apologize up front that I tend to use the words rule set and grammar interchangeably.
At least to me, they mean the exact same thing. But really what I'm talking about is that,
you know, I mentioned, you know, machine learning a couple of times here, you have to go ahead and
train a grammar on existing password data set. Now, you may want to have, you know,
you know, different grammars for different targets that you're trying to target. So if you're trying
to target, you know, a web application catering to younger people, you might want to train it on
passwords that resemble that. If you're trying to target, you know, corporate passwords, you might
want to train on corporate passwords instead, and use those rule sets against target passwords that
you're specifically you think will match that. So you can have as many rule sets as you want
to be able to really kind of fine tune your cracking session there.
So the default one rule set that comes with the PCFG password cracker was actually trained on
a subset of 1 million passwords from the Rocky data set, which came out in 2008. And it was
against web passwords. So there wasn't really any strong password requirement whatsoever.
I've been thinking about updating that. So if you have a good data set that you think that
I should use for that there, I'm open to hearing about that there to make it a little bit more
effective. But that being said, the Rocky data set is still extremely effective even to this day.
It's just, you know, Blink-182 is not nearly as popular. So after you choose the data set that
you want to use, though, now you go ahead and start generating guesses. So it's a Python program.
So you just run, you know, Python 3. You run the PCFG guesser.py tool from the repo.
You get it in the ender rule set. By default, it's default. So if you don't go ahead and
specify that there, it'll go ahead and use the Rocky data set. And then you go ahead and specify
session name as well. By default, this is default as well. And so the session is used to restart a
password cracking session. So if you have to cancel it for whatever reason, you can go ahead
and restart it back up again. So I really want to kind of highlight, though, that the PCFG tool set
is only a password guess generator tool set. It will generate password guesses. It'll generate
those password guesses in probability order. So it'll start with the most probable password guess,
second most probable password guess, and keep on going down the line. It will not actually
hash and crack any passwords. So you need to use another password cracking tool set for that there.
You know, both JohnDeRipper and Hashcat work are basically any other password cracking tool
that accepts, you know, guesses in from, you know, the standard input there. As I mentioned earlier,
I'm on team JohnDeRipper. So I'm going to go ahead and use JohnDeRipper for pretty much all
my examples here. But you can totally use Hashcat as well. So in order to do this here, you run the,
you know, the previous command that, you know, I talked about. And then you run, you know, pipe it
into, for example, John. And on John, they have, you know, a command called standard in, so that
you type that in there. And instead of running data from like a word list, or generate your
password guesses, it'll go ahead and use the password guesses that are piped into it instead.
And you're cracking passwords. That's really all there is to it.
So there's definitely optimizations for actually using this in the real world, though.
So the first thing I really kind of want to highlight is a lot of times you want to know
what the status of a cracking session is. So the challenge when you are using the pipe command,
though, is if you go ahead and hit the enter button on your keyboard, instead of sending the
enter button to JohnDeRipper, it's going to go ahead and forward that to my tool instead.
So you might want to be able to, you know, get JohnDeRipper to output a status report.
The way that you do that is you send a siguser1 signal to JohnDeRipper.
If you're writing this on a Linux system here, you just type in kill-siguser1 and then the
process identifier JohnDeRipper. And when you do that and hit enter, it'll be just like hitting
enter on the JohnDeRipper itself and it'll go ahead and output the status output of its
current cracking session. So now you can do things like, okay, not only see the passwords
are getting cracked, but you can see like the number of hashes, the total number of hashes
are cracked so far. You can see like, for example, the guessing speed. So in this case,
it's making about 4 million guesses a second. And then you can see like how long it's been
running and, you know, all the other, you know, options as well. So I want to kind of dig into
that one, you know, output of that cracking session though, because I think, you know,
this really kind of helps demonstrate kind of some of the power of using the PCFG.
Because normally when you're, this here is showing the passwords as they get cracked.
So you can kind of see that it's not just going ahead and, you know, figuring out one rule and
then exhausting that rule and then going to the next rule like you would see in the more
traditional password cracking session. Instead, it's creating much more fine grained rules
and iterating between all those depending on what the current probability of it is here.
So when you see these passwords being cracked, it's kind of fun to try to figure out like
how did the underlying system, you know, generate that password guess? Why is it making the guess
right now that that it is? So kind of if you look up, you know, initially here,
like this is pretty easy. Okay, it's just taking like some five letter word.
I apologize, my microphone just died here. So fun doing, you know, DEF CON remote.
So, you know, it's using five digits, you know, or five letter words plus four digits here.
Moving on, though, this is a kind of an interesting one here is SES is cool.
So I looked into my input dictionary and SES is cool was not in my input dictionary or my train
set at all. And I found out that it was actually you doing multi words for this year. So it was
combining, you know, SES and then it's cool. So one kind of cool thing about this, and I'll talk
to us about this a little bit later, is that instead of, you know, going ahead and breaking
this up into three words, like we normally would think about it there, it actually broke it up into
two different words. So SES is cool. So that way you can go ahead and go through it and say like,
is, you know, Katie cool? Is, you know, Allie cool? Is Bob cool? Because there's a lot of cool
people out there. So it can go ahead and iterate through those there and try that type of, you
know, mangling rule for it there. And what's really cool about this is that it learned that
is cool is a common word from the train set itself. So I didn't actually ever program in
that logic into it. It learned it by itself by looking at the training data, which is,
as I said, pretty cool. But you can see after that there, it kind of went into brute force.
It wasn't the pure brute force, and I'll talk about the different types of brute force here,
is actually combining some very short words, kind of like a combinator attack.
But still, you know, it's able to kind of get that out that way. And then it went ahead and
tried, you know, words with, you know, special characters, the same special characters at the
beginning and the end of them too. And that's, you might be able to see that in a traditional
password cracking session, but you actually have to have a rule in order to generate that. And
trying to create those rules is a real pain. So you won't see those in, you know, most, you know,
common publicly available rule sets, but it was able to learn that from the training data, which
was, I thought, pretty cool as well. So down here, and I'm kind of, you know, need to get off
the screen here, but you can see it's trying some longer words. But these are actually, while they're
normal words here, it's actually generated them via the multi-words as well. So like finger plus
nail, or 90 plus 9. And once again, this is kind of really useful because now you don't have to
have things like 99, 98, 93, you know, in your data set, your word list as well, because it's
generating those on the fly. Kind of going down a little bit further here, this is your more
traditional kind of rule here. It's just two digits plus capitalizing the first letter of a word.
So you can see it's starting to do that. But, you know, settings is a pretty uncommon password,
word to be able to use. So it's trying it later in a cracking session here.
And now it's even combining even more mangling rules. So it's trying, doing a multi-word,
you know, of wood plus fish and, you know, Tara plus Don, and adding digits to the end of that
as well. So you can see how it starts stacking these different rules together. And I kind of
want to highlight that this cracking session has been going on, as you can kind of see from the
status output, for about 13 minutes. So all the really easy passwords have already been cracked.
It's already guessed, you know, 1, 2, 3, 4, 5, 6, and password 1, 2, 3, 4, 5, and so on.
So these are starting to get into more of the, you know, the fuzzier of the rule sets that you
might not normally see in a normal password cracking session. So as I mentioned a little
bit earlier, if you hit the enter button, it's going to go to my program, not, you know, John
the Ripper. But there's a lot of information that I want to be able to provide to people about the
cracking session. So whenever you hit enter, or basically any of these R keys here, it'll display
an output of what it's currently doing. So you can try to figure out, you know, whether you want to
continue it, whether it's working correctly, and whether it's kind of doing what you want it to do
as well. So kind of going through this here, I hit enter twice. And so you can kind of see how
it's generating, you know, these password guesses as doing it here. So the first one, you know,
it's, you know, basically going ahead and trying to combine two words. So it's a multi-word type
of attack. And you can, if you dig into like the real details of it there, you can kind of see
that it's trying like the hundred and forty-third most probable word with no capitalization. And
it's combining it to the ninety-third most probable four-letter word with no capitalization
as well. So you can see that the probabilities it assigns to like even individual words and
stuff like that is very, very fine grained. So it's going to try some words, and then like do
other mangling rules and stuff like that. And they'll go back to the less probable words later
on in the cracking session. So now this next one here, it's kind of a little bit, I'll try to get
out of the way or something like that. You can see that it switched to a real brute force attack using
OMEN, ordered Markov enumerators. And I'll talk about that a little bit later there.
But really, I kind of want to highlight though that it's trying, you know, more traditional cracking
rules. So it's like, you know, combining words. And then it's switching to brute force. And then
it'll switch to another mangling rule after this here. And then it'll just keep on going
based upon whatever the current probability is. Now, as I said, I really struggle with documenting
my code. So I try to go ahead and add as much documentation into the runtime behavior of it
as possible. So if instead of hitting, you know, enter or anything else along those lines, you hit
H and hit enter, it'll provide, or just H actually, it'll provide a SAS report output of what all
these different fields here mean. And that SAS report actually is much longer than even, you
know, displayed on the screen here. But it explains what all those different, like, letters like A5 or
C5 actually stand for. The one thing that I kind of really want to highlight though is this one
metric here called probability coverage. Because since the PCFG password cracker creates guesses
in probability order, it starts with the very high probable passwords, and it goes to less
probable passwords and less probable passwords. And the model that it has will basically never
finish. It'll just keep on figuring new combinations of words to go through to it.
So a real challenge becomes, you know, when do you go ahead and give up on a cracking session?
So you haven't cracked the password. When should you go ahead and, you know, kill this off and try
some other cracking type of attack that might be more successful? Or when do you go ahead and just
choose to say, I'm not going to crack this password and move to a different case? So this probability
coverage is a very fuzzy metric that I tried to develop to try to just give you a little bit of
kind of a rule some about when that should be. So what this metric says is that if the target
password is the same probability distribution as the password data set that I trained, and if my
grammar and how to model how these passwords are created was exactly correct, this is the
probability that we cracked this password. Now, neither one of those assumptions is actually true
in real life. You know, the probability model of the password you're trying to crack is probably
very different. You know, the grammar that I generate and train on is absolutely not perfect.
But at least, as I said, it kind of gives you a rule of thumb to say, okay, you know, this is
starting to get a little bit high. It says, you know, I had like a 90% chance to crack this password.
I haven't cracked this password yet. Maybe I should go ahead and give up. And you'll notice
this number jumps up really high initially because it's making, you know, high probability
password guesses. And then it slows to a crawl to almost like no advancing after you get to,
you know, like 70 or 80, 90 percent, you know, completion there. So this is kind of really good
to be able to figure out, you know, where can I go ahead and devote that, you know, that one single
CPU and that RAM to somewhere else there. So another usage tip I just kind of want to highlight
is that sometimes the cracking dynamic when it comes to speed is completely reversed.
So you might be trying to crack very, very computationally password hashes, expensive
password hashes, or a lot of like, let's say, assaulted hashes, in which case you're really
only making, you know, a couple guesses a second. Well, this generator is generating, you know,
let's say, you know, between like, you know, 100,000 and like 4 million guesses a second.
So it gets backlogged. And basically, essentially freezes while it waits for to be able to send more
guesses to the password cracking program. So occasionally, if you hit enter, it won't actually
display set the status or it'll take a while to display the status. And that's kind of usually
what's happening. So if that's happening, and you're kind of curious whether the password
cracking session has crashed or not, I recommend going back to earlier advice about sending a
signal to, let's say, John Ripper, and just seeing how that's doing there
in order to make sure that your password cracking session is still running.
So, as I talked about, you know, multi-word feature has probably been the biggest,
you know, addition to the new 4.0 rewrite. And it has completely shocked me how effective this
has been here. So I won't get talk too much details about it. But the one thing I want to
really kind of stress, though, is that it is not language specific at all. It learns all what
constitutes a word from the training set that you're giving it there. So it'll pick up things
like new band names or proper nouns that are really hard to specify inside the language
dictionary or whatever new Pokemon just came out. And identifies patterns like, you know, I love,
and stuff like that. So this is very useful for being able to, you know, you know,
try to target new, you know, password hashes. So, as I said, it's not language specific.
It works best with, I would say, kind of like European English type languages.
It really struggles still with some of the other languages like Mandarin. But that is something
absolutely that I really want to focus on more going forward here. It's not perfect. It's
definitely a work in progress. So there is a balance between, you know, creating, you know,
false positives and matches here. If you don't see some of the base words in the train set by
themselves, it won't identify them. But it's something that is evolving. And part of the new
pull request that I just received from somebody else actually has some improvements to this here
that I'm really excited about, again, pushing to Maine. So one of the other big features that have
been added recently here is ordered Markov enumerator. And the whole reason why I talked
about this is that a similar approach can be taken for pretty much anything. So if someone
creates a better cracking attack or cracking mode, it can totally be incorporated into a PCFG style
attack. I'll be a little bit like the Borg in that respect. But the real challenge is to be
able to figure out how to assign a probability to a password guess. So if you can assign a
probability to a password guess, I can probably incorporate it into a PCFG.
So just kind of in the last little bit here, I really kind of want to highlight
some additional tricks that are very useful when it comes to cracking passwords. So the first one
here is this skip root flag in the PCFG. And basically what this does is disable omen guest
generation. And that's not to say that omen guest generation is something that's bad to do.
It certainly definitely helps increase the success of a password cracking session.
But this is a way to paralyze your attack. So if you're having another system that's going ahead
and really cranking through your brute force attack, you might want to go ahead and do all
your brute force on that other system or on that other thread. And then run the PCFG guesser really
just to focus on the word mangling rules instead. So in order to do that, all you need to do is
just when you run it, just type in skip root. Another flag that's really kind of useful is
the all lower flag. And what this means is it'll stop doing any sort of case mangling on the
password guesses. So let me try to move my picture just a little bit here just to make it easier to
read as I go back. I apologize. Okay. So a lot of times you may want to not go ahead and do case
mangling inside of PCFG itself. And one reason might be that the hash that you're targeting is
case insensitive like landman. That's not probably the best example, though, because if you're
cracking landman hashes, you're not using PCFG in order to do that there. You're just going ahead
and brute forcing that sucker and taking it out that way. Where it's more likely, though, is that
case mangling is very distinct for how people do it there. So if someone does a certain type
of case mangling, they have a tendency to keep on using that strategy for all their other passwords.
So when you start doing things like targeted password cracking, you may not want to go ahead
and just go ahead and do what everyone does. You want to really make a really specific case
mangling for that particular individual. In that case, the better way to do this is that JohnRipper
supports a really powerful feature called Pipe. So what the Pipe does is instead of just going
taking the guesses in from standard input and running them as is, you can apply additional
rules on top of that like you would do in a traditional password cracking dictionary type
of attack. So you can specify your very specific case mangling rules inside JohnRipper's rule set
and then pipe the lowercase password guesses right into JohnRipper and have JohnRipper
capitalize it itself. And that can be very powerful when you have an idea of what type
of case mangling you want to be able to target. So I of course moved it to the wrong portion here.
Let me move my screen again here. I apologize.
Some kind of improvements. As I mentioned, there was an amazing pull request that was
submitted to me with a bunch of new features. I'm slowly incorporating them into the core,
but I actually have the features available as their own kind of tool called segmenter.py.
I mean, and I apologize if I mispronounce your name because I've only seen it written.
But Chun-Wan Wang submitted this here and it really impresses me there.
So probably the biggest feature I'm really excited about is leet-speak replacement. This has
been a feature that has been kind of my white whale as far as implementing. And it's just
every single time I've gone through it, it's just not been very effective. But that's
currently incorporated into this tool he has called segmenter.py. That's been included in the
repo that will go ahead and try to parse that information out. So I'm looking at getting that
incorporated into my core trainer and getting that incorporated into password cracking sessions in
order to be able to really target that there. He also improved some of the multi-word detection.
So he made that better. And then he also has incorporated some new approaches into the
password score, which is a different tool that you can go ahead and submit your password into
the password score. And they'll tell you what the probability your password is,
which is kind of nice as well. So all credit goes to him for this. I'm really impressed with this
here. And if anyone else is looking at helping out too, I'm all about that. So thank you very
much once again for that there. Okay, so let me move my screen around again here. Okay, so
the next thing I currently want to talk about here is the compiled PCFG guesser. So I've been
talking about the Python tool set all along right now. So the compiled PCFG guesser is a completely
different, you know, fork. And as you kind of get the name there, it's still being written in Python,
it's written in compiled C code. It's a little bit harder to get actually installed and running
simply because when you start talking about compiling your code, you know, it runs great
on my machine, but it has challenges elsewhere. I tried to go ahead and use a Hashcat build make
file for this. So if you can build Hashcat on your computer, you have at least a better chance
of being able to go ahead and get this running as well. But if you have problems, please, you know,
reach out to me on the GitLab or GitHub site, and I can try to help you fix those there.
So I will say that the trainer portion, it will always be in written in Python.
I just like writing Python too much to change that over. So basically, you'll go ahead and
create train rule sets with the Python trainer, but then copy them over to be used in the compiled
version here. Also, the compiled version has a tendency to lag in features from the Python tool
set, because once again, I like writing Python. I'm not the best C coder in the world. So basically,
if I write a hello world program, it's going to have like five buffer overflows and, you know,
a segfault. So take what you will there, but I'm making this available. If someone wants to
write a better one, I'm totally open to that as well. But, you know, it doesn't have save restore,
it doesn't have status outputs, and it has no omen guest iteration. So all that being said,
you know, why bother with this here? And really, at the end of the day,
the main reason is it's about 20 times faster than the Python tool set. And I've always heard
that, you know, C code is faster than Python. But when I saw that, I was like, holy crap.
So I will be up front. I'm actually, even with all these limitations, when I'm cracking passwords,
I'm using the compiled C version now much, much more often than I'm using the Python one there.
So because that 20 speed improvement is hard to beat for most password cracking sessions.
So now I'm going to talk real quick about training passwords. So I've been talking about this a lot
here. And there's a lot of different reasons why you want to go ahead and create a new password
training set there. So language is a huge one. So you want to be able to train on passwords that
are similar to the target that you're trying to target. And another big one is that corporate
passwords are very, very different than you'll see from websites. And I'm sure you probably heard
CoreLogic talk about this before, you know, yesterday. But that's something that, you know,
is very evident. So if you're trying to target corporate passwords, you probably do want to go
ahead and train on corporate passwords versus going ahead and training on passwords for some
gaming website. So another reason to go ahead and train it, though, is if you're targeting a specific
password creation policy, or you know which mangler rules your target prefers. So one way to be able
to really target that there is to train only on passwords that match that training set there.
And there's other things you can do, like the password rules, or the, you know, the grammar
that I generate. I made sure that I didn't include anything like a CRC check or any of those
sanity checks into it there. So you can actually open up the files themselves, they're just text
files, and start editing the probabilities of different things in them by hand, too. So if you
say, like, oh, this is one word I really want to go ahead and make it, like, highly probable,
but I don't want to go have to train on a whole new training set. You can just go ahead and open
it up, put that word in there, give it whatever probability you want, and that will just be read
in and used in your password cracking session there. So the other reason to train on a password
train set there is it generates a bunch of information and extracts a lot of information
from that password set. So it's really useful to be able to analyze a new dump that you have
accessible to you there. So for example, it'll pull out, like, common emails, it'll pull out
dates and websites, and try to help you figure out where did this, you know, password data set
come from there. So the next question, of course, is, you know, where do you go ahead and, you know,
get these password data sets from? So there's a lot of challenges with this, too, because a lot
of data sets are not optimal when it comes to training on. So I don't know if you know of
hashes.org, but it's a really great site for being able to download all these, you know, dumps as
they come out here. So for example, let's say you want to go ahead and train on this data set here.
I'm not going to try and pronounce the name of that site, because I'm sure I'll just horribly,
horribly mangle it there. But when I did some googling about this site here, it was a site for,
you know, new college students trying to find a job in China. So that's kind of an interesting
data set there that you might want to be able to use in order to train for correct passwords here.
So if you download something from, like, hashes.org, the first and most important thing is select the
plain text option to train your rule set on, because you don't want to include the hash
part of your training set, because then you'll think it's part of the password and just it goes
poorly there. So one other thing I really kind of want to highlight here, and this is a
feature that I'm hopeful to be able to get added to the PCFG tool set, but I was informed by the
owner of the site here that they actually do some additional things for encoding non-UTF-8 characters
that my trainer will not fully parse correctly. So that's something I need to add in too,
so that it goes ahead and, you know, uses the correct character encoding for non-English
passwords. So I just want to put that warning out too for trying to train this on things like
Mandarin. But one problem with a lot of these dumps here is the first one is they don't contain
duplicate guesses. So duplicate guesses are really important when it comes to trying to figure out
what the probability of password is, because if you don't have duplicate guesses, 1, 2, 3, 4, 5,
6 looks like a very just random string. So that's useful, but I will say when you run longer
cracking sessions with PCFGs, that lack of duplicates becomes less and less important,
because you've already exhausted all the really probable password guesses. The one issue though
is that the OMEN portion really does struggle without the duplicates. So you might not want
to go ahead and enable OMEN guessing if you train on a data set that doesn't contain any duplicate
guesses there. The other problem with these dumps is that they only contain the passwords that have
been cracked. So basically you don't know or learn anything about the passwords that haven't been
cracked. That's not a deal stopper, but just it's useful to keep that in mind there, that the
cracked percentage is going to be very useful when it comes to figuring out how good a data set is
in order to create a new rule set. So in order to train on a password data set there, really
you just apply some program once again. You just give the name of the rule set that you want to be
able to train it on, as well as the password data set that you want to train it on as well.
And it'll go ahead and run in order to do all the parsing and stemming of the
password data set here. So it will try to auto detect what that encoding is,
but when in doubt, set it to be UTF-8 because the encoding really does matter quite a bit there.
So the first pass it takes through the data set there, it learns all the character
frequencies and base words for multi-word detection. So it actually makes a couple
different passes through the same data set in order to learn more and more and more about it
there. The second pass it goes through there, it'll do much of the real parsing of the password. So
it figures out things like, you know, keyboard walks, alpha strings, you know, letters, how
probable like the digits are and stuff like that. So most of the stuff you think of traditionally
when you talk about, you know, what their probabilities of different, you know, things
are, it does on the second run through. And it actually goes ahead and makes a whole third run
through then to see about how effective things like, you know, OMEN would be for cracking
passwords. So that kind of gets back to how OMEN generates the probability it's assigned with its
different levels there. And so this takes a while. So if you're cracking, you're training it on a
million passwords, you know, it's done in like a minute or two. If you're training it on a billion
passwords, it takes significantly longer. And it has to keep all this data in memory. So if you're
training on some of these really gigantic data sets there, it's just not going to work. So one
thing you might want to do is just select a subset of that password set, you know, chosen
randomly in order to train your rule set on instead. So after you're all done with that,
though, it'll display statistics about the data set you just trained upon too, which are really
kind of useful to figure out, you know, where it came from. So, you know, password lengths and stuff
like that. But one thing that I've been kind of added that I found really useful is it'll display
kind of like the top URLs, which are usually at the beginning, the top of them are like, you know,
web, you know, email, email account information. But if you start getting down a little bit,
you'll can actually see usually what the website is because people have a tendency to use the
website in their password. I also highlight the dates that it finds it in there as well,
because that's useful kind of trying to date when that password data set got leaked.
Now, I want to kind of highlight that there's a long tail when it comes to the dates.
I'm sorry. Because people, you know, create passwords before, you know, the password data
set gets stolen. So, you'll see a lot of passwords for years before the data set actually gets
down. But if you start kind of going down it a little bit, you can say, okay, that's probably
about where the cutoff was for when this password data set was, you know, disclosed.
So, kind of one last thing I really want to talk about real quick is that I am trying to get this
to work with other cracking modes there. So, one of the, you know, really popular cracking modes
used is called Prince. So, Prince basically takes a lot of different words and just combines them
all together and makes lots of guesses based upon that. But one challenge with Prince is that it's
very dependent upon the input word list that you give it to it there. Because the word list needs
to have, you know, high quality words in it. But it also needs to have a level of kind of cruft in
there too. Just because if you want to go ahead and, let's say, add the number one to the end of
a word, you have one in your word list by itself. But the challenge is, the larger your word list
is, the more words it's trying to combine. And then, you know, it starts to have issues there
as well. So, we have all this probability information about how a password was generated.
So, maybe we can go ahead and use this to create very bespoke word lists for like a Prince-style
guessing session there. So, I created another tool called Princeling that basically just does that
there. So, it goes... I'm sorry, my microphone just went out again there. But yeah, so it creates a very,
you know, high quality word list there and does it automatically. Because one thing I like
about Prince is it's the kind of the attack that I run when I want to goof off. So, like, you know,
password cracking, sometimes it takes a lot of brain cells because you're kind of looking at
how you're cracking. You're trying to optimize your cracking session. And Prince is like, I have
no idea what I want to do. I want to go watch Tiger King on Netflix. Let's just go ahead and just
launch this off and come back and see if it was successful. And Prince is usually actually quite
successful. So, it's a pretty good tool to be able to use. And by anything that you can do in order
to automate Prince even more, I'm all for, which is why I went ahead and created that.
So, I'm going to go ahead and stop the live stream here. And hopefully, I'll be on Discord there in
order to answer any questions that you have. I hope you enjoyed this. I hope this was helpful.
And once again, you know, thank you for attending the Password Village here at DEF CON Safe Mode.