It's like somebody else already has control over the stream.
Oh, there it went. Now it's still doing the thing.
And there we are.
And there we are. All right.
Welcome back. So we have another session.
This time is going to be with our friend here, Jack Baker.
Welcome, Jack.
Thank you. Glad to be here.
All right. And also we've got Fallible here as well. Hey, Fallible.
Hello.
So Jack was going to tell us a little bit,
at least he did tell us a little bit in his presentation,
about how he kind of attacked game servers.
Jack, do you want to kind of give a little bit of an introduction of yourself
and just kind of a super brief overview of your talk
so that other people might get more interested in it as well,
on top of some of the other questions that we're already seeing
in the TrackOne Live QA channel?
Yeah, for sure.
So my name is Jack Baker.
I don't have much of a background to talk about.
This is my second time speaking at DEF CON.
Both times have been about hacking video games.
And for my talk this year,
it was basically about finding bugs in game engines.
So basically looking at the core software
that you build video games on
with the hope that finding bugs in the engine
means finding bugs in a thousand different games,
basically amplifying that effort.
So I had a reasonable amount of luck
looking at Unreal Engine 4 from Epic Games
and UNet, the core networking protocol of Unity.
And I talk about four of those bugs.
And I give some POCs, typical DEF CON stuff, I guess.
Yeah, it was all pretty awesome stuff in there.
Falvo, do we have any questions coming in yet for Jack?
We have a very first question over here.
I like this one. This is from Hawkeye Dank.
Do you believe that most game servers slash developers
are heading towards a zero-trust architecture for newer games
in order to better protect the game servers?
That is a good question.
So I would actually argue that to some extent
that has already been done.
If I were to go back a bunch to when I was a teenager
trying to learn to hack video games,
I remember that there was sort of this philosophy
that the only online game you really couldn't hack
was RuneScape.
And the reason for that is RuneScape is about as close
as you'll get to just a pure thin client
in a big video game.
And so you couldn't really manipulate the client
to do things like teleport or whatever
because it was just basically just button presses.
In that case, though, people still managed to
quote-unquote hack RuneScape,
but it was in the form of logical bugs.
It wasn't trust being placed on the client.
So to answer that question,
I think that you absolutely can make a zero-trust
game engine as is,
but you're balancing performance and security.
And if you go too far on the security side,
you end up with a game like RuneScape,
which is very low, very slow,
with not a whole lot going on.
It's very difficult to apply that to something like,
I don't know, Fortnite, where you've got a thousand,
no, I guess a hundred people shooting guns.
You've got all this math going on.
It's just a different scale of a problem.
But we're getting much better at it.
Game engines have gotten significantly better
at taking that trust away.
So yeah, I think that's where things are going to keep moving,
is taking that trust away and finding clever ways
to make servers that can still handle a first-person shooter,
still handle Fortnite,
but don't have to put trust on the client to do that.
And from the perspective of game hackers,
hacking will just become more of a process
of finding logical bugs,
rather than doing what we typically do now,
which is exploiting those trust relationships.
Well, that's interesting.
So a slightly different angle of a question
coming from Blackfoot over here.
Why do you think there is so little security knowledge share
between game companies?
Example, little documentation
and hardening of UE4 dedicated servers.
That's a pretty good question.
If I had to guess, it would just be because
game engines are such big pieces of software
that when you develop your own game engine,
you're sort of going to try to hold it close to your chest.
You don't really want to give your competitors
a competing game engine the keys to the kingdom, so to speak.
So architecturally, most game engines are built
from the ground up.
There's not a lot of shared code,
and that's just because it's a big investment
and no one wants to share that.
It's sort of a boring business answer, I think,
but it's really just Unreal and Unity
and whoever else don't want to share their work with each other.
So in your presentation,
one of the things that you were showing
was kind of like the teleporting and the fast running.
I remember at one point in there,
you're just kind of showing somebody running at the normal speed
and then the exploit that you were able to find
where somebody's zipping across the screen.
In addition to running,
do you think there are other things
that people could use that kind of thing for?
Maybe be able to shoot faster, shoot more?
Maybe what other kind of things have you thought of
that somebody could use that type of exploit for in their games?
Yeah, absolutely.
So with that particular bug,
I found what I think is really interesting.
It's basically a bug where you can use the unique properties
of some special floating point values
to basically confuse the server into doing something it shouldn't.
And so I have actually tried basically that exact technique
on other games, other commands and stuff like that.
And you can usually get something unpredictable to happen.
I think I mentioned in my talk,
a lot of times it's not a good thing.
A lot of times you're just harming yourself,
but you can usually cause the game to do something it shouldn't.
And so, yeah, I think these logical bugs,
especially like mathematical bugs, if you can find those,
are a really good way to exploit games
because while the bugs aren't the most common,
they are very predictable.
They're not a memory corruption bug
where you have to worry about layout of the memory
and all those complicated things.
And they tend to, I guess,
slip by the architectural protection.
So in that case, like the bug you talked about,
Unreal Engine has a pretty robust way to prevent speed hacking.
And using just a couple quirks of,
basically, quirks of decimal numbers,
you can cause it to do something it shouldn't.
And that is super useful.
So that particular kind of bug, I'd say, yes,
definitely applies in other places.
It's just a matter of looking for it.
And I guess that's it.
I don't know how I was going to end that sentence.
No, I really liked what you showed there,
especially the things that we find in other places
across security and other types of hacking.
You go, well, okay, let's divide by zero.
Let's use not a number.
Let's use these things that are the outsides
of this language that's being used
and see what happens when you throw it at the system.
And it's just nice to see,
in the niche that you found yourself in,
a lot of the same techniques work.
Yeah, I guess one other thing I like about that bug
is that it's not something that's fixed by, say,
oh, we're going to stop writing things in C
and start writing them in Go or Rust or something.
These are bugs that still exist even in,
quote-unquote, safe languages.
And I think in more than just video games,
that's where this sort of exploitation is going.
And that's why I like that bug so much.
All right, so Hawkeye has another question for you.
He asks, have you noticed any new trends
of input validation in high-user games like Fortnite?
Yeah, I mean, I keep going back to the same example,
but Unreal Engine 4 uses a really,
really robust system for validating user input
and specifically user movement
in a way that scales really well,
but also basically eliminates your ability
to manipulate your movement in any significant way.
And so essentially the broad strokes are basically
taking authority over your character's location,
putting it on the server,
and giving the client the possibility to call functions
in order to request that you move.
And that way, the server has the context to say,
well, you're trying to move too fast.
I'm not going to allow that.
I'm going to limit movement to a specific range.
And so really, I think the entirety of evolutions
in the security of game engines recently
has just been on giving more context to the server
so we can say, well, that's not right.
I'm going to stop that from happening
and ensuring that the client doesn't have the trust
to just overrule the server.
I can't remember whether you said in your presentation
if you've talked to the engine companies
about these vulnerabilities and if they have fixed
the exploits that you found.
But if they do see somebody trying to send
like your NAN, not a number, at the server,
what types of things would you think
that they should do to that?
Should they kind of just adjust or ban?
What are your thoughts for how people could...
what should happen to them?
Yeah, so three of the four bugs I talk about are fixed.
And the speed hack one, the NAN one, was one of those.
It was actually a pretty easy fix.
They just put a tiny little check in there that says
if this is NAN, just abort, get out of there.
But in general, I would say
what they should...
I would recommend possibly that if you're a game engine developer
or you're working on Unreal and you think,
oh, this could be a problem, it's not unreasonable to just say
you shouldn't be able to use NAN in any argument.
That's not going to happen under normal circumstances.
And just essentially eliminate that from the serialization
and deserialization process.
Unreal doesn't do that because it wants to consider itself
sort of impartial to that.
Whatever goes in one end is deserialized on the other end.
But I think it would be a pretty reasonable assumption to say
if you're getting certain types of, say, floating point numbers,
this is blatantly invalid.
And we can probably just toss it out
before we even consider anything else.
Just a general input validation on what's coming in.
Typical boring stuff.
Isn't it that way?
Well, so I don't really want to give away the punchline
to your presentation because it fits so well,
but we'll just kind of hand wave that and say,
so when you've been doing this, how often do you get booted
from whatever game you're playing?
Or how many times have you had your accounts banned?
When you're building your own stuff,
you get banned less than you might expect.
I did a talk last year on hacking, basically, browser games.
And for one of the demonstrations, I took a game.
I don't want to say its name. I've done enough to them.
But I took this game and it has a global scoreboard
and I just set my score to 1, 3, 3, 3, 3, 3, 3, 3, something, 7,
and username to HiDefCon.
And they banned me pretty quickly.
I think that was pretty obvious.
But in general, if you're building your own game hacks,
there is a pretty good chance you're going to get away with it.
So I get banned pretty rarely.
So congratulations or I'm sorry.
However you feel about that.
It's a mixed bag.
Where do you also often kind of see the line here along the same,
I guess similar to what Fallible was just talking about,
where the old question about research versus going over the line,
because essentially you are affecting other users, potentially,
as well as that game engine.
So where is kind of your line on how far you'll go with your research?
Or is it because it's in the name of research, you'll just go all the way with it?
I mean, I won't do anything that's, I think,
directly affecting someone spending money or someone or anything like that.
I'm not stealing items from people in online games.
That certainly crosses the line.
But, you know, if someone's in a lobby of a first person shooter
and someone wall hacks through and, you know, no scopes them in the head,
I don't think they've been significantly harmed in any way.
So I think that is within moral bounds for me.
Excellent.
Can you explain some of the technical setup that you have
in order to capture the data that you're sending across?
And what tools are you using? What's your lab look like?
Yeah, so working with Unreal Engine especially,
I needed to actually buy a new desktop to do
because on the laptop I was working on,
it took most of the day to compile Unreal from source.
And I was doing that several times a day.
So that was not good math working out there.
So essentially, I do all of my Unreal development on Linux,
which is a little less supported, but it's just more comfortable for me.
And the other, I guess the other thing that's really a challenge
is the specifically Unreal source is so big
that trying to open it in most like project editors is, it does not work.
VS code gets by just a little bit, but even then it gets tripped up
when you try to go to this function or something like that,
it'll end up, it'll spin its wheels for five minutes and say,
yeah, couldn't find it, even though you know it exists.
So I wish I had a better setup.
I'm sure people who do actual video game development
have better experiences with building Unreal from source.
But I guess my lab is just, it's a system 76 desktop
with the highest boxes checked on everything,
just so that I can compile Unreal as many times as I want
and it doesn't take me all day.
So in that way, you're looking at the guts of the code,
you're really building it each time.
It's not a situation of you're just capturing everything in Wireshark
and modifying it somehow.
No, I wish it was a little easier.
With Unet, you can get away with just looking at the traffic in Wireshark.
Unreal is pretty difficult because everything is like bit aligned,
so you'll have a string that starts four bits in
and just looks like gibberish.
So you really do have to be looking at it at the low level
to really make any sense of it.
That makes sense.
And how does somebody get started with that kind of thing?
Because this sounds extremely technical and you have to know quite a bit.
How does somebody, where would be the entry point
if somebody was interested in this kind of research
in order to start figuring these kinds of things out,
or if they wanted to just even look at how games work?
Yeah, well, I mean, all the code is out there for Unreal Engine,
but I don't think that's something you should just jump straight into,
especially if you're not pretty familiar with C++,
because it is a complicated code base.
I think the best thing you can do to understand hacking video games
is to build them.
It's sort of a, I don't know, cliche answer,
but that is what teaches you, oh, this is how that works.
This is how I'd write it.
This is probably how it looks on the back end.
Also, I would say if you're going to look at games
written in a particular engine, Unity games are pretty good
because they're usually built, well, they're built with C Sharp
and they're typically easy to decompile back to C Sharp
with something like DN Spy, if I'm pronouncing that correctly.
However, that's verbalized.
So you can actually look at the code of the game, modify it,
see what happens, and basically hack games
without too much actual binary reverse engineering
or anything like that.
So we got a question from Jay that I'm going to obscure this a little bit.
The question is way too specific,
so let's go more along the lines of where else would be interesting to look?
If you were done with the research that you've done so far
and you were looking for a new product,
what would be the things that would draw you to a new product?
Yeah, so one thing I would say is that when you look at Unity,
there are a lot of different multiplayer solutions.
So I looked at UNet, which is sort of the... it's provided by Unity,
but there are third-party solutions that are pretty common.
Photon, Mirror, I haven't looked at those at all,
and I think that there's a pretty solid chance
that the same kind of bugs apply there.
And then I think most of the research I did,
obviously I'm just looking at the core engines for games,
but looking at specific games, you can find these exact same bugs.
They just may not apply to other games.
So I think most of this research still applies.
It just needs to be targeted either at different games
or different multiplayer protocols, even different game engines.
Game Maker, there are...
Well, I think I've looked at the two more common game engines.
There are other engines out there.
In other words, just find another big target and play,
or just go and learn something.
If you happen to be using a game, then this might be a good opportunity.
Hawkeye Dank asked another question
about the outcome of your work,
and he wants to know if you've managed
to pull any bounties for this.
Yeah, I turned down bounties for a few things
because I wanted to talk about them.
If I was really min-maxing it, I probably would have accepted bounties
for a few more, but you never know.
So yes, Unity and Epic Games
were both really good about this.
Epic has
HackerOne program.
Unity, you want to just go straight to their security
at unity3d... whatever, I don't want to give you the wrong email.
Security at email address.
But both were awesome to work with.
I would highly recommend it,
and yes, there is definitely money to be made.
And you've looked at the Unreal
and Unity engines. Are there any others that you've looked at
or found bugs with?
I have done a fair amount of reverse engineering
on GameMaker, which is one of the
more common engines, certainly less than Unity
or Unreal. Not from a security perspective,
just from a how do I hack this game
or how do I mod this game
aspect.
I guess one of the problems is that some of these
smaller engines don't provide you the same
tools. They may not have an entire
networking solution, so there's less
places for bugs to be hiding. That's why
I think Unity and Unreal were such good targets.
But I do think there is opportunity for bugs
in those other engines. I just really haven't done the work
in that context.
And have you also looked at the individual games itself?
And what's really the difference between trying to find
vulnerabilities in the game engine and a specific game?
I guess usually the difference
is whether or not you're going to have an easily accessible
source. So Unreal Engine games,
like reverse engineering an actual released Unreal Engine game
can be pretty tedious because you're not looking at the source.
You're not typically even looking at something with symbols.
You're just looking at a gigantic blob in
Ida or Ghidra. But I have
had some success there. I don't want to burn bridges, so I'm
just going to leave it at that.
But I guess the main
difference is just how much
effort goes into reverse engineering.
This sounds
like a pretty specific question, but we'll toss it out there.
So thoughts on cheat engine or memory scanners
as a toolkit from beginners? That's from
Ethics.
Cheat engine is awesome. It's how I started learning.
My talk last year was on essentially applying
a lot of the methods and techniques of cheat engine to
WebAssembly, so games in your browser, basically.
So I think cheat engine is great. It is
a problem that as
game engines get more secure,
those techniques don't work as well.
To give a general example, what cheat
engine generally does, what you're usually doing with it, is
you look for something like your health and you say, OK, I'm looking
for something in memory that's 100.
Then I take some damage and say, OK, of all those values that used
to be 100, which one is now 87
or something like that? And that helps you find
the value in memory that corresponds to your health.
And then you can manipulate that.
Give yourself more health, whatever.
But as games take trust away from the client,
that process doesn't work anymore. You can manipulate your health
client side, but the server still knows you have 87
health, not 10,000 or whatever.
So, I do think
if you're looking at the right games, that is a great way
to get into game hacking and understand those basic concepts.
But it's not going to work on
PUBG or something where someone has
put a conscious effort into making it secure
and something that's come out in the last
five years, a decade, something like that.
And how much now are the developers trying to go
with more, I don't know if it's the right term for it,
the thin client, so that people can't really
make changes in the client side or
in their mobile device phone and give myself a million lives
or a million gems or coins or anything like that
and trying to keep all of that on the server. How much of
that do you kind of see with the games
and the servers? In my experience,
it's really just whatever the engine does,
the developer will follow. Unreal
sets you up really well to
build an architecture where those game hacks
don't work or are exceptionally difficult.
And so you see developers doing a pretty good job.
Unity does not set you up quite
as well from the start, and so you'll see the majority
of those, I guess we'll call
simple attacks, do often work.
That's not to say that you can't build
a secure architecture on Unity. It just
means that Unity's not doing the work
for you, and like I said, developers
will typically follow what the engine does.
So as the core software, as
the water level rises, developers get better at it as well.
Cool. Is there anything that
you didn't get a chance to put into your presentation that
is additional research that you might have done, or are there any
kind of other areas that you think other people might be able
to add on to the research that you've done?
Yeah, so a couple things come to mind. The first
is the floating point bugs I talked about.
I think that can apply a lot.
I very briefly mentioned this, but
that attack does actually apply to Unity.
The difference is that you don't need to speed
hack Unity. You can just, or at least by
using the core components that are provided for you, you can just
tell it, I'm at x, y, z. And if you tell it
I'm at nan, nan, nan, you get
teleported to this weird dimension where everything is
infinitely small. It's kind of cool. It's not very useful.
So I think that kind of attack applies
to a lot of things, and I'd like to see
where else it goes. Something else that I feel
like I wanted to do, but I didn't quite
find all the pieces, was I found
a number of memory corruption bugs, like buffer
overflow, stuff like that.
And a lot of those seemed like
they would be good for turning something into remote code
execution, but the problem that I was always running into
is I couldn't imagine a situation
where you were looking at a game server and you knew what it
looked like on the back end. So a memory corruption exploit
requires some knowledge of
memory, of the layout of memory, where
code is, gadgets, stuff
like that. And while you may have a copy of the client, you don't have a
copy of the server. And so even if you can bypass
all those difficult protections, ASLR,
stuff like that, I just couldn't imagine
a scenario where I could turn
one of these into an RCE exploit on
an actual server. And so that's why I
felt a little disappointed in myself, because I had what
felt like good bugs, but it didn't
feel practical to turn them into
an exploit that would work in the real world.
So I would love to see someone actually pull that off.
So we are right down
to the bare end of this thing. So two questions.
One is, would you be willing to
post places that people can find you later, whether that
be an email or a website or Twitter?
We can do that in the TrackOne channel at the end of this.
And two, if you were to leave us
with a call to action, something you would like us to
consider as we're moving forward. I've been asking this of everybody.
I kind of like this question. What should we take away from
this and in our own work? And how can we
translate your stuff into ours? What would you want us to do?
Yeah, so absolutely.
I will post my GitHub. It is on the slides
as well, but I'll post it again. My email address is there too.
Hit me up. I've already had a few people.
I've had a couple of really interesting conversations, so I really
appreciate it when people reach out. And
I don't know. If I could ask for anything,
it would be that if you're
a security person, you work pen testing or whatever,
try hacking video games. It's fun.
I know you probably like video games because you're a nerd
and you're at DEF CON. So am I.
Try it. It's rewarding.
It gets your adrenaline pumping. And getting banned
is fun too. So that's my
recommendation. If you haven't ever gotten banned from a
video game, give it a shot. Awesome.
Well, thank you so much for doing this, Jack. Appreciate you
taking the time to be with us and really enjoyed your presentation.
Hopefully everybody else goes and checks that out.
So thank you so much. Yeah, thank you guys. This was fun.
All right. We will be back
in about 30 minutes with another speaker.