00:00:00.000-->00:00:07.958
>>Hi all, uh I’m Tim, or Vulc@n
more often uh d uring this
weekend of the year, anyway. And

00:00:07.958-->00:00:15.958
uh, if you are here to see the,
the CFT, CTF [laughter] uh
talk, with uh, the last 20 years of,

00:00:15.958-->00:00:26.125
of organizers, which incidenta
lly is- is all the years. So uh,
um we have from the beginning

00:00:26.125-->00:00:27.458
all the way to the ones that are
currently organizing it on, here
on, on the panel. And, um I

00:00:27.458-->00:00:31.583
believe the current organizers
are, this is the last one.
Right? So, so stepping down this

00:00:31.583-->00:00:38.125
year. And every, uh few years,
the, the current organizers step
down and there’s um some,

00:00:38.125-->00:00:43.917
process for passing the baton.
And uh one of the reasons that,
that this talk is on the on the

00:00:43.917-->00:00:48.542
docket on this year is to um,
answer questions of people who
think they might want to step up

00:00:48.542-->00:00:53.333
and take the baton and also to
have sort of an archival
reference for when this happens

00:00:53.333-->00:01:05.458
in the future. Um, so uh again
I’m Tim, uh I’m going to um, do
my best to um, moderate, uh

00:01:05.458-->00:01:11.292
these folks up here, when we get
to the panel time. But I’m also
going to uh, sort of lay a

00:01:11.292-->00:01:17.792
foundation of, uh um, sort of
the basics of what uh CTF is.
I’m not going to try and make

00:01:17.792-->00:01:23.375
too many assertions. Now, uh I
am sort of inherently biased
being one of the organizers for

00:01:23.375-->00:01:29.708
uh few years. And also being a
participant for a few years. Um
but I’ll make uh every effort to

00:01:29.708-->00:01:34.917
um, sort of remain neutral up
here and pass the questions off
to our panel of, of experts

00:01:34.917-->00:01:40.458
here. Now the talk is geared
towards um organizing CTF. So,
I’ll do a little bit of

00:01:40.458-->00:01:44.792
explaining about what CTF is,
but it’s from a organizer
perspective, right? And some of

00:01:44.792-->00:01:50.792
us, even do CTFs from time to
time, uh professionally, uh but
in most cases these slides are,

00:01:50.792-->00:02:03.792
are geared specifically to what,
uh what CTF means in the context
of Def Con. So, uh what is CTF?

00:02:03.792-->00:02:10.583
Right? So, we’re not going to
cover uh, what is CTF, in depth.
But at, at the basics it’s a

00:02:10.583-->00:02:15.667
cyber security based, uh capture
the flag contest, right? So,
it’s just like flags when you

00:02:15.667-->00:02:20.833
play with hills and kids and
stuff. Except the, the flags are
digital, uh also called an

00:02:20.833-->00:02:28.375
exercise or an event or a game.
Um, typically it’s all geared
towards the industry and some

00:02:28.375-->00:02:33.958
minimum bar of proficiency or
ideally some bar of excellence
in fields of, of cybersecurity.

00:02:33.958-->00:02:40.042
So, there’s different models
for, for organizing this. And
we’ll go over a couple then, uh

00:02:40.042-->00:02:45.000
in a bit. But to sort of stress
different things and as an
organizer you get to decide how

00:02:45.000-->00:02:51.417
you’re going to stress the, the
different uh sort of areas of
excellence. Um, CTF exercises

00:02:51.417-->00:02:56.333
are becoming um much more
common. So, these days you can
almost play about every other

00:02:56.333-->00:03:00.208
weekend. Uh, if you’re okay
playing in these remote
competitions that happen all

00:03:00.208-->00:03:06.583
over the world. So, there’s uh,
uh circuit so to speak. And um,
there’s also uh, uh multitude of

00:03:06.583-->00:03:12.000
contests that sort of just tack
on CTF at the end where it’s
very domain specific. Like the,

00:03:12.000-->00:03:18.583
um social engineering CTF or uh,
um ICS CTF or something like
that. And some of those bear

00:03:18.583-->00:03:23.792
more resemblance to the game
we’re going to talk about today
than others. Um but uh, in some

00:03:23.792-->00:03:27.750
cases this really just means
contest, in other cases there
really are flags that uh can be

00:03:27.750-->00:03:33.917
captured. So, going all the way
back to the basics for capturing
the flag we need to talk about

00:03:33.917-->00:03:38.292
like what the flag is, right?
What’s the thing that needs to
be captured? But from an

00:03:38.292-->00:03:43.375
organizer's perspective that
actually has uh, uh there’s,
there’s a lot to sort of suss

00:03:43.375-->00:03:48.625
out there. Um, is it just random
text? Well if you just use
random text, then from a

00:03:48.625-->00:03:53.417
participant perspective it’s
difficult to understand whether,
uh you’ve actually found the

00:03:53.417-->00:03:58.333
flag or whether it’s just uh,
random text right? That’s uh,
encrypted or compressed or

00:03:58.333-->00:04:03.125
something like that. It’s
somewhat difficult to know when
you’re done and when you’ve

00:04:03.125-->00:04:08.333
found the flag. Um, from the, uh
uh, also from an organizer
perspective it might result in a

00:04:08.333-->00:04:13.792
lot of guessing. Because the
participants end up with this
sort of um denial of service

00:04:13.792-->00:04:17.958
sort of situation where people
are just guessing repeatedly
hoping that they’ve found the

00:04:17.958-->00:04:21.708
flag but they’re not really
sure. Um, so then the, the next
thing you do is you try and add

00:04:21.708-->00:04:25.500
some structure. Ya know, it’s a
certain size, it’s a certain
format, it has a particular

00:04:25.500-->00:04:31.208
prefix. Um, all of these things
sort of limit that space of
brute forcing right so now you

00:04:31.208-->00:04:35.042
can increase the likelihood of
guessing correctly for people
who do want to guess. Um, you

00:04:35.042-->00:04:40.542
also open the door to all kinds
of, of difference defenses.
Like, consider writing a, a IDS

00:04:40.542-->00:04:44.875
signature for uh, for the prefix
that you know is going to be
there because the flag has a

00:04:44.875-->00:04:50.083
specification. Uh, so there’s
these, uh trade-offs that happen
when you, even define something

00:04:50.083-->00:05:04.000
as basic as, what is a flag. Um,
so, uh even beyond defining sort
of the specification for what a

00:05:04.000-->00:05:09.417
flag is, there’s uh, um, uh, a
lot of other mechanics that
happen around the flag. So,

00:05:09.417-->00:05:13.042
flags are stolen and you sort of
have to prove that you’ve,
you’ve actually stolen it right?

00:05:13.042-->00:05:16.500
So you have to go back and
assert to the organizers that
you’ve successfully stolen it

00:05:16.500-->00:05:23.625
and over the years that has
really matured uh over time from
um, uh verbal or writing on

00:05:23.625-->00:05:28.375
paper, to email, to uh now you
sort of have rich-ish web APIs.
Right? Solid services where

00:05:28.375-->00:05:34.667
there’s structure and, and
protocols around how to, uh sort
of automatically or in automated

00:05:34.667-->00:05:43.417
fashion submit flags to uh web
APIs or scoreboard servers. Um,
and then there’s all of these

00:05:43.417-->00:05:47.958
sort of game decisions that have
to be made. Some of them made by
the participants where, flags

00:05:47.958-->00:05:54.625
might be shared. Um, uh you have
collusion between teams for pot-
uh potential collusion. Um, do

00:05:54.625-->00:06:00.083
flags have a preset, predefined
constant value or do, does it
change over the time? Maybe

00:06:00.083-->00:06:04.917
being diluted as more people
have stolen flags from easier
services. Um, do flags expire?

00:06:04.917-->00:06:09.042
If you steal one at the
beginning of a 48-hour period is
it worth the same amount at the

00:06:09.042-->00:06:13.958
end of the 48 period. Is it
worth anything? Um all of these
things sort of go into the

00:06:13.958-->00:06:19.667
mechanics of how that particular
game is going to operate. And uh
adjust the game strategies that

00:06:19.667-->00:06:28.958
the participants are going to
adopt. So then what is Def Con
CTF specifically? Well there’s

00:06:28.958-->00:06:32.875
no shortage of quotes. These are
just the top hits from when you
do a Google search but um, the

00:06:32.875-->00:06:37.708
assertion that I think most of
the people up on this panel
would say is that Def Con is one

00:06:37.708-->00:06:43.792
of the, the highest regarded
CTFs out there. Right? And uh,
it’s kind of grown over the

00:06:43.792-->00:06:51.042
years from being one of the, the
oldest and uh sort of one of the
only to be the, the, the world

00:06:51.042-->00:06:56.792
series or the best of the best.
Right? Um so typically the
phrase Def Con CTF, if you hear

00:06:56.792-->00:07:01.333
somebody saying it they’re
referring to the on-premise
version of the game which starts

00:07:01.333-->00:07:08.167
tomorrow morning, right? At 9am
and goes til Sunday, goes
throughout the weekend. Um, in

00:07:08.167-->00:07:17.208
the beginning there were the
Goons. Or more specifically
there was Myles. Ok. And um, so

00:07:17.208-->00:07:22.167
Def Con uh CTF was one of the
oldest and longest running. Uh
there are some others that have

00:07:22.167-->00:07:29.292
been around for uh, for a while.
Like the UCSB’s ICTF comes to
mind. Um, but even ICTF is, is

00:07:29.292-->00:07:35.292
fairly restricted until just
this year was sort of academic
only. Right? And they opened up

00:07:35.292-->00:07:41.958
for you guys, right? So as a, as
a pre qualifier they, they
opened it up so it’s not

00:07:41.958-->00:07:46.000
academic affiliated. But, but
Def Cons got a long history of
being very approachable and very

00:07:46.000-->00:07:51.458
open and um, it’s one of the
longest running CTFs. It’s also
one of the oldest contests

00:07:51.458-->00:07:58.708
that’s still running uh at Def
Con. Um, so back at Def Con 4,
uh was the first one. It was

00:07:58.708-->00:08:03.542
just called capture the flag or
network capture the flag I think
a little bit later. And um,

00:08:03.542-->00:08:16.708
certainly grown uh, over time.
So while this Dec Con 25, it’s
CTF uh 22. Uh, uh oh. There we

00:08:16.708-->00:08:22.792
go. Um so if you look at uh, uh
quick a timeline, again in the
beginning there were Goons. Um,

00:08:22.792-->00:08:28.542
sort of later on, uh teams more
formally organized and sort of
had a multi-year approach to

00:08:28.542-->00:08:39.333
organizing. We may switch to
different computer yet. Around
the year 2000 it was clear that

00:08:39.333-->00:08:45.208
organizing needed to be a team
sport. Let’s go back one and
see, less, less epilepsy.

00:08:45.208-->00:08:55.208
[laughter] Um, now, now CTF, now
with less, less epilepsy. Um,
sort of around uh 2000 it was

00:08:55.208-->00:09:01.042
clear that organizing needed to
be a team sport, right. Uh there
were other hacking competitions,

00:09:01.042-->00:09:06.833
um HOPE for example had
something. But there was nothing
that really had, uh sort of the

00:09:06.833-->00:09:17.333
network component, the defensive
component. Yeah, you got one
more, see if this will last, ya

00:09:17.333-->00:09:26.125
know. Alright so in the way of a
road map, which might stay there
for a few seconds, nope. Um,

00:09:26.125-->00:09:38.292
let’s go back and forth here.
Uh, quick, look at it, uh so in
the way of a road map there’s

00:09:38.292-->00:09:43.708
sort of different milestones
that happened over the years. Uh
in the beginning. >>Hey Tim, can

00:09:43.708-->00:09:48.458
I interrupt you for a second?
>>I’m sorry >>Just me, just me,
Uh I just wanted to say, he’s

00:09:48.458-->00:09:53.042
had 2 problems here that are
totally classic what happens
every year when you come with

00:09:53.042-->00:09:58.083
new technology. And that happens
every year with organizing these
contests. We don’t’ know

00:09:58.083-->00:10:02.917
necessarily who’s going to be in
the audience. What words to use
to say to talk to make things

00:10:02.917-->00:10:08.500
work right and we definitely
don’t know what cables to bring.
[laughter] >>Um but they do

00:10:08.500-->00:10:14.500
provide testing harnesses in the
speaker room with do work fine.
It’s fixed. You had to go kick

00:10:14.500-->00:10:23.958
the cable apparently. It’s
unfixed [laughter] [off mic
discussion]. There’s so much in

00:10:23.958-->00:10:29.958
between here and there [off mic
discussion]. Yeah, we’ll just,
we’ll just skip over it. The uh,

00:10:29.958-->00:10:37.125
there’s sort of different
milestones that happen
throughout the years right.

00:10:37.125-->00:10:41.458
There’s, there’s the realization
that teams are going to be
competing, right even though it

00:10:41.458-->00:10:46.208
wasn’t necessarily organized as,
as a team competition. There’s
the realization that teams are

00:10:46.208-->00:10:50.500
required for organizing. Right.
It’s too much for 1 individual
or a couple of individuals to

00:10:50.500-->00:10:55.625
handle. Um, fast forward to
there’s so many people that want
to play that there needs to be

00:10:55.625-->00:10:59.542
some sort of qualifying round in
order to gate the people that
can actually make it to what’s

00:10:59.542-->00:11:03.208
now known as the finals of the
Def Con CTF that’s on premise.
And then, uh the sort of

00:11:03.208-->00:11:08.333
different things that happen
sort of along the way. Like the
inclusion of the game outside

00:11:08.333-->00:11:16.875
the game, the the meta game and
the moving to IVP6 and badges
that have uh game code on the

00:11:16.875-->00:11:21.042
badges. Uh, and then last year
was actually the first year and
perhaps the only year that the,

00:11:21.042-->00:11:26.792
the architecture and the format
of the game were announced like
long ahead of time so you could

00:11:26.792-->00:11:32.042
actually spend time
developing tools as a team and
come in with tools that you knew were going to

00:11:32.042-->00:11:36.250
work. Right. >> We actually, we
actually announced last year
that we were having uh, a custom

00:11:36.250-->00:11:39.917
architecture. It’s like a 9-bit
middle ending thing,
monstrosity. >>Middle ending is

00:11:39.917-->00:11:43.625
not actually a thing by the way.
[laughter] >> No, No. [laughter]
>>Oh, it totally is now. It is

00:11:43.625-->00:11:48.708
now. >>Yeah it is now. >> Um, so
yeah, now we actually have a
completely custom architecture

00:11:48.708-->00:11:52.250
that the teams just got this
morning, they just got the
manuals this morning. >>So, so

00:11:52.250-->00:12:00.583
preparation before 9am is
perhaps of limited futility,
right? Um, so getting back to,

00:12:00.583-->00:12:04.958
to what is capture the flag if
we kind of grossly break them
down into 2 categories. This is

00:12:04.958-->00:12:08.792
again been covered in other,
other presentations so we won’t
spend too much time on it but

00:12:08.792-->00:12:14.583
the 2 broad categories are, are
attack defend and uh game board
or Jeopardy style. And the big

00:12:14.583-->00:12:18.458
thing in attack defend is that
the participants are directly
connected to each other. So

00:12:18.458-->00:12:23.625
defense actually becomes a big
component of the game. Uh also
service level availability or

00:12:23.625-->00:12:30.167
SOA uh becomes part of the game,
the, the part that need to be
vulnerable and able to be

00:12:30.167-->00:12:36.000
attacked have to be there and
running. Uh so everybody is
typically level-set. Uh think of

00:12:36.000-->00:12:39.458
a VM that has custom software
and is distributed to all the
teams. There’s different ways to

00:12:39.458-->00:12:43.875
do that but that’s a good
analogy and ultimately it's
composed of a set of challenges

00:12:43.875-->00:12:47.542
and that’s another thing that’s
matured over time. The concept
of challenges or services and

00:12:47.542-->00:12:54.792
these are sets of custom
software that uh typically run
with some amount of concurrency.

00:12:54.792-->00:13:00.417
Um there’s debate on how many
challenges, uh concurrently make
a good CTF or not. So, uh

00:13:00.417-->00:13:05.250
conversely the Jeopardy style is
much like the show, right you’ve
seen the show. It’s a grid of

00:13:05.250-->00:13:11.250
questions, um participants are
not connected to each other so
defense is typically um not part

00:13:11.250-->00:13:15.708
of these games and you solve a
series of challenges where the
order might be determined, there

00:13:15.708-->00:13:20.208
might be say control by the
leader by the organizer, and the
categories might be completely

00:13:20.208-->00:13:24.125
arbitrary or they might be
designed with uh certain ya know
learning objectives or goals,

00:13:24.125-->00:13:29.500
such as gating for, for CTF,
demonstrating proficiency in, in
subjects that, that would uh

00:13:29.500-->00:13:33.292
work well when you get here for
the weekend. There’s also
hybrids and things that don’t

00:13:33.292-->00:13:39.958
quite fit but broadly these are
the 2. Uh, so Jeopardy style
world-wide or just

00:13:39.958-->00:13:46.333
proportionally with the number
of CTFs that happen is by far
the most common. Um it's uh

00:13:46.333-->00:13:51.708
possibly more diverse within a
single event because defining
what a question is, uh is very

00:13:51.708-->00:13:56.833
broad. Uh whereas you can find
what a software service is in an
attack defend game is perhaps

00:13:56.833-->00:14:04.083
relatively uh confining. Um,
it’s arguably easier to, to
organize for several reasons.

00:14:04.083-->00:14:10.792
So, so today Def Con, when you
say Def Con CTF it means a thing
hear, uh there’s typically a

00:14:10.792-->00:14:17.208
Jeopardy style one that gates
everybody as a qualifying round.
Um so this was introduced in uh

00:14:17.208-->00:14:24.292
2004, Caezar, you did
qualifying, right? >> Uh, turns
out yeah, we were just debating

00:14:24.292-->00:14:28.000
that, discussing that. >>No,
Caezar was like we didn’t do
quals. I was like, I remember

00:14:28.000-->00:14:32.542
playing in your quals so I’m
pretty sure you did. >>Yeah so
the last, so on the, on the

00:14:32.542-->00:14:35.708
chart it should show but you
can’t see it because it causes
epilepsy. Um, but I think the

00:14:35.708-->00:14:39.958
last year that data hackers-
>>Yeah that was all DD back in
the day. Like that was

00:14:39.958-->00:14:47.833
literally >>Although Myles
brought us a long, DD
implemented so much stuff it,

00:14:47.833-->00:14:53.625
you really would have to
understand the guy that, how
amazing he could pump out a user

00:14:53.625-->00:14:58.667
mobile and X router system with
double back. >>Right
>>Firewalling and stuff, like in

00:14:58.667-->00:15:02.792
a night, in a night, in an
evening. >>They, they also had
some pretty crazy constraints in

00:15:02.792-->00:15:07.542
their quals too like, I remember
specifically playing and, and
not being able to actually make

00:15:07.542-->00:15:11.625
any kind of connection into that
box or out of that box, other
than the literally just terminal

00:15:11.625-->00:15:16.542
IM. And so we had to write like
GU encoder and decoder routines
into like telnet so that we

00:15:16.542-->00:15:21.458
could move binaries to the box.
>>There, there first qual was
just a race to finish 8, 8

00:15:21.458-->00:15:24.458
levels of the challenge, that
was it, 8 questions. >>Yeah it
was totally linear. Yep, yeah

00:15:24.458-->00:15:29.875
exactly. >>So, we’re having a
sad realization up here that
anything with pictures appears

00:15:29.875-->00:15:35.375
to anger the projection system.
Uh which is unfortunate because
there’s a lot of pictures in

00:15:35.375-->00:15:39.958
most of this, so. [laughter] Um,
so you’ve seen the board. This
is what the board looked like

00:15:39.958-->00:15:47.375
when DDTEK [inaudible] had quals
in 2006 anyway. And uh there’s a
starter question, there’s a hand

00:15:47.375-->00:15:51.792
that kind of points off
somewhere, gotta download a file
or go to some web service. And

00:15:51.792-->00:16:02.333
uh, maybe I should try and
switch computers here. Uh but
unlike Jeopardy uh the leader

00:16:02.333-->00:16:07.833
controls the next question but
unlike Jeopardy, uh other uh
teams can sort of catch up and

00:16:07.833-->00:16:15.417
answer the, uh the previous
questions. I might just try and
switch briefly here. >>We could

00:16:15.417-->00:16:19.792
probably fill the space. >>Yeah
go for it. >>We’re gonna talk
about stuff while he works.

00:16:19.792-->00:16:24.417
That’s usually how organizing
CTF is. >>I was going to say
yeah it’s pretty, it’s pretty

00:16:24.417-->00:16:27.250
AD-HOC to begin with. Basically
something, something goes awry
every time. This, this is on-

00:16:27.250-->00:16:29.958
this is its own sort of like
micro example of how CTF
actually works in real life.

00:16:29.958-->00:16:33.958
Because it turns out nothing
goes to plan. It’s the old like,
you know planning is useful or

00:16:33.958-->00:16:38.250
plans are useful of some sort of
thing. >>And pulling out that
Tim was a part of DDTEK. And

00:16:38.250-->00:16:43.375
DDTEK CTF’s were famously
broken. [laughter] >>Oh yeah.
There was one year I think you

00:16:43.375-->00:16:46.458
completely erase the scores from
one day. And we were, we were
upset with this. >>Yeah. >>Yeah,

00:16:46.458-->00:16:54.583
and I actually, uh, playing CTF
uh most of my CTF experience was
under DDTEK. My 2 black badges

00:16:54.583-->00:17:00.167
was from when in their first and
final year. Um but a funny story
we actually were, playing quals

00:17:00.167-->00:17:05.792
or organizing quals, uh I once
wrote a, a mock web server with
a CGI script underneath it. And,

00:17:05.792-->00:17:10.958
uh there’s, the bug was in the
CGI script but all the, the web
script kitties were like, yay a

00:17:10.958-->00:17:15.167
web challenge. [laughter]. And
then uh, so they started
uploading like PHP shells and

00:17:15.167-->00:17:19.958
python shells and one team in
particular, uploaded a python
shell connect back with a

00:17:19.958-->00:17:25.042
hard-coded IP address and port.
So like, kind of, let me play
with this. So, I killed it since

00:17:25.042-->00:17:29.625
it wasn’t actually part of the
game. Or necessarily part of the
game and they did it again.

00:17:29.625-->00:17:34.917
Alright, this is what’s going to
happen. Killed it, just
immediately started kicking back

00:17:34.917-->00:17:39.250
to them and then uh, finally it
kicked back and they started
typing commands. And me trying

00:17:39.250-->00:17:44.792
to L S uh slash. And then like,
you know, cat something, uh let
me type some other stuff.

00:17:44.792-->00:17:49.542
Eventually realized they were
talking to somebody buy uh,
[laughter] but uh yeah

00:17:49.542-->00:17:52.625
[inaudible] so much fun >>Yeah
it took an amazing amount of
self-control to not mess with

00:17:52.625-->00:17:55.542
everybody else’s exploits
essentially like, people would
land stuff on boxes and like

00:17:55.542-->00:17:59.417
watching it happen you could
like connect to it faster than
they could. And so like not

00:17:59.417-->00:18:03.500
going in and grabbing everybody
else’s shells when you’re the
organizer just for fun was

00:18:03.500-->00:18:06.958
actually like a massive, massive
self-control. >>The reason I did
though was because it wasn’t

00:18:06.958-->00:18:10.042
necessary to, to land the
challenge. >>Right >>So, that
was actually the year, so I

00:18:10.042-->00:18:14.375
don’t know if you guys know
Geohab he popped the servers by
figuring out that, dot bashRC

00:18:14.375-->00:18:23.083
was writable by general users.
So, he wrote cat, uh flag in
the- slash temp slash something.

00:18:23.083-->00:18:27.458
So he just waited until somebody
else exploited it cause they
were just going to run bash

00:18:27.458-->00:18:31.542
because it wouldn’t run bash so,
whatever and so he just waited
and just saw a that file pop up

00:18:31.542-->00:18:35.208
in temp and got the flag. It’s
the way to do it. >>We actually,
we actually had a whole, we had

00:18:35.208-->00:18:39.375
a whole um, we had a whole like
sub like focus area that we,
that we called the dirty tricks

00:18:39.375-->00:18:43.417
department. >>Yeah. >>Which was
basically like trying to figure
out all the, all the herent

00:18:43.417-->00:18:46.875
conditions in everybody else’s
exploitation methodologies. So,
like I remember one year there

00:18:46.875-->00:18:51.208
was a team that was doing really
really well but they were always
like, their payload would

00:18:51.208-->00:18:56.792
actually cat the flag in a file
in temp and so we could just
like walk around behind them and

00:18:56.792-->00:19:03.583
mop up all the flags that they
were getting too. >>Nice >>One
thing that I found, though is

00:19:03.583-->00:19:06.750
uh, nevermind are you ready?
>>I’m ready >>Alright >>Want to
finish your thought? >>No, no go

00:19:06.750-->00:19:11.458
ahead. >>Alright, too much of a
thought. >>Yeah, you’re in the
middle of your thing, go ahead.

00:19:11.458-->00:19:16.500
>>Um, I will say that I walked
into the, the speaker room
earlier today with my uh, laptop

00:19:16.500-->00:19:20.833
that’s running Linux and plugged
it in and everything worked and
then I didn’t tell anybody I was

00:19:20.833-->00:19:24.792
like, oh linux and it finally
works, it understands how to do
it. >>It’s the year of Linux on

00:19:24.792-->00:19:28.792
the desktop. >>So I went in
there later and I tried it again
[laughter] and it worked again

00:19:28.792-->00:19:32.583
and I got up here. I’m like, oh
no, and now I have a mac plugged
into the projector and it’s

00:19:32.583-->00:19:38.125
working fine. Um, so I guess
it’s still not. >>Go back and
show your timeline, I thought

00:19:38.125-->00:19:41.500
that was uh, that was a good,
you should flip back to your
timeline. >>Yeah agreed. >>Oh

00:19:41.500-->00:19:47.917
yeah. >>Nobody got to see that.
>>Yeah, I’m a few slides down
now. Um, great so now we have

00:19:47.917-->00:19:58.833
timelines with no epilepsy. Um,
so uh I- I, kind of went over a
few of the um the parts and this

00:19:58.833-->00:20:02.458
is by no means exhaustive it's
just sort of like the things
that stood out in, in, in my

00:20:02.458-->00:20:06.083
memory and sort of uh worked
well for the slide desk right.
Um, but uh, it definitely has a

00:20:06.083-->00:20:11.542
lot of the, the pieces that
matured over time with the, the
qualifier introduced in 2004 and

00:20:11.542-->00:20:19.417
then the tradition that
continues now with returning
the, the return in champion. So

00:20:19.417-->00:20:23.542
uh, so Kenshoto would be
inviting the returning champion
back for the first time in 2005

00:20:23.542-->00:20:30.000
and there, there’s sort of these
uh, I don’t know, um dirty,
dirty tricks or, or twists or

00:20:30.000-->00:20:35.667
things that sort of make it hard
to um, uh come in prepared
right. So, like one of the

00:20:35.667-->00:20:39.625
things, this year makes it hard
to come in prepared is that it's
en- entirely custom architecture

00:20:39.625-->00:20:44.083
that wasn’t announces until 9am
this morning right so that’s
like a way that preparation is,

00:20:44.083-->00:20:51.750
is difficult. Um, some other
examples we’ve got going back in
2006- to submit keys over DTMF.

00:20:51.750-->00:20:55.500
>>We, we were just talking too.
I remember, I remember basically
everybody, so we had, we had

00:20:55.500-->00:21:00.583
deployed like a, an actual like
I think it was an asterisk
>>Asterisks >>Yeah, and so

00:21:00.583-->00:21:04.833
basically all the tokens, all
the keys were just ya know,
really big entirely numeric

00:21:04.833-->00:21:09.167
digits and so we were making
everyone like, they would pick
up a phone and it was like press

00:21:09.167-->00:21:14.042
1 to submit a key, press 2 to
talk to a member of Kenshoto.
And so all the teams had to like

00:21:14.042-->00:21:19.458
run out into Vegas trying to
like buy modems and like had to
like write their own DTMF like

00:21:19.458-->00:21:23.833
scripts to submit tokens.
Because otherwise the first few
tokens that were being submitted

00:21:23.833-->00:21:27.958
by people, they were first of
all, they were like 32 digit
numbers, like they were big. And

00:21:27.958-->00:21:32.333
everybody’s like [beep beep
beep] [laughter] [beep beep]
s**t I f****d it up [beep beep

00:21:32.333-->00:21:38.667
beep] [laughter] >>Specifically
Hangout >>Yeah exactly. And then
yeah, and you couldn’t submit

00:21:38.667-->00:21:42.458
more than one token per call so
you had to actually, hang up the
phone and do it again.

00:21:42.458-->00:21:48.875
[laughter] >>We, we had so many
tokens we exceeded the available
bandwidth to submit our tokens

00:21:48.875-->00:21:52.417
we couldn’t submit them fast
enough. >>Yeah so everybody ran
out, everybody ran out to Fry’s

00:21:52.417-->00:21:55.542
to try and buy, to try and find
modems and like. >>Which also
has ranging ramific- >>And

00:21:55.542-->00:21:59.875
people are like how, how do I
connect something over serial to
my like modern laptop. >>Which

00:21:59.875-->00:22:03.750
also has like interesting
ramifications because then next
year like teams would buy all

00:22:03.750-->00:22:12.792
the modems like just to >>Yeah
and we didn’t use modems next
year. [laughter] >>Good, they

00:22:12.792-->00:22:15.000
have good return policies.
[laughter} >>They do. >>Uh, but
there’s, there’s other things

00:22:15.000-->00:22:19.042
that sort of, uh go through the
times. Uh, multiple hosts, uh
IPv6, a lot of people were um,

00:22:19.042-->00:22:25.583
sad face when it was all IPv6
and they had a lot of IPv4 tools
and shell code and so forth. Um

00:22:25.583-->00:22:31.417
so, cust-, so MSP430
architecture on the badges, sort
of midway through, uh legitbs

00:22:31.417-->00:22:35.333
and, and, and again this year
with the custom architecture.
So, sort of the, some of the, um

00:22:35.333-->00:22:41.083
milestones by no means
exhaustive, um but um as you
already said, the one that I

00:22:41.083-->00:22:44.750
want to focus on now is the, the
qualifying round. Because that
gets into the, they style of

00:22:44.750-->00:22:48.208
game that’s game board style.
>>Actually, before you jump off
that slide can I, can I like

00:22:48.208-->00:22:52.208
mention one other thing? >>Yeah
>>I think there’s an interesting
pattern that’s shown uh as a

00:22:52.208-->00:22:56.417
result of this slide. Which is
the like burnout cycle of
hosting CTF. If you notice,

00:22:56.417-->00:23:00.958
almost all of those are the
same-ish like maybe off by 1
year here or there or whatever.

00:23:00.958-->00:23:06.250
Because the, just herculean
level of effort that goes into
creating CTF in general

00:23:06.250-->00:23:10.917
literally causes like internal
drama and burns out teams over
time. >>It’s, it’s a second job.

00:23:10.917-->00:23:15.875
>>It’s, it’s literally, so like
our as an example during, during
Kenshoto CTF we had full time

00:23:15.875-->00:23:20.375
jobs and actual really life
stuff to do besides you know CTF
and then had an 8-month

00:23:20.375-->00:23:25.750
development cycle every year
for, for CTF. And by the end of
it we were just like nope, not

00:23:25.750-->00:23:28.792
doing this again. >>How many
hours a week did you work? >>Oh
my God it was literally,

00:23:28.792-->00:23:32.750
literally a second full time job
for almost the entire team.
>>Wow, we had DD so we went

00:23:32.750-->00:23:37.917
[laughter] 4 hours a week. >>I
know, right? >>And he worked.
>>And you guys were like, DD go

00:23:37.917-->00:23:40.792
do a bunch of stuff. [laughter]
>>What’d you do this week? >>So
um. >>What’s our contest like?

00:23:40.792-->00:23:47.833
>>Uh Lightening who wrote the uh
architecture for this started in
2015. >>Yeah >>So >>Um, to be

00:23:47.833-->00:23:54.083
fair it took Intel a lot longer.
[laughter] Um, so if you kind of
uh, uh fast forward, the the

00:23:54.083-->00:23:58.458
previous slide showed a Kenshoto
board form ’06. Here’s another
one from ’08. I think maybe even

00:23:58.458-->00:24:04.167
the same code or the same style
was used in 2007, it looks very
similar anyway. Um, but you can

00:24:04.167-->00:24:11.750
see that the, the board is, is
sort of uh, um staying the same,
static, as Kenshoto evolved. Um

00:24:11.750-->00:24:16.875
you, you do get to the fine,
sort of the areas of excellence
in the categories, um there,

00:24:16.875-->00:24:20.833
there’s uh, uh an aspect of the
organizer side of how
approachable do you want the,

00:24:20.833-->00:24:25.083
the, the open qualifier. Anybody
can sign up right. So how
approachable do you want it to

00:24:25.083-->00:24:28.958
be for novice players? But
typically, that’s solved with
uh, sort of a trivia category

00:24:28.958-->00:24:34.667
and uh, and also the, the lower
point value ones, uh where
they’re, they’re very

00:24:34.667-->00:24:38.625
approachable to uh.
>>Approachable enough that, that
for all the years we hosted it

00:24:38.625-->00:24:43.958
the trivia 100 had the same
answer. [laughter]. It was, well
it was >>Hack the planet!

00:24:43.958-->00:24:47.833
>>There ya go! [laughter] It was
literally hack the blank. >>So
uh >>Blank the planet. >>The

00:24:47.833-->00:24:51.792
last  >>Hack blank planet >>And,
and our last version was blank
blank blank blank [laughter]

00:24:51.792-->00:24:56.500
>>Yeah. Yeah. They submitted a
version blank blank blank and
people just submitted hack the

00:24:56.500-->00:25:00.875
planet. >>We actually have a, we
have a baby’s first category
now. >>Yep >>So it’s all like

00:25:00.875-->00:25:04.625
baby’s first heap exploit.
>>Yep, yep. >>So through the
years, there’s uh, the next uh

00:25:04.625-->00:25:10.417
set or organizers. DDTEK, um
obviously different uh look and
feel but, but very similar.

00:25:10.417-->00:25:15.667
>>Really? >>Very [laughter] >>Uh
similar semantics. It’s still
Jeopardy, it’s still categories.

00:25:15.667-->00:25:20.125
Um, you’re still choosing the
area of excellence so to speak.
So it, its, when you go around

00:25:20.125-->00:25:24.458
the other clocks um, it’s it’s
sort of all the same things
right. Crypto, cross site

00:25:24.458-->00:25:28.458
scripting, buffer overflow, heap
overflows, right. The, the, the
boxes open into to any of the

00:25:28.458-->00:25:37.208
security related fields. Um, and
then the, the current organizer
right, um have a, a slightly

00:25:37.208-->00:25:45.167
more Japanese game show right.
This is one of their, their
earlier ones from uh 2013. Um,

00:25:45.167-->00:25:48.958
but the concept of qualifying
the teams through this gate has,
has persisted since it started.

00:25:48.958-->00:25:53.708
And it really uh filters down to
the, the handful of teams that
are gonna be present here >>Do

00:25:53.708-->00:25:57.625
you guys know >>Uh at Def Con
>>Do you guys know anybody,
anybody who’s been an organizer

00:25:57.625-->00:26:00.750
and stuff, do you guys know like
off-hand the total number of
like, I actually clicked, I am

00:26:00.750-->00:26:03.833
beginning to play. >>Yeah
>>Quals? Cause those numbers
>>If you guys will >>Are

00:26:03.833-->00:26:07.750
impressive >>Stop interrupting
we’ll get them. >>Oh, okay. >>Um
I, I should have said at the

00:26:07.750-->00:26:11.083
beginning that, that I made
these slides and they haven’t
actually see them, so there

00:26:11.083-->00:26:14.958
might some >>True story
>>Disagreement and, but I’m, I’m
trying to be, uh to be honest

00:26:14.958-->00:26:19.625
and, and faithful, I’ve uh, I’ve
kept everybody in the dark.
[laughter] So the basic

00:26:19.625-->00:26:25.333
organization of the game is the
game, um has remained the same
and um. You know other CTFs

00:26:25.333-->00:26:29.417
around the world sort of take a
similar approach so they modify
like how you get through the

00:26:29.417-->00:26:32.542
board and so forth. So, one
thing you might’ve noticed, the
uh, the astute observer, through

00:26:32.542-->00:26:36.875
the boards that we’ve looked at
some of the same teams keep
showing up. Like everybody up

00:26:36.875-->00:26:40.833
here and anybody out there
that’s a team know that. But you
see the, the sort of regulars

00:26:40.833-->00:26:45.333
the familiar faces that come
back. And um, teams have really
started persisting over time.

00:26:45.333-->00:26:50.000
Uh, um back even when uh the
Ghetto Hackers, were, were
playing and then, and then uh

00:26:50.000-->00:26:56.750
organizing. And uh these are
teams that, that sort of prepare
and, and practice and test tools

00:26:56.750-->00:27:00.500
and, and things year round. So
it’s, >>Except for us >>It’s on
a cycle >>Not the ghetto.

00:27:00.500-->00:27:06.042
>>What’s that? >>The ghetto did
not. [laughter] Sort of I mean a
few hours a week we, we, we

00:27:06.042-->00:27:16.542
drank a lot. >>You had DD. >>I
said that teams existed Um, so
they develop and maintain tools

00:27:16.542-->00:27:20.458
and teach processes and so forth
and you can sort of track them.
These, these are just points in

00:27:20.458-->00:27:23.500
time so it doesn’t say anything
about their, their final placing
in quals. But you see these same

00:27:23.500-->00:27:30.500
teams kind of over and over both
in quals and also in finals. Um,
so so much that, that it’s

00:27:30.500-->00:27:35.958
actually tracked somewhat
formally now. So, this is uh, a
website called CTF Time. There’s

00:27:35.958-->00:27:39.958
other ones but this one is sort
of emerged as the, the
predominant one and it’s, it’s

00:27:39.958-->00:27:45.042
sort of mostly opt-in. So, it’s
you, you might not be as tracked
as you want to be, but um when

00:27:45.042-->00:27:49.917
the results are posted publicly
the teams are known and, and
based on, ya know pattern

00:27:49.917-->00:27:53.792
matching on the IDs they’re sort
of matched together. But um
individuals can sort of

00:27:53.792-->00:27:58.083
affiliate with the team and, and
opt in. But uh, um but these are
tracked over time right so these

00:27:58.083-->00:28:04.750
teams persisted since uh, um for
some it’s again a second job
kind of situation. Uh there,

00:28:04.750-->00:28:08.875
there’s formulas, there’s
there’s APIs you could use to
extract the information and

00:28:08.875-->00:28:13.250
verify that you’re, your
rankings are correct. Um, even
so, still subjective right.

00:28:13.250-->00:28:17.917
Somebody’s still saying, oh that
CTF was harder than this one by
some metric right. And the, the

00:28:17.917-->00:28:22.500
formulas still had to be plugged
in. Um there’s still interesting
opportunities for cooperation

00:28:22.500-->00:28:29.250
and and collusion and what
happens when you switch teams
and all that sort of thing. Um,

00:28:29.250-->00:28:35.708
so how many will participate?
Right. >>Woohoo! >>So we’ll go
back to the uh the, the

00:28:35.708-->00:28:40.292
timeline. And uh, you can see
the, in the in the bar graph, I
don’t know if you can read the

00:28:40.292-->00:28:44.667
numbers very well but the, the
largest bar is 414 and the
smallest bar is 162. I didn’t

00:28:44.667-->00:28:48.042
have numbers from the, the early
Kenshoto days. Uh, I don’t know
how that worked. >>Yeah I don’t

00:28:48.042-->00:28:51.125
think we do either. [laughter]
>>Uh hey I couldn’t, I didn’t
have them and then I couldn’t

00:28:51.125-->00:28:56.250
find them so. But you can, you
can see there’s a, a bit of a
trend line right in the teams.

00:28:56.250-->00:29:00.625
And um, it’s important to know,
which I think Visi was getting
at, is how many teams are

00:29:00.625-->00:29:05.292
actually participating. Which is
vastly different than how many
teams registered. Right. So, I

00:29:05.292-->00:29:10.250
think last year there were 1500
teams that registered I think
Vito said. So, 1500 teams

00:29:10.250-->00:29:14.750
registered 276 actually
submitted something, right.
Which isn’t a perfect metric,

00:29:14.750-->00:29:18.625
might, there might be some
people that tried to play and
just couldn’t figure out hack

00:29:18.625-->00:29:22.917
the planet, right. And they
didn’t submit anything. >>Right
>>It’s possible so it’s not a

00:29:22.917-->00:29:27.833
perfect metric. Um but these,
this is a, a good approximation
of how many teams are playing

00:29:27.833-->00:29:33.042
right. Not how many people,
teams can have many people. Uh
they probably have at least 8 in

00:29:33.042-->00:29:36.917
most cases but. >>Or like 50 if
you’re School of Root >>Uh
[laughter]. We don’t

00:29:36.917-->00:29:44.333
discriminate. >>There’s no rules
in CTF >>Equal opportunity. Um
so you can see a trending up um

00:29:44.333-->00:29:49.917
there’s, there’s sort of other
caveats in that uh registering
shadow teams might have some

00:29:49.917-->00:29:54.292
advantage depending on how uh
the game is structured and how
flights have values so having

00:29:54.292-->00:30:00.167
extra teams on your team might
be useful. Um and, and so forth
we can even in some cases dilute

00:30:00.167-->00:30:06.875
flag values strategically. So
then, the, the other flip side
of how many will participate.

00:30:06.875-->00:30:10.708
That’s how many play in open
qualifiers. But how many are
actually going to play when you

00:30:10.708-->00:30:14.875
get, uh to Las Vegas right.
There’s logistics in setting up
the right amount of tables and

00:30:14.875-->00:30:21.750
how many prizes and uh what kind
of uh orchestration you need to
run. So, um, when the ghetto

00:30:21.750-->00:30:27.167
hacker started there were 8.
Right? And there was always 8.
And where did 8 come from? >>Um

00:30:27.167-->00:30:31.417
I think we had, no it was
probably Myles. >>That’s how
many teams we could fit tables

00:30:31.417-->00:30:37.250
in the room. >>Okay. Fire code
essentially. >>It was fire code
and like lines of network and so

00:30:37.250-->00:30:41.625
we built some really nice PVC
structures to run the networking
cables, that’s where 8 came

00:30:41.625-->00:30:45.917
from. >>Yep that, my >>Did you
guys >>My working theory was 8
port switch, or 8 port hub at

00:30:45.917-->00:30:51.333
the time. [laughter] >>There was
never, th- you guys never had, I
think we were, I think we ended

00:30:51.333-->00:30:55.208
up being the first ones to like
im- implement like a team table
limit. I don’t think you guys

00:30:55.208-->00:30:59.792
had that right? Because we, we
also saw the instances of teams
8 or 10 or however many teams

00:30:59.792-->00:31:03.917
were in the ballroom at the
time, we would see 1 table with
like 3 dudes and another table

00:31:03.917-->00:31:08.750
with like literally people like
crowded around with laptops and
stuff. >>Yeah, yeah there were

00:31:08.750-->00:31:13.458
all starting fire code
considerations there too. >>Like
MUXT >>So anyway you can see

00:31:13.458-->00:31:17.500
that one of the things that
DDTEK wanted to do as organizers
is expand the competition

00:31:17.500-->00:31:23.375
acrossed uh several dimensions
and one of them was size. So,
uh, it kinda ramped up a little

00:31:23.375-->00:31:27.167
bit. Uh the first one should
probably have an asterisk in
that, there weren’t actually, 10

00:31:27.167-->00:31:31.708
uh viable teams. There was one
sort of deceptive team, so
really, only 9 teams in the

00:31:31.708-->00:31:35.833
first year. And then uh the, the
last year that DDTEK ran it
there were 20 and that was, they

00:31:35.833-->00:31:40.625
were, we were sort of gently
pushed because it was Def Con 20
to have 20 teams. Um, and then

00:31:40.625-->00:31:47.875
LegitBS maintained for a little
while has then settled on, on 15
in the most recent years. Um, so

00:31:47.875-->00:31:53.042
how many teams will support is,
is another question that you’d
have to ask as uh, as an

00:31:53.042-->00:31:57.708
organizer. Alright, so this
competition once you get to
Vegas and these 10 teams or 15

00:31:57.708-->00:32:03.042
teams uh get here. Um, how are
you going to implement this
attack defend game, right?

00:32:03.042-->00:32:09.000
So fundamentally uh you’re
gonna be concerned about
scoring. They’re, uh 4, uh I don’t know,

00:32:09.000-->00:32:13.167
here there’s 4 basic components
but there’s going to be basic
building blocks right. There’s

00:32:13.167-->00:32:17.417
offense, there’s defense, uh if
you have defense you probably
need some amount of SOA. And

00:32:17.417-->00:32:20.583
then there’s this concept of
like other points. With like
bonus points granted to the

00:32:20.583-->00:32:24.917
breakthroughs, or, or other
stuff. So offense is going to be
stealing flags or corrupting

00:32:24.917-->00:32:28.083
flags right you can, you can
defeat and take one or you can
burn one or whatever. And these

00:32:28.083-->00:32:34.292
have to be combined in some way
to come up with a score that you
can then use to declare a

00:32:34.292-->00:32:39.625
champion, right. So, score might
be offense times defense times
SLA. That has pros and cons,

00:32:39.625-->00:32:42.583
right. If, if any of those is
zero your total score is zero so
you could drive to zero real

00:32:42.583-->00:32:47.792
quick. So, if you have no
offense do you get no points? If
you have no defense do you get

00:32:47.792-->00:32:50.708
no points? Maybe that’s the kind
of game you wanna run. I don’t
know. Uh maybe you do, uh, uh

00:32:50.708-->00:32:57.208
more of a summation, right? If
you add them all together now a
zero score in any one of them

00:32:57.208-->00:33:01.333
doesn’t drag you to zero but it
also doesn’t have that big of an
effect so you can have um

00:33:01.333-->00:33:05.708
slightly more complex formulas
where SLA maybe is the
multiplier. Meaning you have to

00:33:05.708-->00:33:10.833
allow your opponents to play uh
in order to score any points,
right? If you have zero SLA it

00:33:10.833-->00:33:14.125
means nobody can attack you,
nobody else can sort of play the
game. So, you don’t, you want to

00:33:14.125-->00:33:22.250
encourage the game to be played.
Um, uh and then uh, these need
to work over time, right, we

00:33:22.250-->00:33:25.750
have the concept of rounds,
you’re gonna to play all weekend
so now this has to added up over

00:33:25.750-->00:33:31.500
these uh fixed, or variable
length, 5 minute, 15 minute
rounds, uh whatever. So now your

00:33:31.500-->00:33:38.708
round score is the summation of
these. Uh, um uh the round score
is the formula and the game

00:33:38.708-->00:33:42.292
score is the summation. And then
you might have multiple services
right, these concurrent

00:33:42.292-->00:33:46.458
services, these, these 10
vulnerable things that are
running and your host that needs

00:33:46.458-->00:33:49.292
to be added in there. So maybe
you end up with the formula,
that looks like this down at the

00:33:49.292-->00:33:52.208
bottom. This is by no means the
right formula or even one that
will even work well but you kind

00:33:52.208-->00:33:58.208
of get the idea. There there’s
some uh mechanics that go into
devising the formula that you

00:33:58.208-->00:34:02.625
need to um have the score.
There’s value in, in having this
well-defined ahead of time. Many

00:34:02.625-->00:34:10.375
other methods. Um, once you even
have these formulas figured out,
you need to measure it all.

00:34:10.375-->00:34:16.958
Right? So you have uh, SLA, uh
so this might be a port scanner
but now they’re much more robust

00:34:16.958-->00:34:23.333
pulling, right. And uh, that’s
very uh service specific,
testing different code paths.

00:34:23.333-->00:34:29.250
Um, if a flag is corrupted for
offense the organizers need to
tell somehow. Um, there’s,

00:34:29.250-->00:34:34.417
there’s a lot of sort of game
uh, uh strategies around uh sort
of permanently overriding other

00:34:34.417-->00:34:38.500
people’s flags and things like
that. Um, some moderate CTF’s
employ custom kernels and, and

00:34:38.500-->00:34:43.417
hypervisors and traps and all of
these things in order to detect
that a override has happened.

00:34:43.417-->00:34:48.542
Similar protections for reads.
And defense is sort of the
absence of offense right? You

00:34:48.542-->00:34:53.000
must be doing good because
nobody is successfully attacking
you. So, none of this is set in

00:34:53.000-->00:34:56.667
stone, every year there’s a new
opportunity to revise and come
in with, with new scoring

00:34:56.667-->00:35:01.708
methods. Um, there are more
questions here that, that uh um
will need to be answered, right.

00:35:01.708-->00:35:06.875
Will there be bonus points? Uh
will the scores always increase?
How important is offense? So

00:35:06.875-->00:35:12.292
forth. >>Some of the important
parts about what you’re saying,
um like uh, some of the formulas

00:35:12.292-->00:35:18.000
he was just talking about, if I
turn off my computers and start
hacking, then the games over

00:35:18.000-->00:35:23.583
right? It’s not fun. Uh it it’s
really difficult to convince
security people to do blue team

00:35:23.583-->00:35:29.500
when they’re here for a red team
contest. Uh that, that figuring
out how to game things has been

00:35:29.500-->00:35:34.958
the evolution of the game.
Really >>Absolutely >>Making it
fun >>Absolutely. That’s the

00:35:34.958-->00:35:39.292
whole dirty tricks department. I
mean we had, we had dudes
basically going and like, so

00:35:39.292-->00:35:44.083
when we were playing our dirty
tricks department was doing
stuff like, we had a guy that uh

00:35:44.083-->00:35:48.458
did physical pen testing for a
living like his, he was the guy
that talked his way into data

00:35:48.458-->00:35:52.125
centers and did all that kind of
thing. And so, he had like a
high and tight, he was you know

00:35:52.125-->00:35:55.583
ex-military whatever. So he’d
like roll around in like camo
and like you know camo pants and

00:35:55.583-->00:35:59.542
like a, you know Def Con like,
Def Con regulation black shirt
or whatever. And he would

00:35:59.542-->00:36:03.958
actually, like never ever talk
to us. We would send someone
away from the table to like make

00:36:03.958-->00:36:08.667
drops and like meet with him.
Who, he was rolling around doing
things like brushing a CD off

00:36:08.667-->00:36:12.583
like, brushing burnt CDs off
someone else’s table and he had
like duct tape rolled in a loop

00:36:12.583-->00:36:18.083
on the bottom of his shoe and he
would walk with a limp so that
he wouldn’t like stick it down. But

00:36:18.083-->00:36:22.958
like camo guy walking with a
limp, like no big deal so he
would literally go by tables

00:36:22.958-->00:36:29.792
brush CD-ROMS off, stick them to
the bottom of his shoes and walk
away with them. [laughter]

00:36:29.792-->00:36:37.417
>>Yeah. Yeah. There’s, there’s
many colors to that.
>>Definitely >>Um, so these

00:36:37.417-->00:36:42.417
services that are running
concurrently, you need to
define, you need to define how

00:36:42.417-->00:36:46.750
much you’re going to define
them. Right. Is there a spec for
this? Like, how, what kind of a

00:36:46.750-->00:36:49.292
box are they gonna fit in? The
more of a specification you have
for a service, the easier it is

00:36:49.292-->00:36:53.458
to uh, to automate the test and
the deploy, and the redeploy,
and to rebuild and also to

00:36:53.458-->00:36:56.292
outsource it turns out. Because
if you have a spec you can get,
tell somebody to write against

00:36:56.292-->00:36:59.708
the spec if this’, if it’s an
open box you might come back
with something that’s really

00:36:59.708-->00:37:04.708
hard to, to organize around. Um,
but lots of decisions about how
many you’re going to run, what’s

00:37:04.708-->00:37:10.833
their point value and so forth.
Um, need to decide how teams are
going to interact with the

00:37:10.833-->00:37:13.875
infrastructure, so they’re
connected, together, there’s
some way that they’re

00:37:13.875-->00:37:17.458
interacting with each other. Um
but they’re also going to have
to, to interact with all the,

00:37:17.458-->00:37:22.042
the, the pie- the, the bits and
pieces that you’ve assembled.
Um, so will they operate their

00:37:22.042-->00:37:25.417
own defended host? Is it a VM?
Um will they route through the
table, right. Do they have an

00:37:25.417-->00:37:30.208
uplink that’s on a table? Do
they get a tap? Is the tap
delayed? So forth. There’s lots

00:37:30.208-->00:37:34.375
of uh, um decisions that have to
be made about how the, the game
is going to be represented and

00:37:34.375-->00:37:41.000
how it’s going be interfaced
with. Um, one thing that I
think, I think everybody another

00:37:41.000-->00:37:47.208
thing that’s continued over time
is having a, a, an immense um
desire to protect the integrity

00:37:47.208-->00:37:52.125
of the game. Right. And there’s
lots of ways that you can, you
can take that to heart and, and

00:37:52.125-->00:37:56.583
for, um for >>Specifically it
needs to be just hackable
enough. [laughable] >>Right,

00:37:56.583-->00:37:59.542
right it’s apparently, I mean
it’s already a hostile network,
right it’s already hostile

00:37:59.542-->00:38:05.000
people. Uh it’s designed to be
vulnerable right. Uh but you
wanna have a fair game, you want

00:38:05.000-->00:38:08.708
to make sure that when you’re
crowning the champion you’re
sure that that’s the champion.

00:38:08.708-->00:38:11.750
That it’s the right person,
you’re confident in your scoring
you’re confident in your

00:38:11.750-->00:38:15.792
infrastructure. Um so there’s
there’s lots of different
pieces. This is another slide

00:38:15.792-->00:38:21.000
that’s by no means exhaustive,
um but you wanna take, uh I
think that at least up until now

00:38:21.000-->00:38:26.042
[audio cuts] taken to protect
the integrity of the game. Um
and the non-technical side,

00:38:26.042-->00:38:30.625
there’s, there’s even things
about table positioning, right.
Is it unfair to put one table

00:38:30.625-->00:38:34.708
sort of like with their backs to
the doors, when you’re opening
the room? Uh is it unfair to

00:38:34.708-->00:38:39.000
give 1, 1 team a particular ID
that might result in nulls in
their shell code? Because of

00:38:39.000-->00:38:44.875
their network subnet and things
like that, right. So, you really
want to ha- um strive to be

00:38:44.875-->00:38:50.167
fair. >>Who did that exactly?
[laughter] >>When we, when we
designed the uh, uh that custom

00:38:50.167-->00:38:55.167
badge he mentioned uh
unfortunately we, so we gave
index numbers to people who

00:38:55.167-->00:39:00.958
placed in our qualifier. And PVP
got first so we gave them the
zero index. Uh, unfortunately

00:39:00.958-->00:39:08.167
their index was not exploitable
on the RF network. >>So that’s
one, one instance. Like

00:39:08.167-->00:39:13.792
everyone’s got their instance.
>>Oh for sure. >>Um, so plan for
failure, right. Like in this

00:39:13.792-->00:39:19.667
case, bring a Mac that can also
do your presentation. Um but uh,
and this is the kind of thing

00:39:19.667-->00:39:22.750
where I’m sure we can just go
down the table and everybody can
talk about the different

00:39:22.750-->00:39:26.458
failures but uh one of the
failures that happened in, in
this picture, I’m pretty sure

00:39:26.458-->00:39:30.792
this is the right picture was
uh, a sort of essential scoring
database had a hard disc just

00:39:30.792-->00:39:34.583
totally tank, right, right in
the middle of the competition,
right. So, you have to have, uh,

00:39:34.583-->00:39:40.625
uh ideally some resiliency um
some like fail over, some backup
plans right. Like uh the, sort

00:39:40.625-->00:39:47.125
of the game must go on
mentality, right. Um, so what
will the rules be? So, I think

00:39:47.125-->00:39:54.500
uh, typically historically the
rule is no physical attacks
right? And this is really, to

00:39:54.500-->00:39:57.208
keep people from physically
being hurt. This is actually
there and I think there’s good

00:39:57.208-->00:39:59.625
reasons >>Well and also >>To
have that there >>Don’t cut our
cables because we had to do

00:39:59.625-->00:40:02.458
that, like that would cost us
money so don’t’ cut our stuff.
>>Yeah, so there’s, there’s

00:40:02.458-->00:40:06.083
other things right, there’s,
there’s >>Those were not the
rules when we played. >>It sort

00:40:06.083-->00:40:08.917
of expands into things
[laughter} like, don’t mess with
the infrastructure right, don’t

00:40:08.917-->00:40:13.333
destroy personal property right,
don’t toss >>Messing, messing
with the infrastructure used to

00:40:13.333-->00:40:17.042
be like part of the game. >>That
was the game >>Along time ago,
that was the game. >>That was

00:40:17.042-->00:40:20.375
the game. >>Along time ago, we
hacked >>Well this is like >>We
hacked part of the scoring

00:40:20.375-->00:40:23.542
system and basically like I
remember, I think it was DD that
came up to the table and he’s

00:40:23.542-->00:40:27.375
like, that’s really cool, knock
it the f**k off. [laughter] Like
>>Did you get, did you receive

00:40:27.375-->00:40:30.958
bonus points for your efforts?
>>Yeah, yeah definitely I mean
that was always the thing is you

00:40:30.958-->00:40:34.083
kind of reward people for doing
something novel and new. I
remember uh, I remember the year

00:40:34.083-->00:40:39.625
that um we had uh, again sort of
the de facto rule, don’t cut
cables. Right? Cause, obviously

00:40:39.625-->00:40:43.083
you could walk around with a
razor blade on your shoe and
like cut cables all day long and

00:40:43.083-->00:40:47.000
so one of the things that we had
said was you know basically
denial of service isn’t actually

00:40:47.000-->00:40:50.708
that cool, right? So, like do
something cool. And if you’re
trying to do something cool and

00:40:50.708-->00:40:54.250
you mess it up, like okay we’ll
have a little bit of forgiveness
and it will be alright. So, I

00:40:54.250-->00:40:59.333
remember um I think it was
School of Roots captain John
Boss comes up to me, comes up to

00:40:59.333-->00:41:04.625
me and he’s like so Visi we were
trying something really neat, I
was standing, kind of the tables

00:41:04.625-->00:41:09.167
were like this and actually this
was a lessons learned but we had
uh, the sort of ring of tables

00:41:09.167-->00:41:12.792
for the uh, for the people
running, you know for Kenshoto
in the middle. And the tables

00:41:12.792-->00:41:17.042
had skirts sort of on both sides
you know like the hotel does.
And so, we’re standing there

00:41:17.042-->00:41:22.792
next to the table, you remember
this story now. [laughter].
Right, so we’re standing there

00:41:22.792-->00:41:28.000
and this was actually the DTMF
year. So, we’re standing there
and he’s like so, what if a

00:41:28.000-->00:41:32.833
friend of mine theoretically was
trying to do something really
cool but kind of messed it up

00:41:32.833-->00:41:39.500
and now maybe a team needs a new
cable? And I’m like, what’d you
do? So he’s like well you see,

00:41:39.500-->00:41:43.000
and he lifts up the skirt of the
table and there’s a dude
underneath our table like right

00:41:43.000-->00:41:47.000
next to me and he’s like, well
we haven’t been able to get him
out for like 2 hours because we

00:41:47.000-->00:41:52.500
managed to get him under there
to try and, so they had gone and
bought a 900 megahertz phone and

00:41:52.500-->00:41:57.875
were trying to wire it into one
of the other teams pots line to
literally broadcast whatever

00:41:57.875-->00:42:04.542
tokens they were submitting over
like 900 megahertz analog. So,
they apparently messed up

00:42:04.542-->00:42:09.417
splicing it in ya know and like,
like happens but then they also
couldn’t actually find a window

00:42:09.417-->00:42:13.708
to get the guy out from under
our table covertly and so he had
been under there for like 2

00:42:13.708-->00:42:18.375
hours. [laughter] And so like,
John, John Voss lifts up the
skirt. He’s like, so we’ve got

00:42:18.375-->00:42:24.625
this dude under your table and
I’m like. That’s about it right?
Something like that. Oh, there

00:42:24.625-->00:42:32.500
he is, yeah man. So, like, so,
basically, we were like, yeah
that’s, that’s actually really

00:42:32.500-->00:42:39.208
cool like, and well played you
know, I’m getting people under
our table. So, we just ran a new

00:42:39.208-->00:42:44.917
line and didn’t uh, didn’t cause
a problem but basically that
whole dirty tricks thing, like

00:42:44.917-->00:42:45.667
messing with the infrastructure,
definitely was part of our, our
game. >>On, on totally unrelated

00:42:45.667-->00:42:49.375
fact, when infrastructure moved
to dedicated rooms I may or may
not have been in balconies, the

00:42:49.375-->00:42:55.333
rooms locked at some point.
>>But the balconies didn’t?
Yeah. [laughter] Yeah, uh who

00:42:55.333-->00:42:58.875
was it that climbed across from
one of the balconies to the
other to get into the Kenshoto

00:42:58.875-->00:43:02.833
skybox one year? I- >>Uh, I
don’t know >> I think that was
School of Root 2 >>Yeah, I don’t

00:43:02.833-->00:43:09.833
remember. Um we’re talking about
cultural influence, actually
right before the talk there were

00:43:09.833-->00:43:15.292
some, some bits of culture
slides so again this sort of an
incomplete slide but um,

00:43:15.292-->00:43:20.708
culture is, is, is interesting
in that um there’s a lot of um,
of barriers right especially as

00:43:20.708-->00:43:25.083
this becomes an increasingly
international right
there’s teams from, from all

00:43:25.083-->00:43:28.625
around the planet that played,
not only in quals but also in
finals these days. And there’s,

00:43:28.625-->00:43:32.875
there’s clearly the language
barrier, there’s clearly the
popular cultures. Like uh, uh,

00:43:32.875-->00:43:37.333
um the hacker movies from the
80’s like some of them weren’t
as internationally popular as

00:43:37.333-->00:43:42.042
others so like, those questions
don’t resonate as much. Time
zones and holidays right. Like

00:43:42.042-->00:43:47.458
there’s, there’s, there’s
different things to try and keep
in mind. Um, they tend to show

00:43:47.458-->00:43:50.375
up during uh, trivia and
question starters and things on,
on the right-hand side. Just

00:43:50.375-->00:43:55.375
generally when interfacing with
the teams and when designing uh
services that have ASKI

00:43:55.375-->00:43:59.625
protocols, right. The words
sometimes have meanings to, to
different cultures. Um, there’s

00:43:59.625-->00:44:04.792
also the cu- culture that’s
influenced by CTF, where the
community uh sort of leaves it’s

00:44:04.792-->00:44:11.458
mark uh in various ways. So
generals and producers and
actors and, and so forth uh

00:44:11.458-->00:44:16.333
actually come through and visit
the CTF room and, and want to
have sort of their own

00:44:16.333-->00:44:21.500
individual uh briefing and their
own sort of explanation and you
have no idea of the ramification

00:44:21.500-->00:44:27.333
of those conversations are going
to be and how that sort of
spirals out right? Uh so here’s

00:44:27.333-->00:44:33.125
uh, uh a team mentioned, uh
School of Root mentioned in a,
the HBO show. Um, another sort

00:44:33.125-->00:44:39.208
of thing to broach by the
organizers is um how are you
going to engage everybody right?

00:44:39.208-->00:44:43.583
Not only the participants but
also the, the folks that are
walking around and the average

00:44:43.583-->00:44:48.167
human attendee. So should there
be ambiance, should there be a
distractions, should there be

00:44:48.167-->00:44:50.458
attraction, like should there be
music at all? How loud should
the music be, should there be

00:44:50.458-->00:44:52.667
videos, should there be
scoreboards, visual effects.
>>Really, really, loud.

00:44:52.667-->00:44:57.583
[laughter] >>Uh there’s probably
contention here about that. Um,
but, but otherwise if you don’t

00:44:57.583-->00:45:02.958
consider these things it just
ends up being a bunch of people
in a fairly dark room staring at

00:45:02.958-->00:45:06.917
computers, right? And uh the
teams don’t really have any
physical interaction with each

00:45:06.917-->00:45:10.667
other. The attendees don’t
really know what they’re looking
at, so handouts and so forth.

00:45:10.667-->00:45:14.625
So, this gives way to
visualization. So, uh back in
uh, I- I- think this is about

00:45:14.625-->00:45:21.042
2002, I don’t really know. I
think this is a ghetto created
scoreboard. >>Yeah, I think it

00:45:21.042-->00:45:26.667
is. >>Um, but we had uh
scoreboards right, projected up
to the, the wall. Um, here’s

00:45:26.667-->00:45:32.042
>>One of the, one of the
problems with showing scores is
uh, it can be really easy to get

00:45:32.042-->00:45:36.042
away with a win in the first
day. And you c- you might have
teams start to lose interest.

00:45:36.042-->00:45:41.167
This is part of you know, do
scores always go up? This was
part of uh, this is a zero-sum

00:45:41.167-->00:45:45.833
game and you can see the teams
in the bottom have a little bit
of a red bar and the bigger

00:45:45.833-->00:45:51.042
green bars are the, the top, top
games. Eh, eh at some point
you’re trying to keep people

00:45:51.042-->00:45:57.250
interested even if in reality
you think there’s no point, they
can’t win anymore. >>I don’t

00:45:57.250-->00:46:01.333
know if it was us who started it
but so the first day we showed
scores uh live scores, the

00:46:01.333-->00:46:05.833
second day we only show relative
positions and finally the third
day we just played back the

00:46:05.833-->00:46:08.333
previous >>I think there were,
there were sort of a refinement
process on that. We had

00:46:08.333-->00:46:11.792
something where like, I think
for the last 4 hours we didn’t
show the scoreboard, something

00:46:11.792-->00:46:16.333
like that too, yeah. >>Yeah,
yeah, I think everybody has
their own variation on that

00:46:16.333-->00:46:20.667
theme, I think you’re, I think
you’re a little bit more extreme
than has been in the past. Um,

00:46:20.667-->00:46:27.875
there’s some uh, some boards
kind of through time. Uh, here’s
uh, a Kenshoto board with the,

00:46:27.875-->00:46:33.333
the ordering and uh the team
relative placement on a line
chart over, over time. Um, a

00:46:33.333-->00:46:39.042
little bit later still a very
similar scoreboard. Uh, even
later in the Kenshoto time you

00:46:39.042-->00:46:43.708
still have a similar scoreboard
but it’s also rotating with ones
that have um, uh I guess

00:46:43.708-->00:46:46.875
innovative in some sense. Like
showing different types of
information. So, the one down in

00:46:46.875-->00:46:51.042
the corner is, is kind of
showing what’s left on the
table. These are the points that

00:46:51.042-->00:46:57.125
are, are still sort of up for
grabs. The services that need
some more attention. Um, if you

00:46:57.125-->00:47:01.667
open your data you get
visualizations from others, so
here’s some quals data that was

00:47:01.667-->00:47:07.958
taken by some uh, some uh sort
of non-organizer folks and
graphed over time and it’s sort

00:47:07.958-->00:47:13.167
of easy in, and different types
of visualizations to see um, how
the scores progress. So here you

00:47:13.167-->00:47:17.958
can see relatively quickly a lot
of teams spike up and one sort
of is, is in the lead for a long

00:47:17.958-->00:47:22.208
time but then there’s still a
relatively dramatic come from
behind a few hours before the

00:47:22.208-->00:47:27.458
competition, where the red line
cuts up. So, this uh it’s, it’s
quals so it’s a little bit

00:47:27.458-->00:47:29.875
different but this is an example
of that come behind scoring that
you like to embody in the game.

00:47:29.875-->00:47:36.208
Uh, if you want to encourage the
availability for a come behind.
>>Teams, there were also teams

00:47:36.208-->00:47:40.542
that were like fairly heavy into
like using their lead
psychologically >>Oh yeah >>To

00:47:40.542-->00:47:47.667
win the game, like team [audio
cuts] [laughter] that spent the
last 2 hours of, of CTF one year

00:47:47.667-->00:47:53.500
uh, very, very publicly and very
sort of flagrantly having all
their people just play guitar

00:47:53.500-->00:48:00.833
hero. Mostly just as like uh,
you know you don’t even need to
try anymore because we’ve got

00:48:00.833-->00:48:06.083
this. >>Whenever we say a
certain team we mean Chris Eagle
and School of Root >>Yeah pretty

00:48:06.083-->00:48:10.375
much >>Every time >>Just so it’s
clear >>A certain team is always
them. >>So there’s a new, uh new

00:48:10.375-->00:48:16.042
organizer took over, right, the,
the scoreboard is uh, still um,
relatively basic, it’s

00:48:16.042-->00:48:20.375
displaying the same sort of
information. This is sort of one
of the things that the game ends

00:48:20.375-->00:48:24.042
up being more important for the
organizers so the visualizations
end up being sort of back

00:48:24.042-->00:48:27.667
burnered and back burnered and
then like cobbled together at
the end, right? And, and that

00:48:27.667-->00:48:33.083
code doesn’t persist uh,
generally from organizer to
organizer. So, um you see this,

00:48:33.083-->00:48:36.583
this trend where like the new
organizer that takes over, takes
over and then it’s like sort of

00:48:36.583-->00:48:41.792
back to square one and it kind
of builds back up, so uh, DDTEKs
early boards looked like that.

00:48:41.792-->00:48:48.250
Um, then you know later on they
had much more, um, appealing
graphics, uh that uh displayed

00:48:48.250-->00:48:53.250
some of the same information but
also like that’s a place holder
screen on the side. And then in

00:48:53.250-->00:48:59.167
the end you, you sort of see
the, um the the more appeal
ing side. And some attempts at uh, trying

00:48:59.167-->00:49:03.542
to display types of information
that are kind of hard to
consume. Like the bubble chart

00:49:03.542-->00:49:08.125
where it shows team uh versus
team action. This one is
particular in the numbe- number

00:49:08.125-->00:49:12.000
of writes per service. So, that
big blue circle is showing that
like one team is like massively

00:49:12.000-->00:49:16.000
over writing on one particular
other team instead of evenly
distributing their attacks

00:49:16.000-->00:49:21.500
across. Um, and then you know
like gadgets and stuff like
that. And then another new team

00:49:21.500-->00:49:26.625
came, comes, takes over still
relatively basic. They’ve taken
a different approach where it’s

00:49:26.625-->00:49:30.500
sort of a graph based um thing
where the, the edges display
types of information but it’s

00:49:30.500-->00:49:36.583
still sort of basic blocks, not
particularly appealing that
advances over time, sort of

00:49:36.583-->00:49:41.667
similar information but much
more appealing. >>Oh I’m saying
that, that the, the blocks going

00:49:41.667-->00:49:45.000
across were live. That was our
first iteration. I just saw the
next one, it’s got much better

00:49:45.000-->00:49:49.833
>>Oh, yeah, yeah, yeah, that’s,
that’s sort of the point. Right.
It sort of evolves over time.

00:49:49.833-->00:49:55.375
And uh, and, and as an organizer
you have an opportunity to
decide how you’re going to try

00:49:55.375-->00:49:58.750
and convey information and what
information you’re going to
convey right. Because you want

00:49:58.750-->00:50:03.167
to engage the audience but uh,
in some respect it actually also
inform all the teams that are

00:50:03.167-->00:50:08.125
sitting there present, right?
So, in some way this acts as a,
a, an intrusion detection

00:50:08.125-->00:50:11.875
system. You can tell somebody’s
attacking you and, right so
that’s why you have the staged

00:50:11.875-->00:50:15.833
thing where it’s like live and
then it’s like delayed and then
it’s like gone. Or however you

00:50:15.833-->00:50:20.542
phase it out. But these are
considerations that you have to
have as a, as a organizer. Um so

00:50:20.542-->00:50:28.167
then there’s the, I’m just gonna
rip through these pretty quick
so we can get the questions.

00:50:28.167-->00:50:30.917
There’s the expansion into
physical space, we talked about
this a little bit already

00:50:30.917-->00:50:36.875
alright with the, uh going into
the custom badges that are
actually codable. Uh the, the

00:50:36.875-->00:50:41.042
meta game. Lock picking is super
common in lots of CTFs, like as
a physical aspect to get to your

00:50:41.042-->00:50:45.625
passwords, your keys to start
the game or whatever. Um,
another physical uh thing that

00:50:45.625-->00:50:49.500
was tried is a service that was
in the game actually controlled
physical things outside the

00:50:49.500-->00:50:55.417
game. Um, so like uh, this is
the robotic chicken fight. Um
that was a, a service in 2011.

00:50:55.417-->00:51:00.958
And uh, that was, it, it was
sor- sort of an ancillary to
slide, one of the teams actually

00:51:00.958-->00:51:05.958
went out and got like game
controllers and, and pro- like,
had an adapter interface to the

00:51:05.958-->00:51:09.333
service where they could
actually automatically activate
the service and then control it

00:51:09.333-->00:51:16.167
with their PlayStation 4
controller or something. Um,
interesting thing, right? Um, so

00:51:16.167-->00:51:21.958
tradition, uh so future
organizers, I, I think there’s a
lot of aspects of tradition. If

00:51:21.958-->00:51:25.375
you’ve been a player or close to
the organizers you get some of
that, trying to like document

00:51:25.375-->00:51:30.083
some of that here. Um but
there’s, there’s certainly a
desire to keep Def Con CTF the

00:51:30.083-->00:51:33.917
best in the world. There’s
certainly a desire to keep it
fair, fun and innovative. Um

00:51:33.917-->00:51:38.583
there’s, there’s always desire
to engage the audience. You
never know what the next

00:51:38.583-->00:51:41.583
generation of players is going
to be or where they’re going to
come from. You really, really

00:51:41.583-->00:51:45.708
want to have that open to
everybody. Um there’s logistics,
there’s game banners and team

00:51:45.708-->00:51:49.458
banners. The winner typically
gets to bring the game banner
home as part of their spoils.

00:51:49.458-->00:51:54.208
Um, there’s swag, there’s
t-shirts and stickers and so
forth. Typically, announcements

00:51:54.208-->00:51:58.958
are happening on, on April 1st.
Right that’s just the, the
thing, you know when quals is

00:51:58.958-->00:52:04.875
going to happen. Um, fortune
cookies started. I think, I
think uh Kenshoto started that

00:52:04.875-->00:52:08.875
because they had a balcony and
there was a desire to throw
something off the balcony. So,

00:52:08.875-->00:52:12.917
there was the, the Sunday
tossing of the cookies. >>That’s
why we >>Oh yeah, the >>We get

00:52:12.917-->00:52:15.583
coins >>Fortune cookies
>>They’re much bigger and they
hurt worse. >>We had, we had the

00:52:15.583-->00:52:19.542
an- like the annual Sunday
tossing of the cookies. >>Right
>>Yeah. >>And so just some of

00:52:19.542-->00:52:24.083
those are here back in Def Con
15, some of these are DDTEK
ones. DDTEK incorporated some of

00:52:24.083-->00:52:30.125
the challenges into some of the,
um, uh fortunes. So, there’s
stickers, right. Kind of a

00:52:30.125-->00:52:35.125
hacker thing, there’s laptop
stickers and everybody’s got
their stickers. There’s coins as

00:52:35.125-->00:52:41.708
uh, [inaudible] mentioned. Uh,
now, sort of one of the longest
lasting traditions. But there’s

00:52:41.708-->00:52:45.750
one for, for every year going
back a few years. And then
there’s sort of >>Holy s**t

00:52:45.750-->00:52:49.708
really? >>Other things,
[laughter] right. >>An actual
CTF tattoo. >>Uh, stress sheep

00:52:49.708-->00:52:57.125
and so forth. Teams bring stuff,
you can’t stop them. They bring
stuff, they do stuff, um,

00:52:57.125-->00:53:02.167
particularly like the sheep
>>This slide should have been
titled, will there be sheep?

00:53:02.167-->00:53:09.000
>>Um, there was uh, they
modified the stress sheep to
have the LED eyes. Um, so teams

00:53:09.000-->00:53:12.833
have to prepare, uh for a couple
people who already brought up
that, uh this is sort of a

00:53:12.833-->00:53:16.167
second job. It’s a multi-month,
it might be an all-year round
thing, like when do you start

00:53:16.167-->00:53:19.167
planning for next year? It’s a
soon as this year's over kind of
thing. Uh so there’s a lot of

00:53:19.167-->00:53:24.500
preparation, there’s a setting
of the servers the
configurations, building all the

00:53:24.500-->00:53:28.208
services, building the
infrastructure if you don’t
already have it. Um and then if

00:53:28.208-->00:53:31.750
you do, like the meta games
stuff, you have to set up all
the meta game stuff, you have to

00:53:31.750-->00:53:37.708
program the badges, you have to,
we did uh, um DDTEK interfaced
with all the human badges. So, I

00:53:37.708-->00:53:41.875
think if you have a human badge
from DEFCON 18, you actually
have CTF code in the firmware.

00:53:41.875-->00:53:46.625
So, that’s like a whole other
layer of, of uh of working with
other parts of DEFCON. >>Did you

00:53:46.625-->00:53:50.333
guys end up having the thing
where like every year where you
swore off doing it again? Like

00:53:50.333-->00:53:53.917
every year, we were like we are
never doing this again. And then
like a couple months later

00:53:53.917-->00:53:58.375
you’re like, well maybe. >>Well
we have some like >>If
we do this year next year it

00:53:58.375-->00:54:03.375
will be so much cooler.
>>There’s a couple, like even
now I think, eh maybe you could

00:54:03.375-->00:54:07.125
do it next year. And no, it’s
written down, we’re done.
>>Yeah, it was like the year we

00:54:07.125-->00:54:10.167
stopped, the year that we
stopped hosting was literally
the year that nobody stood up

00:54:10.167-->00:54:14.417
and was like, yeah, we’re doing
this. Like everybody was just
like, eh. >>Yeah, we we

00:54:14.417-->00:54:18.375
definitely got drunk at quals
and like, yeah, let’s, let’s do
this, we’ve got this. >>Yep

00:54:18.375-->00:54:22.208
>>Next morning, no, no, no.
>>Not, no >>Just forget what we
said. [laughter] >>So we’re

00:54:22.208-->00:54:26.917
talking about how much effort
there is like year-round. When
you get there on site, there’s

00:54:26.917-->00:54:30.417
an amazing amount of things that
have to happen behind the
scenes, and I say proportionally

00:54:30.417-->00:54:33.458
most of what you’re going to be
looking at as an organizer is
something like this. You’ll see

00:54:33.458-->00:54:36.583
like the empty rooms with no
participants in it, you’ll be
watching some of your, your

00:54:36.583-->00:54:40.083
buddies terminate network cables
across the room as you’re trying
to make sure all of your

00:54:40.083-->00:54:44.500
services work. >>That’s
happening right now. >>And um,
and this, this is the view that

00:54:44.500-->00:54:48.542
you get. Or you get a view in
the back room, your, your
infrastructure servers or

00:54:48.542-->00:54:51.750
something like that. Right. Uh,
what the >>Also I think, I think
since you’re, since you’re on

00:54:51.750-->00:54:54.958
the topic of preparation, it’s
also pretty important
to point out organizers of CTF, the

00:54:54.958-->00:55:02.500
entirety of the infrastructure
and the code and the logistics
um has always been completely

00:55:02.500-->00:55:08.208
something provided by the people
actually running CTF. Like our
actual, Kenshiro's logistics

00:55:08.208-->00:55:14.750
bill every year out of pocket
for us was like 20 grand. >>Yeah
>>Like that’s why I say don’t

00:55:14.750-->00:55:17.000
cut our cables, like seriously.
>>Cause >>They’re our >>Because
they’re our cables [laughter]

00:55:17.000-->00:55:21.792
>>I don’t know >>So if we’re
talking >>How much it costs just
to ship all the stuff out here.

00:55:21.792-->00:55:24.583
We had >>Yeah, yeah shipping
costs, shipping costs alone. The
pelican cases and all crazy

00:55:24.583-->00:55:27.958
nonsense. >>If we’re talking
proportionally, partially what
the team [audio cuts] something

00:55:27.958-->00:55:31.458
like this. Cause again, like
the, the table are meant for
fire code or due to prizes

00:55:31.458-->00:55:37.292
there’s only 8 black badges. Um,
uh the teams are much larger
than that or some teams are. And

00:55:37.292-->00:55:42.333
you see, uh you know extra hotel
rooms with wiring all over it
and you see the, um you’ll be

00:55:42.333-->00:55:47.000
staring at IDA screens and
debugger screens and have your
persistent servers that have

00:55:47.000-->00:55:50.667
UPSs strapped to the bottom of
them so you can wheel back and
forth as the game goes live.

00:55:50.667-->00:55:56.333
[laughter] So, so why do people
play? Right? This is like, these
last couple slides sort of like,

00:55:56.333-->00:56:02.833
make you wonder like, why? Like,
right? Um so why do people play?
Uh well there’s challenge, uh

00:56:02.833-->00:56:07.458
the, there’s some prizes
especially around the world,
some of the CTFs are getting

00:56:07.458-->00:56:11.583
like 30-->000 dollar, 50-->000
dollar prizes, right? It’s not
quite like eGames. But um like

00:56:11.583-->00:56:16.458
you, you can’t make a living at
it but you can make some money.
Uh what if you just couldn’t get

00:56:16.458-->00:56:21.250
into talks at Def Con?
[laughter] Right? So, you’re
stuck there, right? >>I know

00:56:21.250-->00:56:24.500
right? It’s ridiculous. >> I’m
convinced there’s some people,
have logical disorder >>Glory,

00:56:24.500-->00:56:27.792
glory is the reason you play.
>>Glory, yeah. >>So the really
reason right? >>And that’s why

00:56:27.792-->00:56:31.208
it started, capture the flag,
was to have a chance to go head
to head. >>And you get the flag

00:56:31.208-->00:56:36.083
badge >>Glory >>It’s a black
badge and a jacket. That’s why
you play. >>Yep >>Yep >>Yep,

00:56:36.083-->00:56:42.708
oldest, oldest [audio cuts]
like, around. >>So why, so why
do organizers organize? Right?

00:56:42.708-->00:56:46.792
So this is actually the
question. >>Masechism >>yeah
this is the question, this is

00:56:46.792-->00:56:52.625
the first question I wanna, to,
to hand off to the panel. Before
we do that, just very quickly

00:56:52.625-->00:56:57.458
um, you, you roughly know who's
up here but uh, to, um to
introduce them sort of

00:56:57.458-->00:57:03.208
officially. Uh, Visi, uh wave
your hand. Yeah. So Visi’s
sitting >>I crowd sourced my bio

00:57:03.208-->00:57:06.958
because I’m s**t at writing
bios. So, this is all the
Twitter responses to what should

00:57:06.958-->00:57:11.583
my bio be. >>Yep, so he crowd
sourced the bio, he was the
organizer, uh sort of uh, uh

00:57:11.583-->00:57:19.375
main or chief organizer for
Kenshoto. Uh, Chris Eagle, yeah
down there on the end. Uh so he

00:57:19.375-->00:57:24.042
was uh, uh um a player and then
a organizer and then a player
again. Uh lots of CTF experience

00:57:24.042-->00:57:29.667
from, from both sides. Um,
literally wrote the book on uh
[laughter] on IDA Pro. >>Oh

00:57:29.667-->00:57:35.208
nice, dude hook a brother up >>I
know, right? >>I got you. Don’t
you worry. >>Riley >>We’re,

00:57:35.208-->00:57:39.333
we’re rebooting. Sorry. >>Caezar
>>One second our monitors are
not working. They’re giving

00:57:39.333-->00:57:43.042
epilepsy out. >>So Caezar is
talking >>Momentary technical
difficulties >>Right now. So

00:57:43.042-->00:57:48.208
he’s, he’s part of the Ghetto
Hackers who, who won 3 times and
then became the first formalized

00:57:48.208-->00:57:54.250
organizer and really brought it,
brought it to the next level.
Also known for uh, for uh, C, uh

00:57:54.250-->00:58:00.208
Caezar's challenge um, sort of
an annual thing at. >>21 years I
have thrown a party on Saturday

00:58:00.208-->00:58:07.583
night because uh, behind a
masochist and doing CTF wasn’t
enough. [laughter] Uhm find me

00:58:07.583-->00:58:13.583
at any time and I will hand you
a puzzle and if you can solve
it, which is not very hard this

00:58:13.583-->00:58:22.375
year, uh if you can so, solve it
then I buy you drinks on
Saturday. >>Uh, Hashen. Um, uh,

00:58:22.375-->00:58:27.875
uh also, excuse me, 2-time
champion. Uh unlike the other
champions, what 2 different

00:58:27.875-->00:58:30.958
teams, right? >>2 different
teams >>2 different teams
>>First ones Meta Gods slash

00:58:30.958-->00:58:35.083
Team Awesome the second one's
Samurai TA >>Yeah, there you go.
>>Um, and um one of the current

00:58:35.083-->00:58:38.750
organizers, uh 2013 so we’re not
really sure when they are gonna
stop. >>This year, definitely.

00:58:38.750-->00:58:45.208
>>Yeah we’re not sure when
they’re gonna stop. [laughter]
Uh, and uh and Myles. Right. Um,

00:58:45.208-->00:58:50.083
Myles is, is uh responsible.
He’s like the reason that we’re
all here, right. Cause this

00:58:50.083-->00:58:56.125
wouldn’t have happened if he
wouldn’t have started it so. Um,
>>I know that our presentations

00:58:56.125-->00:59:00.000
not gonna do it but uh, I’m
gonna- it’s time for everybody
to give this guy a big round of

00:59:00.000-->00:59:02.125
applause. >>Specifically
>>Capture the Flag >>Yeah.
[applause] >>It’s Myles’ fault

00:59:02.125-->00:59:14.750
[applause] >>Hacking, hacking is
basically a modern watchable
sport. I mean there are Twitch

00:59:14.750-->00:59:18.708
feeds now for this kind of
thing, right? And mostly it’s
all because of the, the bucket

00:59:18.708-->00:59:26.292
that you started kicking. >>And
uh, and with that I’d like to um
kick it back off and I think the

00:59:26.292-->00:59:30.667
first question will be, so why
do organizers, organizers
organize and let’s go uh, in

00:59:30.667-->00:59:36.375
order, right? In chronological
order. And uh then this room
didn’t get set up uh well, we

00:59:36.375-->00:59:40.958
don’t have mics so if you have
a question, uh you have to come
up here and you can either take

00:59:40.958-->00:59:45.458
the mic or, or I can repeat it.
And also I’ll put up uh, um I’ll
try this thing where you can put

00:59:45.458-->00:59:53.875
up a URL and you can like send
questions to the podium. We’ll
see how that works. >>I would

00:59:53.875-->00:59:57.667
not trust a single person in
this room for that. >>Yeah
>>Alright, goes through Google

00:59:57.667-->01:00:02.583
stuff. >>So Myles. >>So Myles,
yeah >>Why CTF? >> Why, why
organize? >>Because there wasn’t

01:00:02.583-->01:00:08.417
a way to figure out who was best
and not practice out on the live
internet where the con would get

01:00:08.417-->01:00:12.125
shut down. [laughter] Now you,
we forget that >>Who was, so
you’re saying it’s harm

01:00:12.125-->01:00:21.083
reduction? [laughter] >>For the
public >>Totally, and you forget
that like at first, I- I- went

01:00:21.083-->01:00:23.667
behind him because I was
seriously worried if my
employer's heard about Capture

01:00:23.667-->01:00:28.417
the Flag I might be fired. And I
worked as a security guy. And
this last year I was in the

01:00:28.417-->01:00:32.708
Smithsonian as a Black Badge
there, I mean this is really
different. So yeah, it was harm

01:00:32.708-->01:00:37.167
reduction, it was head to head,
it was a chance also if you had
controlled the environment that

01:00:37.167-->01:00:42.250
you could start throwing in some
stuff that explains it to the
general, about why would someone

01:00:42.250-->01:00:48.417
stand in front of a screen for
so long. Well because it’s
really cool and there’s these

01:00:48.417-->01:00:51.042
things, it’s a puzzle that
they’re solving. So, it’s clear
to me that we needed something

01:00:51.042-->01:00:56.458
with a big more showmanship, and
also chance where attackers and
defenders could go head to head.

01:00:56.458-->01:01:04.208
And that it didn’t break the
rest of the network and throw
out the con. >>Yeah, uh, for,

01:01:04.208-->01:01:12.417
for us we came and um, we felt
like we knew enough about
security to not necessarily need

01:01:12.417-->01:01:18.917
to see every single talk so we’d
catch [laughter] a couple of
talks and enjoy it and then we’d

01:01:18.917-->01:01:24.292
kind of wander over and see
Myles’ game and say, ya know,
gee, so I just set down at a

01:01:24.292-->01:01:30.083
table. Every person at that
table today is a dear friend of
mine today and like we go out

01:01:30.083-->01:01:35.000
together all the time, we live
near each other we’ve all move
to, to live near each other. So,

01:01:35.000-->01:01:40.500
my social life is Myles’ fault.
Uh the Ghetto Hackers was formed
because we didn’t have any

01:01:40.500-->01:01:45.167
paper, cause we didn’t bring
pens or paper or notebooks or,
or hardly anything. Uh so we had

01:01:45.167-->01:01:51.750
a napkin and uh mascara and uh
we wrote IP addresses that were
available on the network. That

01:01:51.750-->01:01:58.583
was our, our group of friends
came out of us sitting around
each other. And, for us uh after

01:01:58.583-->01:02:02.708
the first year, the first year
we just sort of tried and saw
how it was and, and then after

01:02:02.708-->01:02:08.708
that there was kind of this fire
of, well you know, if we just
stored every exploit we know.

01:02:08.708-->01:02:13.500
And that w-, that was the
contest back then, was like how
many exploits can you bring in a

01:02:13.500-->01:02:18.458
searchable useable format. And
uh, and i- j-, i- b-, it just
became this, this passion to

01:02:18.458-->01:02:23.417
actually just get better. So,
for, for us we didn’t
necessarily try to win as much

01:02:23.417-->01:02:30.250
as we tried to be c-, get good.
Like get good scrub. That, we
were scrubs and, and we came and

01:02:30.250-->01:02:36.458
you kind of made a crucible and
a bunch of hackers popped out.
>>And then why did you run it?

01:02:36.458-->01:02:44.458
>>Why did we run it? Uh, we ran
the contest uh, after the third
year, uh somebody in one of the

01:02:44.458-->01:02:50.500
teams said that they hated us
and we were cheating bastards
and that they didn’t wanna play.

01:02:50.500-->01:02:56.083
>>Hackers, hello. >>We, we, we
took it as a great honor, and
pride? And but, but there was

01:02:56.083-->01:03:01.792
something uh, a, spark in, in,
in, in our, among us that, um
made us jump up and volunteer to

01:03:01.792-->01:03:08.667
take the game to a new level.
And so uh we got up on stage and
talked about coming back the

01:03:08.667-->01:03:14.333
next year and making a new
contest. And Myles was part of
that contest as well. Uh and,

01:03:14.333-->01:03:21.583
and that was how we kind of came
to the uh, scoreboard and, and,
and sort of all the th- some of

01:03:21.583-->01:03:27.542
the the trappings that are the
beginnings of what these guys
who won the contests that we

01:03:27.542-->01:03:32.875
threw and we handed the reigns
to them. >>That’s actually kind
of been a tradition. I know it

01:03:32.875-->01:03:35.625
was kind of touched on earlier
but there is also a, you end up
feeling this sort of sense of,

01:03:35.625-->01:03:40.417
of belonging and ownership to
the game. Um, when you’re al-
when you win it also. Um, and

01:03:40.417-->01:03:44.792
you have your own designed on
how you want it to be or how you
think it should be so, like

01:03:44.792-->01:03:50.042
transitioning to the reason that
like Kenshoto uh decided to run
CTF for a few years. I think

01:03:50.042-->01:03:54.292
there’s, I mean there’s a number
of facets but for us watching,
watching the Ghetto Hackers and

01:03:54.292-->01:03:59.583
watching the coalescents that it
caused in the hacker community
specifically. And the and the

01:03:59.583-->01:04:04.208
crucible effect actually is
super huge, like I, we would
watch people literally have a

01:04:04.208-->01:04:09.250
reason, have drive to learn all
these new exploit mechanisms,
and, and all of these like

01:04:09.250-->01:04:15.250
details about- part and
whatever. So, for us actually I
think that the biggest thing

01:04:15.250-->01:04:21.083
that, that caused us to actually
run CTF was uh, we were
concerned initially uh that it

01:04:21.083-->01:04:25.042
would be taken over instead of
by someone that is about the
hacker community, that it would

01:04:25.042-->01:04:29.958
be taken over by a university or
a corporation or whatever like
that and we were very concerned

01:04:29.958-->01:04:34.292
about that initially. [laughter]
Turns out, ya know, off we go,
modern times and all that. But

01:04:34.292-->01:04:39.917
um, but the other reason um, the
other reason that Kenshoto
stayed in the game, the reason

01:04:39.917-->01:04:44.542
that, not, not the first year
but the reason for the years
after that um, was that we

01:04:44.542-->01:04:48.917
really felt that it was sort of
an unmeasured uh like an
unmeasured resource, right? So

01:04:48.917-->01:04:53.292
as an example, um in our game we
were sort of one of the first
people to be like, don’t bring

01:04:53.292-->01:04:55.958
any of those exploits because
they’re not going to work
because it’s going to all be

01:04:55.958-->01:04:58.917
entirely custom software with
challenges that you have to
reverse engineer and exploit and

01:04:58.917-->01:05:04.417
land payloads against like, in
that weekend. And for us that
was um, it was really important

01:05:04.417-->01:05:08.333
for us because we felt that,
that was a, that was something
where people hadn’t stretched

01:05:08.333-->01:05:13.042
far enough yet. The, the bar
wasn’t high enough and we felt
like um we felt like we wanted

01:05:13.042-->01:05:18.542
to continue to be able to push
that higher and higher. So, that
eventually not everyone was

01:05:18.542-->01:05:21.333
making it. So as an example
actually there’s also, what,
what do they call it? I think

01:05:21.333-->01:05:24.833
they rename amateur CTF but it’s
called something else now.
>>Open CTF >>There’s several now

01:05:24.833-->01:05:29.375
>>Open CTF, right >>Yeah open,
project too >>So, but like that
>>Subtle >>As an example

01:05:29.375-->01:05:33.708
>>Subtle humble brag there >>Oh,
yeah right but so, that is
essentially formed out of the

01:05:33.708-->01:05:38.083
fact that we pushed, we tried to
push the game into actually
measuring the real redline of

01:05:38.083-->01:05:42.500
what people were capable of in a
weekend. And that, that really
drove us for the years after the

01:05:42.500-->01:05:46.542
first year that we hosted. The
first year was cause we were
super concerned that it was

01:05:46.542-->01:05:49.583
gonna become this like corporate
sponsorship kind of thing. Um
and after that it was because we

01:05:49.583-->01:05:54.167
thought that the, the, the
performance that we’d seen uh,
warranted further measurement.

01:05:54.167-->01:06:00.833
Higher and higher so. >>And that
is a perfect transition to him
to Mr. >>Right >>Cause we, we

01:06:00.833-->01:06:04.792
had a lot of the same reasons.
So, we played in Myles’ game, we
played in Caezar’s game, we

01:06:04.792-->01:06:15.583
played in Visi’s game, and uh we
loved every iteration um I teach
and it turned out that the game

01:06:15.583-->01:06:20.208
as it evolved was it, was a
great microcosm of the security
space. Uh in which to uh conduct

01:06:20.208-->01:06:24.042
uh teaching an- and learning. Uh
my students got really excited
about it so w- w- we loved the

01:06:24.042-->01:06:30.958
game as it was, loved uh, the,
the game uh the last year we
played. Uh with Kenshoto and

01:06:30.958-->01:06:36.583
when they stepped down we were
also very worried that ya know,
some company would come in and

01:06:36.583-->01:06:40.500
commercialize this thing. Uh,
and it, we looked around and
would say, well who's gonna to

01:06:40.500-->01:06:43.292
run it. We could, we couldn’t
imagine who might run it.
>>That’s exactly >>I think there

01:06:43.292-->01:06:47.042
was one other that might’ve run
it, we didn’t know what they
would pitch. Uh so we made our

01:06:47.042-->01:06:52.833
pitch. I think another team made
their pitch um, and uh we
basica- [off-mic comment]

01:06:52.833-->01:06:59.750
[laughter] >>But you lost. And
>>Our, our [laughter] out, our
pitch, our pitch was a lit- was

01:06:59.750-->01:07:04.083
basically if you don’t’ give it
to us it’s gonna suck.
[laughter] >>Ours was if you

01:07:04.083-->01:07:11.333
give it to them it will suck.
[laughter] Um, uh and we really
like the game and wanted to see

01:07:11.333-->01:07:14.833
it continue as much the same
vein, of course it didn’t’ occur
to us that we wouldn’t get to

01:07:14.833-->01:07:19.625
play for 4 years. Um, and then
>>Playing is way more fun then
run- >>Why we kept running it

01:07:19.625-->01:07:26.250
was more or less, well, we
screwed that one up, maybe it’ll
be better next year. Um, I think

01:07:26.250-->01:07:30.375
we might have got it, we got it
close to right the >>Your last
year >>Fourth time >>Was good

01:07:30.375-->01:07:34.250
>>Yeah we got it close to right
the four- it actually started on
time. >>It did? You didn’t

01:07:34.250-->01:07:37.292
delete all of our scores.
>>Yeah, so um >>Chr, Chr, Chris.
Before, before you, before you

01:07:37.292-->01:07:41.542
stop Chris, uh, how did it
change your team’s experience,
your school’s experience to go

01:07:41.542-->01:07:46.833
from playing, to running it, to
playing. What d- do you play
differently because you ra-,

01:07:46.833-->01:07:53.792
because you ran it. >>No, uh,
playing to running is a really
tough thing so uh, to e- it’s an

01:07:53.792-->01:07:59.375
entirely different mindset to
become an organizer. Right, so
yeah, if you’re, if it you’re

01:07:59.375-->01:08:03.750
doing ya know different type
stuff and then now you’ve got to
turn around and write it. You’ve

01:08:03.750-->01:08:07.417
got to write with an entirely
different mindset. Uh writing a
challenge is no easy task. I-

01:08:07.417-->01:08:12.167
it’s easy to put one bug i- in
the challenge of a very specific
nature. It’s a little more

01:08:12.167-->01:08:15.708
difficult not to put other bugs
in that you >>it’s really
>>Didn’t mean to >>Yeah, it’s

01:08:15.708-->01:08:21.125
really hard to write secure
insecure code. [laughter] >>Yeah
>>Just, just exploitable enough

01:08:21.125-->01:08:24.583
>>Yeah [laughter] and uh >>And,
and >>So >>How many of you have
found exploits in code you’re

01:08:24.583-->01:08:29.042
adapting? Cause I know I was
finding exploits out there, it’s
like wow that’s been vulnerable

01:08:29.042-->01:08:32.333
since the 90’s. >>We actually
>>I’m the first one >>Had 2
challenges that, that literally

01:08:32.333-->01:08:36.083
just incorporated a library that
we knew had an o’day in it.
>>Yeah and we wrote challenges

01:08:36.083-->01:08:40.417
like that too. You take things
from real world and you try to
bake it into this challenge, uh,

01:08:40.417-->01:08:44.083
w- we didn’t want to drop
mountains of software, like ok
here’s a patch you find the

01:08:44.083-->01:08:50.708
o’day. Uh [laughter] which was
easier, ya know, 10 years ago,
but uh, eh so yeah everything

01:08:50.708-->01:08:55.042
was pretty stripped down and
you, you try to build the bug
you want people to hit and you

01:08:55.042-->01:08:59.583
try not to have other bugs. Um
so, so from running to
organizing th- that mindset was

01:08:59.583-->01:09:05.875
hard for some people to, to uh,
uh a- adapt to. So, we did lose
a lot of people actually from

01:09:05.875-->01:09:08.667
the, from the playing side.
They, I- that’s what they want
to do. They want to keep on

01:09:08.667-->01:09:13.417
playing. I don’t blame them. Um,
but yeah, we were a much smaller
group when we uh, during the

01:09:13.417-->01:09:17.667
time we spent running it. >>That
tells into another question.
This things actually working so

01:09:17.667-->01:09:23.625
we’re getting questions that are
coming in. Uh so, uh you wanna
answer HJ >>Oh sure >>And then

01:09:23.625-->01:09:25.500
I’ll follow >>I think I think uh
>>Into the next question >>Short
and sweet anything he can do, I

01:09:25.500-->01:09:32.292
can do better. [laughter] >>We
played um TEK’s game >>Uh and
you got 2 black badges out of

01:09:32.292-->01:09:37.208
it, uh sorry. >>We played their
game, got 2 black badges and saw
what they did. And knew how we

01:09:37.208-->01:09:42.458
were a**holes and how to beat
ourselves. And decided we could
make an interesting game.

01:09:42.458-->01:09:45.667
>>Masochism >>What was that? >>I
was just gonna say it all boils
down to masochism. >>Yeah, yeah,

01:09:45.667-->01:09:50.333
like you know what, yeah we, we
made people upset about how we
beat their defenses. Anyway, um

01:09:50.333-->01:09:54.625
so yeah same, actually same
reason. Like we played their
game and thought we could do

01:09:54.625-->01:09:59.417
better or do things differently
and take an it an interest,
interesting direction for the

01:09:59.417-->01:10:03.333
community. So, >>And, and
they’ve done an amazing job and
I will say that one thing as an

01:10:03.333-->01:10:06.542
organizer, the only thing I’ll
add is the only fun you can have
in organizers is trying to f**k

01:10:06.542-->01:10:10.167
with the players. >>Totally
true, absolutely 100 percent
true. >>So whether it’s a

01:10:10.167-->01:10:13.958
challenge you designed or some
new twist, the badge challenge,
the >>I really support >>Get a

01:10:13.958-->01:10:21.500
kick out of >>Or challenges, or
challenges that were actually
purposely not actually

01:10:21.500-->01:10:23.417
exploitable. There was a bug but
it, like couldn’t be landed.
>>No w- we, that’s why, our are

01:10:23.417-->01:10:25.708
always exploitable. [laughter]
You thought there was a bug and
now you want to say you were

01:10:25.708-->01:10:29.208
f**king the players. >>What’s
that? >>Yours are always
exploitable are zero. >>Oh that

01:10:29.208-->01:10:33.000
was an accident. >>For one team,
yeah >>For one team, yeah that
was th- it was exploitable. >>So

01:10:33.000-->01:10:36.833
the follow-on question which uh
perhaps I’ll ask uh Chris first
since you already started

01:10:36.833-->01:10:40.833
answering it. Um, how big are
the organizing teams? And you
can sort of take that with, with

01:10:40.833-->01:10:45.708
the transition from player to
organizer and everybody else can
answer it. >>Well we went from

01:10:45.708-->01:10:53.792
about 900 players down to
[laughter] uh, you’d probably
accord it topped out around 10.

01:10:53.792-->01:11:00.208
But you know, but there was you
know even a subset to that, that
was a little more active. But uh

01:11:00.208-->01:11:02.542
for the 4 years we ran it. >>We
were around 10 as well, uh
>>Yeah, we, Kenshoto was around

01:11:02.542-->01:11:05.625
8, we had a couple outside
contributors. >>Yeah but
composed is funny, like people

01:11:05.625-->01:11:10.292
who write challenged people who
organize, the guy I knows
actually the head, uh he’s, he’s

01:11:10.292-->01:11:15.125
more like having to deal with
uh, Def Con stuff. We’ve got one
guy who's just an amazing

01:11:15.125-->01:11:19.333
network guy. No joke, he is the
person that Cisco calls when
they can’t answer a problem. Not

01:11:19.333-->01:11:27.375
even j- not even joking. >>Yeah,
we were um, as many as 14 but we
did ya know we made movies, we

01:11:27.375-->01:11:31.167
did all sorts of stuff but it
was really only I think um 6 or
8 people that were the core of

01:11:31.167-->01:11:37.000
our team. >>You were 1. >>Yeah
>>Yeah >>My- Myles was >>Army of
1 >>At, at the end was it still

01:11:37.000-->01:11:42.208
1 or you drawn in a couple
people at the last [inaudible]?
>>It was 1 >>Yeah >>And that was

01:11:42.208-->01:11:47.542
why it was time to hand it on.
>>Yeah, you lasted, you lasted
longer, you’re a good many

01:11:47.542-->01:11:52.417
Charlie Brown. Um, so it turns
out that this thing allows you
to vote, there’s actually like

01:11:52.417-->01:11:59.042
most popular questions. Um, what
is the most unexpected way
someone solved a challenge? So,

01:11:59.042-->01:12:03.750
there’s like an intended path
and then there’s like unintended
paths, so. >>Yeah there’s a lot

01:12:03.750-->01:12:08.125
of those. >>I mentioned they
used free BSG jailbreak
[laughter] to get out of our

01:12:08.125-->01:12:12.208
jails. But, the we, we watched
them do it and they were so
inept when they go out. >>Yeah,

01:12:12.208-->01:12:16.250
people, people bringing o’days,
people bringing o’days and then
completely messing up using them

01:12:16.250-->01:12:24.250
correctly, would be a, that’s a
good one. >>My favorite one was
I had managed to let’s see,

01:12:24.250-->01:12:30.708
Quake was the hot game in Def
Con 4, 5 so I managed to talk
its software to donating Quake

01:12:30.708-->01:12:37.167
licenses to the Capture the Flag
contest. Like Quake servers and
a couple station setups so that

01:12:37.167-->01:12:42.375
people could play Quake and then
I got free video games out of
it. So, all good. Um and so

01:12:42.375-->01:12:46.583
someone came up to me and said,
you know we’ve dialed servers
but it’s not on the network. We

01:12:46.583-->01:12:49.875
can attack the quake server and
one of the people gets so p**sed
off he goes over and reboots the

01:12:49.875-->01:12:58.583
machine and that causes it to
reboot so we can run our RC
script. I said, go for it. >>We

01:12:58.583-->01:13:05.625
uh, we convinced a team to
surrender. [laughter] Give us
all their points by telling them

01:13:05.625-->01:13:10.667
that we were about to win even
though they were ahead. And so
they joined and all became

01:13:10.667-->01:13:14.042
Ghetto Hackers. >>Actually, I’m
gonna, I’m gonna relay uh
something we were talking about

01:13:14.042-->01:13:17.875
earlier act- an anecdote that we
were talking about specifically
in the Ghetto Hackers playing.

01:13:17.875-->01:13:23.333
So, it wouldn’t be an instance
of Caezar having seen this occur
as a novel way to score points

01:13:23.333-->01:13:27.458
to win the game, um but having
executed a novel way to win the
game, uh they basically

01:13:27.458-->01:13:34.292
convinced the CTF organizers at
the time that they needed, they
needed store a half rack of

01:13:34.292-->01:13:39.500
equipment that they had, so they
had ya know, your little like uh
what would it be 20, 20-ish u

01:13:39.500-->01:13:47.250
rack. Um, and they had a
completely built out entire
faceplates of real servers and

01:13:47.250-->01:13:50.500
stuff and they like stuffed a
person into the half rack and
actually got them to store it in

01:13:50.500-->01:13:54.167
the room with all the rest of
the CTF equipment and
infrastructure overnight.

01:13:54.167-->01:13:58.458
Because, well we need a place to
put this, right? And so
obviously out creeps person

01:13:58.458-->01:14:04.292
>>Single roots all the machines
>>Single roots all the boxes
>>Reboots them all >>Peace

01:14:04.292-->01:14:08.417
>>Puts them in single user mode,
roots them all. We come back in
the morning and then not

01:14:08.417-->01:14:12.125
prepared or worked or done
anything the night before. We
come in with like, I, we’re

01:14:12.125-->01:14:20.000
pretty been drinking all night,
uh the contest is over so let’s
all go home. And we got a whole

01:14:20.000-->01:14:28.042
bunch of points and it was, yeah
it was pretty good. The contest
was not over. >>Oh no I >>We did

01:14:28.042-->01:14:30.417
not go home >>I already
mentioned the, the bashrc one
which was fantastic. When we

01:14:30.417-->01:14:34.042
were playing, we once snuck the,
the root search for one of the
teams and got first blood on

01:14:34.042-->01:14:42.458
every service. That was during
your, your game. >>Uh, this
worked pretty good I can just go

01:14:42.458-->01:14:48.167
to the next most popular one.
Um, so how do you come up with
unique uh flags or challenges I

01:14:48.167-->01:14:53.833
think, uh I think it means
challenges or services uh every
year, aside from following CVE’s

01:14:53.833-->01:14:57.583
or meeting content from other
>>That was actually a serious
source of exhaustion. >>Is is

01:14:57.583-->01:15:00.833
>>Like >>Actually >>Like mental
exhaustion of attempting to come
up with services. And we used to

01:15:00.833-->01:15:04.750
even like, we would literally
scrap 2 or 3 services every year
because they wouldn’t’ get

01:15:04.750-->01:15:08.042
deployed correctly or they
wouldn’t get done. But like
coming up with like unique idea

01:15:08.042-->01:15:12.042
of like, uh here’s a web service
that helps you make a sandwich
like or whatever. >>Well and,

01:15:12.042-->01:15:15.667
and now with 20 years of Def Con
history >>Exactly, now you wanna
like do something original like

01:15:15.667-->01:15:20.583
>>And that’s just Def Con
history that’s not talking about
every other CTF >>All the other

01:15:20.583-->01:15:24.500
CTFs yeah, for sure. >>100 plus
probably per year, how do you
actually be unique, you can’t

01:15:24.500-->01:15:27.292
really be unique anymore. >>NO
definitely now >>That’s why >>We
would actually, we would

01:15:27.292-->01:15:31.583
actually basically, the real
truth of the story how we
produced our services is mostly

01:15:31.583-->01:15:35.500
about getting a bunch of people
in a room, getting a bunch of
whiskey in the room too, and

01:15:35.500-->01:15:40.000
like just bulls**tting until
funny ideas came out. And then
like codifying that list into

01:15:40.000-->01:15:44.250
services. And then, and then
going in and deciding what kind
of exploit goes into it, what

01:15:44.250-->01:15:48.667
kind of whatever. But every one
of those services had a little
>>Oh and for us, what we’ve done

01:15:48.667-->01:15:53.083
is we’ve really introduced
multiple architectu-
architectures. Uh, 3 years ago I

01:15:53.083-->01:15:59.833
think we had 5 different
architectures uh, in 1 game and
it was just XX6 and, and NIFs

01:15:59.833-->01:16:05.750
there was, we even had a Windows
IoT arm challenges running a
powerPC shell, or something like

01:16:05.750-->01:16:09.875
that, I don-. And this year of
course I’ve mentioned we have
the, the custom architecture

01:16:09.875-->01:16:13.708
that we built. But it really is,
we’ve had to push it >>Nobody’s
quite explained this. Custom

01:16:13.708-->01:16:18.000
architecture means >>Oh right
>>A system for storing bits in
electronics [laughter] >>Right

01:16:18.000-->01:16:21.500
>>So imagine >>Instruction
stacks >>They implemented a CPU
>>Processor >>Yeah we

01:16:21.500-->01:16:28.583
implemented a CPU >>A processor
of VM all up on top of nothing.
It’s, it’s um whole cloth, it’s

01:16:28.583-->01:16:33.458
fantastic. >>Lightening actually
did it, but seemed to handle it.
>>Epic >>Yeah it is pretty good,

01:16:33.458-->01:16:37.708
um yeah, it’s pretty much, we’ve
had, and the caliber of people
who are playing are just

01:16:37.708-->01:16:41.500
phenomenal, like it, they
frighten me. People like Load
Key and GeoHash, scary good. So

01:16:41.500-->01:16:49.708
the, the challenges, the level
that we have to do for the
difficulty has astronomically

01:16:49.708-->01:16:54.375
increased. Um the, the medium
challenges back in the day are
now easy challenges for

01:16:54.375-->01:16:58.958
everybody. >>Yeah, tools, tools
have gotten so much better too
you don’t want to write a, a

01:16:58.958-->01:17:04.417
challenge that’s gonna be auto
solved by a tool, under, ya know
under a minute. >>Auto solve you

01:17:04.417-->01:17:06.667
just, yeah >>That’s a good
segway for >>You have to keep up
with the state of the tools and

01:17:06.667-->01:17:13.500
uh try to find the weaknesses in
the tools so that the, you still
will get the human you just

01:17:13.500-->01:17:18.833
gotta do the deep dive. >>That’s
a good segway into the next,
actually set of questions.

01:17:18.833-->01:17:23.958
There’s uh several related to
uh, CGC and, and sort of
automation. Um lessons learned

01:17:23.958-->01:17:28.500
from, from Cyber Grand
Challenge. It’s been a year, was
it a success. >>Hackers are

01:17:28.500-->01:17:34.083
obsolete [laughter] >>Failure,
I’m looking for one, I sorted
these and now I can’t find one

01:17:34.083-->01:17:40.375
that I saw earlier. There’s this
one, mostly for Visi, how many
years until a computer wins Def

01:17:40.375-->01:17:45.542
Con CTF, right? >>Uh, that’s.
>>So the, the thing that the,
the thing the computers have

01:17:45.542-->01:17:50.417
trouble with right, is the dirty
tricks department. Um, and so I
think it will be awhile unless a

01:17:50.417-->01:17:56.542
game is designed specifically
for the computer system, I mean
that’s really, that’s really

01:17:56.542-->01:18:00.250
kind of where we are at like
right now I think CGC was a
really good example of this just

01:18:00.250-->01:18:05.042
massively forward in technology
but at the same time you have to
keep in mind that um it was a

01:18:05.042-->01:18:09.417
reduced instruction set and a
reduced sys call set and all
these other things that add a

01:18:09.417-->01:18:13.250
really sufficient level of
complexity that I think there’s
still a lot of room, a lot of

01:18:13.250-->01:18:18.792
wiggle room for like the human
dirty tricks department kind of
mindset. So, I think it’ll be a

01:18:18.792-->01:18:23.625
little while yet. But it depends
kind of on the structure of the
game, so if elements of the game

01:18:23.625-->01:18:28.333
require creative th- require
creative thought and unique
approaches um, I think that’ll

01:18:28.333-->01:18:33.167
continue to kind of be an arms
race. And, and >>I only think it
matters if they have arms. If

01:18:33.167-->01:18:37.667
you have to actually go walk,
pick something that the computer
won’t do that well. >>Well and

01:18:37.667-->01:18:41.167
actually that’s a really good
example, because, because you
guys were notorious specifically

01:18:41.167-->01:18:46.292
for incorporating into the CTF a
whole bunch of side games, where
those side games were like,

01:18:46.292-->01:18:51.500
every team gets issues this
ancient hard drive that was like
gigantic and, they’re gon- that

01:18:51.500-->01:18:55.500
hard drive is painted your
team's color and it’s going to
be out at the Def Con shoot

01:18:55.500-->01:19:01.417
which I think still happens,
right? >>Myles >>Yeah >>Myles
>>Yeah that, the incorporate Def

01:19:01.417-->01:19:04.750
Con shoot was mine, >>Right
>>That turned out to be a really
bad idea [laughter] >>Right,

01:19:04.750-->01:19:08.958
well so basically I think a team
got a, a team got some bonus
points for like having like

01:19:08.958-->01:19:13.458
basically punched a hole with a
bullet like through the most
center point of the drive or

01:19:13.458-->01:19:17.958
something like that. Um,
computers aren’t gonna do that
for a little while yet, so.

01:19:17.958-->01:19:23.042
>>Th, the other aspect with
computers is you have to freeze
the game, right chess is,

01:19:23.042-->01:19:26.792
doesn’t change. Chess is chess,
is been chess forever and a
computer knows how to play chess

01:19:26.792-->01:19:30.375
and if you wanna >>How, how
would the computer, how would
the computer deal with this new

01:19:30.375-->01:19:33.667
architecture, >>Yeah, so
>>Right? >>You know >>Read the
manual >> We wanna see something

01:19:33.667-->01:19:36.375
different every year, we wanna
throw curve balls every year and
its, unless you advertise that

01:19:36.375-->01:19:41.083
some amount of time in advance,
like the CGC version that you
guys did last year, uh it's, I

01:19:41.083-->01:19:46.458
think in Def Con CTF in
particular I don’t think we’ll
ever see it because that’s not

01:19:46.458-->01:19:50.750
the game we humans want to play.
>>All, all we’ll ever need is
64K of RAM right? [laughter]

01:19:50.750-->01:19:57.667
>>So uh, it, it probably wasn’t
clear earlier but 4 people up
here were involved in, in the

01:19:57.667-->01:20:03.750
CGC competition some way and
then for those that aren’t aware
Def Con CTF last year was mostly

01:20:03.750-->01:20:09.250
more or less CGC compatible,
right and the winner of CGC the,
the machine was a player in the

01:20:09.250-->01:20:14.417
CTF competition last year which
kind of ties all these questions
back into the panel. So, uh it’s

01:20:14.417-->01:20:19.250
sort of a related question and
um I won’t direct at anybody in
particular. But, so it’s been a

01:20:19.250-->01:20:23.958
year right, CGC was last year
and the CTF that had the
computer was um, was a year ago.

01:20:23.958-->01:20:30.333
Um, was that a, a success, a
failure was it sort of um, a, a
>>I would say that >>History >>I

01:20:30.333-->01:20:34.750
would say it was definitely a
f-, a success. I really am a fan
of the fact that uh anger was

01:20:34.750-->01:20:40.917
really open sourced. Um, I think
>>Pushed tools >>Right getti-
th- getting these tools out to

01:20:40.917-->01:20:45.208
th- the general community is
fantastic. The fact that anybody
can go and use these and learn

01:20:45.208-->01:20:52.708
how to write software, write
tools, to automatically RE and
go towards uh exploitation is

01:20:52.708-->01:20:58.667
fantastic. Um, when you can m-
lower the bar s- uh to entry it
really makes it that much

01:20:58.667-->01:21:05.625
better. >>Learning uh, learning
how the teams did their job last
year is probably the best thing

01:21:05.625-->01:21:11.875
that anyone in the audience, any
of us could imagine doing for
their career for the next year.

01:21:11.875-->01:21:17.083
>>An, and to be honest even
>>State of the art >>Uh, th-
future of ARRI just in normal

01:21:17.083-->01:21:21.292
careers is gonna be automated,
it’s go- software’s becoming so
incredibly difficult, some

01:21:21.292-->01:21:26.500
amount of automation is going to
be required, just to get even
the low hanging fruit anymore.

01:21:26.500-->01:21:31.458
So incor- we’re actually, we
have to incorporate this idea
into CTF so that we can still be

01:21:31.458-->01:21:36.417
representative of the wider
security community. >>Yeah, I
think that’s, the mistake people

01:21:36.417-->01:21:40.625
make when they look at the CGC
is to think that the goal was to
build a purely autonomous system

01:21:40.625-->01:21:44.875
and that wasn’t the goal. The
goal was advance the state of
the art in software a- analysis,

01:21:44.875-->01:21:49.500
and wh- what, ya know we’ll see
last, we saw last year, we’ll
see it this year, is that uh,

01:21:49.500-->01:21:54.917
software automatons assisting
humans make it, ya know making
humans better at what they’re

01:21:54.917-->01:21:58.125
doing, is is probably where
we’re going. The best chess
playing systems in this world

01:21:58.125-->01:22:01.417
are hybrid systems that pair
computers with the best players.
>>Yep >>Not, not purely

01:22:01.417-->01:22:06.708
computers. >>Yeah, I think, I
think we’ve seen that too in
like, the a, even other CTFs now

01:22:06.708-->01:22:11.333
all with modern things like
anger being applied in an
automated way to catch some of

01:22:11.333-->01:22:15.833
the low hanging fruit or point
out areas of the code that’s
like, this probably like, we

01:22:15.833-->01:22:19.625
don’t, it can’t necessarily
automatically generate an
exploit for every condition yet.

01:22:19.625-->01:22:23.458
>>So we’ve actually had
challenges that required um,
anger or some automation,

01:22:23.458-->01:22:26.458
automated reverse engineering.
That was >>Absolutely >>Like I
think it was called a Thousand

01:22:26.458-->01:22:30.417
Cuts by Vito. Where you do, you
were given a thousand binaries
really fast and you had to be

01:22:30.417-->01:22:34.750
able to exploit them in computer
speed, not human speed. Um
there’s even a challenge on

01:22:34.750-->01:22:38.958
pwnable dot KR, called AEG where
it requires you, you to download
it and it gives you a random

01:22:38.958-->01:22:46.875
binary every time and you have
to, uh auto RE it and that,
write the exploit. >>So how

01:22:46.875-->01:22:52.208
international has CTF become?
Um, when did international teams
start showing up and do

01:22:52.208-->01:22:56.375
Americans still stand a chance?
>>One of the, one of the first
uh things that I remember

01:22:56.375-->01:23:03.000
happening was uh, Dillon
Kanabraun from Hack in the Box.
Uh reached out and asked uh, if

01:23:03.000-->01:23:10.000
he could take CTF obviously, do
it yourself I mean do whatever
you want but, uh that was uh

01:23:10.000-->01:23:18.167
taking CTF to Kuala Lumpur,
Asia. Uh, in I think 2001. So
it’s, it’s been coming for a

01:23:18.167-->01:23:23.000
long time, it’s been s-
diversifying and spreading
rapidly uh, it’s I think one of

01:23:23.000-->01:23:29.250
the most true and honest ways
that hackers can really be
better than each other. Because

01:23:29.250-->01:23:35.833
we’re not very good I think
otherwise at knowing like am I
good enough. And being able to

01:23:35.833-->01:23:41.542
say, well I’m better than that
guy is a huge, huge uh
foundation to stand on >>Yeah,

01:23:41.542-->01:23:45.125
we have uh, competitors from all
over. We’ve got Korean and
Taiwan, Taiwanese and Chinese

01:23:45.125-->01:23:51.458
and m- most major areas of, of
the world are representative,
like are represented and um,

01:23:51.458-->01:23:59.958
they actually, 2 years ago Def
Corp from was it Korea, actually
won our game, so it was a, one

01:23:59.958-->01:24:04.458
of the first times that I can
remember, and outside one, they
were mostly Danish um that

01:24:04.458-->01:24:08.875
they’ve won CTF. >>I remember
actually it kind of, it kind of
like goes back to some of the

01:24:08.875-->01:24:12.333
slides you were talking about
earlier with the language
barrier stuff. I think the first

01:24:12.333-->01:24:19.333
like fully like non-English
speaking team um was, was f- was
a couple of the Korean teams

01:24:19.333-->01:24:23.167
that were fielded early on in
the Kenshoto CTF and like we
actually had to really

01:24:23.167-->01:24:27.500
specifically sit down and think
like, what are we gonna do about
this? Because we don’t want to

01:24:27.500-->01:24:32.250
create a game, because the game
isn’t about English, the game is
about bits, right? And so like

01:24:32.250-->01:24:37.000
we actually tried to create like
e-, we actually went through
several iterations of attempting

01:24:37.000-->01:24:41.958
to create like pictographic
representations of like stealing
a key, key submission and things

01:24:41.958-->01:24:45.417
like that with like, like you
know stick figures and stuff
like that, because the language

01:24:45.417-->01:24:50.875
barriers, is so significant
that, um, I remember actually
one of the um, the captain of

01:24:50.875-->01:24:55.458
the Korean team at the time, um
they did amazingly well in
quals. They like whomped

01:24:55.458-->01:25:00.042
everyone in quals that year.
They show up, uh at the actual
game and they had a lot of

01:25:00.042-->01:25:04.458
trouble but it was mostly about
understanding what was going on,
and the and the actual mechanism

01:25:04.458-->01:25:08.208
of the game. And the, the
captain of that team came up to
me and was like, I think this

01:25:08.208-->01:25:14.292
next year we’re gonna work on
hacking English. >>Nice >>Uh
some of those stickers, I think

01:25:14.292-->01:25:18.583
that might have become Ken’s
stickers later on. >>Yeah, no
they were totally stickers. >>So

01:25:18.583-->01:25:21.583
at the top there you can see
>>Oh yeah, actually, no that’s,
that’s exactly some of what I’m

01:25:21.583-->01:25:25.792
talking about. We were like,
yeah. >>So, I mean clearly you
didn’t make those right then but

01:25:25.792-->01:25:33.125
like that became a thing for
later. Um so I guess speaking of
that uh, what’s with all the,

01:25:33.125-->01:25:38.167
the Japanese or like sort of
Asian references, right? So,
like when Myles ran it, it had

01:25:38.167-->01:25:43.958
like the big >>It’s his fault
>>The, the red uh >>Did ever
pick out uh, Myles >>Japanese

01:25:43.958-->01:25:48.125
thing, right >>No I think that
was actually >>It was Caezar
>>Back that up, oh ok >>We did,

01:25:48.125-->01:25:55.333
uh we decided that we wanted to
give away a championship belt.
So, we gave away the Root Fu

01:25:55.333-->01:26:00.875
Championsh- it, it was like a
boxing belt, like >>Like a
literal like WWF like belt

01:26:00.875-->01:26:08.750
>>Yeah, yeah, so we made up this
whole concept of Root Fu. And it
was gonna be, um ya know a

01:26:08.750-->01:26:12.667
measure of how good you could
compete against other people.
And it basically turned into

01:26:12.667-->01:26:19.458
hacking and and, and the whole
thing fell down but as a theme.
You know we had um, uh all the

01:26:19.458-->01:26:24.250
cultural references, in, in, in
Blade Runner and hackers, and I
mean the connection to

01:26:24.250-->01:26:29.667
>>CyberPunk >>Yeah CyberPunk, uh
yeah Chatsubo. Uh I think I was
just kind of the gestalt hacker

01:26:29.667-->01:26:35.542
meme at the time. Um as were a
whole lot of bad ideas and
things that we’re growing past.

01:26:35.542-->01:26:43.042
So, we don’t have to live in
that world forever but uh I
think it was just the way that

01:26:43.042-->01:26:45.833
things uh, everything grew up
then. >>We kind of just followed
suit. I mean Kenshoto actually

01:26:45.833-->01:26:49.625
the like the name is in Japanese
so. For us there was that tie in
but it was mostly out of the

01:26:49.625-->01:26:55.625
influence of that same sort of
cyber punk picture that you guys
had created. >>And, and one of

01:26:55.625-->01:27:02.792
our uh, founding members uh
Praisin’ Angel uh is Korean, uh
woman who uh fits in a half rack

01:27:02.792-->01:27:09.333
it turns out. [laughter] Uh, and
she and uh Data Angel uh did all
of the what we call the Ghetto

01:27:09.333-->01:27:14.875
News Network. We did um news
videos that were kind of in um
the Blade Runner if you remember

01:27:14.875-->01:27:19.250
the movie the Blade Runner on
the screens there’s uh, some
kind of Asian commercials I

01:27:19.250-->01:27:23.333
think over like a soda. Um
>>I’ve seen it yeah. >>We kind
of took that image and we just

01:27:23.333-->01:27:28.792
blew it out and made a whole
fake news system. >>They had her
basically congratulating teams

01:27:28.792-->01:27:35.833
on not f**king up the network as
I recall correctly. >>Uh, yeah
so I just, had some back up

01:27:35.833-->01:27:39.958
slides to show some of that,
there’s the, I can’t remember
what it's’ called but it’s like

01:27:39.958-->01:27:45.208
the Japanese for uh, um, uh
something religious the red
thing in the background there.

01:27:45.208-->01:27:52.958
And uh, certainly the Japanese
stuff continues uh over time.
Um, so what’s the oh I wanted

01:27:52.958-->01:27:58.917
to, this one. So how do you find
middle ground uh, in the
challenges between the ones that

01:27:58.917-->01:28:04.333
are too easy and the ones that
are too hard? >>Everything is
too easy. >>Yep >>Hackers are so

01:28:04.333-->01:28:08.000
much better when they’re under
pressure than we are when we’re
sitting around trying to be

01:28:08.000-->01:28:13.292
cool. >>The way our, we had to
modify our scoring algorithm, w-
the way we do it, is uh so for

01:28:13.292-->01:28:17.750
quals everything starts out at 5
points except for baby's first,
and the more people solve it the

01:28:17.750-->01:28:22.250
less it becomes worth. So it
just kind of, they were
self-correcting. >>Yep >>Cause

01:28:22.250-->01:28:25.917
we, we’re it’s funny if you
write a challenge it’s really
hard to judge how hard it is

01:28:25.917-->01:28:29.667
when you write it yourself.
>>So, so wait it’s 5 points when
I get it but then if so many

01:28:29.667-->01:28:33.292
people follow along, then I lose
my points? >>Right so its >>Yeah
that’s fantastic >>We also made

01:28:33.292-->01:28:37.125
>>So much smarter >>We made
other people in Kenshoto g- um
actually do all the challenges

01:28:37.125-->01:28:41.333
with no prior knowledge of them
and so >>Testing >>Yeah testing,
essentially >>Time is the

01:28:41.333-->01:28:44.542
problem >>Because the problem is
it’s also really it's really
easy to implement a challenge

01:28:44.542-->01:28:48.875
where there’s some like leap of
faith that you didn’t realize
you made and then other people

01:28:48.875-->01:28:53.417
wouldn’t and aren’t part of the
logical analysis progression.
And so that being the case like

01:28:53.417-->01:28:57.667
we, we had found challenges,
actually there were 1 or 2 in
your guys’ game too, and

01:28:57.667-->01:29:01.542
there’ve been others and
specifically in, usually in
quals rounds. Where like there’s

01:29:01.542-->01:29:04.958
some like leap of faith required
if you, actually I think the, I
think the best examples are

01:29:04.958-->01:29:11.833
listen, listen to the crazy a**
explanations of the Def Con
badge challenges where they’re

01:29:11.833-->01:29:17.208
like, I realize that these dots
were actually morse code of
GEOlocation things that were

01:29:17.208-->01:29:21.417
airports that then if you
arrange the airports and then
reverse their letters and then

01:29:21.417-->01:29:24.417
just all these things that
you’re like, why did you think
to do that in the first place?

01:29:24.417-->01:29:28.500
And so, like we didn’t, we
didn’t want to have challenges
that were, that were, why did

01:29:28.500-->01:29:31.792
you think to do that in the
first place. We wanted there to
be this sort of thread through

01:29:31.792-->01:29:38.000
them and, and essentially the
way we imp, enforced that was
just entirely, here you go other

01:29:38.000-->01:29:41.917
person on Kenshoto like go break
this and solve it. >>It’s also
really easy to make questions

01:29:41.917-->01:29:47.667
that are easy for your top 1
percent. And so, it’s like no
one gets it and the top 1

01:29:47.667-->01:29:51.375
percent gets it in 5 minutes and
then where’s the rest of the
contest? >>Well but that’s, so

01:29:51.375-->01:29:54.542
for the, for the real game
that’s what we were shooting for
cause we were trying to we were

01:29:54.542-->01:29:58.917
basically, if anyone was topping
out that R and solving all the
challenges they weren’t hard

01:29:58.917-->01:30:04.458
enough, right? So, for us, we,
we considered it a failure in
any year if someone solved all

01:30:04.458-->01:30:09.375
the challenges. And so, I think
that they like the pushing of
that scale to, to being higher.

01:30:09.375-->01:30:14.625
We didn’t really concern
ourselves in the real game with
that approachability or, or

01:30:14.625-->01:30:18.250
middle ground but in quals we
did because that’s the thing
that lie everybody gets to play

01:30:18.250-->01:30:22.542
everybody gets to be inspired
and do something, right >>One of
the things that we did for quals

01:30:22.542-->01:30:27.250
was we really, prior to us they
had things lie forensics, trivia
and because those weren’t really

01:30:27.250-->01:30:33.792
necessary for the actual finals
game, um we only did, ya know
Puntables, we did like gorilla

01:30:33.792-->01:30:39.167
programming, things that will,
you have to know in order to
play, uh succeed in finals. So,

01:30:39.167-->01:30:44.333
forensics where you have to go
and take ya know the first bit
of every sector of the NTFS dri-

01:30:44.333-->01:30:49.542
or section ah that was terrible.
That was also before you did
flag open brace cl- closed

01:30:49.542-->01:30:53.458
brace. That’s when you were
guessing. >>Yeah, we left those
in because we wanted be, we

01:30:53.458-->01:30:58.542
wanted quals to be accessible to
a large number of people. >>Yeah
it was a conscious decision for

01:30:58.542-->01:31:02.333
us too >>That’s why we added
baby's first because we wanted
that same approachability

01:31:02.333-->01:31:11.083
without it still be you know
directly uh, ac- or uh be
directly applicable to the game.

01:31:11.083-->01:31:20.625
>>Playskool my first tp exploit
>>Yeah >>Um, you guys ever seen
evidence of collusion?

01:31:20.625-->01:31:25.208
[laughter] >>So, uh >>We
literally >>I was going to say
>>Colluded with teams >>Teams

01:31:25.208-->01:31:29.208
>>All the time >>Won via
collusion >>We only won by
collusion [laughter] >>Uh

01:31:29.208-->01:31:34.000
where’s, where’s Zeon is he
here? >>So >>There he is
[laughter] >>So I think at, at

01:31:34.000-->01:31:40.583
this stage in the game for us
most people have at least begin
treating it like a gentlemen’s

01:31:40.583-->01:31:47.208
game. Like, we genuinely want to
see who is the best and other
teams feel the same way. For

01:31:47.208-->01:31:51.500
example, uh a few years ago,
there was a team for some reason
who put a wiki up on the game

01:31:51.500-->01:31:55.750
network and that wiki had their
passwords on this, and one of
the, the teams actually the

01:31:55.750-->01:32:00.583
[inaudible] officers came up and
said hey guys, we saw these guys
had a wiki up you should

01:32:00.583-->01:32:04.333
probably tell them. And that was
during finals, where they could
have gone just completely wiped

01:32:04.333-->01:32:09.625
all their flags, and just ran
the board. But it’s really
become, uh, uh like I said a

01:32:09.625-->01:32:12.875
gentlemen’s game. So, for the
most part >>Whereas like,
whereas like in our game like we

01:32:12.875-->01:32:16.958
were owning other people’s
laptops and stuff too. Like if
it was connected to the network.

01:32:16.958-->01:32:20.917
>>So that’s sort of breaking
outside of the intended path,
but that’s not really collusion.

01:32:20.917-->01:32:24.875
Right? That’s >>No, so, but
there was, there was legitimate
team, there was legitimate team

01:32:24.875-->01:32:29.667
collusion at least in the, in
the, in one of the ghetto hacker
wins and, and uh, I mentioned

01:32:29.667-->01:32:33.417
Zeon and Evans, stuff like that
from back in the day. Because
it’s that’s actually really hard

01:32:33.417-->01:32:38.708
thing from a game mechanic to
detect and so we actually had,
so I mentioned the idea of like,

01:32:38.708-->01:32:42.083
ha, ha to mess with the teams we
had services that weren’t
exploitable but they were

01:32:42.083-->01:32:47.458
actually canary services because
if someone submitted a token
from that, from another team

01:32:47.458-->01:32:52.458
either the game mechanism was
broken and they managed to get
tokens they shouldn’t’ be able

01:32:52.458-->01:32:57.125
to or they’re colluding. So, we
were very concerned about that
because it’s almost impossible

01:32:57.125-->01:33:01.250
to totally prevent right. If
you’re just like slide a couple
of tokens by here and there

01:33:01.250-->01:33:06.292
between 2 teams to that they can
like get, get way far ahead you
make it so it’s a race between 2

01:33:06.292-->01:33:10.500
rather than 8 or 10 right.
>>Yeah for us the, the biggest
problem for collusion happens in

01:33:10.500-->01:33:16.375
quals. So, you have less control
over quals than what’s going on
in the game itself because it’s

01:33:16.375-->01:33:20.542
not on your own infrastructure,
they’re not in a room >>I
actually think uh the, our game

01:33:20.542-->01:33:24.917
mechanic that we do where if you
submit more than eh, s- submit
the keys, uh that the score

01:33:24.917-->01:33:30.875
drops for everybody. Um you’ll
see, so back when I, a- actively
played I had a team myself I

01:33:30.875-->01:33:36.875
would go and test the keys and
see if it worked. And however,
that’s actually a bad move now.

01:33:36.875-->01:33:40.625
Because if you were to do that,
you actually dropped the score
for everybody, including your

01:33:40.625-->01:33:44.250
own team. So, if you start
colluding there’s a possibility
that somebody will actually pull

01:33:44.250-->01:33:49.000
ahead of both of you. So,
>>Right >>Yeah because you end
up cutting those >>Strategy,

01:33:49.000-->01:33:53.250
right? Because now the people
that are, you could delude flags
that are more important to other

01:33:53.250-->01:33:56.292
teams. >>Yeah >>Right >>But if,
but if you >>HJ is that specific
to quals? >>This is specific to

01:33:56.292-->01:33:59.125
quals >>Yeah >>But if you’re on
top then you’re on top, right?
>>Right >>So that’s not gonna

01:33:59.125-->01:34:05.292
affect you much. >>I mean we, we
built collusion detection in the
quals because we had unique cha-

01:34:05.292-->01:34:09.000
challenges were tweaked in a, in
a small way or everybody had
their ya know, their own

01:34:09.000-->01:34:14.208
service, their own accounts.
>>Monitoring source IP addresses
of submissions and >>Yeah

01:34:14.208-->01:34:17.375
>>People would say I can’t log
into my account, that’s because
you’re using someone else’s

01:34:17.375-->01:34:20.458
creds and you’re not supposed to
see those creds unless you’ve
been sharing or the keys were

01:34:20.458-->01:34:25.875
different per team. So, we, we
did set up some services wh-
that were specifically to detect

01:34:25.875-->01:34:29.417
teams that were sharing answers.
>>It takes a massive amount of
work >>I th- I think the, the

01:34:29.417-->01:34:33.458
conclusion is, that uh as an
organizer you’re increasingly
playing game theory. We’re

01:34:33.458-->01:34:40.292
playing a game. Everybody else
is doing an activity. [laughter]
We make an activity and for us

01:34:40.292-->01:34:43.708
it’s a game and maybe that’s why
we like to >>So that’s why you
organize because you want to

01:34:43.708-->01:34:46.750
play the game. >>Kina yeah
>>Well we know the tricks for
the game, so we try to beat

01:34:46.750-->01:34:52.750
them. >>Yeah, for sure >>Uh so
here’s a perhaps interesting
question so why is, uh the

01:34:52.750-->01:34:58.333
finals uh attack defend style,
why don’t you just continue
Jeopardy style? >>Oh, because

01:34:58.333-->01:35:03.208
that’s not hacking. >>Yeah so
>>Because, because reality.
>>Yep >>Cause that’s bulls**t.

01:35:03.208-->01:35:07.708
>>If, If, if we wanted it to be
a puzzle game it would be a
puzzle game. We want it to be a

01:35:07.708-->01:35:11.458
hacking game. >>It, there’s
almost zero pressure in
Jeopardy, right? >>Exactly >>You

01:35:11.458-->01:35:14.333
sit back >>Exactly >>And you
answer questions and you’re in
your living room. >>Yeah

01:35:14.333-->01:35:17.625
>>Attack defense, you, you
there’s a reactive nature to
that and that, that can cause

01:35:17.625-->01:35:21.417
people to >>And, and, and we’ve
always wanted to make a contest
that everyone could start.

01:35:21.417-->01:35:27.958
Everyone could start down the
road to winning Def Con CTF and
we needed something that’s hard

01:35:27.958-->01:35:33.583
enough that it actually slows
people down who aren’t ready.
>>So, it adds another game

01:35:33.583-->01:35:39.208
mechanic in terms of, nice, for
example defense-like patching. I
can patch my binary but there’s

01:35:39.208-->01:35:43.625
a chance that it will now fail
poles. So. >>And the whole SOA
discussion from earlier. >>We’ve

01:35:43.625-->01:35:49.000
also added uh a game mechanic in
our game where we have a
concurrency for patches, so if

01:35:49.000-->01:35:54.417
you patch something and all you
do is add ya know 40 hex, 40 to
the st- side of the stack well

01:35:54.417-->01:35:58.208
that’s published now so everyone
can see what you just patched.
>>If you take nothing else away

01:35:58.208-->01:36:02.375
from this conversation that is
the piece of sheer genius that
we’ve been missing this whole

01:36:02.375-->01:36:08.792
time since we started playing.
>>Agreed >>Is when you add
defense everyone else has an

01:36:08.792-->01:36:13.375
opportunity to do the same. That
turns it >>Yep, I think really
into a game finally, for the

01:36:13.375-->01:36:20.333
players because now do I accept
your patches blind? >>Oh PPP,
like they definitely back doored

01:36:20.333-->01:36:26.875
patches, like they’ll, they’ll
patch something and then like,
yeah go ahead use our patches.

01:36:26.875-->01:36:30.458
See what happens. >>Yeah and
like purposely deploying patches
that then cause other people to

01:36:30.458-->01:36:33.208
try and analyze those to figure
out what was going on. Purposely
deploying a harmless patch that

01:36:33.208-->01:36:37.542
doesn’t actually necessarily fix
anything but sufficiently
complicates something >>Or

01:36:37.542-->01:36:41.125
>>That you use, that you use
resources of the other team
having to try and reverse

01:36:41.125-->01:36:45.250
engineer, like what the hell
does that patch mean? >>I’ve
definitely heard of, uh patches

01:36:45.250-->01:36:49.042
with Q&U bugs in them. >>Yep
>>Yeah y- y- you can create an
obfuscated code contest sub

01:36:49.042-->01:36:57.292
contest as a player now while
you use the system. >>Which,
which really ultimately becomes

01:36:57.292-->01:37:03.042
like team captain leadership
triage contest really because.
>>Who came up with this idea?

01:37:03.042-->01:37:06.708
>>So it was actually uh the guys
who did CGC. >>Yeah >>It was
there idea. >>It came from CGC

01:37:06.708-->01:37:17.792
>>Thought >>Just amazing, just
the most important thing that
happened. >>Um, how bout, what’s

01:37:17.792-->01:37:25.167
the uh, uh I can’t find the
exact wording, what’s the
hardest question or service that

01:37:25.167-->01:37:30.542
each of you have ever fielded in
your tenure? >>Man >>That’s hard
>>Super tough, yeah >>Yeah and,

01:37:30.542-->01:37:33.875
that’s eh no preparation you
just have to come up with it but
like which one of your >>I would

01:37:33.875-->01:37:38.667
say every new CTF is so much
harder than the old ones, that
listening to anybody but Hawaii

01:37:38.667-->01:37:44.625
John is probably a waste of time
cause the new contests are just
so much harder. >>Well so I’ll

01:37:44.625-->01:37:50.208
give you >>Uh, I, I invented a
CPU in order to write bugs on,
um uh, is such a high bar. >>One

01:37:50.208-->01:37:56.375
of our guys uh, Solier it was-
was uh, J Dub, um he wrote a
MUD. H- a full featured MUD,

01:37:56.375-->01:38:06.000
where you had to go in and
collect certain uh numbers of
items from each, like from b- uh

01:38:06.000-->01:38:11.458
NPC’s and then that’s your shell
code. So th-, in the place,
then, then you actually cause,

01:38:11.458-->01:38:18.042
you trigger the overflow and the
number of items you have of
different types becomes your um,

01:38:18.042-->01:38:21.708
your code. [laughter] Yeah, it’d
be your shellcode direct. >>So
we had a MUD, we had a MUD but

01:38:21.708-->01:38:27.875
you could hit A L a lot of times
and then hit enter and it would
crash. [laughter] >>Yeah no

01:38:27.875-->01:38:32.625
>>See the game is not the same
thing >>I- well what was really
fun, he put, uh he made us all

01:38:32.625-->01:38:38.125
God mode characters, um so we
get to walk around in it and, it
was really funny because if

01:38:38.125-->01:38:42.708
somebody attacks you, you can’t
stop attaching back it just
happens automatically so

01:38:42.708-->01:38:46.583
eventually you realize, alright
we don’t hit these guys. Because
if you hit them I just die >>We,

01:38:46.583-->01:38:50.708
we had a, we had a MUD a couple
years but it was all just about
like uh the social aspect of

01:38:50.708-->01:38:55.708
quals it was where everybody
would talk and, and hang out and
stuff during quals, um but

01:38:55.708-->01:38:59.958
effectively we ended up like
killing all the players a couple
of times by accident, like some

01:38:59.958-->01:39:03.708
monster that one of the other
Kenshoto people was creating
like got loose and we didn’t, I

01:39:03.708-->01:39:07.208
don’t know. >>Oh it S-, it was
S-, so Seer Goon the guy that
wrote the MUD is also the guy

01:39:07.208-->01:39:13.542
who uh developed the badge that
you saw up there. So that badge
he designed and we floated

01:39:13.542-->01:39:18.750
ourselves on a hot plate. Like
not even joking, put it on
there, did all the solder and

01:39:18.750-->01:39:22.750
LE, the components and floated
on a hot plate and, that was,
you had to exploit it over rf,

01:39:22.750-->01:39:28.792
you can send them all text
messages back and forth.
[laughter] >>Yeah >>So it’s

01:39:28.792-->01:39:33.667
like, it’s like a twist question
that came in from the audience.
Uh what about the most elaborate

01:39:33.667-->01:39:37.917
challenge that you were able to
dream up but that couldn’t quite
pull off. >>Man >>It’s like

01:39:37.917-->01:39:40.958
still on the to-do list >>Uh let
me make sure I can act- >>A
really good scoreboard

01:39:40.958-->01:39:46.750
>>Actually >>Uh the >>Ours was
>>Scoreboard >>The uh, ou- ours
was the uh World Series of Poker

01:39:46.750-->01:39:52.500
printer challenge. >>Oh yeah I
remember that one >>Oh yeah
that’s right >>That that’s when

01:39:52.500-->01:39:55.250
we started to destroy that
thing. You’re like, no, no, no
it’s not our printer >>W- we

01:39:55.250-->01:39:59.917
>>Yeah, I remember that one >>I-
one year at the Rio, we, we,
rolled in right after the world

01:39:59.917-->01:40:04.125
series of poker had ended and in
the back rooms were all the
printers that they had used for

01:40:04.125-->01:40:07.583
the World Series of Poker's and
we just wheeled one out and put
one on every table. Everyone had

01:40:07.583-->01:40:14.542
an Ethernet [laughter] we ran
cable to every table. And uh,
yeah we wanted to work that into

01:40:14.542-->01:40:21.000
the game but didn’t quite get
there. >>So [off mic comment]
>>Y- yeah some of them got fully

01:40:21.000-->01:40:25.500
disassembled and never
reassembled and >>Yeah >>I think
>>There’s, there’s actually some

01:40:25.500-->01:40:28.917
significant analysis on the
internet of some of those
printers >>Pay for a few

01:40:28.917-->01:40:35.167
printers I think >>Uh Guynar you
here? No. I was trying to ask if
I could mention his challenge

01:40:35.167-->01:40:40.792
but I guess I’ll go ahead it’s
in a text message. So, he came
up with an awesome idea, uh GDB

01:40:40.792-->01:40:47.000
over phone. So, you dial in and
says, this is your prompts and
he actually got a decent way

01:40:47.000-->01:40:53.125
into the challenge, like it, I
ya know press 1 for R aright.
These are you registers,

01:40:53.125-->01:41:00.083
alright. Ya know, step. There
was a [laughter} But we
couldn’t, like the, the band-

01:41:00.083-->01:41:02.208
bandwidth and throughput just
wasn’t enough for that one.
>>Operator, operator >>My God

01:41:02.208-->01:41:08.333
that’s great, that’s great
>>Real person [off-mic comment]
>>Well >>And then, and then do

01:41:08.333-->01:41:11.500
the teams have to implement
like, like audio lang- language
processing to sort of automate

01:41:11.500-->01:41:22.750
the debugger? >>Hang, hang on,
hang on yeah at Goa at NolCon uh
I was a, I was really proud of

01:41:22.750-->01:41:26.750
one of the kids there he made a
DTMF only attack for IVRs. He
found a sql injection attack in

01:41:26.750-->01:41:33.042
the default template for IVRs
running asterisk asterisk. So he
had a compressed DTMF string

01:41:33.042-->01:41:41.667
that he could play into an IVR
and get it to read out the
username and password of all of

01:41:41.667-->01:41:46.042
the accounts allowed to edit the
system. >>Awesome >>And the
phone came out, it, h- h- he

01:41:46.042-->01:41:51.333
used his own voice so it was
hard for me to understand, but
uh nothing to do with our game

01:41:51.333-->01:41:56.083
but uh but you can make DTMF
attacks. >>Yeah >>Turns out it
takes only about I don’t know 50

01:41:56.083-->01:42:04.000
milliseconds or something to get
a digit through. >>Um, when did
black badges start being

01:42:04.000-->01:42:11.667
awarded, was it always 8? >>I
don’t know >>Uh >>Caezar >>We
got the first one, um I think

01:42:11.667-->01:42:17.333
they were I don’t know actually
when they, black badges came out
[off-mic comment] >>I’m saying

01:42:17.333-->01:42:24.000
they were, when the team was I
think the largest ever team was
like th- tw >>20 people >>20 odd

01:42:24.000-->01:42:27.500
people and DT’s standing there
with like 1 black badge, like
[laughter] uh, yeah. >>Yeah so

01:42:27.500-->01:42:33.042
he went and grabbed and he could
like spare 8 total so >>That’s
hilarious >>That became 8 of us

01:42:33.042-->01:42:37.417
got black badges the rest of
them did not >>Cause yeah, we
pick 8 people for our team, we

01:42:37.417-->01:42:43.542
picked 8 people for our team for
the first year because you guys
did and you probably picked it

01:42:43.542-->01:42:46.417
because at some point he just
had 8 badges. That’s kind of
funny >>8 teams, 8 badges >>Well

01:42:46.417-->01:42:50.833
alright >>8 players >>That was,
that was based on, that was also
based on the like table layout

01:42:50.833-->01:42:55.000
at the time, like there were
like 8 approximate seats without
whatever and so we basically

01:42:55.000-->01:42:59.417
decided that you couldn’t have
more than that at the table >>I
think that’s still the same we,

01:42:59.417-->01:43:03.000
we, we limit the size of the
table just because uh, just
managing it all’s a pain.

01:43:03.000-->01:43:07.500
>>Which I mean doesn’t really
prevent large teams but what it
does do is gives large teams the

01:43:07.500-->01:43:12.375
same problem that large
organizations in real life have.
Which is left hand right hand

01:43:12.375-->01:43:17.250
communication problems and
organizational problems because
like 20 people sitting at a

01:43:17.250-->01:43:22.750
table can still kind of coalesce
and self-organize but 20 people
divided into 2 groups sitting in

01:43:22.750-->01:43:27.292
2 different rooms can’t >>In the
real world you can buy a
fractional T3 or, or a you know

01:43:27.292-->01:43:32.375
a private line and have it run
and have somebody put up some
VPNs and buy some keys and walk

01:43:32.375-->01:43:37.708
them over there, all these
things, here you’ve got 48 hours
to end the contest from the time

01:43:37.708-->01:43:41.292
you know where you’re gonna be
and getting a remote team
connected is >>Yeah limited the

01:43:41.292-->01:43:46.125
effectiveness of a large team
was one of our driving ideas
>>Yep >>And the way we’ve done

01:43:46.125-->01:43:51.500
that is that we actually don’t,
wh- when the game first started
like, during DDTEK they would

01:43:51.500-->01:43:54.333
just give you all services
immediately except for maybe a
few handful excep- exceptions,

01:43:54.333-->01:44:00.125
now we’ll drop one service so
sure you have 50 people, let all
50 people look at 1 service,

01:44:00.125-->01:44:04.667
good luck, alright have fun.
>>Limiting parallel >>Limi- yeah
li- limiting parallelism is what

01:44:04.667-->01:44:08.792
we do and then ya know maybe la-
maybe later we’ll do ya know 2,
2 more services or something

01:44:08.792-->01:44:11.417
>>Yeah we never thought about
anything like that I wish,
honestly I wish we had because

01:44:11.417-->01:44:19.458
limiting, limiting team size was
a huge motivator for us too.
>>For a certain team >>2

01:44:19.458-->01:44:24.917
actually >>Uh it’s >>You, you
and Shellfish >>It’s sort of
interesting because it also

01:44:24.917-->01:44:28.833
plays into the, sort of
distribution int- inside of a
team, right? How many rockstars

01:44:28.833-->01:44:32.292
are there and how, how fast does
it drop off and like what is the
curve >>We also started

01:44:32.292-->01:44:37.208
implementing what we do cal-, uh
we patch a service so like real
world we’ll implement the basic

01:44:37.208-->01:44:42.625
functionality of a service and
then y- maybe mid-day on
Saturday, >>Change it >>A

01:44:42.625-->01:44:47.208
completely new one and the
original bug is patched, so now
you have to pull the person who

01:44:47.208-->01:44:52.625
was originally the, the facto
expert on that, that service and
they now have to do this new

01:44:52.625-->01:44:56.625
thing because do you really
wanna ramp up somebody else on,
on the same service or, hold

01:44:56.625-->01:45:00.167
them? So yeah we’ve, we’ve
really spent a lot of time
thinking about these game

01:45:00.167-->01:45:07.875
mechanics and, and how to make
it um, more playable, useful for
the real world. >>Yep >>Uh so

01:45:07.875-->01:45:14.875
how many women have
participated? >>I know that um,
um More Smoke Leak Chicken has

01:45:14.875-->01:45:18.792
at least had female
participation. Um unfortunately
not enough. >>Yeah >>We had more

01:45:18.792-->01:45:23.667
than 25 percent every year that
we’ve played. Uh we, we uh did
not try to make a conscious

01:45:23.667-->01:45:30.250
inclusion of people, we picked
the best people we knew and in
Seattle that’s a mixed group.

01:45:30.250-->01:45:34.708
>>We had women with us, uh not a
huge percentage but we had more
and more with us every year with

01:45:34.708-->01:45:38.708
School of Root. >>Imagine they
just didn’t want to smell us. We
start stinking after a day.

01:45:38.708-->01:45:42.792
>>Par- participation
participation-wise there’s been
a lot of mixed team, um but by

01:45:42.792-->01:45:50.333
far it’s obviously uh been
lopsided. >>What about it as
organizers. >>Uh so, uh

01:45:50.333-->01:46:00.500
Lightening, uh eh- her name is
Jewel, the person that wrote
this. >>Wow, again it’s just a

01:46:00.500-->01:46:04.958
full stop. >>Phenomenal, yeah
like custom architecture
implemented by a woman for

01:46:04.958-->01:46:08.792
absolutely, yeah. >>We’re not
looking for a winner, I’m just
c- [laughter] >>No, n- >>No, no,

01:46:08.792-->01:46:12.042
no >>No, that’s >>We found one
>>That’s >>You weren’t looking
for it but we found it, yeah we

01:46:12.042-->01:46:17.333
found the winner of CTF
>>Exactly >>And it’s up here
with Lightening >>Um, I feel

01:46:17.333-->01:46:22.917
like just putting this up here
even though it’s clearly a troll
but Frank wants to know. He’s

01:46:22.917-->01:46:26.708
hacked the Gibson and only got a
partial download of the garbage
file so can you help him look

01:46:26.708-->01:46:30.500
through it? >>I think >>There’s
a person in the audience who
does not know what this means

01:46:30.500-->01:46:33.750
and we just found this out right
before we walked to this room
[inaudible] >>Hack the Planet

01:46:33.750-->01:46:37.542
>>Or something >>Blank, blank,
blank right? >>I kind of have to
do it >>Yeah >>It’s like

01:46:37.542-->01:46:41.875
>>Blank, blank, blank >>It’s
like the only question that’s
been posted that wasn’t

01:46:41.875-->01:46:46.417
anonymous so I felt like I had
to actually put it up there
[laughter] Um so here’s a

01:46:46.417-->01:46:50.833
curious question at least to me,
uh did you do dry-runs and like
how many people would

01:46:50.833-->01:46:54.792
participate in the dry-runs if
you did do >>We never had time
>>We can’t, w- time plus we

01:46:54.792-->01:46:58.250
can’t do dry-one, runs. >>We did
cause we really only trust
ourselves. >>We did dry-runs of

01:46:58.250-->01:47:03.250
individual services >>Okay yeah
>>Right. So, the, the the idea
of the dry-run thing of like

01:47:03.250-->01:47:07.625
here other member of Kenshoto or
whatever like try to solve this
challenge but other than that

01:47:07.625-->01:47:11.750
>>Internal >>No way >>Data
testing I guess you’d call it
>>Yeah, yeah unit, unit testing

01:47:11.750-->01:47:17.333
and not integration or systems
testing. >>No >>Like systems go
online at the real game >>Test

01:47:17.333-->01:47:22.167
them >>Yeah, exactly and we had
to make changes like, like last
second all the time, like every

01:47:22.167-->01:47:27.875
year. >>M- yeah, we did it as
well, we did it for our, all of
our automated polling scoring

01:47:27.875-->01:47:31.375
all of that stuff, I mean th-
the second part of the question
doesn’t make sense to me, right.

01:47:31.375-->01:47:35.875
How did you stop any info
leaki-that’s an outside thing.
[inaudible] >>Some sort of beta

01:47:35.875-->01:47:46.208
run or something, more people
have, it sounds like >>We didn’t
have >>No way >>It was only

01:47:46.208-->01:47:48.125
team, right >>No way >>It wasn’t
like we opened it to public beta
or something like that >>No way

01:47:48.125-->01:47:51.667
>>It, it was only our internal
dev team. >>Yeah >>It’s, it’s
probably one of the reasons his

01:47:51.667-->01:47:55.542
code hasn’t even transacted from
hand to hand really. Is, is just
because we all feel like, at

01:47:55.542-->01:48:02.042
least I think we felt like, we
wanted to play Myles’ game and
then once we kind of got on the

01:48:02.042-->01:48:05.667
inside we didn’t want to be
tainted, we didn’t want to taint
anybody else we wanted all of

01:48:05.667-->01:48:12.500
our friends to be able to enjoy
the game that they loved. >>Yep,
totally. >>Got 5. Oh, we’re

01:48:12.500-->01:48:19.875
done? Okay. Program says another
9 minutes. But we can be done if
you want us to. [laughter] >>My

01:48:19.875-->01:48:24.167
beers empty >>Mine too >>Alright
last, last >>No more beer >>Last
question then >>Let’s go >>Uh,

01:48:24.167-->01:48:30.792
what qualities do you admire
most in the teams that compete?
>>Fury, like there are teams

01:48:30.792-->01:48:35.792
that just like throw down and
they don’t do anything else.
They don’t sleep. They don’t

01:48:35.792-->01:48:39.542
eat. >>Do you remember the mad
>>Passion >>Hackers >>Yeah >>I
was going to say like I admire

01:48:39.542-->01:48:44.708
no whining [laughter] >>Yeah,
like as, as an organizer that,
that becomes a big problem. I

01:48:44.708-->01:48:50.208
agree like that’s yeah, that’s a
big issue but just the fury the
passion of it. >>Like, the fact

01:48:50.208-->01:48:55.000
that you can go and you spend ya
know 12 hour, 10 hours in front
of a PC and then you go and

01:48:55.000-->01:48:59.458
spend all night in front of,
doing the same thing. It’s
pretty hard core like, like I

01:48:59.458-->01:49:03.458
kind of admire all of it. Yeah
>>Yeah the endurance, the i- it
really is at some point an

01:49:03.458-->01:49:06.875
endurance feat. >>You know
what’s funny is thinking about
that is you’re right now th- the

01:49:06.875-->01:49:10.542
cost entry is, well they get
free entry now I don- it wasn’t
when you guys, was the case but

01:49:10.542-->01:49:16.000
they get 8 badges per team but
before when you’d have to pay
>>We didn’t even get badges >>Oh

01:49:16.000-->01:49:20.125
yeah. When you pay, you go and
you come here like especially
people coming here from far

01:49:20.125-->01:49:26.958
away. Russia and Japan, and they
spend their entire weekend in
front of a computer, the exact

01:49:26.958-->01:49:31.750
same thing they do at home and
they come in here and do it so
uh >>Th, the yeah. Aside from

01:49:31.750-->01:49:34.500
that the thing that always
shocked me as an organize-, time
to solve, is the thing that I

01:49:34.500-->01:49:38.042
always looked at right? Some,
something drops like radically
faster than you expected it,

01:49:38.042-->01:49:40.917
>>Oh yeah >>That that’s like an
instant >>That’s frightening
>>Wow >>Sometimes >>Yeah,

01:49:40.917-->01:49:43.958
agreed, agreed. >>Yeah th- and
the level it’s not just the
stamina, like yeah I can drink a

01:49:43.958-->01:49:50.167
bunch of Mt Dew and see, I put
in the 10 months getting to this
>>That’s true a lot of people

01:49:50.167-->01:49:54.042
bring, now you have to bring
custom stuff, for >>Right
>>Example when h- he was

01:49:54.042-->01:49:59.292
mentioning that if you pull a
flag and you don’t encrypt it,
it’s gonna get caught, cause oh,

01:49:59.292-->01:50:02.458
that’s one thing we mention too
is that pe- all the teams have
access to the packet captures,

01:50:02.458-->01:50:06.458
so they see all the data coming
across the network and they
control what’s uh, coming in and

01:50:06.458-->01:50:12.083
out of the box, so, um if you
don’t encrypt your, your data
back and forth in your shell

01:50:12.083-->01:50:15.625
code, your shellcode has to
implement the encryption, then
you’re gonna fail. >>Alright, do

01:50:15.625-->01:50:23.917
you have something? >>Uh, uh
just, just one of the things in
time, time to crack uh, I don’t

01:50:23.917-->01:50:31.458
know if it was you guys uh, 1 of
our routers leaked 1 bit of
information and the scoreboard

01:50:31.458-->01:50:36.792
had a different TTL, off by 1
from th- >>I remember, no that
was, that was, I remember >>Was

01:50:36.792-->01:50:39.500
that you? >>That was us, yeah
>>Was that you guys? Aight
>>Yeah >>The TTL on the

01:50:39.500-->01:50:44.000
>>Basically we could tell, we
could tell apart the packets
that were your engine testing

01:50:44.000-->01:50:48.292
that the service was online from
everyone else. >>So they
firewalled only the players

01:50:48.292-->01:50:53.875
>>Nobody else can get >>Percent
SLA, zero percent defense >>Yeah
>>An- and they did this in maybe

01:50:53.875-->01:50:58.958
I don’t know, maybe 7 minutes.
From the time, we handed them
their discs, they were. It was

01:50:58.958-->01:51:04.458
amazing >>We profiled TTL and
your agents on your >>Yeah >>We
missed that one we didn’t, yeah

01:51:04.458-->01:51:10.167
>>Alright, we’ve got the, the
cutoff so um the, the slides
will be posted and uh, th- the

01:51:10.167-->01:51:14.500
video will be posted so it's, uh
thank the panelists and uh I
suppose we’ll be around for a

01:51:14.500-->00:00:00.000
little bit. >>Thanks for coming
>>Yeah thanks for coming
[applause]