WEBVTT 00:00.000 --> 00:04.240 We are at the DEF CON 30 CTF run by Nautilus. 00:04.240 --> 00:09.380 And the teams have been doing crazy things with a few issues along the way. 00:09.380 --> 00:11.180 But it is... that's this year. 00:11.180 --> 00:15.820 Because you have the teams, and then you have this whole live aspect where teams come on over here. 00:15.820 --> 00:20.440 Can you just tell me, high level, how did you design this CTF? 00:20.440 --> 00:21.760 Like, what are the teams actually playing? 00:21.760 --> 00:26.780 Well, let's step back a second and say this is your first year doing the DEF CON CTF. 00:26.780 --> 00:36.880 Although many people on Nautilus have designed CTFs before, or in this particular CTF, what is Nautilus doing this year for the CTF? 00:36.880 --> 00:39.860 That's either different, or what are your takes on it this year? 00:40.160 --> 00:46.220 So, what we're doing here is a relatively traditional CTF. 00:46.560 --> 00:49.380 The CTF has evolved over the years. 00:49.820 --> 00:52.100 The general structure has stayed the same. 00:52.100 --> 00:55.100 So there's N teams, this year there's 16 teams. 00:55.200 --> 00:58.420 Each team gets access to their own server. 00:58.760 --> 01:00.340 All the servers are identical. 01:00.340 --> 01:05.240 Those servers are running executables on them that provide services. 01:05.280 --> 01:07.560 Those services are vulnerable. 01:07.620 --> 01:09.640 They're stuff that we have created. 01:09.640 --> 01:12.980 None of the teams have seen these things before. 01:13.420 --> 01:27.880 And they reverse engineer those services, find vulnerabilities, write exploits, and throw the exploits at other teams to get text tokens off of the game boxes, which are called flags, and then they submit them to our scoreboard. 01:27.880 --> 01:33.040 Sure, so this is... and we can get a little more technical, or use some internal jargon. 01:33.040 --> 01:35.480 This is a typical attack and defense CTF. 01:35.480 --> 01:38.860 So the core of the CTF is a typical attack and defense. 01:38.860 --> 01:45.080 So they have servers that they're defending against other teams, and servers that they're attacking that the other teams have. 01:46.040 --> 01:56.060 So this is where I like to get into it, which is the actual challenges, if you will, or the actual exploitables on the machines are new every year. 01:56.060 --> 02:00.900 And everyone has a different take on what should or should not go in. 02:01.280 --> 02:02.140 And what... 02:02.660 --> 02:04.660 you've already burned a bunch of challenges. 02:04.660 --> 02:09.540 Are there things that you're particularly proud of this year that you may have designed and got put on? 02:10.420 --> 02:20.560 So we had one challenge this year called Web 4 Factory from Itsun, one of our... one of the Nautilus members who's new to organizing CTFs, or at least new to organizing DEF CON CTF. 02:20.800 --> 02:24.160 And I think it was deeply, deeply hilarious. 02:24.340 --> 02:30.080 As the Web 4 Factory implies, it's a Java applet, or a Java program, not a Java applet. 02:30.200 --> 02:41.240 It's a Java server that has, you know, deeply nested classes, like teams entered the applet, or the job... I'm too old, I'm thinking Java applets all the time. 02:41.240 --> 02:44.220 But they went to like institute.nautilus.web4factory. 02:44.220 --> 02:49.220 factory, all the way down this, you know, horrible Java chain of classes. 02:49.780 --> 02:51.880 That was one that we ran through yesterday. 02:52.080 --> 02:57.960 And I thought it was funny enough that I made a custom sticker for it in kind of secret without telling anybody about it. 02:58.040 --> 02:59.020 That's fantastic. 02:59.020 --> 03:00.840 I heard you like classes with your classes. 03:00.840 --> 03:01.860 Oh yeah, yeah, yeah. 03:01.860 --> 03:05.540 So we put a class factory in your class factory, so you can factory classes while you factory classes. 03:05.540 --> 03:09.200 Oh my... you've been practicing that a little bit too much, haven't you? 03:09.980 --> 03:16.640 So we also heard that you guys... well, I mean, we've been having some power issues and things have been going on and off throughout the conference. 03:16.640 --> 03:19.720 Have you guys got hit by any of the infrastructure issues? 03:20.160 --> 03:25.380 Honestly, Defcon has provided like awesome power, awesome network. 03:25.620 --> 03:29.540 Unfortunately, most of the hardware issues that we've had were our own. 03:29.940 --> 03:34.740 We didn't have some of the servers kind of correctly configured before we brought them here. 03:35.020 --> 03:43.320 And so we did run into an issue yesterday where we had to retire a challenge early in order to basically keep the load on the boxes down. 03:43.420 --> 03:52.060 And this led to us spending all night actually reconfiguring all of the hardware so we could run today's challenges more smoothly, which did fortunately work. 03:52.300 --> 03:56.600 So would you say this year is going smoother or less smooth than a typical CTF? 03:56.600 --> 04:02.240 Because that sounds... I've run my own and that sounds like something that pops up in most cases. 04:02.240 --> 04:06.320 So both Vito and myself have actually run the Defcon CTF before. 04:06.320 --> 04:07.860 Vito's pretty old hat at it. 04:07.860 --> 04:13.560 I ran this last in 2017 with the Legitimate Business Syndicate. 04:14.080 --> 04:21.300 And relative to the year that I was hosting, this has gone a little bit more rough. 04:21.300 --> 04:26.020 It's our first year, whereas I was helping the Legitimate Business Syndicate in their last year. 04:26.020 --> 04:30.760 So they had had, at that point, four previous years to kind of perfect everything. 04:30.760 --> 04:32.780 So the first year is usually kind of rough. 04:32.780 --> 04:34.360 You're building everything from scratch. 04:34.360 --> 04:36.580 You're kind of getting all your tech together. 04:36.780 --> 04:40.200 This is the first time you've really had to have teams interact with it. 04:40.560 --> 04:45.000 But I don't know if your experience... My experiences are broadly similar to that. 04:45.000 --> 04:45.620 We always... 04:45.620 --> 04:51.460 it felt like as LegitBS, Legitimate Business Syndicate, we always had to do some kind of cleanup after the first day. 04:51.620 --> 04:56.180 Something would unexpectedly break or, you know, scoring would break. 04:56.340 --> 04:59.420 Checking for, you know, service uptime would break at some point. 04:59.440 --> 05:01.660 And there's always a little bit of fix-ups. 05:01.740 --> 05:15.960 I think 2016 was a particularly rough year for us because we went in... that was the Cyber Grand Challenge year where we supported the winner from the DARPA competition playing our game as an autonomous computer. 05:16.340 --> 05:20.340 And that was a different enough game from our previous years that we had a ton of problems. 05:21.860 --> 05:24.960 That was, I think, the year we didn't start our finals on time. 05:25.160 --> 05:28.080 And there were just lots and lots of little issues that year. 05:28.140 --> 05:31.900 I mean, you keep talking about the problems, which is... like, they are interesting. 05:31.900 --> 05:44.480 The thing which is fascinating to me is that you have legitimately the best hackers in the world in this room spend their entire year training kind of for this event. 05:44.480 --> 05:51.100 They pre-qualify, they get in here, and you have to design an event in such a weird way. 05:51.100 --> 05:53.380 Because you're not designing secure software. 05:53.380 --> 05:55.320 That's what's so weird about this. 05:55.340 --> 05:58.100 You're designing a maze. 05:58.100 --> 06:04.440 An insecure software enough where players can solve it in a finite amount of time. 06:04.660 --> 06:14.780 I am curious, though, when you design a solution, I think of the movie Inception where he says, give me a maze that you have to solve in 30... it takes me exactly one minute to solve. 06:15.040 --> 06:17.260 How has it been when you design challenges? 06:17.260 --> 06:20.580 Because I know you have this whole live aspect which throws a loop. 06:20.640 --> 06:23.040 You're designing a challenge where you go, this has to be solved. 06:23.040 --> 06:25.600 I expect this to be solved in one hour. 06:25.660 --> 06:36.060 How do you go about gauging how difficult some of these challenges are when you're dealing with legitimately the best hackers arguably in the world for binary exploitation and things like that? 06:36.240 --> 06:41.280 So for us, I feel like there's a large aspect of that for people on our team is experience. 06:41.280 --> 06:44.340 And there's a lot of CTF playing experience on our team. 06:44.460 --> 06:52.520 So for some of the challenges, if somebody's not sure about how difficult it's going to be, we pass it to a legitimately great CTF player that's playing with us. 06:52.760 --> 06:59.020 Shellfish for the last few years, their powerhouse has been Phish, who has thankfully joined Nautilus Institute. 06:59.580 --> 07:01.720 And Phish is one of the best reversers in the world. 07:01.720 --> 07:08.540 Phish is working hard every day in his normal job on better software for analyzing software. 07:08.820 --> 07:13.480 So he's an incredible play tester, and he's not the only incredible play tester on our team. 07:13.480 --> 07:15.240 How do you balance? 07:15.560 --> 07:21.400 There's this whole key... I love playing CTS, playing the NSEC when there's so many great ones out there. 07:21.400 --> 07:32.840 And when I've played them, there's this weird balance that happens between security through obscurity, which leads to mostly frustration on the team, because it leads you down these rabbit holes. 07:32.840 --> 07:40.740 And the challenge starts becoming, how do I overcome all these red herrings to find the toad of truth? 07:40.740 --> 07:46.120 So full disclosure, we used to help run Defcon Darknet, which is a more introductory CTF. 07:46.120 --> 07:51.060 It gets people into the con, gets them around the con, gets them towards these more advanced things. 07:51.060 --> 07:54.280 And the question for us is always that red herring question. 07:54.280 --> 08:02.140 Do you worry so much at this level about the red herring problem where someone picks up on the wrong thing, or is that their fault? 08:02.700 --> 08:08.500 So in terms of our game, the larger attack-defense CTF, we don't really worry about that at all. 08:08.500 --> 08:11.700 The best teams are not going to get tripped up by those kind of things. 08:11.700 --> 08:15.620 They're not going to get lost down some sort of unexploitable rabbit hole. 08:15.620 --> 08:20.240 They've got enough experience to kind of be able to feel these things out quickly and hone in on the right stuff. 08:20.380 --> 08:26.160 For the live CTF event, that is a consideration, because you want the challenge to be solvable within an hour. 08:26.160 --> 08:33.380 And if both competitors are stuck on some, you know, rabbit chase down something that's not exploitable, that's not a good challenge design for that format. 08:33.460 --> 08:35.640 No, I was telling him about Michael. 08:35.640 --> 08:40.820 Oh, so yeah, well, we had a short discussion with Cyphertext about the live CTF. 08:40.820 --> 08:42.260 And of course, he's actually helping. 08:42.260 --> 08:44.540 He's one of the commentators actually running. 08:44.720 --> 08:46.280 How has that been going? 08:46.280 --> 08:55.360 I heard that the one of these last rounds went a really long time, in part because you throw a problem at them, and then it takes them an hour, and then their brain gets fried. 08:55.380 --> 08:59.280 And then you go to your backup challenge, which is supposed to be really, really easy. 08:59.280 --> 09:04.540 But by that point, they're so frustrated that they can't even do that one, which, you know, that this is a thing that happens. 09:04.540 --> 09:06.180 It happens to seasoned professionals. 09:06.460 --> 09:07.860 Has that been happening a lot? 09:07.940 --> 09:10.820 Or is that just been popping up here and there? 09:10.900 --> 09:15.460 So specifically, in the context of live CTF, I haven't seen it happen too much. 09:15.460 --> 09:20.380 I believe that's actually the only challenge of the day that went to, like, a sudden death. 09:20.380 --> 09:26.360 Like, they couldn't solve the original challenge in 45 minutes, so they were provided an easier one and then had to do that one. 09:26.640 --> 09:30.560 I was worried, actually, so I had one of the challenges in live CTF this year. 09:30.560 --> 09:38.800 I was worried that my challenge, we were going to have to switch to a, like, a backup challenge to make sure the round went through. 09:38.800 --> 09:45.760 And I got to be honest, the person who solved it, I wasn't really following what his exploit was when he was doing it. 09:45.760 --> 09:47.240 He was so fast on the keyboard. 09:47.240 --> 09:50.520 Like, it was like watching a StarCraft player or something, but with, like, code. 09:51.060 --> 09:57.180 And I thought that we were going to have to, you know, give out one of these, you know, easier challenges kind of thing. 09:57.180 --> 09:58.600 And then he just he got it. 09:58.780 --> 09:59.600 It was great. 09:59.600 --> 10:07.620 So this brings up the question, which is what have been the pinnacle moments that have happened so far in the CTF? 10:08.240 --> 10:19.580 My favorite so far has have been especially live CTF related, but every time there's applause on the room or somebody jumping up or somebody cheering, that is something I always really enjoy to see in a CTF. 10:19.640 --> 10:31.460 I don't, I haven't been super paying attention to see if that's been happening with First Blood on any of the regular challenges and the regular part of the CTF, but it's always cool to see when somebody gets that breakthrough and they get excited about it. 10:31.460 --> 10:32.120 I love it. 10:32.120 --> 10:37.020 So has that been happening primarily with the attack and defense or on the live CTF? 10:37.760 --> 10:46.200 So in our game, because it's attack and defend and you have to like have a sustained attack, it's not like one little rush of like, yes, I scored a point. 10:46.200 --> 10:48.480 You're scoring points continuously. 10:49.060 --> 10:51.880 There's not usually as much animation from teams. 10:51.980 --> 10:53.900 It's kind of like, yes, okay, we scored on that one. 10:53.900 --> 10:55.960 Now let's get another vulnerability kind of things. 10:56.180 --> 11:01.980 And our challenges for the main game are designed to have more than one vulnerability inside of them. 11:01.980 --> 11:09.900 So the reason we do that is because we want teams to, you know, patch one vulnerability, but still be vulnerable in another way. 11:09.900 --> 11:16.620 And that, you know, keeps them like looking at this stuff for hours and hours and hours trying to figure out how do we, you know, figure this out. 11:16.780 --> 11:28.900 One of my favorite things from our game is two of our top teams earlier today were in kind of this like interesting, both teams were leading. 11:28.900 --> 11:31.040 They both had exploits on a service. 11:31.040 --> 11:41.440 They both had patched that service, but they were going back and forth, basically dossing each other's like service, like finding ways to make it do things that weren't intended. 11:41.440 --> 11:46.800 And then we have like an SLA checker that's like, hey, is your service up and responding to things correctly? 11:46.800 --> 11:50.480 They were finding ways to basically like get it to be in weird states. 11:50.640 --> 11:56.440 It was interesting seeing them go back and forth, kind of like making each other lose points and getting points off of other teams. 11:56.440 --> 12:00.960 And like, for a sustained period of time, it was definitely these two teams kind of going at it. 12:00.960 --> 12:03.960 And that's really cool to see in our game. 12:03.960 --> 12:18.960 So I remember during the Cyber Grand Challenge year that it was announced at the end that a number of the competitors had found a bunch of what would amount to zero days or just unintended bugs in the challenges that were discovered. 12:19.000 --> 12:24.700 Has that happened at the CTF yet where there was some kind of exploit that was just not expected? 12:24.960 --> 12:26.660 Yes, that has happened. 12:26.660 --> 12:28.980 There was a challenge we released earlier today. 12:28.980 --> 12:30.100 I think it's called Monbo. 12:30.340 --> 12:34.320 And there was a heat bug that we didn't intend to be in the challenge. 12:34.420 --> 12:36.380 And a team found it and exploited it. 12:36.380 --> 12:37.080 It was great. 12:37.080 --> 12:38.440 That's exactly what we want. 12:38.660 --> 12:40.900 Do you ever see escalation with the teams? 12:40.900 --> 12:49.920 And what I mean by that is you have, I imagine like myself in my day job, even where I'm bashing my face across something over and over and over and I'm getting nowhere. 12:49.920 --> 12:54.200 And then you start going, all right, Jello against the wall, I'm going to try this and I try this. 12:54.200 --> 13:02.000 And then they escalate almost to the point where the rules get a little wishy-washy and the fact where like they're doing stuff they're not really supposed to. 13:02.280 --> 13:03.960 Has any of that happened? 13:05.420 --> 13:09.600 Yeah, so we have had some team... it's hard running a hacking competition, right? 13:09.600 --> 13:17.320 Because it's really hard to enforce rules when the whole point of hacking is that you're getting around like rules basically in a program. 13:17.820 --> 13:23.060 We have had a couple of teams that have... so I mentioned the dossing earlier. 13:23.060 --> 13:27.920 If you're specifically targeting like a patch that another team has made, that's kind of cool. 13:27.980 --> 13:33.120 But if you're running like a fork bomb on a bunch of servers that's going to like take down the whole game, that's not so cool. 13:33.320 --> 13:40.080 So we kind of have some loose, you know, spirit of the game kind of rules that are kind of like, hey, make sure people can still play. 13:40.240 --> 13:43.280 You know, make sure that we're not like attacking our infrastructure. 13:43.280 --> 13:45.860 We put a lot of time and effort and work into this. 13:45.860 --> 13:52.880 It's really frustrating to have our stuff come crashing down just because, you know, somebody decided to run a fork bomb or something. 13:52.880 --> 14:01.760 It's fascinating me, as you said, with hacking challenges, you're designing them to be broken and then you have kind of the semblance of rules. 14:01.760 --> 14:07.220 Because I remember when you're like, oh, goodness, there's a wild back where the scoreboard gets hacked into and things like that. 14:07.360 --> 14:15.600 With this challenge, how are the teams really dealing with, let's say, the size limitation? 14:15.600 --> 14:17.300 With this, you have open. 14:17.300 --> 14:20.840 Are you finding a mass difference in the sizes of teams? 14:22.500 --> 14:26.200 We aren't actually completely sure about how big some of these teams are. 14:26.380 --> 14:42.760 One of the things I was most worried about, and I promise this is an actual answer to your question, kind of in a roundabout way, I was really, really worried that the 16 tables in this room were going to have one person holding a VPN open and responding to events in the room for everybody else elsewhere. 14:42.760 --> 14:47.540 Instead, what we've seen is most of the seats at these tables are full, like all eight of them. 14:47.900 --> 14:52.040 Every time I look around, it's a busy room, and I really, really love to see that. 14:52.080 --> 15:04.260 We don't necessarily know how many players are up in hotel rooms or back at home, but I can't imagine that it would be wildly out of place to assume anywhere between eight people or 40, maybe. 15:04.500 --> 15:15.220 One of the problems with bigger teams is kind of endemic to all software engineering or all engineering in general, where the bigger your team, the more communication that has to get done back and forth between groups. 15:15.340 --> 15:24.800 And since we haven't had, you know, more than about half a dozen challenges open at any time, there's dubious value to having, you know, 80 people working on, you know, one thing. 15:24.800 --> 15:26.600 That brings up a question, though. 15:26.900 --> 15:32.800 Of the teams that are doing well, what do you think they're doing that the other teams aren't doing? 15:32.800 --> 15:35.180 Are they just getting luckier with the challenges? 15:35.180 --> 15:41.940 I mean, they're not luck, but you get the idea where you're finding those problems a little quicker, or do you think it's a communication thing? 15:41.940 --> 15:43.700 What do you think separating the teams? 15:44.640 --> 15:51.300 Something that I find is really important when you're playing a CTF is to get into this mindset where you're failing very quickly. 15:51.300 --> 15:56.380 So you try as many things as possible in a short period of time. 15:56.440 --> 16:15.640 And what I have found as a player, and also watching as an organizer, watching all of these players, is that the best teams, the best players tend to be people who are able to very quickly iterate through a number of, or, you know, like different scripts or things to try things out, 16:15.640 --> 16:17.880 to see what they think is going to work. 16:18.300 --> 16:27.960 And so I think that a lot of the teams who are doing better are just able to make these iterations to adapt much more fast than the other teams. 16:28.400 --> 16:35.420 As an ancillary to that, somebody, you know, in this conversation maybe was watching me write a SQL query to analyze some scoring event earlier today. 16:35.600 --> 16:41.780 And I definitely was in the zone there for, you know, a minute or two, like, you know, type a clause, hit enter, see if it works. 16:41.780 --> 16:43.240 Type another clause, hit enter. 16:43.240 --> 16:47.960 And it felt really, really good to just blast it out like that, step by step, getting to a destination. 16:48.000 --> 16:52.160 So I want to get into a little bit of... it is a game, right? 16:52.160 --> 16:56.120 And I always like the game design aspect of it. 16:56.120 --> 17:00.160 And this is really hard because I'm not a PvP gamer. 17:00.160 --> 17:03.600 That's why I typically don't attend attack and defense CTFs. 17:03.600 --> 17:04.920 He is a PvP gamer. 17:04.920 --> 17:06.060 I mean, no one's perfect. 17:06.720 --> 17:07.880 But I like PvE. 17:07.880 --> 17:11.080 I like a more Jeopardy style and I like a more learning CTF. 17:11.080 --> 17:12.500 That's for me. 17:12.740 --> 17:21.820 But how much thought do you give into how players are going to interact with each other when you're developing the challenges? 17:21.820 --> 17:24.060 Because I have to imagine that's one of the biggest... 17:24.060 --> 17:28.080 as an attack and defense, it's all about how teams interact with each other. 17:28.300 --> 17:32.200 How do you go through the process of designing some of those challenges? 17:32.200 --> 17:34.060 I know that's a big question. 17:34.060 --> 17:37.540 But is there anything in particular that you want to talk about? 17:37.560 --> 17:39.600 Yeah, it is super broad. 17:39.920 --> 17:46.720 And I think the game design aspect is something that CTF organizers don't always consider. 17:46.960 --> 17:50.800 And for the attack-defense CTF, which there really aren't very many. 17:50.800 --> 17:55.280 Not only is this the biggest and best one, but there really aren't too many alternatives. 17:55.300 --> 17:58.320 Most CTFs are kind of a Jeopardy board of challenges. 17:58.320 --> 18:01.200 You just kind of pick one and it's kind of like a speed run. 18:01.200 --> 18:04.700 You're just trying to do it as fast as you can, faster than other teams, that kind of thing. 18:05.520 --> 18:10.820 So for our game, there's a lot of interactions between players. 18:10.820 --> 18:13.820 So we have to, for example, model scoring very carefully. 18:14.800 --> 18:22.880 We don't want one team that's solving one challenge to run away with the game because we make it too many points versus another challenge. 18:22.880 --> 18:27.000 So we have to be very careful about how many points is a flag worth? 18:27.140 --> 18:29.180 How many flags can be redeemed? 18:29.180 --> 18:30.900 How does that look over time? 18:31.100 --> 18:33.660 So modeling all of those things are really important. 18:33.720 --> 18:37.660 And then also just kind of trying to encourage things with points. 18:37.880 --> 18:39.820 So we have a king of the hill challenge. 18:40.480 --> 18:48.680 And one of the design considerations in that is we want people to try to win in the little king of the hill rounds. 18:48.920 --> 18:56.060 So in an earlier iteration of that design, we said, well, a win is worth two points and a tie is worth one and a loss is worth zero. 18:56.180 --> 18:59.680 But then winning is not so much better than tying. 18:59.680 --> 19:07.140 So it might be better for you to just try and tie with everybody, spend more of your time on other challenges, things like that. 19:07.140 --> 19:13.680 So we adjusted it so that a win was worth three points to give the winner a much bigger edge on things. 19:13.680 --> 19:18.180 Similarly, I mentioned this concept of SLA, where you have to keep your services up. 19:18.180 --> 19:27.140 The SLA penalty is much larger than the amount of points you score on somebody to ensure that teams keep their services up. 19:27.260 --> 19:29.720 Because we want them to be attacking each other. 19:29.720 --> 19:39.240 And we want the main method of defense to be patching the service, like fixing the bug, rather than just taking the service down so that you don't lose any points by being scored on. 19:39.240 --> 19:55.140 Well, so this was this was actually huge, because in prior years, the CGC year, for example, there was an after analysis of what was the most effective way for your thing to game the game, that's my blog post. 19:55.140 --> 20:00.380 Yeah, if I remember correctly, it was like, if you had just left your services up the whole time, you would have finished third. 20:00.400 --> 20:00.860 Right. 20:00.860 --> 20:02.140 And so that's, that's fascinating. 20:02.140 --> 20:06.020 But you don't necessarily know the rules of the game while you're playing it. 20:06.020 --> 20:10.280 So it's really hard to game it and not just basically roll the dice. 20:10.460 --> 20:14.540 In fact, Alex has a story about how he won his black badge. 20:15.280 --> 20:17.240 Oh, is this the one where I was preparing? 20:17.240 --> 20:17.880 You're saying? 20:17.880 --> 20:18.540 No, no, no. 20:18.540 --> 20:19.860 Yeah, the pairing one. 20:20.460 --> 20:23.140 It's because it's very similar in a different sense. 20:23.140 --> 20:23.300 Yeah. 20:23.300 --> 20:33.840 So in DEF CON 22, that was the year I won my black badge is at the dark that there was tons of different points you could get for going all over con building your badge and things like that. 20:33.860 --> 20:40.840 And when I actually started looking at the math, I realized that pairing with badges was it was weighted a lot. 20:40.840 --> 20:44.320 It was weighted a lot more than I felt it probably should. 20:44.460 --> 20:47.240 So I ended up sitting in the solder. 20:47.240 --> 20:56.300 Well, it's a hardware hacking village because the soldering skills village didn't exist at the time, but just sitting there and helping people assemble their badges and meeting them. 20:56.300 --> 20:57.940 And I had to meet a ton of people, which was great. 20:57.940 --> 20:59.980 But then I would sit down and work on challenges. 20:59.980 --> 21:06.880 The moment someone assembled their badge like, hey, let me go test your badge and then sink it on up and test the badge. 21:06.880 --> 21:11.080 It was actually really funny because the badge was only capable of holding 50. 21:11.400 --> 21:14.780 And what ended up happening was I didn't realize that. 21:14.780 --> 21:18.640 So I got like 70 badges to realize I was starting to overwrite them. 21:18.640 --> 21:22.480 It was a lot of fun looking at the point engines. 21:22.520 --> 21:27.200 That almost feels like a situation where it was working as intended. 21:27.340 --> 21:29.780 So, you know, you got to meet a lot of people. 21:29.780 --> 21:31.540 You got to help a lot of people with soldering. 21:31.580 --> 21:35.700 At tour camp a few weeks ago, there was a event called bureaucracy. 21:36.220 --> 21:49.560 And I kind of realized early on, it was like, oh, this is a way to get like people who are new at this camp a fun game to play where they have to go around the camp, collect stamps from different groups of people, learn how to like pulse dial a phone by wrapping on the hook switch, 21:49.560 --> 21:58.080 and get them to meet new people for, you know, a one of those like awful like soccer practice conical paper cups of, you know, rainier lager. 21:58.580 --> 22:03.400 So the question I had, though, in this context is because this is a common problem. 22:03.400 --> 22:12.120 Once you game the game and you figure out what's the most effective way for me to get the most points in the most rapid way, the speed run issue, right? 22:12.140 --> 22:25.440 Do you guys do anything to combat that in the sense that one of the things Darknet did was we didn't publish what things were worth, and we didn't necessarily publish your scores until much later on, or we give you a time frame where you could see that. 22:25.440 --> 22:30.160 Have you done anything to combat the gaming the game, or is that just part of the game at this level? 22:30.760 --> 22:43.040 Yeah, I think my philosophy on it is if there is a way to game the game that is not in the spirit of the game, then the game design is fundamentally broken and needs to be addressed. 22:43.840 --> 22:50.480 And I know in past years, like the CTF here has like hidden scores and things like that. 22:50.760 --> 22:56.360 We're, you know, we're still toying with hiding some of the scores tomorrow to, you know, increase anticipation kind of thing. 22:56.360 --> 23:06.580 But I think my philosophy is there's no sport, there's no other competitive endeavor I can think of where, oh, we don't know who's winning in the fourth quarter because we turned the scores off or something, right? 23:06.580 --> 23:17.980 Like, you know what the score is, that's knowing the score does not lessen the anticipation of, you know, seeing teams perform and be scoring and like, you know, win the game. 23:17.980 --> 23:20.340 So let me ask you a wrap-up question, unless you... 23:20.340 --> 23:28.140 Oh, no, I was just going to say, I know just speaking personally, I know that was a frustrating aspect to me when the scores go down, because then it just also encourages teams. 23:28.140 --> 23:32.640 I mean, more in Jeopardy style, where it encourages teams to go, okay, we're going to hold on to points. 23:32.940 --> 23:37.240 Where, and here it's a little harder, but I know that can add some frustration. 23:37.240 --> 23:41.280 I mean, I don't know, maybe in boxing they don't show you who wins. 23:41.500 --> 23:46.940 So to wrap up, this will not go live until tomorrow morning. 23:46.940 --> 23:49.380 When does the next round start? 23:49.780 --> 23:52.020 So the next round starts at 10 a.m. 23:52.020 --> 23:54.380 We're going to run to 2 p.m. 23:54.380 --> 23:55.780 and then we have to end. 23:55.780 --> 24:08.980 Live CTF might go a little bit longer, but we need to get our scores into the black badge thing, like tell people, yeah, to Dark Tangents, so we can do all the pomp and circumstance of awarding the winners and things like that. 24:08.980 --> 24:15.320 So do you have any surprises for the teams on the final day, or is it going to be more of what they've been seeing already? 24:17.100 --> 24:19.040 I mean, it wouldn't be a surprise, would it? 24:20.300 --> 24:22.680 I think that's the best answer they can get. 24:22.680 --> 24:23.420 It's a good point. 24:23.420 --> 24:24.700 Thank you so much. 24:24.700 --> 24:27.940 I am so excited to see some of the write-ups that come from this. 24:27.940 --> 24:32.000 Thank you for watching, and as always, hack on!