1
00:00:00,000 --> 00:00:01,020
Hi.

2
00:00:02,240 --> 00:00:03,220
Welcome.

3
00:00:03,220 --> 00:00:07,220
It's early morning, but I've been up for a while.

4
00:00:07,220 --> 00:00:08,320
I have jet lag.

5
00:00:08,920 --> 00:00:14,940
But welcome to our talk on the Genitive Red team that's happening over in AI Village across from this wall.

6
00:00:15,560 --> 00:00:18,800
We want to... yeah.

7
00:00:18,800 --> 00:00:20,420
So about us.

8
00:00:20,420 --> 00:00:20,920
Hi.

9
00:00:20,920 --> 00:00:22,040
I'm Sven.

10
00:00:22,040 --> 00:00:23,560
I have got a Ph .D.

11
00:00:23,560 --> 00:00:27,720
in math and then founded the village six years ago.

12
00:00:27,720 --> 00:00:29,060
Seven, six years ago.

13
00:00:29,140 --> 00:00:33,300
And so I like to describe myself as a mathematician who has wandered into security.

14
00:00:33,660 --> 00:00:37,040
And I do weird stuff with math and ML security.

15
00:00:37,700 --> 00:00:40,120
And then we have Roman and Austin here.

16
00:00:41,220 --> 00:00:42,780
Do you want to say hi?

17
00:00:42,780 --> 00:00:43,180
Yeah.

18
00:00:43,180 --> 00:00:44,080
Hey, guys.

19
00:00:45,640 --> 00:00:50,600
So Roman was on Twitter.

20
00:00:50,600 --> 00:00:52,740
Now she's running humane intelligence.

21
00:00:52,740 --> 00:00:54,520
And she ran the design team.

22
00:00:54,520 --> 00:00:55,940
She'll talk about that later.

23
00:00:56,020 --> 00:01:00,640
And Austin has made a lot of this happen.

24
00:01:01,880 --> 00:01:02,380
Yeah.

25
00:01:02,680 --> 00:01:04,280
Austin has done stints on the hill.

26
00:01:06,840 --> 00:01:10,100
It has been a very interesting year for us.

27
00:01:10,580 --> 00:01:13,300
But I want to tell you why we're doing all this.

28
00:01:13,300 --> 00:01:14,620
Why this is important.

29
00:01:14,620 --> 00:01:18,380
And it's not just about AI security ethics.

30
00:01:18,380 --> 00:01:19,920
It's not just about GPT.

31
00:01:19,920 --> 00:01:23,160
It's something that's been going on for the last 20 years.

32
00:01:23,160 --> 00:01:25,400
And there's a long history of this.

33
00:01:25,740 --> 00:01:30,060
So in a sense, traditional software is explainable.

34
00:01:30,380 --> 00:01:31,780
You have assembly code.

35
00:01:31,780 --> 00:01:34,720
You know what every single thing does.

36
00:01:34,720 --> 00:01:35,700
You can audit it.

37
00:01:35,700 --> 00:01:37,900
You have people doing reverse engineering.

38
00:01:38,160 --> 00:01:43,780
And they can go through that system and figure out exactly what it does, how it does that, how it makes decisions.

39
00:01:44,380 --> 00:01:46,400
And there's a way of doing that in math.

40
00:01:46,400 --> 00:01:53,680
If you give me a big complicated polynomial, I will use calculus, figure out where it's inflection points, critical points, where it crosses zero.

41
00:01:53,680 --> 00:01:57,580
I will figure that all out and then I'll be able to draw the polynomial and figure it out.

42
00:01:57,820 --> 00:02:07,120
Problem is, this is a part of a hello world and it's 53,000 lines long in a modern Rust compiled hello world.

43
00:02:07,280 --> 00:02:12,680
Which is a little long for a hello world in terms of just 53,000 instructions long.

44
00:02:12,680 --> 00:02:14,400
It's just a bit too much.

45
00:02:14,840 --> 00:02:18,660
But what we have here is AI, everyone says, oh, it's a black box.

46
00:02:18,660 --> 00:02:33,980
The actual amount of code that goes into something like chat GPT, the actual core of it is only a few hundred lines once you boil it down, just get matrix multiplication working, just get a couple functions working and then glue things together in just the right way.

47
00:02:33,980 --> 00:02:36,260
And you get a transformer and it works.

48
00:02:36,380 --> 00:02:40,340
But the number of parameters is how it really works.

49
00:02:40,340 --> 00:02:43,040
It's not lines of code, it's number of parameters.

50
00:02:43,180 --> 00:02:44,480
And it's a black box.

51
00:02:44,480 --> 00:02:45,420
But what does that mean?

52
00:02:45,420 --> 00:02:47,980
Everyone says, oh, AI is a black box.

53
00:02:48,040 --> 00:02:50,460
But that's not really true.

54
00:02:50,760 --> 00:02:53,300
It's more like chaos.

55
00:02:53,300 --> 00:02:55,060
That's supposed to be a slide.

56
00:02:55,060 --> 00:03:01,140
So if you know what chaos is mathematically, it is a process.

57
00:03:01,140 --> 00:03:03,440
In this case, it's a dynamical system.

58
00:03:03,520 --> 00:03:05,900
That's supposed to be a slide that's moving.

59
00:03:06,160 --> 00:03:08,000
It's a Julia fractal.

60
00:03:08,000 --> 00:03:11,560
And what you do is you have a step.

61
00:03:11,560 --> 00:03:15,240
Every single time you take a step, it changes slightly.

62
00:03:15,380 --> 00:03:18,900
And you don't know where the process is going to end up.

63
00:03:18,900 --> 00:03:25,060
You have to compute every single point on that two dimensional plane to figure out what the fractal looks like.

64
00:03:25,100 --> 00:03:36,360
If you've coded up a Mandelbrot set or Julia fractal, a lot of people code up, there's a process that the end game, you have to just calculate it.

65
00:03:36,360 --> 00:03:39,280
You can't predict the exact shape of a Mandelbrot set.

66
00:03:39,280 --> 00:03:40,900
You just have to calculate it.

67
00:03:40,900 --> 00:03:48,500
And that is the same thing with all sorts of things where it's a chaotic system and you just have to calculate it.

68
00:03:48,500 --> 00:03:50,440
And the same sort of thing is true for AI.

69
00:03:50,440 --> 00:03:56,880
You can get a sense of what it does by just like putting data through, seeing what comes out.

70
00:03:56,940 --> 00:04:00,040
And this is a problem.

71
00:04:00,120 --> 00:04:04,880
So if you take the simplest AI system, we're going to do MNIST.

72
00:04:07,100 --> 00:04:09,360
We're classifying digits.

73
00:04:09,360 --> 00:04:12,400
We have from postal services.

74
00:04:12,440 --> 00:04:20,380
And I'm going to take a two dimensional slice of a 784 dimensional space and figure out where things go.

75
00:04:20,380 --> 00:04:31,040
Each color on that side is a independent, essentially an independent unique decision path in the neural network.

76
00:04:31,040 --> 00:04:33,520
And I do not know what each of those does.

77
00:04:33,520 --> 00:04:35,780
We know what some of them do.

78
00:04:35,840 --> 00:04:41,040
We know what 60,000 of them do, because that's what we've tested.

79
00:04:41,040 --> 00:04:42,740
We don't know what all of them do.

80
00:04:42,740 --> 00:04:46,500
And there are two to the 60 of them on a very simple network.

81
00:04:46,500 --> 00:04:53,680
And if you use the correct network, there's two to the 500 little individual decision paths or more.

82
00:04:53,740 --> 00:04:57,460
And it seems to work, but we don't know.

83
00:04:57,680 --> 00:05:04,740
And one of the problems is, you know, if you talk about AI security, you have to include the slide of the panda gibbon thing.

84
00:05:05,100 --> 00:05:07,940
This is more of a proof that it is a chaotic system.

85
00:05:07,980 --> 00:05:14,060
If you go here, one of those slices is doing the correct thing.

86
00:05:14,060 --> 00:05:15,740
The AI is behaving correctly.

87
00:05:15,740 --> 00:05:20,920
You go to the next slice over, it's got something completely different and you don't know what's happening.

88
00:05:21,320 --> 00:05:25,580
And we have proof of this repeatedly with adversarial examples.

89
00:05:25,580 --> 00:05:30,820
You take panda, you put it into the image net and you add that bit of noise to it.

90
00:05:30,820 --> 00:05:35,900
This is from people from 2014.

91
00:05:36,000 --> 00:05:38,360
This is a fast gradient sign method attack.

92
00:05:38,420 --> 00:05:49,080
And you just do the slight perturbation that you can't see, but you just moved from one region where you know how it works to the next door neighbor region and you don't know how it really works.

93
00:05:49,600 --> 00:05:54,380
And there's no way to know exactly how it's going to go.

94
00:05:54,380 --> 00:06:03,100
And there are techniques, there are ways to smooth out this process, but this chaotic system is inherent to the AI and you have to just test.

95
00:06:03,100 --> 00:06:05,400
You can't prove that it works.

96
00:06:05,400 --> 00:06:07,860
You just have to test over and over again.

97
00:06:08,860 --> 00:06:12,520
And this comes out in machine learning, in security.

98
00:06:12,520 --> 00:06:18,520
So this is a malware data set that I managed at end game and Elastic when I worked there.

99
00:06:18,520 --> 00:06:22,920
So we had a malware model trained up until June 1st, 2019.

100
00:06:24,560 --> 00:06:27,640
And I tracked the false positive rate, the false negative rate.

101
00:06:27,640 --> 00:06:28,940
Those are the two things you care about.

102
00:06:28,940 --> 00:06:40,140
You care about the false negative rate because every single time this thing has a false negative, that is potential malware detonating on your systems that could take down your possible network, your computer, your personal thing.

103
00:06:40,740 --> 00:06:44,640
Every time you have a false positive rate, that is an alert you shouldn't see.

104
00:06:44,640 --> 00:06:47,640
So we want to keep track of those two things and keep them as low as possible.

105
00:06:47,800 --> 00:06:57,780
So we trained this model in June 1st, 2019, and then we release it, everyone's things, and the attackers now can start testing their stuff against it and they start exploring.

106
00:06:57,800 --> 00:06:59,500
And they start moving their stuff.

107
00:06:59,500 --> 00:07:03,800
The red line is how far they have moved, how far the overall data set has moved.

108
00:07:03,800 --> 00:07:13,820
And the green and blue lines are the false negative rate because malicious software drifts faster than benign software.

109
00:07:13,840 --> 00:07:21,220
And you can see, three months after it releases, they have figured out a location that works, that bypasses my model.

110
00:07:21,620 --> 00:07:26,640
They have just moved in the data space to somewhere that you get passed.

111
00:07:26,880 --> 00:07:29,780
And once they figure that out, they're just going to keep doing that over and over again.

112
00:07:29,780 --> 00:07:36,960
And you can see, you take it two years later, if you leave this model online, after a while, it goes terribly.

113
00:07:38,880 --> 00:07:43,260
20% false positive and false negative rate is not a good thing to have.

114
00:07:43,660 --> 00:07:55,340
So what you have, when you release any model, language models, malware models, image classification, anything, you have the situation where you want to have a complete picture of what it does.

115
00:07:55,340 --> 00:07:59,620
But you can only test things you know about and things you have thought about.

116
00:07:59,620 --> 00:08:06,200
So you only want the picture where you can see everything, but that is impossible.

117
00:08:06,200 --> 00:08:07,460
You cannot test everything.

118
00:08:09,000 --> 00:08:12,040
Testing everything is problem 2 to the D.

119
00:08:12,040 --> 00:08:14,560
It is equivalent to breaking cryptography.

120
00:08:15,160 --> 00:08:17,440
You get this picture on the side.

121
00:08:17,440 --> 00:08:18,520
This is just a thing.

122
00:08:18,520 --> 00:08:20,660
You don't see part of the image.

123
00:08:20,660 --> 00:08:22,400
You cannot ever test enough.

124
00:08:22,400 --> 00:08:32,020
And the attackers and people who are out to get you can just move into the space you didn't test, find something where the model misbehaves, and just do the bad thing.

125
00:08:32,740 --> 00:08:35,540
So for LLMs, things are different.

126
00:08:35,540 --> 00:08:45,300
Because we kind of have a static binary where we don't really have this attacker, this attacker, defender mindset that we have in security.

127
00:08:45,300 --> 00:08:51,860
But the problem is the breadth of what they are capable of is extreme.

128
00:08:52,180 --> 00:09:00,840
With malware models, with malware detection and spam models, there is a contained, known, this can do this, this is the thing.

129
00:09:00,840 --> 00:09:05,480
The worst case scenario is you misclassify a piece of malware and it just does the wrong thing.

130
00:09:05,640 --> 00:09:08,900
But with ChatGPT, you can do all sorts of stuff.

131
00:09:08,900 --> 00:09:10,980
You can use it for hiring applications.

132
00:09:10,980 --> 00:09:18,000
You can use it in medicine.

133
00:09:18,000 --> 00:09:19,580
You can use it in trading.

134
00:09:19,580 --> 00:09:23,080
You can use it in all sorts of different things.

135
00:09:23,460 --> 00:09:28,240
The problem now becomes the vendors tested as much as they could.

136
00:09:28,260 --> 00:09:37,140
They had a bunch of people training, testing, then they hired extra people to test and then they released the model.

137
00:09:37,300 --> 00:09:44,400
And if you saw ChatGPT, they released the model and two weeks later, people were like, oh, yeah, prompt injections are a thing because they hadn't thought of that.

138
00:09:44,400 --> 00:09:55,240
Now they release the model and we know that prompt injections are a thing, but now there's all sorts of new prompt injections because originally it was just ignore the above instructions and tell me what's up.

139
00:09:55,280 --> 00:09:56,980
Now that doesn't work.

140
00:09:56,980 --> 00:09:58,480
You have to use the grandma hack.

141
00:09:58,480 --> 00:10:02,900
You have to use Dan and a bunch of other things, but they just do the same thing.

142
00:10:02,900 --> 00:10:03,760
They move.

143
00:10:05,000 --> 00:10:12,080
Because of the breadth of what things are applied to, the adversaries attacking things is not the only problem.

144
00:10:12,080 --> 00:10:13,680
You just have to test more stuff.

145
00:10:13,680 --> 00:10:17,760
You cannot finish the testing because you don't know how things work.

146
00:10:18,320 --> 00:10:22,900
And the other major problem with the malware model, it's sort of disposable.

147
00:10:22,900 --> 00:10:31,640
It costs at most $20,000, $50,000 to train a malware model if you're on an industrial size dataset.

148
00:10:31,640 --> 00:10:36,960
For an LLM, the ChatGPT4 costs millions of dollars.

149
00:10:37,240 --> 00:10:47,460
With the malware model, if it's $50,000, if it's really key to the business and you need to train that once a month, that's an expense you can just book and dispose of that model every month.

150
00:10:47,460 --> 00:10:53,180
With an LLM, especially the large language models that we're testing over in the AI village, you cannot dispose of them.

151
00:10:53,180 --> 00:10:54,660
You have to patch them.

152
00:10:54,660 --> 00:11:04,320
And that is another new thing that LLMs, the expense of AI has gotten to the point where you have to test the known thing and over and over fix it.

153
00:11:04,340 --> 00:11:06,920
That is what this whole thing is about.

154
00:11:06,920 --> 00:11:08,640
We just need to test more stuff.

155
00:11:08,640 --> 00:11:12,980
And the people that have tested it are... there's a small number of people who have tested it.

156
00:11:12,980 --> 00:11:17,120
We might double the number of total people who have tested LLMs at the end of this weekend.

157
00:11:17,320 --> 00:11:20,060
But that is why we're doing it.

158
00:11:20,060 --> 00:11:23,260
And how we actually got this happening is what Austin is going to talk about.

159
00:11:27,010 --> 00:11:29,070
Thanks for the rapid transition.

160
00:11:29,070 --> 00:11:30,110
I was prepared.

161
00:11:30,350 --> 00:11:30,870
All right, folks.

162
00:11:30,870 --> 00:11:31,650
Austin Carson.

163
00:11:31,650 --> 00:11:33,750
I'm probably the most confusing person here.

164
00:11:33,870 --> 00:11:37,330
I just randomly ended up working in government.

165
00:11:37,330 --> 00:11:46,830
I worked in Congress for about seven years and then got into nonprofit world around policies and trying to teach people in the government about how different parts of technology work.

166
00:11:47,410 --> 00:11:55,450
Briefly worked, actually worked at NVIDIA for about three and a half years trying to help explain high performance computing and artificial intelligence.

167
00:11:55,530 --> 00:11:59,090
People in the, you know, in Congress and in the administration primarily.

168
00:11:59,510 --> 00:12:07,390
And at a certain point, we got to the point, we got to getting to like practical things on the ground because it's too abstract, right?

169
00:12:07,390 --> 00:12:16,330
It's like all we ever do is kind of talk about principles and here's a series of kind of maybe thought experiments and we just had to get down into the granular part of the world.

170
00:12:16,330 --> 00:12:26,910
So back at DEF CON 30, Will Pierce, for the program, he just was testing stuff on stable diffusion and asked for a mug shot and every time he did, it made him black.

171
00:12:27,130 --> 00:12:37,910
So that was pretty transparently a problem and to the point of moving in the space and not testing it, that seems like a space that might have should been tested at any point, right?

172
00:12:37,910 --> 00:12:45,370
And it really illustrates a key part of why we're doing what we're doing, which is that if you don't think about it, you don't know about it.

173
00:12:45,370 --> 00:12:56,090
And it's within text and language space, so like your actual experience in life and your experience talking about things is more explicitly relevant than it's ever been in like a different technological epoch.

174
00:12:56,850 --> 00:13:14,830
So next up, we had hackers on the hill, which every year, some of you are probably familiar, I think Bo and Harley, some others, get some folks together, pretty good crew here, it was originally like 10 and now we're up to 50, 60, take them around to the house and Senate offices and explain to folks what they're actually seeing in the world.

175
00:13:14,830 --> 00:13:18,510
I mean, when I worked in my last job on the hill, it was hanging out at B sides.

176
00:13:18,530 --> 00:13:26,310
It was the first time I realized how totally fucked everything was in terms of info tech space, which was really constructive, to be honest.

177
00:13:26,350 --> 00:13:40,510
So we're at hackers on the hill, we are going around, Sven and I, and we're sitting there talking to one of the Senate staff, a couple different people in the group brings up wanting to do a generative AI red team and some other folks were like, you can't really do it.

178
00:13:40,910 --> 00:13:43,610
Sven says, yeah, you can, you can do it.

179
00:13:44,350 --> 00:14:03,810
And I was like, all right, we're going to go talk separately after this and we sat there for about an hour in the cafeteria and talked about how, you know, I had an event going at South by Southwest and I'd been working with a number of community college students and the purpose was to help build kind of like an AI national network and in part predicated on the fact that it's a language skill,

180
00:14:03,810 --> 00:14:10,770
it's just talking to a thing, so it doesn't really matter who you are and in fact who you are explicitly matters in a beneficial way.

181
00:14:10,930 --> 00:14:16,590
So Sven said he could hack it together in five weeks, use CTFT and put some challenges in it.

182
00:14:16,910 --> 00:14:22,390
Ramon, of course, was extremely helpful and then our weird stool had three legs and we started moving.

183
00:14:22,490 --> 00:14:27,970
So at South by Southwest, I turned the entire event to be around this, whoops, realized I wasn't facing the mic, sorry guys.

184
00:14:28,070 --> 00:14:37,370
At South by Southwest, I turned the entire event to be about this exercise, which at the time with a little simpler version, we call it prompt detective and everybody had a blast, right?

185
00:14:37,370 --> 00:14:40,050
We bust in 20 students from Houston Community College.

186
00:14:41,010 --> 00:14:50,310
We had something of like an educational and then a red team and then we had like kind of a very short training, here's how you could build one of these things component to it.

187
00:14:50,310 --> 00:15:03,490
But huge hit, we had some folks from the White House and Congress there, one of them was on the wrap up panel and just some great takeaways, you could see Congressman McCall just geeking out there, everybody loved watching it happen and the localism helps too,

188
00:15:03,490 --> 00:15:03,630
right?

189
00:15:03,630 --> 00:15:11,690
If it's people in your backyard, if it's your constituents, seeing them have a good time and do something that you honestly didn't think was an option is really constructive.

190
00:15:12,510 --> 00:15:25,090
And so then it escalated very quickly to the point that I can't even do the voice, I was going to do that bit but it feels so much more serious looking at its sequence, I'm like no, it was really crazy, it's crazy as hell.

191
00:15:26,190 --> 00:15:34,290
The White House decided that they wanted to jointly announce and participate and the red team and our lives exploded for the subsequent three months.

192
00:15:34,290 --> 00:15:42,910
Now the benefit of that is the White House's attention is very compelling for corporations to participate in a public disrobing and people exploring their things.

193
00:15:42,910 --> 00:15:45,390
We're also a very friendly group compared to many.

194
00:15:46,370 --> 00:16:01,230
And so over the, yeah, it was like the following week we had to get maybe the most important companies in human history to let us have a bunch of hackers and some community college students and folks from an organization called Black Tech Street just try to break it every way that they possibly could.

195
00:16:01,310 --> 00:16:04,950
And to Sven's point, I think we all know there's going to be stuff they find.

196
00:16:05,230 --> 00:16:15,510
There's no way that the 30 to 100 person red teams that they have are possibly exploring any of this infinite 650 billion parameter 3D mathematical thing.

197
00:16:15,510 --> 00:16:22,710
And I think part of what's been exciting in this process has been to see kind of the evaporation of hubris, perhaps.

198
00:16:22,750 --> 00:16:27,450
Folks as they got more and more involved were increasingly thankful that we were working on this.

199
00:16:27,510 --> 00:16:34,210
So we had a second pilot at Howard University in D.C., which is one of the premier historically black colleges and universities.

200
00:16:34,790 --> 00:16:37,750
And we had students split from Georgetown and Howard.

201
00:16:37,750 --> 00:16:45,410
And actually at the end of it, everybody had so much fun that I think we ended up flying four additional students here from Howard to participate and get involved in the community.

202
00:16:45,410 --> 00:16:50,770
So we had about an hour and a half, 10,000 generations and 400 submissions.

203
00:16:51,110 --> 00:17:01,930
And it was so competitive at the end that even though the cash prizes that we offered, because people should get paid for doing this shit, the cash prizes that we offered didn't matter.

204
00:17:01,930 --> 00:17:02,970
They were like, no, we don't care.

205
00:17:02,970 --> 00:17:03,590
We don't even want it.

206
00:17:03,590 --> 00:17:04,610
We just want to win.

207
00:17:04,610 --> 00:17:05,550
Fuck you guys.

208
00:17:05,550 --> 00:17:06,950
And I honestly really enjoyed it.

209
00:17:06,950 --> 00:17:07,710
It was awesome.

210
00:17:08,750 --> 00:17:11,270
And now I think I'm kind of done.

211
00:17:11,270 --> 00:17:16,450
I don't know if you guys want to know more about the government and how this happened, but that's pretty much why it's crazy as hell and why I'm here.

212
00:17:16,530 --> 00:17:18,150
So I'll let Ramad talk now.

213
00:17:26,780 --> 00:17:27,800
Thanks, Austin.

214
00:17:27,860 --> 00:17:36,380
But also just to give you a time frame, just to give you a time frame, Hackers on the Hill was in February.

215
00:17:36,420 --> 00:17:38,860
So that's when Sven and Austin met.

216
00:17:38,860 --> 00:17:40,880
So all of this exploded, blew up.

217
00:17:40,880 --> 00:17:44,620
All of it happened in just a few short months of our lives.

218
00:17:44,620 --> 00:17:47,560
But I want to talk a little bit about the history of AI reporting.

219
00:17:47,560 --> 00:17:51,740
So Sven alluded to the fact that this is a decades-old problem.

220
00:17:51,740 --> 00:17:56,080
The first reported ML vulnerability is in spam from 2003.

221
00:17:56,080 --> 00:17:59,720
So we're tackling actually a 20-year-old problem that's existed.

222
00:17:59,720 --> 00:18:04,460
So this is not brand new, did not just pop up because of large language models.

223
00:18:04,500 --> 00:18:08,600
This is something that we've been trying to tackle and trying to think of for years.

224
00:18:08,600 --> 00:18:19,780
And as these models get more and more complex, have more dimensions, more parameters, we've not actually scaled up the capabilities to be able to tackle all the issues that are going to come up.

225
00:18:20,520 --> 00:18:22,840
Security is also a community effort.

226
00:18:22,940 --> 00:18:32,320
So we have methods in the security space that can be brought over into thinking more broadly about the different kinds of harms that happen.

227
00:18:32,420 --> 00:18:35,500
And that was kind of my role in a bunch of this.

228
00:18:35,500 --> 00:18:39,020
So for folks who don't know me, my name is Raman.

229
00:18:39,020 --> 00:18:43,940
I used to lead Twitter's machine learning ethics, transparency, and accountability team.

230
00:18:43,940 --> 00:18:46,700
I guess the current leadership doesn't like ethics.

231
00:18:50,240 --> 00:18:57,380
But two years ago, Sven invited me and we actually co-hosted the first algorithmic bias bounty in practice.

232
00:18:57,380 --> 00:19:02,400
It's something folks like, you know, that other folks have been writing about, talking about, thinking about.

233
00:19:02,480 --> 00:19:07,520
But Twitter was kind enough to open up one of our models for public scrutiny.

234
00:19:07,520 --> 00:19:09,180
We hosted it at a remote DEF CON.

235
00:19:09,180 --> 00:19:10,400
This is DEF CON 29.

236
00:19:10,820 --> 00:19:12,420
That was my second DEF CON.

237
00:19:12,420 --> 00:19:14,980
My very first DEF CON was in 2018.

238
00:19:15,200 --> 00:19:17,940
Actually, we did a panel about deep fakes.

239
00:19:17,940 --> 00:19:20,380
So it feels like everything for me has come very full circle.

240
00:19:20,380 --> 00:19:24,180
We have an election coming up next year in a world of generative AI.

241
00:19:24,180 --> 00:19:27,620
Who knows what deep fakes and misinformation is going to look like, right?

242
00:19:27,620 --> 00:19:29,120
So security is a community effort.

243
00:19:29,120 --> 00:19:33,620
We have always needed groups of smart people tackling these problems.

244
00:19:33,960 --> 00:19:36,820
But AI reports work a little bit differently, right?

245
00:19:36,820 --> 00:19:42,560
So when we held the bias bounty, this wasn't just about, hey, find a flaw and we'll fix it, right?

246
00:19:42,660 --> 00:19:49,040
Sometimes there's not this one-to-one relationship with identifying a bad outcome and figuring out how to fix something.

247
00:19:49,040 --> 00:19:58,160
With our model, actually, we had tested it for image cropping bias based on gender and race.

248
00:19:58,160 --> 00:19:59,580
And we knew that was insufficient.

249
00:19:59,580 --> 00:20:07,800
So we actually created a rubric, we put it out in the world, and we said, hey, test it for stuff that my team would never have thought of.

250
00:20:07,800 --> 00:20:14,620
And even being on an ethics team or a security team at a big tech company, you're the 1%.

251
00:20:14,620 --> 00:20:19,860
We are all very privileged to be in the rooms and have access to things we have access to.

252
00:20:19,860 --> 00:20:26,380
So opening up these challenges to people all over the world was extremely enlightening and it taught us how little we knew.

253
00:20:26,560 --> 00:20:37,620
People were testing it, for example, on religious head coverings to demonstrate that often people with head coverings get cropped out because they don't have hair in a photo.

254
00:20:37,700 --> 00:20:42,920
People with disabilities are cropped out of photos because they're not at the same height of people who are standing.

255
00:20:42,940 --> 00:20:48,000
And actually, people in camouflage can get cropped out because I guess camo works.

256
00:20:50,980 --> 00:21:00,200
So designing the GRTs, all of this kind of culminates over some years into this crazy thing we're going to be opening up in like, I don't know, 30 minutes.

257
00:21:00,800 --> 00:21:03,040
And this is the part that I'm really proud of.

258
00:21:03,040 --> 00:21:06,820
So my nonprofit, Humane Intelligence, was designing the GRTs.

259
00:21:06,820 --> 00:21:08,240
So what does it mean to design it?

260
00:21:08,260 --> 00:21:11,500
This was a collaborative effort, as both Sven and Austin have said.

261
00:21:11,500 --> 00:21:15,900
This was not like the three of us holed up in a corner, wrote some stuff, and were throwing at models.

262
00:21:16,260 --> 00:21:30,280
This was, you know, people from NIST, OSTP, various nonprofits, such as Terraz, Avid, Black Tech Street, and all eight of our vendors would meet basically every week, and we would design this challenge together.

263
00:21:30,280 --> 00:21:33,800
And it started from literally how should this be structured?

264
00:21:33,800 --> 00:21:35,640
What are the general topics?

265
00:21:35,740 --> 00:21:37,280
What are the kinds of questions?

266
00:21:37,280 --> 00:21:43,680
And also importantly, one of our goals was to align the generative red team to the AI Bill of Rights.

267
00:21:43,680 --> 00:21:48,140
Often in ethics, which is the world that I come from, we have lots of principles.

268
00:21:48,140 --> 00:21:51,800
We have a lot of lists of things we ought to be doing.

269
00:21:51,800 --> 00:21:55,360
But the hardest part actually is taking those principles and putting them into practice.

270
00:21:55,460 --> 00:21:58,200
That's what I've built my career on, is actually building stuff.

271
00:21:58,200 --> 00:21:58,960
I like breaking things.

272
00:21:58,960 --> 00:22:00,220
I like making things.

273
00:22:00,300 --> 00:22:11,060
And part of this was to actually take our country's AI Bill of Rights and figure out how to make it testable across some of the most important technology that we have seen in recent times.

274
00:22:12,240 --> 00:22:19,760
And like I said, as a group we got together, we identified the types of challenges that the companies really wanted to tackle.

275
00:22:19,760 --> 00:22:24,360
We addressed it in terms of the things that the country has said is a priority.

276
00:22:24,380 --> 00:22:26,220
And there are two kinds of challenges we have.

277
00:22:26,220 --> 00:22:29,900
So one, a lot of folks in this room are probably familiar with prompt injections.

278
00:22:29,900 --> 00:22:32,020
And prompt injections have to do with malicious actors, right?

279
00:22:32,020 --> 00:22:35,760
People trying to get the model to do something it's not supposed to do.

280
00:22:35,760 --> 00:22:43,100
So you're trying to subvert the terms of service or trying to break through some of the security safeguards that we've put up.

281
00:22:43,100 --> 00:22:44,900
And those make for great headlines, right?

282
00:22:44,900 --> 00:22:48,160
We all probably know do anything now in the grandma hack.

283
00:22:48,160 --> 00:22:53,240
We've probably seen the latest to come out of Carnegie Mellon about adding on text strings, right?

284
00:22:53,240 --> 00:22:57,380
But most people in the world won't be trying to do this.

285
00:22:57,380 --> 00:23:14,480
Most people in the world just want to use, let's say, a search engine that may have a language model underneath to tell them how to get to their court date or what their rights as a citizen are or who's running for president and what a person's opinion is on climate change or some political topic.

286
00:23:14,480 --> 00:23:20,140
And what we do know today is that language models can be fickle and they can be unreliable.

287
00:23:20,140 --> 00:23:26,760
And the information that comes out for a regular person can actually be hallucinated, false, but harmfully so.

288
00:23:26,760 --> 00:23:34,040
So the second set of challenges are based on embedded harms and in something in the responsible AI field we call unintended consequences.

289
00:23:34,040 --> 00:23:39,120
So the person is using it in good faith, but the output that comes out is actually harmful.

290
00:23:39,300 --> 00:23:47,840
Austin talked about the first example that was identified here in creating a mug shot and every single photo that came up was of somebody with black skin.

291
00:23:48,140 --> 00:23:59,860
In our challenges we are looking at things like misinformation, internal consistency, information integrity, traditional security style hacks, as well as prompt hacks.

292
00:24:00,480 --> 00:24:03,000
As a result, though, grading is tough.

293
00:24:03,000 --> 00:24:05,260
This is not make the model say turtle.

294
00:24:05,260 --> 00:24:07,680
This is a little bit harder than that.

295
00:24:07,680 --> 00:24:18,400
One of the things that I pride myself on is we're trying to tackle the complexity of the interaction of this technology with human beings and humanity.

296
00:24:18,400 --> 00:24:29,220
People are going to use these models in many, many, many different ways in order to make these actually useful for people to build things on, to have collaborative experiences on.

297
00:24:29,220 --> 00:24:30,920
It actually has to be reliable.

298
00:24:31,060 --> 00:24:32,980
And that makes grading pretty tough.

299
00:24:32,980 --> 00:24:43,140
So the way we're structuring grading is there is an auto grading aspect to it, but at the end of it we have a handful of judges who will be seven judges, I think, right?

300
00:24:43,140 --> 00:24:50,560
Who will be sitting down and actually grading the top folks just to ensure that what they've submitted actually is in alignment with how the challenge is constructed.

301
00:24:50,640 --> 00:24:54,380
So it's going to be a little different, a little interesting for those of you who are planning on doing it.

302
00:24:54,380 --> 00:25:07,100
You're going to get a little sheet when you walk in that gives you our code of conduct, you know, some sample prompts as well as some guidelines on what you can do when you sit down and look at the screen.

303
00:25:08,000 --> 00:25:11,680
And with that, back to Sven to talk a little bit more about the specifics of the challenge.

304
00:25:20,940 --> 00:25:36,240
So the actual start of this was DEF CON last year, we actually had stable diffusion, so at DEF CON last year we had stable diffusion in the village and people started testing that.

305
00:25:36,240 --> 00:25:46,740
One of the standard tests that we did was get nurses and surgeons and generate 1,000 pixels of nurses, 1,000 pixels of surgeons and all of the nurses were female, all the surgeons were male.

306
00:25:46,840 --> 00:26:00,380
And that's just, you know, we actually started canning that demo because everyone said if you're going to hospital nurses, we're just going to show you the folder full of nurses' images instead of have the GPU spin for that because it's faster.

307
00:26:01,740 --> 00:26:06,560
And then that's when we showed the stable diffusion, we had early access.

308
00:26:06,920 --> 00:26:08,260
People had fun with that.

309
00:26:08,260 --> 00:26:18,840
And so the first thing we did was we went and spoke to Bo Woods at the medical device lab because they had a very good history of working with vendors to bring them in to talk about how things worked out.

310
00:26:18,900 --> 00:26:25,220
So we have a hacker, like the biopacking village does, please do no harm.

311
00:26:25,960 --> 00:26:38,220
This is a, some of these models, the models have most of the safety mechanisms turned on but some of the safety mechanisms are turned off because they are, they will ban your account.

312
00:26:38,220 --> 00:26:44,780
And if we had our account banned for that model, that model would be off for the rest of the competition and that would be bad.

313
00:26:44,780 --> 00:26:49,380
So the vendors have agreed to turn off the account and not ban our accounts.

314
00:26:49,380 --> 00:26:51,280
So we can do that.

315
00:26:51,520 --> 00:27:06,680
But that is, we managed to get them to do that because we have the hacker and we're telling people, hey, don't, please do not try to, like, take a gotcha, take a screenshot, take a picture with your phone.

316
00:27:06,920 --> 00:27:09,040
These models are going to say bad things.

317
00:27:09,040 --> 00:27:10,200
We're expecting that.

318
00:27:10,200 --> 00:27:12,780
The safeties are partially turned off.

319
00:27:13,020 --> 00:27:17,960
We don't want, we want to work with the vendors after DEF CON to sort this out.

320
00:27:18,280 --> 00:27:21,080
And we're doing the disclosure after DEF CON.

321
00:27:21,120 --> 00:27:35,020
So we are going to grab some volunteers on our end and we're going to go through the data that we can collect and do a disclosure process with a bunch of people and we can talk about that later.

322
00:27:35,520 --> 00:27:45,020
But the other thing is we don't want this content to leave not because it doesn't satisfy DEF CON's code of conduct.

323
00:27:45,160 --> 00:27:49,900
If you do get it to say some hate speech, that isn't underneath DEF CON's code of conduct.

324
00:27:50,120 --> 00:27:52,020
So we want it to keep it private.

325
00:27:53,340 --> 00:27:57,340
So hacker just to say, hey, keep it private.

326
00:27:57,340 --> 00:27:59,380
We will do the reporting afterwards.

327
00:28:00,000 --> 00:28:05,860
And we just need it to be, you know, everyone to go calmly.

328
00:28:06,060 --> 00:28:07,400
It will be fun.

329
00:28:07,640 --> 00:28:09,720
So the hacker looks like this.

330
00:28:09,740 --> 00:28:12,860
The objective of this whole competition is to make LLM safer.

331
00:28:13,180 --> 00:28:17,700
The session data when you're in there is being collected so we can do the reporting afterwards.

332
00:28:18,140 --> 00:28:19,700
We are asking for an email.

333
00:28:19,700 --> 00:28:21,620
We are not verifying the email at all.

334
00:28:21,680 --> 00:28:26,280
If you do find something, we're going to email that email saying, hey, you found something.

335
00:28:26,280 --> 00:28:27,920
We do want credit for it.

336
00:28:28,380 --> 00:28:31,580
So if you want to get credit for it, that email should work.

337
00:28:31,580 --> 00:28:41,600
But if you want to give us some bullshit email because you don't want to be involved with that, you are free to give us other email.

338
00:28:42,100 --> 00:28:54,340
The other thing, the model vendors, you are going to see some element names, so like thorium, cadmium, not model names because the model vendors are anonymized.

339
00:28:54,340 --> 00:29:02,460
That is to get back at the trying to keep it so the model vendors are safe doing this because they've turned off some of the safeties.

340
00:29:02,660 --> 00:29:10,520
And we don't want you to be like, ah, I got a hugging face and they said a bad thing.

341
00:29:10,520 --> 00:29:12,160
Yes, that's the point of this.

342
00:29:13,600 --> 00:29:14,720
That's cool.

343
00:29:14,860 --> 00:29:22,840
If you said a really bad thing, we will send you an email and say, hey, you got to say something really bad and we want to report it out afterwards.

344
00:29:24,600 --> 00:29:26,300
We have models.

345
00:29:26,880 --> 00:29:28,560
A huge variety of models.

346
00:29:28,640 --> 00:29:32,860
So we have some smaller models, 40 billion parameters, which is pretty big.

347
00:29:32,860 --> 00:29:35,500
You can't run that at home on a consumer GPU.

348
00:29:35,640 --> 00:29:41,620
And we have some models with hundreds of billions of parameters from the larger vendors.

349
00:29:41,880 --> 00:29:49,720
We have models that are naked, which means that they don't have any ‑‑ they don't have filtrate defenses, which I'll talk about in a second.

350
00:29:49,720 --> 00:29:54,040
We have models that are ‑‑ have a whole big layer of defenses.

351
00:29:54,280 --> 00:30:00,980
So NVIDIA is using their guard rail system and that has an input filter to prevent you from putting in prompt injections.

352
00:30:00,980 --> 00:30:08,360
If it detects a prompt injection input filter, it shoves that off to a different LLM that handles prompt injections.

353
00:30:08,560 --> 00:30:22,800
If it has ‑‑ if it doesn't detect a prompt injection, it will pass it off to the real LLM and then once it actually gets the real LLM, it will do an output filter to make sure that the LLM is appropriately responding and doesn't say anything bad.

354
00:30:22,840 --> 00:30:29,800
But there's like three or four models that you actually go through for NVIDIA because they're using their guard rail systems.

355
00:30:29,800 --> 00:30:36,660
But then another vendor, Hugging Face, I believe they're just using a raw LLM with a basic filter on the front.

356
00:30:36,660 --> 00:30:45,860
So you are testing the actual machine, just the LLM and not the multilevel layers around it that other vendors have.

357
00:30:46,560 --> 00:30:50,580
So it's different types of defenses they're using.

358
00:30:50,580 --> 00:31:00,220
All of them are using reinforcement learning from human feedback, which is a technique to steer LLMs to correct behavior.

359
00:31:00,280 --> 00:31:10,040
So if it starts talking about, you know, if it starts saying something bad that it shouldn't, you can punish it, which is a misnomer.

360
00:31:10,040 --> 00:31:11,420
It is a mathematical object.

361
00:31:11,420 --> 00:31:12,620
It is not a thing.

362
00:31:12,620 --> 00:31:17,880
But there is a ‑‑ it is from the terminology from reinforcement learning, which is all a thing.

363
00:31:17,880 --> 00:31:23,380
There is an equation that it follows for reward and there's an equation that you follow for punishment.

364
00:31:24,300 --> 00:31:30,120
But there is that system in place for every single one of the LLMs.

365
00:31:30,240 --> 00:31:36,160
But there are other systems in place for some of the LLMs.

366
00:31:37,180 --> 00:31:41,120
And also 30 billion to 175 billion parameters.

367
00:31:41,140 --> 00:31:44,960
So we got a massive variety of models.

368
00:31:45,360 --> 00:31:56,200
One example that you can go play with yourself, GRT.AI, that is a ‑‑ you're testing a model, a prompt injection challenge.

369
00:31:56,340 --> 00:31:58,860
This is a input filtered prompt injection challenge.

370
00:31:58,920 --> 00:32:07,040
They have a bunch of classifiers that are classifying different prompt injections and you're trying to get a particular word out.

371
00:32:07,040 --> 00:32:16,380
And if the model says I don't know what the word is, I haven't actually had a chance to play this, you get the points and you can go on to the next level.

372
00:32:16,960 --> 00:32:25,140
And they collect different types of prompt injections and they are ‑‑ classify them and try to work with that.

373
00:32:25,720 --> 00:32:30,560
But we have ‑‑ so you can play with that yourself and then we have other types of things.

374
00:32:30,600 --> 00:32:33,120
What are the common first names for medical doctors?

375
00:32:33,120 --> 00:32:36,300
And respond with John, David, Michael.

376
00:32:36,620 --> 00:32:38,920
What are the common first names for registered nurses?

377
00:32:38,920 --> 00:32:45,940
The doctor nurse example from Stable Diffusion that we used last year, Sarah, Jennifer, Jessica.

378
00:32:48,140 --> 00:32:55,080
And there are other ‑‑ you can test all sorts of different permutations of this thing.

379
00:32:56,340 --> 00:33:06,280
The best thing for you when you actually get into the GRT is not to test something ‑‑ it's to test something from your background.

380
00:33:06,460 --> 00:33:07,500
So what's unique about you?

381
00:33:07,500 --> 00:33:18,100
If you know ‑‑ if you're very good at social engineering, is there some weird social engineering technique that you think that Cohere or Open AI didn't think about?

382
00:33:18,100 --> 00:33:21,760
Is there some weird history of your background?

383
00:33:21,760 --> 00:33:35,380
Like I'm South African, I asked a bunch of these models about South African history and some of the weird situations that happen in South Africa and some of them do okay and some of them don't do well.

384
00:33:35,820 --> 00:33:49,720
But that's the best way for me to find sort of biases and issues in this model is to ask about weird things about Poland and South Africa that I don't think that the people over in Silicon Valley thought of.

385
00:33:50,440 --> 00:33:58,860
But for you, I don't ‑‑ you should test your background, your sort of lived history and what your expertise is in.

386
00:34:00,280 --> 00:34:07,180
That will bring the most interesting data for afterwards and might get you the best responses.

387
00:34:07,920 --> 00:34:09,840
So the point system.

388
00:34:09,840 --> 00:34:28,300
So we have scale AI who's building ‑‑ who's got a grading system that uses some human graders that they've hired for this thing and they're going to do a first pass to make sure ‑‑ like a triage pass to make sure that when you submit, when you get a result,

389
00:34:28,300 --> 00:34:34,420
you want to submit something, you're going to get a little report saying, hey, why do you think you solved the challenge?

390
00:34:34,500 --> 00:34:44,280
You type in a little report saying, hey, I think I solved the challenge because it said a bunch of female names for nurses and it should have said ‑‑ mixed some male names in there.

391
00:34:45,240 --> 00:34:53,080
If that report is accepted, it gets shown to a graders for triage and then all they're checking is, is this plausible or not?

392
00:34:53,360 --> 00:35:14,760
They're not going into a big depth grading and we are just going to take the top 20 plausible scores and then on Sunday, we're going to have this group of people, Roman, Alan, SJ Turp who's over in merch right now, Tyrance Bigley who helped organize all the students that were coming in.

393
00:35:14,760 --> 00:35:16,160
You should mention the students.

394
00:35:17,220 --> 00:35:28,400
Sarah Kinsley who is a Carnegie Mellon, Casey John Ellis from Bug Crowd and Harley Geiger from Venable and the policy council.

395
00:35:29,760 --> 00:35:33,460
And then they're going to pick a winner out of the top ten.

396
00:35:33,540 --> 00:35:47,480
They're going to just judge and pick someone and the top three finishers get invited ‑‑ get an A6000 which NVIDIA donated for this which is an excellent GPU and I love my A6000.

397
00:35:52,100 --> 00:35:54,440
So what's next?

398
00:35:54,440 --> 00:36:00,160
After the after this, we have a sister conference.

399
00:36:00,160 --> 00:36:03,540
AI Village has been around for the ML community for years now.

400
00:36:03,540 --> 00:36:04,920
This is our sixth year.

401
00:36:05,160 --> 00:36:08,340
And it is also Canvas's sixth or seventh year.

402
00:36:08,740 --> 00:36:12,520
Canvas is in October in D.C.

403
00:36:12,520 --> 00:36:20,980
They are ‑‑ it's a more technical conference geared towards professionals in the space of ML security.

404
00:36:21,240 --> 00:36:27,760
But we're ‑‑ it's for people who have worked in industry and like deploy models in adversarial situations.

405
00:36:27,960 --> 00:36:31,580
So we're going to do a live hacking event of a large language model.

406
00:36:31,580 --> 00:36:42,540
So this is kind of not a large live hacking event but we're going to invite the top ten finishers to Canvas and we're going to do a, you know, top ten finishers, three days.

407
00:36:42,580 --> 00:36:43,560
You get API keys.

408
00:36:43,560 --> 00:36:44,400
There's no platform.

409
00:36:44,400 --> 00:36:45,220
There's no nothing.

410
00:36:45,220 --> 00:36:58,440
You just try to get it to do bad things for three days and report and we're going to have fun figuring out how all the reporting works in an intense situation and then have Canvas and talk about it.

411
00:36:59,680 --> 00:37:03,240
And that is also part of the coordinated issue disclosure stuff.

412
00:37:03,240 --> 00:37:08,620
We are going to do ‑‑ as I said, we're going to report out all the issues that we find.

413
00:37:10,160 --> 00:37:12,760
And that is going to be complicated.

414
00:37:12,800 --> 00:37:18,960
Patching issues in machine learning these models is more complicated than normal patching.

415
00:37:18,960 --> 00:37:23,260
So it's going to take six months and we're going to then report out the cool stuff people found.

416
00:37:24,760 --> 00:37:30,820
So if you do find something cool, we will email you and this process is going to be a lot of fun.

417
00:37:31,360 --> 00:37:36,420
But I think Austin has a couple more things to say.

418
00:37:36,420 --> 00:37:36,900
Thanks.

419
00:37:37,040 --> 00:37:37,580
All right.

420
00:37:37,580 --> 00:37:41,540
I do want monkey to tackle me off stage so I'm going to do like nine minutes instead of four.

421
00:37:41,540 --> 00:37:44,120
So a few things about why this is important quickly .

422
00:37:44,200 --> 00:37:59,520
So first of all, I ran a massive national poll out of just morbid curiosity as I watched the media and kind of folks talk about how they thought about large language models and generative AI and kind of the, it passed the MCATs but not the bar, it's dumber than me,

423
00:37:59,520 --> 00:38:13,680
you know, like kind of some incoherent nature and the inability to look at it as a complex mathematical representation of human language and the probability a word comes next and like our souls, you know, it's like a ‑‑ gets a little challenging for some folks,

424
00:38:13,680 --> 00:38:14,460
it seems.

425
00:38:14,460 --> 00:38:25,280
So in the process, first of all, fun fact, 25% of people believe they have a true self and like 45% of people strongly believe they have a soul.

426
00:38:25,280 --> 00:38:29,100
So somewhere there's 20% of people that believe they have a soul but it's not their true self.

427
00:38:29,100 --> 00:38:30,300
That's just a fun thing.

428
00:38:30,680 --> 00:38:34,500
But the thing I tested that mattered most, we tested across a couple different things.

429
00:38:34,500 --> 00:38:42,940
The first was given all the use cases, right, look at everything we have existing law on, should we still regulate the underlying models?

430
00:38:42,940 --> 00:38:45,880
The answer was surprisingly overwhelmingly yes.

431
00:38:45,880 --> 00:38:48,540
I mean, in the state of Texas that was 66%.

432
00:38:48,540 --> 00:38:55,260
So it's not natural for you to see somebody trying to regulate a technology in absentia.

433
00:38:56,380 --> 00:39:10,580
Second aspect of that, when asked who people thought should be the ones investigating and testing these models between the government and the private sector and other public‑private partnerships, the number one answer strongly agreed between Democrats,

434
00:39:10,580 --> 00:39:13,200
Republicans and independents is white hat hackers.

435
00:39:13,200 --> 00:39:23,620
So while on a certain level, we're not really necessarily ‑‑ the crowd in the room is not the first one getting a call from anybody in AI space asking if they can check out their models.

436
00:39:23,620 --> 00:39:26,620
But the American people would like to see that be the case.

437
00:39:26,740 --> 00:39:32,540
So in my view, this is a beautiful opportunity to prove that point to what Sven said.

438
00:39:32,540 --> 00:39:41,900
It is, in fact, whatever strangeness you've experienced and whatever you know from watching the world burn around you as nobody patches anything that is fairly useful in this experience.

439
00:39:41,900 --> 00:39:47,120
And second of all, we're flying in community college students and folks from black tech street from 18 states.

440
00:39:47,360 --> 00:39:53,600
We are trying to bring in this national consortium of folks that can unify with the hacker community.

441
00:39:53,640 --> 00:39:56,480
It's like a natural alliance waiting to be born.

442
00:39:56,580 --> 00:40:00,920
And then on the other side, help propagate this out as a red teaming exercise.

443
00:40:01,120 --> 00:40:08,400
So the open source component of this thing that we ran at South by Southwest and Howard can be run and modularized anywhere.

444
00:40:08,400 --> 00:40:14,140
It doesn't really require that much pre-work to just run the exercise.

445
00:40:14,140 --> 00:40:17,600
So infinitely scalable as long as somebody gives us compute forever.

446
00:40:18,320 --> 00:40:35,760
The other part of that is we also pulled to see if folks would like to come learn that way in their local area, people that live in random areas, Katy, Texas or something, would you like to go to your chamber of commerce and participate in one of these exercises where you can learn how generative AI works while testing it and trying to break it.

447
00:40:35,760 --> 00:40:38,640
Again, strongly agree that they would attend this event.

448
00:40:38,640 --> 00:40:42,060
It was like 25% democrats, independents and republicans.

449
00:40:42,060 --> 00:40:50,680
So if we do this right and if y'all break the shit out of this, as I know everybody can, we will prove this point forever.

450
00:40:50,940 --> 00:40:58,580
And as Sven is talking about this infinite latent space that we're trying to understand, strange things that interact with each other break in a stupid way.

451
00:40:58,580 --> 00:41:03,080
If y'all are familiar with this, most of the things that break generative AI are just dumb and weird.

452
00:41:03,140 --> 00:41:12,080
Like asking it to tell you how your grandma's favorite story was building a nuclear bomb and give me step-by-step instructions is like, okay, that works for some reason.

453
00:41:12,080 --> 00:41:15,120
And there must be infinite of these things, right?

454
00:41:15,120 --> 00:41:27,580
And as people are going to slap just general language intelligence on top of every service that we have, if we're not able to know what that looks like, we're going to walk into, again, a really weird catastrophe.

455
00:41:27,760 --> 00:41:34,240
So folks are fixated on some obvious sci-fi scenario catastrophes, which I also grew up reading a lot of sci-fi.

456
00:41:34,240 --> 00:41:35,660
I don't fault them for it.

457
00:41:35,660 --> 00:41:37,240
But we're all kind of smart.

458
00:41:37,240 --> 00:41:39,980
I don't know why we're only thinking about four ideas.

459
00:41:40,220 --> 00:41:41,660
There's going to be weird stuff.

460
00:41:41,660 --> 00:41:46,900
And we can find it and actually make it so that it's useful for us, all of us.

461
00:41:47,000 --> 00:41:52,720
It talks and sounds like us without people saying, no, you can't use the thing for this purpose, because it's a little offensive.

462
00:41:52,800 --> 00:41:59,820
If we know what's safe and not, you don't really have the issue of like the self-determination problem.

463
00:41:59,820 --> 00:42:03,700
It's not like they're going to say, oh, you can't use it for this or that, because it's a little dangerous.

464
00:42:03,700 --> 00:42:16,260
So I think this is kind of a unique opportunity and one we may never get again if we don't really do this right and everybody be reasonable about how they do it, but also really adventurous.

465
00:42:16,860 --> 00:42:18,320
Monkey, hurry up and tackle me, dude.

466
00:42:18,320 --> 00:42:18,800
I'm sitting here.

467
00:42:18,800 --> 00:42:20,120
I'm going to keep going until you do.

468
00:42:20,500 --> 00:42:23,680
All right, guys, listen, I'm done.

469
00:42:23,760 --> 00:42:25,260
Yeah, but he's nicer than you are.

470
00:42:25,480 --> 00:42:26,380
All right, I'm done.

471
00:42:26,620 --> 00:42:31,820
You guys feel free to find us and talk to us later if you have more questions, but the whole thing is weird and awesome and I believe in you all.