1
00:00:00,210 --> 00:00:02,550
- We've got Stewart Scott talking about

2
00:00:02,550 --> 00:00:04,710
Preserving supply side
vulnerability disclosure.

3
00:00:04,710 --> 00:00:06,730
So please give him a warm welcome.

4
00:00:06,730 --> 00:00:08,980
(applause)

5
00:00:15,600 --> 00:00:17,970
- Hey everybody, can
you guys hear me fine?

6
00:00:17,970 --> 00:00:19,710
Awesome. My name's Stuart Scott.

7
00:00:19,710 --> 00:00:21,090
I'm an assistant director

8
00:00:21,090 --> 00:00:23,700
with the Atlantic council's
cyber state craft initiative.

9
00:00:23,700 --> 00:00:25,260
and we were kind of at the intersection

10
00:00:25,260 --> 00:00:27,753
of tech policy and cyber security.

11
00:00:28,882 --> 00:00:30,030
In the past couple of years

12
00:00:30,030 --> 00:00:32,460
we've had a really great
opportunity to work a lot

13
00:00:32,460 --> 00:00:34,470
on software supply chain security,

14
00:00:34,470 --> 00:00:35,303
and do a lot of kind of

15
00:00:35,303 --> 00:00:36,900
translation from the policy community

16
00:00:36,900 --> 00:00:38,910
to tech community and the other direction.

17
00:00:38,910 --> 00:00:41,550
And part of how we're
trying to frame our thinking

18
00:00:41,550 --> 00:00:43,530
is looking at sources of risk

19
00:00:43,530 --> 00:00:45,810
and sources of security.

20
00:00:45,810 --> 00:00:47,790
and one that we've recently
kind of honed in on

21
00:00:47,790 --> 00:00:49,470
as a source of security

22
00:00:49,470 --> 00:00:51,000
has been the kind of pipeline

23
00:00:51,000 --> 00:00:53,100
of external vulnerability research

24
00:00:53,100 --> 00:00:54,930
from researchers and researching entities

25
00:00:54,930 --> 00:00:58,680
to private companies and
open source libraries.

26
00:00:58,680 --> 00:00:59,850
So as I'm sure many of you know

27
00:00:59,850 --> 00:01:02,520
the bug bounty platform
market has been growing

28
00:01:02,520 --> 00:01:05,100
and is projected to hit
some pretty crazy numbers

29
00:01:05,100 --> 00:01:06,690
in the next couple of years.

30
00:01:06,690 --> 00:01:08,250
It's a pretty direct manifestation

31
00:01:08,250 --> 00:01:10,140
of the many eyes theory
of security, right?

32
00:01:10,140 --> 00:01:11,250
More people looking at code,

33
00:01:11,250 --> 00:01:13,500
better chance of finding
and fixing vulnerabilities

34
00:01:13,500 --> 00:01:15,093
before someone exploits them.

35
00:01:16,020 --> 00:01:18,770
And we've also seen some
increasing government adoption

36
00:01:19,783 --> 00:01:20,616
- sorry.

37
00:01:20,616 --> 00:01:21,449
- Okay.

38
00:01:21,449 --> 00:01:22,282
(chuckling)

39
00:01:22,282 --> 00:01:23,610
So government's setting up
their own kind of programs

40
00:01:23,610 --> 00:01:25,590
and given some more clarity to the kind of

41
00:01:25,590 --> 00:01:28,110
legal protections and
guardrails around researchers.

42
00:01:28,110 --> 00:01:30,540
And the whole process is
generally called something like

43
00:01:30,540 --> 00:01:33,450
C V D coordinated vulnerability disclosure

44
00:01:33,450 --> 00:01:35,760
and we'll kind of refer
to it by that throughout,

45
00:01:35,760 --> 00:01:38,040
but it varies between organizations.

46
00:01:38,040 --> 00:01:40,740
And we had a really
interesting opportunity

47
00:01:40,740 --> 00:01:43,350
to look at some anonymized
bug bounty data.

48
00:01:43,350 --> 00:01:45,510
It came up in congressional
testimony in February.

49
00:01:45,510 --> 00:01:46,830
And it got us thinking about kind of

50
00:01:46,830 --> 00:01:49,590
concentrations of this source of security.

51
00:01:49,590 --> 00:01:51,870
And the big question we ask, you know

52
00:01:51,870 --> 00:01:54,000
part of what we do as policy folks

53
00:01:54,000 --> 00:01:55,530
is try to figure out good policy.

54
00:01:55,530 --> 00:01:56,430
But another part of what we do

55
00:01:56,430 --> 00:01:59,010
is try to figure out bad
policy and how to avoid it.

56
00:01:59,010 --> 00:02:00,660
So we started asking ourselves:

57
00:02:00,660 --> 00:02:02,640
What kinds of policies could disrupt

58
00:02:02,640 --> 00:02:05,130
this supply of security,
the supply of research

59
00:02:05,130 --> 00:02:07,260
that coming from a community?

60
00:02:07,260 --> 00:02:09,630
And we had a really excellent opportunity

61
00:02:09,630 --> 00:02:10,890
to dive into that question

62
00:02:10,890 --> 00:02:12,600
and look at it with some data

63
00:02:12,600 --> 00:02:14,610
after the log for Jay disclosure

64
00:02:14,610 --> 00:02:16,560
and kind of the saga around that.

65
00:02:16,560 --> 00:02:18,180
So, at the front end,

66
00:02:18,180 --> 00:02:20,310
and this is a quote from

67
00:02:20,310 --> 00:02:23,100
Carnegie Mellon CBD all
you need to know guide

68
00:02:23,100 --> 00:02:24,367
It kind of describes CBD as

69
00:02:24,367 --> 00:02:26,220
"Picking somewhere between

70
00:02:26,220 --> 00:02:28,320
finding a vulnerability
and never telling anyone,

71
00:02:28,320 --> 00:02:29,580
and finding a vulnerability

72
00:02:29,580 --> 00:02:31,140
and immediately telling the world."

73
00:02:31,140 --> 00:02:34,410
So trying to find that optimal
path of information sharing

74
00:02:34,410 --> 00:02:37,590
that gets you to fix bugs
as quickly as possible

75
00:02:37,590 --> 00:02:40,590
without kind of revealing
exploits before patches are ready.

76
00:02:41,430 --> 00:02:42,750
So the general kind of timeline

77
00:02:42,750 --> 00:02:44,640
around the log for Jay disclosures

78
00:02:44,640 --> 00:02:45,570
was a useful example of

79
00:02:45,570 --> 00:02:48,330
how that kind of information
management happens.

80
00:02:48,330 --> 00:02:50,880
So research at Alibaba cloud
found the vulnerability

81
00:02:50,880 --> 00:02:54,200
and just directly emailed the
Apache kind of maintainers

82
00:02:54,200 --> 00:02:57,630
of that library in November 24th, 2021,

83
00:02:57,630 --> 00:02:58,710
and then followed up

84
00:02:58,710 --> 00:02:59,940
as he noticed that the vulnerability

85
00:02:59,940 --> 00:03:01,470
was starting to make its way more publicly

86
00:03:01,470 --> 00:03:04,200
into some cybersecurity and for a

87
00:03:04,200 --> 00:03:05,250
which is you know kind of prompts

88
00:03:05,250 --> 00:03:08,100
the Apache needs to patch more quickly

89
00:03:08,100 --> 00:03:09,240
and to get people into position

90
00:03:09,240 --> 00:03:12,030
to mitigate and to remediate.

91
00:03:12,030 --> 00:03:13,680
And so within a couple of days of that

92
00:03:13,680 --> 00:03:15,060
second follow up email

93
00:03:15,060 --> 00:03:17,370
Apache starts pushing out their patches

94
00:03:17,370 --> 00:03:19,200
and different kind of
government organizations

95
00:03:19,200 --> 00:03:20,400
start notifying everyone.

96
00:03:20,400 --> 00:03:21,750
So that's a good
management process. Right?

97
00:03:21,750 --> 00:03:24,930
You tell people who need to
know and can fix the problem.

98
00:03:24,930 --> 00:03:27,090
And then once that information
starts becoming more public,

99
00:03:27,090 --> 00:03:28,110
you tell everyone so they can kind of

100
00:03:28,110 --> 00:03:30,390
prepare and do their side.

101
00:03:30,390 --> 00:03:32,430
But then we saw something interesting.

102
00:03:32,430 --> 00:03:35,490
On the 22nd of December, the MIIT,

103
00:03:35,490 --> 00:03:37,350
one of China's kind of ministries of

104
00:03:37,350 --> 00:03:39,750
this is industry and
information technology

105
00:03:39,750 --> 00:03:41,880
sanctioned Alibaba for how they handled

106
00:03:41,880 --> 00:03:42,900
that disclosure process.

107
00:03:42,900 --> 00:03:43,733
They were sanctioned for

108
00:03:43,733 --> 00:03:45,750
not telling the MIIT quick enough

109
00:03:45,750 --> 00:03:46,890
about the vulnerability.

110
00:03:46,890 --> 00:03:48,900
And there are two kind of legal mechanisms

111
00:03:48,900 --> 00:03:51,030
that might have been
part of that sanction,

112
00:03:51,030 --> 00:03:52,410
which involved removing Alibaba

113
00:03:52,410 --> 00:03:54,270
from an information sharing forum.

114
00:03:54,270 --> 00:03:57,030
One was that as part of the
information sharing forum

115
00:03:57,030 --> 00:03:59,670
they had to share information
and they didn't do that.

116
00:03:59,670 --> 00:04:01,680
The other that we wanted to look at was

117
00:04:01,680 --> 00:04:03,780
there is a law in the books in China,

118
00:04:03,780 --> 00:04:06,090
in effect since September, 2021

119
00:04:06,090 --> 00:04:08,520
requiring companies that
find vulnerabilities to

120
00:04:08,520 --> 00:04:10,350
immediately tell product owners.

121
00:04:10,350 --> 00:04:13,350
And then within two days
to also tell the MIIT.

122
00:04:13,350 --> 00:04:14,430
So we talked to a couple sources

123
00:04:14,430 --> 00:04:16,920
and it's unclear which
mechanism was kind of

124
00:04:16,920 --> 00:04:19,832
the force behind the sanction,

125
00:04:19,832 --> 00:04:22,710
but pretty clearly the letter of that law

126
00:04:22,710 --> 00:04:23,550
is violated, right?

127
00:04:23,550 --> 00:04:27,270
So you see the little orange
boxes network providers have to

128
00:04:27,270 --> 00:04:30,170
within two days tell the MIIT
of vulnerabilities they find

129
00:04:31,170 --> 00:04:33,540
and this to our mind is a
really convenient case study

130
00:04:33,540 --> 00:04:34,980
for testing this idea of

131
00:04:34,980 --> 00:04:37,500
security supply from
vulnerability research.

132
00:04:37,500 --> 00:04:38,970
So a lot of the reporting focuses on

133
00:04:38,970 --> 00:04:41,100
what happens to vulnerabilities
given to government early,

134
00:04:41,100 --> 00:04:43,350
but that's not what we're
focused on here, right?

135
00:04:43,350 --> 00:04:45,720
China is a huge producer
of this type of research.

136
00:04:45,720 --> 00:04:48,210
So it's very easy, in
theory, for us to tell if

137
00:04:48,210 --> 00:04:50,160
some kind of macro movement happens there.

138
00:04:50,160 --> 00:04:51,480
The law is pretty clear

139
00:04:51,480 --> 00:04:54,330
and we've got some good dates
to test before and after.

140
00:04:54,330 --> 00:04:56,820
And the kind of mechanism
for chilling effect

141
00:04:56,820 --> 00:04:58,320
is also pretty clear, right?

142
00:04:58,320 --> 00:05:00,480
If you're a private
company operating in China,

143
00:05:00,480 --> 00:05:01,313
you have to kind of consider

144
00:05:01,313 --> 00:05:03,240
your internal policies
receiving that information.

145
00:05:03,240 --> 00:05:04,380
And if you're a researching entity

146
00:05:04,380 --> 00:05:06,150
you might wanna wait on disclosing

147
00:05:06,150 --> 00:05:07,290
and even on performing research

148
00:05:07,290 --> 00:05:08,520
until there's more legal clarity

149
00:05:08,520 --> 00:05:10,140
because the law itself is pretty big.

150
00:05:10,140 --> 00:05:11,970
You know, it definitely
applies to product providers.

151
00:05:11,970 --> 00:05:14,370
It's unclear to what extent
it covers researchers

152
00:05:14,370 --> 00:05:16,740
who might be vaguely
affiliated with a company

153
00:05:16,740 --> 00:05:17,880
who might be sponsored by a company

154
00:05:17,880 --> 00:05:18,810
who might work for a company,

155
00:05:18,810 --> 00:05:20,753
but are working on their own free time.

156
00:05:21,660 --> 00:05:23,940
And to be clear, this is not
just focusing on China, right?

157
00:05:23,940 --> 00:05:26,160
We see similar issues going
on in other countries.

158
00:05:26,160 --> 00:05:28,320
It's just not as clear of a case study

159
00:05:28,320 --> 00:05:29,220
for us to analyze, right?

160
00:05:29,220 --> 00:05:31,170
So in Germany, a researcher
found a vulnerability

161
00:05:31,170 --> 00:05:33,450
in an application for election polling

162
00:05:33,450 --> 00:05:34,530
and was threatened with legal action

163
00:05:34,530 --> 00:05:35,940
even though they followed
the right channels.

164
00:05:35,940 --> 00:05:37,710
And something similar happened in Missouri

165
00:05:37,710 --> 00:05:38,543
I've seen a couple of those

166
00:05:38,543 --> 00:05:41,460
" F12 is not a crime"
t-shirts around, right?

167
00:05:41,460 --> 00:05:42,960
Guy looked at a website,

168
00:05:42,960 --> 00:05:45,030
figured out he could pull social
security numbers out of it

169
00:05:45,030 --> 00:05:48,690
and was threatened by the
kind of state government.

170
00:05:48,690 --> 00:05:49,890
But we've also seen government kind of

171
00:05:49,890 --> 00:05:51,570
embrace this approach to security,

172
00:05:51,570 --> 00:05:54,750
more DOJ declining to
prosecute good faith research

173
00:05:54,750 --> 00:05:56,070
whatever that might be.

174
00:05:56,070 --> 00:05:58,260
And ENISA And CISA each pushing for

175
00:05:58,260 --> 00:06:00,510
government agencies and
member states in the EU

176
00:06:00,510 --> 00:06:03,660
to develop their own explicit processing

177
00:06:03,660 --> 00:06:06,180
which is a better idea of what to do.

178
00:06:06,180 --> 00:06:07,590
But this is the main question, right?

179
00:06:07,590 --> 00:06:10,020
Did the RMSV have a supply shock effect

180
00:06:10,020 --> 00:06:12,000
in the Chinese research contributions?

181
00:06:12,000 --> 00:06:13,560
And if so, what did that look like?

182
00:06:13,560 --> 00:06:15,690
Did it trickle into the broader ecosystem?

183
00:06:15,690 --> 00:06:16,523
and are there other shocks

184
00:06:16,523 --> 00:06:18,143
that are worth kind of looking at?

185
00:06:19,680 --> 00:06:20,970
You know, along the way, we got to look at

186
00:06:20,970 --> 00:06:21,960
just some really cool information

187
00:06:21,960 --> 00:06:23,490
about how that ecosystem works

188
00:06:23,490 --> 00:06:24,870
where different companies are kind of

189
00:06:24,870 --> 00:06:26,640
pulling in research.

190
00:06:26,640 --> 00:06:28,800
But the basic selection
of data we looked at

191
00:06:28,800 --> 00:06:30,930
were the acknowledgements in CVEs

192
00:06:30,930 --> 00:06:33,180
and security updates from Microsoft,

193
00:06:33,180 --> 00:06:35,580
Apple, F5, VMWare and Red Hat.

194
00:06:35,580 --> 00:06:37,260
Give us a nice selection
of different products,

195
00:06:37,260 --> 00:06:39,660
rates and some consumer facing things

196
00:06:39,660 --> 00:06:42,483
some very in the racks type things.

197
00:06:42,483 --> 00:06:44,760
And with Microsoft kind of
a broad swath of all that,

198
00:06:44,760 --> 00:06:47,250
as well as some proprietary
and open source code bases

199
00:06:47,250 --> 00:06:50,340
and a nice spread of different
kind of internal processes

200
00:06:50,340 --> 00:06:53,250
for how companies report
this kind of information.

201
00:06:53,250 --> 00:06:54,960
And so we're gonna take that data,

202
00:06:54,960 --> 00:06:57,090
match it against a pretty
simple timeline, alright?

203
00:06:57,090 --> 00:06:59,010
The first RMSV kind of public mention

204
00:06:59,010 --> 00:07:00,690
is July, 2020

205
00:07:00,690 --> 00:07:03,240
laws passed in July, 2021

206
00:07:03,240 --> 00:07:05,280
and then takes effect September, 2021.

207
00:07:05,280 --> 00:07:06,690
And if we have recent enough data,

208
00:07:06,690 --> 00:07:07,620
we can even look at kind of

209
00:07:07,620 --> 00:07:09,360
post first maybe enforcement,

210
00:07:09,360 --> 00:07:10,440
even though we can't really confirm

211
00:07:10,440 --> 00:07:12,540
that it was enforced at that point.

212
00:07:12,540 --> 00:07:14,460
They don't exist in a vacuum as events.

213
00:07:14,460 --> 00:07:15,293
Of course, right?

214
00:07:15,293 --> 00:07:18,002
So the summers of 2020 and
2021 both have a lot of

215
00:07:18,002 --> 00:07:20,100
other cyber activity going on in China.

216
00:07:20,100 --> 00:07:23,010
They have a increased
sanctioning of Chinese companies

217
00:07:23,010 --> 00:07:24,063
by U.S. companies.

218
00:07:24,960 --> 00:07:27,900
And they also have a couple
other regulations being passed

219
00:07:27,900 --> 00:07:28,830
that might interfere.

220
00:07:28,830 --> 00:07:31,680
So even if we see an effect
that lines up with our time,

221
00:07:31,680 --> 00:07:32,760
we do have to be careful about kind of

222
00:07:32,760 --> 00:07:35,660
assigning causation where there
might just be correlation.

223
00:07:37,140 --> 00:07:38,490
But our basic process is pretty simple.

224
00:07:38,490 --> 00:07:40,770
Pull out all that data,
either some scraping texts

225
00:07:40,770 --> 00:07:41,790
or direct downloads,

226
00:07:41,790 --> 00:07:43,230
and huge thanks to those companies

227
00:07:43,230 --> 00:07:44,790
that make that easy to do.

228
00:07:44,790 --> 00:07:47,820
and then parse out a
CBE, a relevant date for

229
00:07:47,820 --> 00:07:49,950
when that kind of disclosure happened,

230
00:07:49,950 --> 00:07:51,570
and then the entity credited.

231
00:07:51,570 --> 00:07:54,090
And then we had a kind of list of

232
00:07:54,090 --> 00:07:55,560
companies and a list of individuals.

233
00:07:55,560 --> 00:07:57,120
We decided to just focus on the companies

234
00:07:57,120 --> 00:07:59,580
both for kind of concern for
the individual's privacy,

235
00:07:59,580 --> 00:08:00,960
and also just for ease of access.

236
00:08:00,960 --> 00:08:02,880
It's much harder to tag an individual,

237
00:08:02,880 --> 00:08:05,280
it might be a Twitter
handle or a GitHub account,

238
00:08:05,280 --> 00:08:06,113
to a location.

239
00:08:06,113 --> 00:08:07,500
But for a company it's
pretty straightforward

240
00:08:07,500 --> 00:08:08,700
and it gives us a good proxy for

241
00:08:08,700 --> 00:08:10,924
the legal context they operate in.

242
00:08:10,924 --> 00:08:12,270
And then we just batch those by month,

243
00:08:12,270 --> 00:08:14,223
clean out the data and analyze it.

244
00:08:15,180 --> 00:08:16,470
And these are the kind of
the five companies again

245
00:08:16,470 --> 00:08:17,970
that went into that process.

246
00:08:17,970 --> 00:08:21,120
And there are a lot of
issues with that data set.

247
00:08:21,120 --> 00:08:22,290
We eventually got it to work,

248
00:08:22,290 --> 00:08:24,030
but there is tons of irregularity in

249
00:08:24,030 --> 00:08:26,370
how that data's stored,
how that's organized.

250
00:08:26,370 --> 00:08:28,260
So apple, for example, will tell you about

251
00:08:28,260 --> 00:08:30,450
CBEs that it remediates and updates.

252
00:08:30,450 --> 00:08:32,756
And those updates might predate the date

253
00:08:32,756 --> 00:08:34,860
that the CBE has kind of
publicly acknowledged.

254
00:08:34,860 --> 00:08:36,630
So there's some back work going on.

255
00:08:36,630 --> 00:08:38,340
Other companies organized by CBE.

256
00:08:38,340 --> 00:08:40,350
Some companies organized
by a little bit of a blend

257
00:08:40,350 --> 00:08:42,570
of like affected product lines.

258
00:08:42,570 --> 00:08:44,310
And there's tons of miscodings

259
00:08:44,310 --> 00:08:47,340
and typos and different
organizational practices.

260
00:08:47,340 --> 00:08:48,990
But, in general, we get a good sense of

261
00:08:48,990 --> 00:08:50,343
where things are happening.

262
00:08:52,020 --> 00:08:53,130
So we're gonna talk a bit about

263
00:08:53,130 --> 00:08:54,570
the Microsoft and the Red Hat data,

264
00:08:54,570 --> 00:08:56,400
where we had some interesting findings.

265
00:08:56,400 --> 00:08:58,020
In the Apple and VMware data

266
00:08:58,020 --> 00:09:00,210
there was no supply shock effect

267
00:09:00,210 --> 00:09:01,590
but we'll show the Apple data a little bit

268
00:09:01,590 --> 00:09:02,423
just to give a sense of

269
00:09:02,423 --> 00:09:04,860
what that absence that
no finding looks like.

270
00:09:04,860 --> 00:09:06,750
And for the F5 data, we didn't find any

271
00:09:06,750 --> 00:09:09,300
even Chinese affiliated
entities participating

272
00:09:09,300 --> 00:09:10,850
in that contribution ecosystem.

273
00:09:12,750 --> 00:09:15,060
So here's a quick look at the kind of

274
00:09:15,060 --> 00:09:17,880
proportional breakdown
by month of contributions

275
00:09:17,880 --> 00:09:21,330
to vulnerability disclosure
in the Microsoft ecosystem.

276
00:09:21,330 --> 00:09:22,860
So looking at the left part of the graph

277
00:09:22,860 --> 00:09:23,880
we get a pretty good idea

278
00:09:23,880 --> 00:09:26,190
of a general story throughout
our data set, right?

279
00:09:26,190 --> 00:09:28,830
Early on the U.S. is kind
of the dominant contributor.

280
00:09:28,830 --> 00:09:31,380
Then as other countries'
IT sectors mature,

281
00:09:31,380 --> 00:09:34,110
as other kind of relationships are built

282
00:09:34,110 --> 00:09:36,240
between the kind of security center and

283
00:09:36,240 --> 00:09:37,710
Microsoft and all those researchers,

284
00:09:37,710 --> 00:09:39,000
And as they develop,

285
00:09:39,000 --> 00:09:40,710
we see more participation internationally.

286
00:09:40,710 --> 00:09:44,160
So this orange line shows
increasing Chinese contributions

287
00:09:44,160 --> 00:09:47,100
but it's true of pretty much
all the other countries.

288
00:09:47,100 --> 00:09:50,850
But what's really interesting
is the steep drop.

289
00:09:50,850 --> 00:09:53,400
Lines up, as you can see, on July, 2020.

290
00:09:53,400 --> 00:09:54,780
So one of our key dates,

291
00:09:54,780 --> 00:09:56,883
and interrupts a pretty clear trend.

292
00:09:57,900 --> 00:10:00,840
So to get a bit more
of a close look at that

293
00:10:00,840 --> 00:10:02,370
we look not just at portions,

294
00:10:02,370 --> 00:10:04,650
but the raw counts of contributions

295
00:10:04,650 --> 00:10:06,300
And the green line at the top is just

296
00:10:06,300 --> 00:10:08,040
the total number of contributions.

297
00:10:08,040 --> 00:10:10,620
It's a little more of
a nuanced story there.

298
00:10:10,620 --> 00:10:12,240
So as you can see, the drop is a bit of a

299
00:10:12,240 --> 00:10:15,300
return to the mean for the
Chinese contribution count.

300
00:10:15,300 --> 00:10:16,890
But it takes place in the context of

301
00:10:16,890 --> 00:10:19,860
overall more contributions
happening at that time,

302
00:10:19,860 --> 00:10:22,830
and not a corresponding
drop in that total amount.

303
00:10:22,830 --> 00:10:25,560
So the trendlines are
a little less dramatic,

304
00:10:25,560 --> 00:10:27,600
but that steep, steep
decline is still there

305
00:10:27,600 --> 00:10:28,710
at one of our key dates.

306
00:10:28,710 --> 00:10:30,870
It is kind of our most
compelling key date.

307
00:10:30,870 --> 00:10:33,600
It's when a lot more
sanction activity happened

308
00:10:33,600 --> 00:10:35,040
between the U.S. and China.

309
00:10:35,040 --> 00:10:36,960
And it's also when a lot of those kind of

310
00:10:36,960 --> 00:10:39,390
new Chinese cybersecurity regulations

311
00:10:39,390 --> 00:10:41,430
were drafted and started circulating.

312
00:10:41,430 --> 00:10:44,370
But, it is also a little
harder to assign causality

313
00:10:44,370 --> 00:10:46,080
to any single one of those laws

314
00:10:46,080 --> 00:10:48,230
because of that kind
of flurry of activity.

315
00:10:49,440 --> 00:10:50,730
The other data set we wanted to show off

316
00:10:50,730 --> 00:10:52,080
was the Red Hat data.

317
00:10:52,080 --> 00:10:52,913
And you'll have to forgive

318
00:10:52,913 --> 00:10:54,720
the very spiky stuff on the left here,

319
00:10:54,720 --> 00:10:58,170
but it went a lot farther
back to about 1996.

320
00:10:58,170 --> 00:11:00,180
And early on, again, very U.S. dominated

321
00:11:00,180 --> 00:11:01,950
and then more countries
start to participate.

322
00:11:01,950 --> 00:11:04,500
We see kind of more
general representation,

323
00:11:04,500 --> 00:11:06,270
but again we found an
interesting decline here.

324
00:11:06,270 --> 00:11:09,660
But not at a date we immediately
had a good explanation for.

325
00:11:09,660 --> 00:11:13,830
So this is between
February and April, 2017.

326
00:11:13,830 --> 00:11:15,270
Steep decline in the proportion of

327
00:11:15,270 --> 00:11:18,360
Chinese contributions to
this research ecosystem.

328
00:11:18,360 --> 00:11:20,190
And again, a kind of a
disruption of a trend line

329
00:11:20,190 --> 00:11:21,540
that's kind of then resumed afterwards,

330
00:11:21,540 --> 00:11:22,950
but at a decreased rate.

331
00:11:22,950 --> 00:11:25,590
And if you look at the contribution counts

332
00:11:25,590 --> 00:11:27,990
rather than the portions,
same story, again.

333
00:11:27,990 --> 00:11:28,950
It is worth noting too,

334
00:11:28,950 --> 00:11:31,050
that because as an open source ecosystem

335
00:11:31,050 --> 00:11:33,000
there's a lot more
individual work going on,

336
00:11:33,000 --> 00:11:35,370
There's a much bigger gap
between the total contributions

337
00:11:35,370 --> 00:11:37,110
and the by country contributions.

338
00:11:37,110 --> 00:11:38,850
But again, same story.

339
00:11:38,850 --> 00:11:39,930
You can see that drop off

340
00:11:39,930 --> 00:11:42,630
and then kind of maintained
throughout is a decreased rate.

341
00:11:42,630 --> 00:11:44,460
So, we had the chance to
talk to some other folks

342
00:11:44,460 --> 00:11:46,800
and they pointed out that
around this timeframe

343
00:11:46,800 --> 00:11:48,870
Red Hat had entered an
information sharing agreement

344
00:11:48,870 --> 00:11:50,370
I believe with Huawe.

345
00:11:50,370 --> 00:11:51,630
So there's something going on there.

346
00:11:51,630 --> 00:11:53,340
We don't know what the exact mechanism

347
00:11:53,340 --> 00:11:56,147
or if that's even the relevant
event, if there is one.

348
00:11:56,147 --> 00:11:58,560
But it is worth kind of looking into.

349
00:11:58,560 --> 00:12:00,000
And then just for final context,

350
00:12:00,000 --> 00:12:02,880
here are the Apple data... data sets.

351
00:12:02,880 --> 00:12:04,800
And, you know, there was no drop off here,

352
00:12:04,800 --> 00:12:06,290
but it is nice to know that kind of, again

353
00:12:06,290 --> 00:12:09,000
very prevalent U.S.
presence in the ecosystem.

354
00:12:09,000 --> 00:12:12,120
And then as other countries kind of tap in

355
00:12:12,120 --> 00:12:13,653
more broad representation.

356
00:12:14,670 --> 00:12:16,380
And here again are the
kind of total counts

357
00:12:16,380 --> 00:12:17,640
rather than the proportions.

358
00:12:17,640 --> 00:12:18,660
It's very spiky data too.

359
00:12:18,660 --> 00:12:20,580
There's some seasonality,
I think, every quarter.

360
00:12:20,580 --> 00:12:21,870
And then every other
month there'd be kind of

361
00:12:21,870 --> 00:12:24,993
an increased amount of
security updates going out.

362
00:12:26,700 --> 00:12:29,550
But the take away from those you know,

363
00:12:29,550 --> 00:12:32,370
very, very crowded graphs,

364
00:12:32,370 --> 00:12:34,260
our main finding was that on the one hand,

365
00:12:34,260 --> 00:12:36,660
there isn't a clear kind of ecosystem wide

366
00:12:36,660 --> 00:12:39,420
supply shock occurring
because of the RMSV yet,

367
00:12:39,420 --> 00:12:40,740
although it's really hard to judge

368
00:12:40,740 --> 00:12:42,270
the exact timelines going on.

369
00:12:42,270 --> 00:12:43,830
But there's definitely potential for one,

370
00:12:43,830 --> 00:12:45,270
given that kind of very steep decline

371
00:12:45,270 --> 00:12:48,240
in Microsoft, that seems easily explained.

372
00:12:48,240 --> 00:12:50,160
And also that less well explained

373
00:12:50,160 --> 00:12:52,740
but still present drop
in the red hat data.

374
00:12:52,740 --> 00:12:54,630
So our main kind of recommendation to

375
00:12:54,630 --> 00:12:56,760
government and policy
makers is to not take

376
00:12:56,760 --> 00:12:59,280
the kind of supply of
external research for granted,

377
00:12:59,280 --> 00:13:01,560
and to find ways to kind of
invest in that community.

378
00:13:01,560 --> 00:13:04,020
Either proactively avoiding supply shocks,

379
00:13:04,020 --> 00:13:05,850
or in general just to
improve kind of the volume

380
00:13:05,850 --> 00:13:08,700
and the spread of
research they're getting.

381
00:13:08,700 --> 00:13:11,100
So we figured about three
kind of different avenues

382
00:13:11,100 --> 00:13:13,290
to approach that for
government and industry.

383
00:13:13,290 --> 00:13:16,350
One is just for governments
and partners of governments

384
00:13:16,350 --> 00:13:17,730
to better harmonize their kind of

385
00:13:17,730 --> 00:13:19,440
CBD processes and guardrails, right?

386
00:13:19,440 --> 00:13:21,420
Providing clarity for researchers

387
00:13:21,420 --> 00:13:22,650
and not having to force them

388
00:13:22,650 --> 00:13:23,910
to worry about jurisdiction, right?

389
00:13:23,910 --> 00:13:26,670
A researcher in Europe,
working on an American product

390
00:13:26,670 --> 00:13:28,020
but perhaps finding a vulnerability

391
00:13:28,020 --> 00:13:29,370
while they're somewhere else.

392
00:13:29,370 --> 00:13:30,442
Just providing kind of clarity and

393
00:13:30,442 --> 00:13:32,880
clear legal protections
for them is useful.

394
00:13:32,880 --> 00:13:34,260
And it doesn't have to be too explicit

395
00:13:34,260 --> 00:13:35,400
that it kinda limits the scope

396
00:13:35,400 --> 00:13:36,842
of researchers and their ability to

397
00:13:36,842 --> 00:13:39,543
look at things in an
interesting and different way.

398
00:13:40,470 --> 00:13:42,210
Another great tool is
just supporting kind of

399
00:13:42,210 --> 00:13:43,380
vulnerability discovery tools

400
00:13:43,380 --> 00:13:45,180
takes a lot of the
burden off of researchers

401
00:13:45,180 --> 00:13:48,630
and gives them kind of new
angles to plug into that work

402
00:13:48,630 --> 00:13:50,160
without having to start from scratch.

403
00:13:50,160 --> 00:13:53,400
And then finally, we
wanted to take a pass at

404
00:13:53,400 --> 00:13:56,160
trying to incentivize work in areas where

405
00:13:56,160 --> 00:13:58,170
either some kind of supply
shock might have happened,

406
00:13:58,170 --> 00:14:00,600
or where there's simply not
a lot of research going on.

407
00:14:00,600 --> 00:14:01,950
In general, if there's you know,

408
00:14:01,950 --> 00:14:03,390
some product type or ecosystem

409
00:14:03,390 --> 00:14:05,130
where there are not a lot of
vulnerabilities being found

410
00:14:05,130 --> 00:14:07,350
any vulnerabilities found
there have more security value

411
00:14:07,350 --> 00:14:09,990
and government and industry
can shape incentives

412
00:14:09,990 --> 00:14:12,420
and, you know, bug
bounty rewards to reflect

413
00:14:12,420 --> 00:14:13,870
that kind of increased value.

414
00:14:14,910 --> 00:14:17,490
But to quickly sum up our
work, we wanted to start

415
00:14:17,490 --> 00:14:20,670
thinking about supply side
vulnerability research,

416
00:14:20,670 --> 00:14:22,110
start thinking about it as a community

417
00:14:22,110 --> 00:14:24,060
that contributes to overall security,

418
00:14:24,060 --> 00:14:27,390
and figure out: Can bad policy
kind of shock that system?

419
00:14:27,390 --> 00:14:28,590
And if so, how to avoid it.

420
00:14:28,590 --> 00:14:30,270
And we found that, you know,

421
00:14:30,270 --> 00:14:32,250
our best case study so far doesn't provide

422
00:14:32,250 --> 00:14:33,960
an overwhelming abundance of evidence

423
00:14:33,960 --> 00:14:35,730
for that supply shock, but
it is kind of compelling

424
00:14:35,730 --> 00:14:37,620
and a little bit troubling.

425
00:14:37,620 --> 00:14:39,960
And fortunately, there are
things that governments

426
00:14:39,960 --> 00:14:42,930
and companies can do to kind
of protect that community.

427
00:14:42,930 --> 00:14:45,570
And that doesn't depend
on there being necessarily

428
00:14:45,570 --> 00:14:47,070
a clear case for a supply shock,

429
00:14:47,070 --> 00:14:49,313
because it's a good investment
to make in security.

430
00:14:50,370 --> 00:14:51,203
And finally,

431
00:14:51,203 --> 00:14:52,410
this is still very much
a working project for us.

432
00:14:52,410 --> 00:14:54,030
We've got some open
ended research questions

433
00:14:54,030 --> 00:14:56,280
about which companies stopped contributing

434
00:14:56,280 --> 00:14:58,410
to the Microsoft ecosystem, for example.

435
00:14:58,410 --> 00:14:59,580
So we'd love kind of, any feedback,

436
00:14:59,580 --> 00:15:00,900
any kind of experience y'all have had

437
00:15:00,900 --> 00:15:03,630
with you know, contributing
to work from companies

438
00:15:03,630 --> 00:15:05,970
either in bug bounty programs or directly.

439
00:15:05,970 --> 00:15:08,310
Happy to answer some questions,

440
00:15:08,310 --> 00:15:10,440
and our emails are up here,
if you'd like to reach out.

441
00:15:10,440 --> 00:15:11,640
And you can check out the full kind of

442
00:15:11,640 --> 00:15:12,600
draft report on our website.

443
00:15:12,600 --> 00:15:14,183
But, thanks so much for having us.

444
00:15:15,152 --> 00:15:17,402
(applause)

445
00:15:21,513 --> 00:15:22,964
- We got a few minutes
to take some questions.

446
00:15:22,964 --> 00:15:23,797
- Yeah, sure.

447
00:15:23,797 --> 00:15:25,710
We have a few minutes, if
you have any questions,

448
00:15:25,710 --> 00:15:27,960
if not happy to give
everyone some time back.

449
00:15:30,720 --> 00:15:32,503
Also, I cannot see out there, so.

450
00:15:32,503 --> 00:15:34,753
(Chuckles)

451
00:15:36,540 --> 00:15:37,800
- Agreed.

452
00:15:37,800 --> 00:15:38,633
- [Man] Yeah.

453
00:15:38,633 --> 00:15:39,466
- Awesome. Thanks so much.

454
00:15:39,466 --> 00:15:40,763
- Thank You.

455
00:15:40,763 --> 00:15:42,808
(applause)