1
00:00:00,000 --> 00:00:12,220
Out of all things in the world, it is now my pleasure to introduce you from the MITRE Corporation, Dan Adonofi and Anthony Singleton to talk about CVEs.

2
00:00:12,360 --> 00:00:15,780
So, Anthony, Dan, take it away.

3
00:00:16,080 --> 00:00:20,260
Thank you very much. Hi, folks. Welcome to DEF CON. Y'all having fun?

4
00:00:21,420 --> 00:00:24,500
Yeah? All right, now I'm going to lay some facts on you here.

5
00:00:24,500 --> 00:00:33,340
Okay, this is, the talk, the whole point of this talk is to give you, the audience, an understanding of, A, what CVE IDs are.

6
00:00:33,940 --> 00:00:41,500
Okay, so it's, there's a lot of misconceptions about what CVE IDs are, how they're used, how they're assigned, why they're not assigned, things like that.

7
00:00:41,680 --> 00:00:45,640
So we're going to talk a little bit about that. We're going to talk about how you get these things.

8
00:00:45,840 --> 00:00:50,000
It's amazingly simple, but it's always, there's always a little catch, right?

9
00:00:50,040 --> 00:00:52,180
So we're going to talk a little bit about what those catches are.

10
00:00:54,500 --> 00:00:58,380
Getting a CVE ID assigned is different from getting it published.

11
00:00:58,820 --> 00:01:05,060
So we're going to talk a little bit about that, and especially addressing why you might see a reference to a CVE ID in the public,

12
00:01:05,160 --> 00:01:09,360
but you're not seeing it in the CVE ID list, and why that is.

13
00:01:11,060 --> 00:01:17,900
We're also going to just mention how you can update and work with CVE to improve the content that we have,

14
00:01:17,900 --> 00:01:20,240
improve the data that we have, so that the whole community benefits.

15
00:01:20,800 --> 00:01:23,200
And then hopefully we'll have time at the end for some questions.

16
00:01:24,500 --> 00:01:31,040
All right. So, first off, anyone in here who doesn't know what a CVE, has never heard of CVE?

17
00:01:32,080 --> 00:01:34,740
Okay, good. Common Vulnerabilities and Exposures.

18
00:01:35,040 --> 00:01:40,920
Okay, so CVE, it is a dictionary of publicly known cybersecurity vulnerabilities.

19
00:01:42,160 --> 00:01:45,960
The whole point of CVE ID is to uniquely identify vulnerabilities,

20
00:01:46,200 --> 00:01:53,400
so that you know when you're talking to a scanning vendor, some other security entity,

21
00:01:53,400 --> 00:01:57,640
who's doing a vulnerability analysis on you, your own internal teams.

22
00:01:57,740 --> 00:02:00,880
When you talk about a vulnerability, you're all talking about the same thing.

23
00:02:01,000 --> 00:02:09,040
It's an apples-to-apples conversation, and you can use the CVE ID to pivot around all the different tools and conversations that might be out there.

24
00:02:09,240 --> 00:02:15,800
If you look at the antivirus world, antivirus software vendors name things each differently.

25
00:02:16,040 --> 00:02:20,600
You know, if you look at different viruses, they have like 10, 20 names, depending on who's named them,

26
00:02:20,620 --> 00:02:22,880
and when they named them, and who they think did it, and all that.

27
00:02:23,400 --> 00:02:28,260
It's very confusing. So CVE was created to address that kind of problem.

28
00:02:29,080 --> 00:02:38,760
And so, again, when you're talking about a vulnerability, you're using a name that everyone understands from tool to tool and from conversation to conversation.

29
00:02:39,180 --> 00:02:39,740
Yeah?

30
00:02:41,740 --> 00:02:42,300
.

31
00:02:53,900 --> 00:02:57,800
I'll talk a little bit more about what data we do have and what we can do in a bit.

32
00:02:57,920 --> 00:03:06,040
So, again, there's a lot of misconceptions about CVE, a lot of misconceptions about what people expect of us.

33
00:03:07,460 --> 00:03:10,040
We are, first of all, a de facto standard.

34
00:03:10,240 --> 00:03:12,900
We're not a standard. IETF never said, let there be CVE.

35
00:03:13,420 --> 00:03:18,060
It was something that MITRE and the U.S. government put together to initially help MITRE, actually.

36
00:03:18,240 --> 00:03:21,180
It was an internal project that the government said, hey, this is a great idea.

37
00:03:21,300 --> 00:03:22,300
We should do something with it.

38
00:03:22,300 --> 00:03:23,440
And it grew from there.

39
00:03:24,080 --> 00:03:25,400
It is, again, a dictionary.

40
00:03:26,300 --> 00:03:27,920
And it acts as that pivot point.

41
00:03:28,100 --> 00:03:32,400
Now, what it is not, CVE is not a vulnerability mitigation itself.

42
00:03:32,580 --> 00:03:38,040
We sometimes get e-mail from people saying, how can I use this CVE to fix my computer?

43
00:03:39,100 --> 00:03:41,200
That's not quite the right question, right?

44
00:03:41,340 --> 00:03:44,340
The question should be, okay, I've identified a CVE.

45
00:03:44,420 --> 00:03:45,940
Where do I get more information about it?

46
00:03:47,260 --> 00:03:49,800
We're not a vulnerability database, okay?

47
00:03:49,960 --> 00:03:52,280
To us, a vulnerability database is something that,

48
00:03:52,300 --> 00:03:56,180
takes a CVE ID, a uniquely named object,

49
00:03:56,420 --> 00:03:59,920
and then enriches the information that they have about it.

50
00:04:00,040 --> 00:04:02,800
So, for example, the National Vulnerability Database, NVD.

51
00:04:03,480 --> 00:04:05,760
They are a consumer of our information.

52
00:04:05,960 --> 00:04:09,100
They take that information in, and then they add to it.

53
00:04:09,180 --> 00:04:10,260
They add CVSS scores.

54
00:04:10,540 --> 00:04:12,000
They add other references.

55
00:04:12,300 --> 00:04:16,880
They add things like risk assessment or attack vectors.

56
00:04:17,120 --> 00:04:20,000
Stuff that we don't necessarily publish in ours.

57
00:04:20,200 --> 00:04:21,160
And that's done purposely.

58
00:04:22,300 --> 00:04:24,400
We're not out there to compete with vulnerability vendors.

59
00:04:24,620 --> 00:04:26,820
We're just there to name the vulnerabilities.

60
00:04:29,400 --> 00:04:32,500
So, again, we don't include risk assessment.

61
00:04:32,700 --> 00:04:33,900
We don't talk about impact.

62
00:04:34,560 --> 00:04:36,140
We don't talk about the fixes.

63
00:04:36,600 --> 00:04:38,300
We don't care if there's a fix or not.

64
00:04:38,640 --> 00:04:41,260
We don't really dive too far into the technical information.

65
00:04:41,540 --> 00:04:44,860
That's not what our service is about.

66
00:04:45,040 --> 00:04:48,940
Now, that information is useful when determining whether or not something is a vulnerability.

67
00:04:49,260 --> 00:04:50,300
And we'll talk about that in a minute.

68
00:04:50,300 --> 00:04:55,220
But it is not required for a CVE ID to be assigned and published.

69
00:04:56,580 --> 00:05:00,280
We also, by the way, we're not there to announce vulnerabilities.

70
00:05:01,080 --> 00:05:06,840
CVE IDs are only published in our list when they have become public already.

71
00:05:06,980 --> 00:05:13,280
There has to be a public reference to the vulnerability somewhere out there on the Internet that we can point to.

72
00:05:13,460 --> 00:05:18,100
Because we don't want to be the first point of publishing for anything.

73
00:05:18,580 --> 00:05:19,340
We don't want to be there.

74
00:05:19,900 --> 00:05:21,360
Like announcing a zero day.

75
00:05:21,460 --> 00:05:22,780
That's not what we are there for.

76
00:05:23,980 --> 00:05:25,160
There's a few reasons for that.

77
00:05:25,620 --> 00:05:28,100
Mainly, we just don't want to become...

78
00:05:30,760 --> 00:05:31,840
There's a lot of legal stuff.

79
00:05:32,040 --> 00:05:32,840
Let's just put it that way.

80
00:05:32,940 --> 00:05:34,320
Our lawyers looked at it.

81
00:05:34,420 --> 00:05:36,300
Our management looked at it.

82
00:05:36,340 --> 00:05:38,300
And they just decided that's not what we want to do.

83
00:05:39,580 --> 00:05:47,300
But this drives a lot of why you might have problems seeing CVE IDs published when they're assigned.

84
00:05:47,660 --> 00:05:49,280
And, again, we'll talk about that in a minute.

85
00:05:49,340 --> 00:05:53,100
So who can assign a CVE ID?

86
00:05:53,980 --> 00:05:58,380
So the primary assigner of CVE IDs is still the MITRE Corporation.

87
00:05:59,480 --> 00:06:04,740
Just so you understand, the MITRE Corporation is a not-for-profit organization.

88
00:06:05,360 --> 00:06:10,160
We run seven federally funded research and development centers for the U.S. government.

89
00:06:11,660 --> 00:06:13,520
CVE work is under one of those.

90
00:06:14,140 --> 00:06:17,380
So MITRE does a whole bunch of things, CVE being one of them.

91
00:06:17,380 --> 00:06:22,860
And we operate all the guts behind the program.

92
00:06:23,080 --> 00:06:28,340
And we administer both the technology, the operational side, and the administrative side.

93
00:06:28,520 --> 00:06:31,300
We work with our sponsors in the Department of Homeland Security.

94
00:06:31,460 --> 00:06:34,500
And we work with stakeholders in the community to provide this service.

95
00:06:35,700 --> 00:06:41,760
In addition to MITRE, we have what we call the CNAs, the CVE Numbering Authorities.

96
00:06:41,940 --> 00:06:46,120
These are organizations that we've given the...

97
00:06:46,120 --> 00:06:53,380
we've delegated authority to to assign CVE IDs based on a common set of rules that we call the CNA rules.

98
00:06:53,580 --> 00:06:54,780
We're very good at naming things.

99
00:06:55,080 --> 00:07:02,420
So these CNA rules define things like who's responsible for what, who should be assigning what,

100
00:07:02,600 --> 00:07:05,700
what information they need to share, what information they need to make public.

101
00:07:05,920 --> 00:07:10,940
Any CNA must have a disclosure policy, for example.

102
00:07:11,260 --> 00:07:14,960
CNAs can be someone like Microsoft, Apple, Cisco.

103
00:07:14,960 --> 00:07:20,400
It can also be research organizations like Cisco Talos or Rapid7.

104
00:07:20,900 --> 00:07:24,440
Rapid7 is actually a hybrid because they have their products and they do their research.

105
00:07:24,820 --> 00:07:31,000
It can also be individual researchers, people who have come to us and are working with us a little more closely

106
00:07:31,000 --> 00:07:37,340
and are producing enough CVEs that it makes sense for them to get a stack of CVEs at the beginning of each year

107
00:07:37,340 --> 00:07:39,680
and then just assign those as they need them.

108
00:07:39,920 --> 00:07:44,940
That's the real...the main benefit of being a CNA is that you get a block of CVEs,

109
00:07:44,940 --> 00:07:51,280
CIDs ahead of time, and you have more control when you're a product vendor, for example,

110
00:07:51,400 --> 00:07:52,820
over the disclosure process.

111
00:07:53,200 --> 00:07:58,080
Right now, if a company isn't a CNA and one of their products has a vulnerability in it,

112
00:07:58,160 --> 00:08:01,220
it's discovered, the researcher comes to MITRE and says,

113
00:08:01,320 --> 00:08:03,240
I found a vulnerability in product X.

114
00:08:03,800 --> 00:08:06,440
If they're not a CNA, we'll say, yep, looks like a vulnerability.

115
00:08:06,940 --> 00:08:08,460
We'll go through the assignment process.

116
00:08:08,840 --> 00:08:14,240
If they are a CNA, if company X for product X is a CNA,

117
00:08:14,240 --> 00:08:19,140
we would pass it to them, and they would have the first crack at understanding the vulnerability,

118
00:08:19,440 --> 00:08:20,320
setting the timeline.

119
00:08:20,600 --> 00:08:24,880
As long as they're following their published disclosure process, they have that control.

120
00:08:25,480 --> 00:08:29,820
So if you work at a company that has a PCERT function, we'd love to talk to you.

121
00:08:30,040 --> 00:08:35,260
If you're a researcher who's doing a lot of vulnerability research, we'd like to talk to you.

122
00:08:35,360 --> 00:08:39,260
If you're a nation state who has a CERT, we'd like to talk to you.

123
00:08:39,320 --> 00:08:42,960
We work with JP CERT CC, Korea CERT CC.

124
00:08:43,160 --> 00:08:44,020
We're talking to Europe.

125
00:08:44,240 --> 00:08:49,400
We're trying to get these out there to different entities that are involved in vulnerability disclosure

126
00:08:49,400 --> 00:08:50,980
to help with that whole process.

127
00:08:51,060 --> 00:08:51,740
That's a whole other talk.

128
00:08:52,720 --> 00:08:54,620
But I just wanted to get that plug in there.

129
00:08:54,760 --> 00:08:57,840
Because one of my hats is the CNA coordinator, so I have to do that.

130
00:09:08,720 --> 00:09:12,060
The CNA ultimately would decide for their own product.

131
00:09:12,340 --> 00:09:13,100
Oh, sure, yes.

132
00:09:13,100 --> 00:09:17,980
The question was, if we're referring to a CNA, who decides if it's a vulnerability or not?

133
00:09:18,140 --> 00:09:20,440
The CNA or MITRE?

134
00:09:20,860 --> 00:09:21,580
So it depends.

135
00:09:22,280 --> 00:09:26,680
If we hand it to the CNA and they look at it and they agree with the researcher,

136
00:09:26,820 --> 00:09:29,760
yes, this is a vulnerability, then it follows the process through.

137
00:09:29,980 --> 00:09:32,400
They look at it and say, no, this isn't a vulnerability.

138
00:09:32,820 --> 00:09:34,060
We have a process.

139
00:09:34,180 --> 00:09:37,220
We have an escalation process where the researcher could come to MITRE and say,

140
00:09:37,320 --> 00:09:41,120
that CNA over there said that what I found wasn't a vulnerability.

141
00:09:41,580 --> 00:09:42,180
Help me.

142
00:09:42,180 --> 00:09:47,260
And what MITRE will do in that case is pull the CNA and say, what's your side of the story,

143
00:09:47,440 --> 00:09:49,160
understand the researcher's side of the story,

144
00:09:49,240 --> 00:09:52,600
and then make a determination based on the information that is available from the researcher

145
00:09:53,000 --> 00:09:54,200
and from the vendor.

146
00:09:54,640 --> 00:10:01,100
So we have the final say as the maintainer of the program,

147
00:10:01,340 --> 00:10:05,860
but we want everyone in the room working to understand if something should be assigned or not

148
00:10:05,860 --> 00:10:07,440
when there is a dispute.

149
00:10:12,180 --> 00:10:13,660
Any other questions before I move on?

150
00:10:16,100 --> 00:10:16,580
Okay.

151
00:10:20,600 --> 00:10:22,760
So there's a value to having a CVID.

152
00:10:22,960 --> 00:10:25,980
That value is you've named something, and once you've named it,

153
00:10:26,020 --> 00:10:30,340
you can coordinate with other people, talk with other people about it,

154
00:10:30,380 --> 00:10:32,380
and be clear about what you're talking about.

155
00:10:33,900 --> 00:10:38,000
But when is the right time to actually go get your CVID?

156
00:10:41,340 --> 00:10:48,160
So first thing, you have to identify a new or previously unassigned vulnerability in any product.

157
00:10:48,500 --> 00:10:48,940
Okay.

158
00:10:48,960 --> 00:10:54,080
Now, we have traditionally only been assigning in a very small scope up until the last year,

159
00:10:54,180 --> 00:10:56,600
and some of you might have hit that wall where you're like,

160
00:10:56,700 --> 00:10:58,500
I found this cool vulnerability in a skateboard.

161
00:10:59,000 --> 00:11:01,000
And we said, well, we're not covering that.

162
00:11:01,420 --> 00:11:02,940
I'm not going to go into why that was.

163
00:11:03,360 --> 00:11:04,120
We've changed.

164
00:11:04,660 --> 00:11:06,600
Anything can get a CVID now.

165
00:11:06,760 --> 00:11:07,980
If you're in the IOT right now, you can get a CVID.

166
00:11:08,000 --> 00:11:09,880
If you're in the room and you find something in there,

167
00:11:10,000 --> 00:11:11,820
you can probably get a CVID for it.

168
00:11:11,940 --> 00:11:15,500
If you're in the automotive hacking area, you can probably get a CVID for it.

169
00:11:15,580 --> 00:11:16,900
We're working with the medical community.

170
00:11:17,240 --> 00:11:19,480
We're working with, again, our international partners.

171
00:11:19,620 --> 00:11:20,160
We don't care.

172
00:11:20,280 --> 00:11:22,120
Why do we think that CNA should rule the world?

173
00:11:23,580 --> 00:11:24,000
No.

174
00:11:24,220 --> 00:11:25,040
No, of course not.

175
00:11:25,200 --> 00:11:28,260
But what we do think is that we have this resource,

176
00:11:28,620 --> 00:11:32,800
and that it's silly that it shouldn't be utilized by anyone else who needs to use it.

177
00:11:33,620 --> 00:11:36,040
CV is a lot like an ISBN number in a book.

178
00:11:36,040 --> 00:11:38,500
Each book has a unique ISBN number.

179
00:11:38,600 --> 00:11:39,720
It's how you identify the books.

180
00:11:39,880 --> 00:11:41,380
And they're used around the world.

181
00:11:42,440 --> 00:11:47,120
People could come up with different ways of naming things in different countries, but why?

182
00:11:47,400 --> 00:11:51,340
If this system works, or it's a system that exists that we can improve on,

183
00:11:51,500 --> 00:11:52,980
you don't need to recreate everything.

184
00:11:53,420 --> 00:11:55,560
So it doesn't matter the product.

185
00:11:55,740 --> 00:11:56,420
Talk to us.

186
00:11:56,620 --> 00:11:57,700
There are some exceptions.

187
00:11:58,080 --> 00:12:01,400
We don't cover software as a service as of yet.

188
00:12:02,220 --> 00:12:04,880
So, for example, if it's a bug in AWS,

189
00:12:04,880 --> 00:12:10,100
it may not get a CV assigned to it.

190
00:12:10,460 --> 00:12:15,960
But if it's software that is something that the end user can control,

191
00:12:16,080 --> 00:12:17,280
that the consumer can control,

192
00:12:17,460 --> 00:12:19,420
or hardware that the consumer can control,

193
00:12:19,760 --> 00:12:23,060
it probably qualifies to get a CV ID.

194
00:12:25,300 --> 00:12:26,980
So let's say you've discovered something.

195
00:12:28,280 --> 00:12:31,940
The first step in a responsible or coordinated,

196
00:12:32,100 --> 00:12:34,040
or whatever term you want to use, disclosure process,

197
00:12:34,040 --> 00:12:37,020
is reaching out to the maintainer of that software.

198
00:12:37,540 --> 00:12:39,540
So if it's a vendor, it would be a vendor.

199
00:12:39,660 --> 00:12:42,360
If it's a project, an open source project,

200
00:12:42,540 --> 00:12:44,260
it's reaching out to the people who run the project.

201
00:12:44,600 --> 00:12:45,660
And get them in the loop.

202
00:12:46,060 --> 00:12:47,220
They should be aware of it.

203
00:12:47,840 --> 00:12:53,340
If they are a CNA, they have a process to get the CV ID assigned for it.

204
00:12:53,740 --> 00:12:57,460
And they will have their own policies about what information they need from you,

205
00:12:57,620 --> 00:13:00,060
what information they're going to give back to you, all that.

206
00:13:00,200 --> 00:13:01,380
And that should all be documented.

207
00:13:02,240 --> 00:13:04,020
And if you run into problems with that,

208
00:13:04,040 --> 00:13:05,380
again, we can help.

209
00:13:05,540 --> 00:13:06,500
We can intercede with that.

210
00:13:07,280 --> 00:13:08,920
If they're not a CNA, though,

211
00:13:10,580 --> 00:13:12,640
you have to make sure that it hasn't already been assigned.

212
00:13:12,920 --> 00:13:14,000
So do a little research.

213
00:13:14,340 --> 00:13:16,380
Do a little checking on the website.

214
00:13:16,540 --> 00:13:18,380
See if there's anyone else who's found this vulnerability

215
00:13:18,380 --> 00:13:20,220
and has assigned a CV ID to it.

216
00:13:21,180 --> 00:13:23,100
If not, then you're good.

217
00:13:24,380 --> 00:13:26,560
Now, some other cases could come up as well.

218
00:13:26,820 --> 00:13:29,140
If you're working with a coordination center,

219
00:13:29,220 --> 00:13:30,780
if it's a big deal, big product,

220
00:13:31,260 --> 00:13:32,620
and you're working with CertCC,

221
00:13:34,560 --> 00:13:35,440
JPCertCC, ICScert,

222
00:13:36,160 --> 00:13:38,500
you want to make sure you're coordinating with them.

223
00:13:39,380 --> 00:13:42,600
It really doesn't matter who assigns it, per se.

224
00:13:43,860 --> 00:13:45,900
But you want to make sure that people aren't assigning

225
00:13:45,900 --> 00:13:47,840
two CV IDs to the same thing.

226
00:13:48,020 --> 00:13:50,120
The whole point is that each CV ID is unique.

227
00:13:51,540 --> 00:13:53,580
So if you're working with a coordination center,

228
00:13:53,760 --> 00:13:55,120
coordinate with them.

229
00:13:55,360 --> 00:13:57,700
It helps make sure that there's less confusion.

230
00:13:59,480 --> 00:14:01,020
Note, and as I said before,

231
00:14:01,020 --> 00:14:03,300
the vulnerability does not have to be public

232
00:14:03,820 --> 00:14:05,180
before you request it.

233
00:14:06,120 --> 00:14:08,660
It won't be published in our list until it's public.

234
00:14:08,900 --> 00:14:10,200
But if you found a vulnerability

235
00:14:10,200 --> 00:14:11,820
and you're going through an embargo process,

236
00:14:12,400 --> 00:14:14,840
having that CV ID early means that

237
00:14:14,840 --> 00:14:16,240
everyone who's involved in the embargo

238
00:14:16,900 --> 00:14:19,220
are going to use that when it goes public,

239
00:14:19,340 --> 00:14:23,440
and can use it when it's being dealt with under the embargo.

240
00:14:23,820 --> 00:14:27,100
So you can ask for it as soon as you realize it'll need one,

241
00:14:27,260 --> 00:14:30,060
but then it won't get published publicly

242
00:14:30,060 --> 00:14:34,640
until the information's been publicly given out

243
00:14:34,640 --> 00:14:36,240
as part of the disclosure process.

244
00:14:36,940 --> 00:14:38,340
So you don't have to wait,

245
00:14:38,460 --> 00:14:41,040
but until it's public,

246
00:14:41,220 --> 00:14:42,740
we don't put it in our list.

247
00:14:47,680 --> 00:14:49,700
So before I move on,

248
00:14:49,800 --> 00:14:51,760
any questions about any of that?

249
00:14:55,100 --> 00:14:56,840
We have about 65 right now.

250
00:14:57,160 --> 00:14:59,420
They range, again, from vendors,

251
00:14:59,420 --> 00:15:02,260
we have a few bug bounty programs,

252
00:15:02,860 --> 00:15:04,820
we have some national certs,

253
00:15:06,300 --> 00:15:08,220
and we're always looking to get more.

254
00:15:10,060 --> 00:15:12,620
Ideally, every vendor is a CNA,

255
00:15:12,740 --> 00:15:14,080
every PCERT,

256
00:15:14,160 --> 00:15:16,980
every vendor that has a mature vulnerability management process

257
00:15:16,980 --> 00:15:18,140
can be a CNA,

258
00:15:18,280 --> 00:15:19,620
and they should be for their own product.

259
00:15:19,860 --> 00:15:25,100
We have one CNA that maybe issues a CV ID a year,

260
00:15:25,240 --> 00:15:26,520
and then we have Adobe.

261
00:15:28,840 --> 00:15:30,440
And with Flash going away,

262
00:15:30,480 --> 00:15:31,440
I think we'll go out of business,

263
00:15:31,520 --> 00:15:31,900
it's horrible.

264
00:15:33,300 --> 00:15:34,360
So, I love Adobe.

265
00:15:36,700 --> 00:15:38,600
So it doesn't matter the size,

266
00:15:38,780 --> 00:15:39,820
it doesn't matter the vertical,

267
00:15:40,700 --> 00:15:41,500
it can be anybody.

268
00:15:41,700 --> 00:15:42,340
Right now it's 65.

269
00:15:42,640 --> 00:15:44,780
In the last year, we went from 25 to 65,

270
00:15:44,920 --> 00:15:46,180
so we're really trying to ramp it up.

271
00:15:46,420 --> 00:15:47,580
And it is a community,

272
00:15:47,720 --> 00:15:49,880
so that if you're working in a PCERT,

273
00:15:49,940 --> 00:15:51,940
say, you get to talk to other PCERTs,

274
00:15:51,940 --> 00:15:56,020
and you get to sort of get around the politics

275
00:15:56,020 --> 00:15:56,900
that might be there.

276
00:15:56,960 --> 00:15:58,440
It's like, why are you talking to our competitor?

277
00:15:58,720 --> 00:15:59,800
You know, you get that sometimes.

278
00:16:00,140 --> 00:16:01,520
We want to create a trust community

279
00:16:01,520 --> 00:16:03,680
within the vulnerability management crowd

280
00:16:03,680 --> 00:16:05,380
so that you all can talk to each other

281
00:16:05,380 --> 00:16:07,680
and coordinate and learn from each other.

282
00:16:08,000 --> 00:16:10,280
Because I think last year at Black Hat, right,

283
00:16:10,400 --> 00:16:12,760
security, we don't compete on security, right?

284
00:16:12,960 --> 00:16:14,480
We believe in that.

285
00:16:14,740 --> 00:16:15,120
So.

286
00:16:16,480 --> 00:16:16,880
Yes.

287
00:16:17,220 --> 00:16:19,480
How do you address,

288
00:16:19,820 --> 00:16:20,980
like, how would you represent

289
00:16:20,980 --> 00:16:21,480
?

290
00:16:33,360 --> 00:16:33,760
.

291
00:16:42,760 --> 00:16:43,160
Right.

292
00:16:43,320 --> 00:16:44,280
So how do you weed out,

293
00:16:44,880 --> 00:16:46,440
how do you weed out duplicates

294
00:16:46,440 --> 00:16:49,500
when what you find may be generically described already

295
00:16:49,500 --> 00:16:50,520
in another CVID,

296
00:16:50,620 --> 00:16:51,460
or that might be something

297
00:16:51,480 --> 00:16:52,220
completely different?

298
00:16:52,640 --> 00:16:53,580
The more information,

299
00:16:53,760 --> 00:16:55,300
the better that you can provide us.

300
00:16:56,580 --> 00:16:58,780
The more information you provide us,

301
00:16:58,860 --> 00:16:59,660
the better, okay?

302
00:16:59,980 --> 00:17:01,340
Because we do have,

303
00:17:02,080 --> 00:17:04,700
you know, we can look at what is public.

304
00:17:04,840 --> 00:17:06,640
We can do some of that analysis.

305
00:17:06,800 --> 00:17:08,760
We hope that you will do that analysis as well.

306
00:17:09,640 --> 00:17:10,780
This is a scaling problem.

307
00:17:11,040 --> 00:17:12,620
If you all in the room came to me today

308
00:17:12,620 --> 00:17:13,340
with one of these,

309
00:17:13,420 --> 00:17:15,180
like, I'd have to get back to you, right?

310
00:17:15,300 --> 00:17:16,660
But if you do some of the due diligence yourself

311
00:17:16,660 --> 00:17:17,420
and come to us,

312
00:17:17,460 --> 00:17:19,600
we'll be able to work much more quickly.

313
00:17:19,600 --> 00:17:22,520
This comes up with things like fuzzing.

314
00:17:23,800 --> 00:17:25,760
We can sometimes get requests

315
00:17:25,760 --> 00:17:28,040
for dozens of CVIDs

316
00:17:28,040 --> 00:17:29,340
from somebody who's doing fuzzing.

317
00:17:30,840 --> 00:17:32,480
It's unclear, based on the information

318
00:17:32,480 --> 00:17:33,360
that we've been given,

319
00:17:33,460 --> 00:17:35,420
whether or not it's actually just one vulnerability

320
00:17:35,420 --> 00:17:37,360
that's been kicked off a bunch of times,

321
00:17:37,440 --> 00:17:40,200
or if it's actually a dozen different vulnerabilities.

322
00:17:40,740 --> 00:17:44,920
Because they haven't actually provided the information

323
00:17:44,920 --> 00:17:46,220
to make that determination,

324
00:17:46,360 --> 00:17:47,680
or they haven't done that due diligence.

325
00:17:47,680 --> 00:17:50,840
We have a set of rules that we follow,

326
00:17:51,000 --> 00:17:53,380
and sometimes we have to make judgment calls,

327
00:17:53,580 --> 00:17:55,120
but that's built into our process.

328
00:17:55,220 --> 00:17:56,100
We try to do it consistently.

329
00:17:57,200 --> 00:17:59,060
But the more information that you have

330
00:17:59,060 --> 00:18:00,280
and you can give to us,

331
00:18:00,340 --> 00:18:01,700
and you can say,

332
00:18:01,780 --> 00:18:04,100
I don't know if this is a duplicate of this other one.

333
00:18:04,180 --> 00:18:04,940
It looks kind of similar.

334
00:18:05,280 --> 00:18:07,740
That is information that, if it's us,

335
00:18:07,840 --> 00:18:09,300
we can look at and analyze.

336
00:18:09,560 --> 00:18:11,100
If it's a CNA, they'll know, certainly,

337
00:18:11,220 --> 00:18:12,880
because hopefully they know their product.

338
00:18:13,440 --> 00:18:13,820
Hopefully.

339
00:18:15,340 --> 00:18:15,900
So, yeah.

340
00:18:15,900 --> 00:18:17,020
It's about...

341
00:18:17,020 --> 00:18:19,260
It's really that due diligence is important.

342
00:18:20,000 --> 00:18:22,500
We don't have access to magic source code

343
00:18:22,500 --> 00:18:23,880
that you all don't.

344
00:18:24,180 --> 00:18:25,910
And even open source software...

345
00:18:26,540 --> 00:18:27,840
I mean, there's a lot of code out there,

346
00:18:27,960 --> 00:18:29,440
and we're a very small team.

347
00:18:29,640 --> 00:18:32,040
So the more you can give to us

348
00:18:32,040 --> 00:18:34,060
that you've sort of generated through,

349
00:18:34,280 --> 00:18:36,740
the faster and easier it is for us to work through it

350
00:18:36,740 --> 00:18:37,960
and to get back to you.

351
00:18:38,580 --> 00:18:38,760
Okay?

352
00:18:40,540 --> 00:18:41,980
So I mentioned these rules.

353
00:18:42,240 --> 00:18:43,280
We call this counting.

354
00:18:44,380 --> 00:18:44,840
Okay?

355
00:18:45,900 --> 00:18:46,960
So counting asks two questions.

356
00:18:47,180 --> 00:18:49,700
Is it a vulnerability per CVE?

357
00:18:50,760 --> 00:18:51,700
Is it a vulnerability?

358
00:18:52,000 --> 00:18:53,060
And if it is a vulnerability,

359
00:18:53,600 --> 00:18:55,560
how many vulnerabilities are there?

360
00:18:56,640 --> 00:18:57,120
Okay?

361
00:18:58,040 --> 00:19:00,060
This is exactly what we were talking about here.

362
00:19:01,280 --> 00:19:03,380
And I'm going to pass it over to Anthony

363
00:19:03,380 --> 00:19:04,820
to talk a little bit more about counting.

364
00:19:05,640 --> 00:19:06,440
Thank you, Dan.

365
00:19:07,200 --> 00:19:07,940
Thank you, Dan.

366
00:19:08,980 --> 00:19:10,800
So the runoff that...

367
00:19:10,800 --> 00:19:12,940
So we have this process in place,

368
00:19:13,040 --> 00:19:14,140
this method in place,

369
00:19:20,160 --> 00:19:21,620
because in the industry,

370
00:19:21,740 --> 00:19:23,080
there's so many different definitions

371
00:19:23,080 --> 00:19:25,100
for vulnerabilities and, again,

372
00:19:25,280 --> 00:19:26,840
even more definitions for bugs.

373
00:19:27,020 --> 00:19:28,460
So instead of trying to come up

374
00:19:28,460 --> 00:19:30,480
with our own definition of the two,

375
00:19:30,660 --> 00:19:33,020
we've set a method called counting

376
00:19:33,020 --> 00:19:35,020
in place to kind of help you figure out,

377
00:19:35,080 --> 00:19:36,260
as Dan talked,

378
00:19:36,560 --> 00:19:38,620
about seeing if the bug is a vulnerability

379
00:19:38,620 --> 00:19:40,780
and then seeing how many there actually are.

380
00:19:41,040 --> 00:19:43,160
And so those are the two phases

381
00:19:43,160 --> 00:19:45,060
that it's actually broken into.

382
00:19:45,420 --> 00:19:45,880
So the first thing we're going to do

383
00:19:45,900 --> 00:19:48,660
is determine the number of vulnerabilities.

384
00:19:49,500 --> 00:19:50,660
So we do...

385
00:19:50,660 --> 00:19:51,880
This is going to be how we do it

386
00:19:51,880 --> 00:19:53,900
at internal CVE.

387
00:19:54,140 --> 00:19:55,580
We start by breaking the report

388
00:19:55,580 --> 00:19:57,080
into the individual bugs.

389
00:19:57,620 --> 00:19:58,580
And then from there,

390
00:19:58,660 --> 00:20:00,260
we determine if those bugs themselves

391
00:20:00,260 --> 00:20:01,000
are vulnerabilities.

392
00:20:01,500 --> 00:20:02,440
For those that do,

393
00:20:02,600 --> 00:20:03,880
we then go to the next step.

394
00:20:04,020 --> 00:20:05,060
But for the ones that don't,

395
00:20:05,060 --> 00:20:06,380
we then put them together and see

396
00:20:06,380 --> 00:20:08,340
if there's a combination of bugs

397
00:20:08,340 --> 00:20:09,600
that will give you a vulnerability.

398
00:20:09,960 --> 00:20:10,940
And then we move on.

399
00:20:11,520 --> 00:20:13,500
Once we have that listed or figured out,

400
00:20:13,640 --> 00:20:15,280
you then determine whether

401
00:20:15,840 --> 00:20:17,580
the vulnerability occurs

402
00:20:18,080 --> 00:20:20,440
due to shared code,

403
00:20:20,700 --> 00:20:22,160
protocol, or standard.

404
00:20:23,100 --> 00:20:24,480
And if it does,

405
00:20:24,560 --> 00:20:26,280
we then merge those vulnerabilities

406
00:20:26,280 --> 00:20:27,080
into one.

407
00:20:27,500 --> 00:20:29,140
Because the issue is now in that

408
00:20:29,140 --> 00:20:30,400
either the shared library,

409
00:20:31,220 --> 00:20:32,240
protocol, or standard.

410
00:20:33,140 --> 00:20:34,040
After you do that,

411
00:20:34,060 --> 00:20:37,080
now you know how many vulnerabilities there are.

412
00:20:38,340 --> 00:20:40,260
Which then leads you into your second phase

413
00:20:40,260 --> 00:20:41,060
of figuring out

414
00:20:41,700 --> 00:20:44,160
does these vulnerabilities warrant a CVE ID?

415
00:20:45,580 --> 00:20:46,100
Well,

416
00:20:46,140 --> 00:20:47,880
you determine that because you have to understand

417
00:20:47,880 --> 00:20:49,360
that there's a certain value,

418
00:20:49,400 --> 00:20:51,500
as we talked about earlier in the slide deck,

419
00:20:51,920 --> 00:20:54,400
what the purpose and value is

420
00:20:54,400 --> 00:20:54,840
of CVE.

421
00:20:55,240 --> 00:20:57,740
So for that, we have to start off by saying

422
00:20:57,740 --> 00:20:59,160
or thinking or determining

423
00:20:59,160 --> 00:21:01,700
whether the information for that vulnerability

424
00:21:01,700 --> 00:21:02,640
will be released.

425
00:21:02,860 --> 00:21:04,940
So it will be public information for people to have.

426
00:21:05,160 --> 00:21:07,260
So it doesn't make sense to have an ID

427
00:21:08,340 --> 00:21:10,120
if you're going to restrict access to the information.

428
00:21:10,500 --> 00:21:12,180
So if no one can know about the vulnerability,

429
00:21:12,640 --> 00:21:13,600
why have an ID?

430
00:21:13,920 --> 00:21:15,240
So that's the first thing you have to know,

431
00:21:15,320 --> 00:21:17,260
is that we're going to release that information publicly,

432
00:21:17,560 --> 00:21:18,940
we give an ID to it,

433
00:21:19,000 --> 00:21:20,920
so now everyone that also talks about that

434
00:21:20,920 --> 00:21:23,880
knows that we're actually talking about this one CVE ID.

435
00:21:24,780 --> 00:21:25,880
The next thing for that

436
00:21:25,880 --> 00:21:27,000
is you have to understand

437
00:21:27,880 --> 00:21:30,100
the community must be able to do something

438
00:21:30,100 --> 00:21:31,020
to mitigate the vulnerability.

439
00:21:31,360 --> 00:21:33,200
So a lot of this comes back to being something

440
00:21:33,200 --> 00:21:34,860
that's considered customer controlled.

441
00:21:35,160 --> 00:21:37,580
Meaning that you release a report

442
00:21:38,340 --> 00:21:38,900
and you agree on this vulnerability,

443
00:21:39,200 --> 00:21:40,800
and now the community can say,

444
00:21:40,960 --> 00:21:42,700
well okay, I know if I apply this patch

445
00:21:42,700 --> 00:21:43,760
I can then remove this vulnerability.

446
00:21:44,120 --> 00:21:45,720
So that's another benefit of having it.

447
00:21:46,400 --> 00:21:48,520
And then the last one is kind of subjective,

448
00:21:48,780 --> 00:21:50,400
but the community has to be concerned

449
00:21:50,400 --> 00:21:51,520
about the security of the product.

450
00:21:52,540 --> 00:21:53,360
Again, it's subjective,

451
00:21:53,580 --> 00:21:54,720
so when you make the request,

452
00:21:54,960 --> 00:21:57,660
you would tell us what the impact of that vulnerability is,

453
00:21:57,760 --> 00:21:59,380
and most likely we'll agree with you

454
00:21:59,380 --> 00:22:00,480
based on your research.

455
00:22:01,700 --> 00:22:04,120
And then again, like we talked about a little bit earlier,

456
00:22:04,300 --> 00:22:06,580
we want to make sure we're not duplicating requests.

457
00:22:08,340 --> 00:22:08,860
We want to make sure there's no,

458
00:22:08,980 --> 00:22:10,020
if it hasn't been assigned,

459
00:22:11,240 --> 00:22:14,040
we'll also work with you if there is a duplicate.

460
00:22:18,220 --> 00:22:18,700
So,

461
00:22:19,740 --> 00:22:21,160
the community must be concerned

462
00:22:21,160 --> 00:22:22,300
about the security of the product.

463
00:22:22,540 --> 00:22:24,120
So yes, it's subjective,

464
00:22:24,380 --> 00:22:25,580
but it can also be very easy.

465
00:22:25,680 --> 00:22:27,720
For example, this bug allows you

466
00:22:27,720 --> 00:22:29,900
to turn the color of the page to blue.

467
00:22:30,980 --> 00:22:31,460
Okay.

468
00:22:31,780 --> 00:22:33,080
Is that a security vulnerability?

469
00:22:34,720 --> 00:22:35,480
It depends.

470
00:22:38,340 --> 00:22:38,800
If you're in a power plant,

471
00:22:38,960 --> 00:22:41,340
and you need to know that if something turns red,

472
00:22:41,560 --> 00:22:43,400
it's bad, and if it's blue, it's good.

473
00:22:43,560 --> 00:22:45,220
Well, if you can arbitrarily change the color,

474
00:22:45,400 --> 00:22:47,060
that's a security problem, right?

475
00:22:47,140 --> 00:22:48,160
So there's like this policy

476
00:22:48,160 --> 00:22:49,480
that you have to take into consideration.

477
00:22:50,420 --> 00:22:52,160
Whereas, if it's like a web browser,

478
00:22:52,280 --> 00:22:52,960
it turns it blue,

479
00:22:53,100 --> 00:22:54,780
that's really more of a bug than a vulnerability.

480
00:22:57,260 --> 00:22:58,980
The community must be able to do something

481
00:22:58,980 --> 00:23:00,020
to mitigate the vulnerability.

482
00:23:01,680 --> 00:23:03,220
That doesn't mean that there is a bug,

483
00:23:03,600 --> 00:23:04,080
I'm sorry,

484
00:23:04,080 --> 00:23:06,160
that doesn't mean that there is a patch already.

485
00:23:06,580 --> 00:23:08,120
It means that it has to be something

486
00:23:08,120 --> 00:23:09,080
under their control.

487
00:23:09,320 --> 00:23:10,860
So again, back to the AWS example.

488
00:23:11,140 --> 00:23:15,280
If there's a bug in AWS's stack

489
00:23:15,280 --> 00:23:17,040
that is above user land,

490
00:23:17,240 --> 00:23:19,240
above where a user can actually

491
00:23:19,240 --> 00:23:20,480
affect something in it,

492
00:23:21,660 --> 00:23:22,240
you know,

493
00:23:22,260 --> 00:23:24,160
that's not something that we'll probably

494
00:23:24,160 --> 00:23:25,300
assign a CVID for.

495
00:23:28,060 --> 00:23:30,200
Because Amazon certainly needs to handle that,

496
00:23:30,340 --> 00:23:31,320
they need to market it,

497
00:23:31,460 --> 00:23:32,580
they need to spin it,

498
00:23:32,620 --> 00:23:33,380
whatever they need to do,

499
00:23:34,240 --> 00:23:35,520
but they're the only ones

500
00:23:35,520 --> 00:23:36,760
who can do anything about it.

501
00:23:36,840 --> 00:23:38,540
Their users might need to be aware of it,

502
00:23:38,600 --> 00:23:39,640
but giving it a CVID,

503
00:23:40,840 --> 00:23:42,180
that doesn't have value.

504
00:23:42,420 --> 00:23:45,140
Giving it some other way of labeling it,

505
00:23:45,200 --> 00:23:45,960
that may have value.

506
00:23:46,060 --> 00:23:47,660
This is a discussion that we're having

507
00:23:47,660 --> 00:23:49,500
with the community in CVE,

508
00:23:49,680 --> 00:23:51,280
and if you have thoughts on that,

509
00:23:51,380 --> 00:23:52,000
we'd love to hear them.

510
00:23:54,280 --> 00:23:56,340
And again, the vulnerability must be public.

511
00:23:56,580 --> 00:23:58,500
One example where this is not the case

512
00:23:58,500 --> 00:24:00,200
is we don't want CVEs to be used

513
00:24:00,200 --> 00:24:01,300
for internal bug tracking,

514
00:24:04,080 --> 00:24:05,980
for code commit,

515
00:24:06,200 --> 00:24:07,320
and so you find a bug.

516
00:24:07,520 --> 00:24:08,900
You don't want to issue a CVID

517
00:24:08,900 --> 00:24:10,600
if you're going to fix it that night

518
00:24:10,600 --> 00:24:12,420
and it'll be gone and it was never seen

519
00:24:12,420 --> 00:24:13,260
by the general public.

520
00:24:14,700 --> 00:24:16,260
Again, there's no value there.

521
00:24:16,520 --> 00:24:19,540
It's only valuable if it's out in the world

522
00:24:19,540 --> 00:24:21,000
and people need to talk about it

523
00:24:21,000 --> 00:24:21,520
and know about it.

524
00:24:39,320 --> 00:24:39,720
Questions?

525
00:24:40,140 --> 00:24:42,200
Let's say it's an old proprietary bit of code

526
00:24:42,200 --> 00:24:44,220
that the community can't affect,

527
00:24:44,480 --> 00:24:45,260
the code, right?

528
00:24:45,300 --> 00:24:45,960
They can't patch it,

529
00:24:46,040 --> 00:24:47,200
the company's out of business, whatever.

530
00:24:48,060 --> 00:24:49,960
They can mitigate the risk, though.

531
00:24:51,820 --> 00:24:53,020
Either turn it off

532
00:24:53,020 --> 00:24:54,880
or put a firewall in front of it

533
00:24:54,880 --> 00:24:56,700
or whatever they have to do to mitigate that risk.

534
00:24:57,020 --> 00:24:58,220
So in that case, yes,

535
00:24:58,280 --> 00:24:59,680
we would have probably assigned to it.

536
00:24:59,740 --> 00:25:02,140
Even if the company doesn't exist anymore,

537
00:25:04,080 --> 00:25:04,640
the vulnerability exists

538
00:25:04,640 --> 00:25:08,060
because the consumer can do something about it.

539
00:25:08,960 --> 00:25:09,920
Not necessarily

540
00:25:09,920 --> 00:25:11,440
as much as they'd like,

541
00:25:11,560 --> 00:25:13,060
but there's something that can be done.

542
00:25:20,220 --> 00:25:21,940
So, all of that

543
00:25:21,940 --> 00:25:23,660
discussion boils down to

544
00:25:23,660 --> 00:25:24,740
this one bullet.

545
00:25:24,920 --> 00:25:26,040
How do you get a CVID?

546
00:25:26,360 --> 00:25:27,160
You go to that website.

547
00:25:28,000 --> 00:25:28,960
Thank you, good night.

548
00:25:30,940 --> 00:25:32,500
It is just that easy.

549
00:25:34,080 --> 00:25:34,780
It's mostly that easy.

550
00:25:36,160 --> 00:25:37,320
There's a form there

551
00:25:38,080 --> 00:25:39,720
and an injection strip.

552
00:25:39,840 --> 00:25:40,940
No, no, there's a form there

553
00:25:40,940 --> 00:25:43,600
and the form asks for specific bits of information.

554
00:25:44,040 --> 00:25:45,760
There is a minimum set of information

555
00:25:45,760 --> 00:25:47,780
we need to start the process.

556
00:25:48,220 --> 00:25:49,820
We need to have a general idea

557
00:25:49,820 --> 00:25:51,860
of what the vulnerability type is.

558
00:25:51,980 --> 00:25:53,840
Now, there isn't a list

559
00:25:53,840 --> 00:25:56,680
of canonical vulnerability types.

560
00:25:56,860 --> 00:25:57,880
Like the definition

561
00:25:57,880 --> 00:25:58,520
for vulnerability,

562
00:25:59,220 --> 00:26:00,660
there's tons of different ways

563
00:26:00,660 --> 00:26:02,600
to cut vulnerability types.

564
00:26:04,080 --> 00:26:05,540
But you just need to describe it.

565
00:26:06,460 --> 00:26:07,080
Buffer overflow,

566
00:26:07,580 --> 00:26:08,580
cross-site scripting,

567
00:26:08,660 --> 00:26:09,260
things like that.

568
00:26:09,340 --> 00:26:10,660
Common terms, that's good enough.

569
00:26:10,840 --> 00:26:12,260
If you want to get into more depth, you can.

570
00:26:13,340 --> 00:26:16,140
We need to know what the product and vendor is

571
00:26:16,140 --> 00:26:18,080
or the project that's affected.

572
00:26:18,380 --> 00:26:20,040
You need to tell us what you found the bug in.

573
00:26:22,480 --> 00:26:24,160
And also the version information.

574
00:26:24,940 --> 00:26:26,700
You need to be clear on

575
00:26:26,700 --> 00:26:27,840
when you talk about version

576
00:26:27,840 --> 00:26:28,700
what you're talking about.

577
00:26:29,160 --> 00:26:30,960
You can't just say all versions

578
00:26:32,640 --> 00:26:33,600
because what if

579
00:26:34,080 --> 00:26:34,900
a new version came out tomorrow?

580
00:26:35,120 --> 00:26:36,240
Would that still be true?

581
00:26:36,760 --> 00:26:38,440
So you want to be very explicit

582
00:26:38,440 --> 00:26:41,720
if it's all versions prior to 1.1.1

583
00:26:41,720 --> 00:26:44,500
or only version 2.5.

584
00:26:44,780 --> 00:26:46,300
If you've only seen the vulnerability

585
00:26:46,300 --> 00:26:47,600
in one particular version

586
00:26:47,600 --> 00:26:49,080
and you don't know about the rest,

587
00:26:49,260 --> 00:26:50,960
just tell us about that one particular version.

588
00:26:51,080 --> 00:26:52,200
You could always update it later.

589
00:26:53,540 --> 00:26:55,400
So that's what we require

590
00:26:55,400 --> 00:26:56,660
to start the process.

591
00:26:57,600 --> 00:26:59,260
If you have other information,

592
00:26:59,460 --> 00:27:00,480
like we were talking earlier,

593
00:27:00,680 --> 00:27:02,060
please give it to us. We'll take it.

594
00:27:04,340 --> 00:27:05,960
Anything that helps

595
00:27:05,960 --> 00:27:07,040
describe the vulnerability.

596
00:27:07,740 --> 00:27:09,240
We don't care

597
00:27:09,240 --> 00:27:11,720
as far as whether it's a vulnerability or not

598
00:27:11,720 --> 00:27:13,280
about things like impact

599
00:27:14,040 --> 00:27:15,320
or the attack vectors.

600
00:27:15,800 --> 00:27:18,040
NVD does and other vulnerability databases do

601
00:27:18,040 --> 00:27:19,540
because that's enriched

602
00:27:20,080 --> 00:27:21,600
information about the vulnerabilities.

603
00:27:23,100 --> 00:27:23,740
But

604
00:27:23,740 --> 00:27:25,520
that helps us understand that

605
00:27:25,520 --> 00:27:27,620
this is a unique vulnerability

606
00:27:27,620 --> 00:27:29,360
because we can compare it with other things

607
00:27:29,360 --> 00:27:30,160
that might be out there.

608
00:27:30,640 --> 00:27:32,260
And also it helps with the counting question.

609
00:27:34,080 --> 00:27:35,680
And public references are always great.

610
00:27:35,860 --> 00:27:37,480
Again, in order for this to be public

611
00:27:38,060 --> 00:27:38,760
on our website,

612
00:27:38,960 --> 00:27:40,720
you have to have public references.

613
00:27:40,860 --> 00:27:42,100
Now if you don't have them yet,

614
00:27:43,120 --> 00:27:44,920
that's okay. You can still submit.

615
00:27:45,020 --> 00:27:46,140
Again, the minimum is up here.

616
00:27:46,400 --> 00:27:48,320
But if you do already have something public

617
00:27:48,320 --> 00:27:49,960
or you found somebody's research

618
00:27:49,960 --> 00:27:51,460
and hey, that should get a CVID,

619
00:27:51,760 --> 00:27:53,440
that's a public reference, that works too.

620
00:27:54,460 --> 00:27:54,900
Okay?

621
00:27:57,160 --> 00:27:59,500
Now, any questions about that?

622
00:28:00,040 --> 00:28:00,480
Yeah?

623
00:28:04,420 --> 00:28:05,020
.

624
00:28:05,720 --> 00:28:06,320
.

625
00:28:10,220 --> 00:28:11,700
So, like I said before,

626
00:28:11,920 --> 00:28:13,880
we traffic only in public information

627
00:28:13,880 --> 00:28:14,720
when possible.

628
00:28:14,920 --> 00:28:17,340
So we do have an embargo policy on our website.

629
00:28:17,580 --> 00:28:19,480
The embargo policy basically says

630
00:28:19,480 --> 00:28:20,440
if you give us information,

631
00:28:21,080 --> 00:28:22,540
we're not going to share it

632
00:28:22,540 --> 00:28:24,920
until the information is made public.

633
00:28:25,480 --> 00:28:27,920
So we actually do sometimes receive information

634
00:28:27,920 --> 00:28:29,200
that isn't in the references.

635
00:28:30,480 --> 00:28:31,000
We publish that information.

636
00:28:31,420 --> 00:28:32,960
But if someone gives us something

637
00:28:32,960 --> 00:28:34,580
and says that it's public

638
00:28:34,580 --> 00:28:36,220
or will be public,

639
00:28:36,420 --> 00:28:38,620
then we'll publish it along with everything else.

640
00:28:40,080 --> 00:28:42,920
We try to stay out of the embargo world, though.

641
00:28:44,240 --> 00:28:45,680
If you're giving us something

642
00:28:45,680 --> 00:28:46,360
that's under embargo

643
00:28:46,360 --> 00:28:47,560
that's not public yet,

644
00:28:48,940 --> 00:28:50,640
we'll take very good care of it,

645
00:28:50,660 --> 00:28:51,000
I swear.

646
00:28:51,200 --> 00:28:52,700
We'll treat it like our own baby.

647
00:28:53,120 --> 00:28:55,580
But again, we don't want to be like

648
00:28:56,220 --> 00:28:57,820
having tons and tons of zero days

649
00:28:57,820 --> 00:28:59,680
just sitting there waiting for people to get to them.

650
00:29:00,480 --> 00:29:01,380
We want to encourage them to get public

651
00:29:01,380 --> 00:29:02,680
as fast as possible.

652
00:29:04,640 --> 00:29:05,460
But we don't.

653
00:29:05,520 --> 00:29:07,980
We have, there's a lot of sausage making

654
00:29:07,980 --> 00:29:08,580
in the background.

655
00:29:08,840 --> 00:29:10,020
The stuff that comes in,

656
00:29:10,040 --> 00:29:12,540
it gets stored off the internet.

657
00:29:13,540 --> 00:29:14,260
In quotes.

658
00:29:14,940 --> 00:29:16,600
The stuff that's on our public website,

659
00:29:16,840 --> 00:29:18,620
there's nothing there that isn't public.

660
00:29:20,220 --> 00:29:21,380
Does that make sense?

661
00:29:23,280 --> 00:29:24,620
And also, by the way,

662
00:29:24,680 --> 00:29:26,160
you can always PGP encrypt it.

663
00:29:26,340 --> 00:29:27,940
If you don't trust how we store it,

664
00:29:28,460 --> 00:29:29,480
that helps a little bit.

665
00:29:30,480 --> 00:29:32,700
But you can do that as an extra bonus

666
00:29:32,700 --> 00:29:33,560
if you want to.

667
00:29:34,640 --> 00:29:34,900
Yeah?

668
00:29:43,820 --> 00:29:44,820
They suck.

669
00:29:51,260 --> 00:29:51,760
Yeah.

670
00:29:51,900 --> 00:29:52,360
Right.

671
00:29:52,480 --> 00:29:55,420
So again, we have an escalation process for that.

672
00:29:55,560 --> 00:29:56,380
So you go to the vendor,

673
00:29:57,000 --> 00:29:57,640
CNA or not,

674
00:29:57,800 --> 00:29:58,780
and you say, hey, I found this bug.

675
00:29:59,100 --> 00:30:01,700
The escalation process only matters if there's CNA.

676
00:30:01,860 --> 00:30:03,240
If you go to the vendor and the vendor says,

677
00:30:03,320 --> 00:30:04,140
that ain't nothing,

678
00:30:04,540 --> 00:30:05,720
you come to us and we give you a CVID

679
00:30:05,720 --> 00:30:07,880
if it follows the counting rules and works.

680
00:30:08,440 --> 00:30:10,660
If it's a CNA, there's a disagreement.

681
00:30:11,040 --> 00:30:12,920
If they sit on it for too long,

682
00:30:13,020 --> 00:30:14,920
if they break their embargo policy,

683
00:30:15,060 --> 00:30:16,300
because some of the vendors say,

684
00:30:16,420 --> 00:30:19,340
it may take up to nine months before you hear anything from us

685
00:30:19,340 --> 00:30:20,500
because we have to analyze this.

686
00:30:20,740 --> 00:30:21,640
You've got to respect that.

687
00:30:21,640 --> 00:30:23,560
Or at least we have to respect that.

688
00:30:23,640 --> 00:30:24,320
We can't stop you.

689
00:30:24,520 --> 00:30:26,940
But we will listen to what our vendors say.

690
00:30:27,120 --> 00:30:28,560
But the minute they break their promise,

691
00:30:28,860 --> 00:30:29,760
you come to us.

692
00:30:29,920 --> 00:30:31,660
And we say, okay, what's the deal?

693
00:30:31,880 --> 00:30:34,100
And we will push it to some kind of resolution.

694
00:30:35,640 --> 00:30:36,160
We're not,

695
00:30:36,820 --> 00:30:39,720
we're going to act as a third party.

696
00:30:39,920 --> 00:30:40,620
We don't,

697
00:30:40,620 --> 00:30:42,800
we want the information to be out there

698
00:30:42,800 --> 00:30:44,060
if it's supposed to be out there.

699
00:30:44,300 --> 00:30:46,940
So we'll do what we can

700
00:30:46,940 --> 00:30:48,200
to understand the situation

701
00:30:48,200 --> 00:30:49,900
and satisfy,

702
00:30:52,020 --> 00:30:53,100
what's best for everybody.

703
00:30:56,940 --> 00:30:59,260
Don't let your vulnerability requests

704
00:30:59,260 --> 00:31:00,300
fall into a hole.

705
00:31:00,500 --> 00:31:02,960
Or your vulnerability disclosures fall into a hole.

706
00:31:03,520 --> 00:31:05,120
We're not a coordination center.

707
00:31:05,460 --> 00:31:06,920
We have friends who are.

708
00:31:07,120 --> 00:31:08,340
If it's a really big deal,

709
00:31:08,480 --> 00:31:10,100
we can go talk to CertCC and say,

710
00:31:10,220 --> 00:31:11,540
hey guys, are you aware of this thing?

711
00:31:12,540 --> 00:31:14,980
But we can also pressure our CNAs

712
00:31:14,980 --> 00:31:17,600
because we do have expectations,

713
00:31:17,920 --> 00:31:18,560
community expectations.

714
00:31:19,160 --> 00:31:20,920
And if they don't follow those community expectations,

715
00:31:21,640 --> 00:31:21,880
we can take them out.

716
00:31:23,200 --> 00:31:24,480
No matter the size.

717
00:31:24,920 --> 00:31:26,340
We've had some very hard discussions

718
00:31:26,340 --> 00:31:27,940
with some of the bigger vendors about this.

719
00:31:30,580 --> 00:31:35,200
So what happens after that?

720
00:31:35,360 --> 00:31:35,760
Sausage.

721
00:31:36,840 --> 00:31:38,860
There's a lot of sausage being made in the background.

722
00:31:39,080 --> 00:31:40,500
But what you see is that you'll get,

723
00:31:40,620 --> 00:31:41,100
when you email,

724
00:31:41,280 --> 00:31:43,360
if you use the web form,

725
00:31:43,500 --> 00:31:45,260
you'll get an email response with a ticket number.

726
00:31:45,660 --> 00:31:47,720
This should seem like nothing unusual

727
00:31:47,720 --> 00:31:49,620
if you've ever used any kind of help desk system.

728
00:31:50,300 --> 00:31:51,540
We take that information,

729
00:31:51,860 --> 00:31:52,640
we review it.

730
00:31:53,280 --> 00:31:55,680
If there's enough information in what you've given us

731
00:31:55,680 --> 00:31:57,060
to assign it,

732
00:31:57,260 --> 00:31:58,860
the content team will say,

733
00:31:59,020 --> 00:32:00,660
here's the CV ID you're going to use.

734
00:32:02,260 --> 00:32:04,780
If it's a CNA covered product,

735
00:32:04,940 --> 00:32:06,160
it goes to the CNA.

736
00:32:06,420 --> 00:32:08,440
Or we help you redirect them.

737
00:32:08,480 --> 00:32:09,160
We don't actually take,

738
00:32:09,240 --> 00:32:10,040
so back to what you were saying,

739
00:32:10,120 --> 00:32:11,640
we don't take that information and forward it.

740
00:32:12,300 --> 00:32:13,460
We'll get back to you and say,

741
00:32:13,460 --> 00:32:14,800
you need to talk to the CNA.

742
00:32:15,040 --> 00:32:15,660
So that way,

743
00:32:15,700 --> 00:32:19,580
if you disclose something that's super sensitive to us,

744
00:32:19,720 --> 00:32:22,420
but you are reluctant to talk to the vendor,

745
00:32:22,620 --> 00:32:24,980
we'll encourage you to reach out to the vendor

746
00:32:24,980 --> 00:32:26,140
and coordinate with them.

747
00:32:26,320 --> 00:32:27,660
We're not going to pass the information

748
00:32:28,240 --> 00:32:30,060
that you've passed us to them.

749
00:32:30,580 --> 00:32:32,220
So we want to keep that,

750
00:32:32,340 --> 00:32:33,980
if there's a concern about sensitivity,

751
00:32:34,240 --> 00:32:36,400
we try to keep that control in your hands.

752
00:32:39,120 --> 00:32:41,300
But if you give us enough information

753
00:32:41,300 --> 00:32:43,440
that not only is it available

754
00:32:43,460 --> 00:32:45,040
as a valid CV ID request

755
00:32:45,040 --> 00:32:46,200
or a valid vulnerability,

756
00:32:46,740 --> 00:32:48,540
and we have all the information we need,

757
00:32:48,600 --> 00:32:49,220
we'll publish it.

758
00:32:49,540 --> 00:32:51,340
We'll just send it up in the database.

759
00:32:52,500 --> 00:32:54,560
Until it is published in our database,

760
00:32:55,080 --> 00:32:57,300
it is marked as reserved.

761
00:32:57,760 --> 00:32:59,100
You've seen this, I'm sure.

762
00:32:59,320 --> 00:33:01,860
You've looked at a CV ID in the CV list

763
00:33:01,860 --> 00:33:03,020
and it just says reserved.

764
00:33:03,680 --> 00:33:04,160
Why?

765
00:33:04,580 --> 00:33:07,120
It's reserved because we don't have enough information

766
00:33:07,120 --> 00:33:09,040
to publish it in our database.

767
00:33:09,200 --> 00:33:10,640
We either don't have the public references

768
00:33:10,640 --> 00:33:12,820
or the information that we've been given

769
00:33:13,460 --> 00:33:14,640
by whoever requested it

770
00:33:14,640 --> 00:33:16,080
was not enough to say,

771
00:33:16,200 --> 00:33:17,700
yes, this is a unique vulnerability.

772
00:33:18,720 --> 00:33:20,240
That's why you see so many of these.

773
00:33:20,320 --> 00:33:21,600
A lot of times people come to us

774
00:33:21,600 --> 00:33:22,180
and they say,

775
00:33:22,260 --> 00:33:24,080
I need a vulnerability for X, Y, and Z.

776
00:33:24,160 --> 00:33:25,820
And we get those first three things,

777
00:33:29,480 --> 00:33:30,540
but that's not enough.

778
00:33:30,720 --> 00:33:32,400
You need the references to publish.

779
00:33:33,260 --> 00:33:35,200
So they get this and they go off

780
00:33:35,200 --> 00:33:37,400
and then they post it on their blog somewhere,

781
00:33:38,040 --> 00:33:39,340
but they never got back to us

782
00:33:39,340 --> 00:33:40,020
with more information.

783
00:33:40,280 --> 00:33:42,480
They never clarified, they never gave us a reference.

784
00:33:43,460 --> 00:33:44,360
So we don't publish until we have that.

785
00:33:49,860 --> 00:33:51,640
So if you see something

786
00:33:51,640 --> 00:33:54,400
and you know there's public information about it,

787
00:33:55,400 --> 00:33:56,700
you can tell us

788
00:33:56,700 --> 00:33:58,940
and we will immediately get it published

789
00:33:58,940 --> 00:33:59,900
if that's what we needed.

790
00:34:00,540 --> 00:34:02,140
And we'll talk about that in a second.

791
00:34:02,260 --> 00:34:04,140
So at this point, again,

792
00:34:04,920 --> 00:34:07,380
it's reserved unless it has enough information.

793
00:34:07,840 --> 00:34:09,820
It is going to stay reserved

794
00:34:09,820 --> 00:34:11,900
until it has been populated.

795
00:34:12,340 --> 00:34:12,820
So,

796
00:34:12,820 --> 00:34:15,180
if you're disclosing something,

797
00:34:15,420 --> 00:34:16,780
you need to notify us

798
00:34:16,780 --> 00:34:17,900
when it's public.

799
00:34:18,420 --> 00:34:19,640
We won't know.

800
00:34:19,880 --> 00:34:22,580
We have some web scrapers out there.

801
00:34:22,620 --> 00:34:24,680
We're trying to find this information proactively.

802
00:34:24,880 --> 00:34:25,800
It's really hard.

803
00:34:26,440 --> 00:34:28,520
Stuff gets published all over the place

804
00:34:28,520 --> 00:34:30,600
and we don't know to necessarily look for it.

805
00:34:30,680 --> 00:34:31,800
We're trying to improve that,

806
00:34:31,900 --> 00:34:33,360
but we need to rely on the community

807
00:34:33,360 --> 00:34:35,940
to tell us when the work they've done

808
00:34:35,940 --> 00:34:37,840
is ready for publication,

809
00:34:38,160 --> 00:34:38,900
ready to be published.

810
00:34:42,820 --> 00:34:43,900
Again, if you have an embargo,

811
00:34:44,480 --> 00:34:45,200
that's fine.

812
00:34:46,960 --> 00:34:48,320
When the embargo ends, though,

813
00:34:48,460 --> 00:34:49,500
you've got to let us know.

814
00:34:49,640 --> 00:34:51,420
We have no idea whether it's under embargo

815
00:34:51,420 --> 00:34:53,380
because we're not part of the embargo.

816
00:34:53,880 --> 00:34:55,000
So you've got to tell us.

817
00:34:59,620 --> 00:35:00,600
So again, as I said,

818
00:35:00,680 --> 00:35:01,940
once we have that public reference

819
00:35:01,940 --> 00:35:03,520
and we have enough information to know

820
00:35:03,520 --> 00:35:05,060
it's a unique ID, it gets published.

821
00:35:05,320 --> 00:35:07,140
And congratulations, you can put it on your resume.

822
00:35:10,120 --> 00:35:10,840
Pretty straightforward.

823
00:35:11,240 --> 00:35:12,120
It actually isn't.

824
00:35:12,820 --> 00:35:13,420
It goes on in the background.

825
00:35:13,540 --> 00:35:14,120
A lot of coordination.

826
00:35:14,360 --> 00:35:16,160
The disclosure processes will vary.

827
00:35:16,360 --> 00:35:17,860
If you are in this business,

828
00:35:17,960 --> 00:35:19,120
if you're doing this regularly,

829
00:35:19,380 --> 00:35:20,960
there are ways to shortcut this

830
00:35:21,420 --> 00:35:24,320
where you can send us information more effectively,

831
00:35:24,520 --> 00:35:25,000
more efficiently.

832
00:35:25,180 --> 00:35:27,580
We can talk about that if you're interested in that.

833
00:35:27,740 --> 00:35:29,600
But if you're just looking for one-offs here and there,

834
00:35:30,100 --> 00:35:32,660
you just have to remember that we need to know

835
00:35:32,660 --> 00:35:33,760
what you know.

836
00:35:33,860 --> 00:35:35,860
We need to know when you're going public,

837
00:35:35,940 --> 00:35:37,220
when you're publishing information.

838
00:35:39,440 --> 00:35:41,360
Okay, we're running a little low on time.

839
00:35:45,080 --> 00:35:46,540
Yeah, we're almost out.

840
00:35:49,440 --> 00:35:50,120
So for,

841
00:35:50,180 --> 00:35:51,160
we're running a little low on time.

842
00:35:51,240 --> 00:35:51,540
So for,

843
00:35:51,720 --> 00:35:53,780
when you come back to us to let us know

844
00:35:53,780 --> 00:35:54,900
to publish what you have,

845
00:35:55,060 --> 00:35:58,040
we kind of encourage you to supply us

846
00:35:58,040 --> 00:35:59,840
with like a description of your research

847
00:35:59,840 --> 00:36:01,200
because you guys know the source code

848
00:36:01,200 --> 00:36:02,760
and you know the research better than we do.

849
00:36:03,920 --> 00:36:05,380
And so we kind of have some things

850
00:36:05,380 --> 00:36:06,660
that are necessary to be in it.

851
00:36:06,760 --> 00:36:08,280
So if you go to our public list

852
00:36:08,280 --> 00:36:10,740
and you'll see that in the description

853
00:36:10,740 --> 00:36:12,720
you'll see that there has to be a product

854
00:36:12,820 --> 00:36:13,860
the version problem type

855
00:36:13,860 --> 00:36:15,840
is described in the description.

856
00:36:16,560 --> 00:36:18,320
And then we also have

857
00:36:18,320 --> 00:36:20,240
wanted to note that

858
00:36:20,820 --> 00:36:22,700
we only publish information that's

859
00:36:22,700 --> 00:36:24,780
that is already public in that

860
00:36:24,780 --> 00:36:25,960
public reference that you give us.

861
00:36:26,120 --> 00:36:28,620
And so any information, any additional information

862
00:36:28,620 --> 00:36:30,900
you may have given us in the request phase

863
00:36:30,900 --> 00:36:32,660
will not be published when you

864
00:36:32,660 --> 00:36:33,580
request your publish

865
00:36:34,540 --> 00:36:36,040
request us to publish the ID.

866
00:36:36,160 --> 00:36:38,680
We only publish information that is reflected

867
00:36:38,680 --> 00:36:39,460
in the reference.

868
00:36:40,200 --> 00:36:42,640
So that's another thing to take note of.

869
00:36:42,820 --> 00:36:45,620
And then we only want relevant information on it.

870
00:36:45,700 --> 00:36:46,760
So we don't want any profanity,

871
00:36:46,920 --> 00:36:49,260
no marketing in the descriptions.

872
00:36:49,700 --> 00:36:51,140
Just things that are relevant

873
00:36:51,140 --> 00:36:52,220
to the vulnerabilities

874
00:36:52,220 --> 00:36:53,640
what we want to have published

875
00:36:53,640 --> 00:36:55,580
in the public list.

876
00:37:01,200 --> 00:37:02,800
No profanity in the description.

877
00:37:03,140 --> 00:37:04,880
And then lastly it has to be in English.

878
00:37:05,800 --> 00:37:07,500
When you send it to the primary CNA.

879
00:37:07,860 --> 00:37:09,120
There are other organizations

880
00:37:09,120 --> 00:37:10,200
such as JPCert

881
00:37:10,200 --> 00:37:12,120
that are looking to be able to

882
00:37:12,120 --> 00:37:13,260
translate for you guys

883
00:37:13,260 --> 00:37:15,640
from whatever language they have to English

884
00:37:15,640 --> 00:37:16,540
when they send it to us.

885
00:37:16,940 --> 00:37:19,860
If you don't have a kind of style

886
00:37:19,860 --> 00:37:21,100
this is what MITRE uses.

887
00:37:21,340 --> 00:37:22,240
These are two templates.

888
00:37:22,600 --> 00:37:25,400
We actually have a public presentation

889
00:37:25,400 --> 00:37:27,440
you guys can actually look at

890
00:37:27,440 --> 00:37:29,080
which will give you more information

891
00:37:29,080 --> 00:37:30,300
on how we do things like

892
00:37:30,300 --> 00:37:31,620
how do we describe versioning,

893
00:37:32,760 --> 00:37:34,620
product types, vulnerability types.

894
00:37:34,900 --> 00:37:37,080
So that's publicly available with examples

895
00:37:37,080 --> 00:37:38,760
if you want to look into

896
00:37:40,200 --> 00:37:41,360
providing this kind of style

897
00:37:41,360 --> 00:37:43,440
when you submit for a publication request.

898
00:37:46,620 --> 00:37:47,680
Just really quick.

899
00:37:47,980 --> 00:37:50,140
As I mentioned we need help

900
00:37:50,140 --> 00:37:51,880
with the updating of this information.

901
00:37:52,120 --> 00:37:54,280
If you see something that needs updating

902
00:37:54,280 --> 00:37:56,060
even if it's not your vulnerability

903
00:37:56,060 --> 00:37:57,320
not something that you found

904
00:37:57,320 --> 00:37:58,300
but you have information

905
00:37:58,300 --> 00:37:59,560
you'd like to see it in there.

906
00:38:00,660 --> 00:38:02,700
Certainly for example the US government

907
00:38:02,700 --> 00:38:05,320
they look at CVE as a source

908
00:38:05,320 --> 00:38:06,800
of what their risk is.

909
00:38:06,980 --> 00:38:08,720
So if they look at our list and say

910
00:38:10,200 --> 00:38:10,840
there's no other information

911
00:38:10,840 --> 00:38:12,840
it's a problem.

912
00:38:13,080 --> 00:38:15,380
They can't do their proper risk assessment

913
00:38:15,380 --> 00:38:16,540
based on our information.

914
00:38:16,840 --> 00:38:19,560
But if they see that there is a public reference

915
00:38:19,560 --> 00:38:20,380
they can let us know

916
00:38:20,380 --> 00:38:21,300
we can publish it

917
00:38:21,300 --> 00:38:23,860
and not only do they benefit

918
00:38:23,860 --> 00:38:26,100
because it works into their own system

919
00:38:26,100 --> 00:38:27,340
the whole community benefits.

920
00:38:28,380 --> 00:38:32,100
It just helps the program be a little more robust.

921
00:38:32,600 --> 00:38:34,100
So you see something

922
00:38:34,100 --> 00:38:36,100
you can update it.

923
00:38:36,400 --> 00:38:38,060
If you see something

924
00:38:38,060 --> 00:38:39,160
and it looks wrong

925
00:38:40,200 --> 00:38:40,940
you can dispute it.

926
00:38:41,120 --> 00:38:42,920
We have a dispute process.

927
00:38:44,580 --> 00:38:45,880
CNA or not

928
00:38:45,880 --> 00:38:47,060
you can dispute anything.

929
00:38:47,380 --> 00:38:49,620
It just shows up typically as a dispute

930
00:38:49,940 --> 00:38:51,700
in the list but it helps enrich

931
00:38:51,700 --> 00:38:52,400
that information.

932
00:38:54,380 --> 00:38:55,240
And before

933
00:38:55,240 --> 00:38:57,360
we finish up

934
00:38:58,580 --> 00:38:59,140
so

935
00:39:00,180 --> 00:39:02,240
we don't care about attribution.

936
00:39:02,340 --> 00:39:03,060
Like I just said

937
00:39:03,060 --> 00:39:05,320
you can send us information whether or not

938
00:39:05,320 --> 00:39:06,280
you discovered a vulnerability.

939
00:39:07,580 --> 00:39:10,060
We don't include credit or attribution

940
00:39:10,200 --> 00:39:11,280
as part of a CV entry.

941
00:39:11,860 --> 00:39:13,120
We're not in that business.

942
00:39:14,600 --> 00:39:16,620
You can send us a reference

943
00:39:16,620 --> 00:39:18,480
and that reference can be a blog post

944
00:39:18,480 --> 00:39:19,800
that says I found this vulnerability.

945
00:39:20,200 --> 00:39:21,580
Great, we'll add it to the thing.

946
00:39:21,660 --> 00:39:22,700
But we're not going to put it in the entry.

947
00:39:23,940 --> 00:39:26,200
We're more interested in these vulnerabilities

948
00:39:26,200 --> 00:39:27,740
getting disclosed

949
00:39:27,740 --> 00:39:29,440
and the information getting out there

950
00:39:29,440 --> 00:39:31,800
than we are about padding people's resumes.

951
00:39:33,800 --> 00:39:35,000
That's not our focus.

952
00:39:35,860 --> 00:39:37,040
It's a nice side benefit

953
00:39:37,040 --> 00:39:38,280
but it's not our focus.

954
00:39:40,200 --> 00:39:41,660
You typically aren't going to see

955
00:39:42,200 --> 00:39:43,420
attribution in a CV entry

956
00:39:43,420 --> 00:39:44,140
and that's why.

957
00:39:45,940 --> 00:39:48,040
For CNAs we will be attributing

958
00:39:48,040 --> 00:39:49,800
which CNA might have

959
00:39:49,800 --> 00:39:51,140
submitted a CV entry

960
00:39:51,140 --> 00:39:52,820
just so you know that

961
00:39:52,820 --> 00:39:55,620
this Apache vulnerability

962
00:39:56,500 --> 00:39:57,960
description was written by

963
00:39:57,960 --> 00:39:59,940
Apache as opposed to written by

964
00:39:59,940 --> 00:40:01,080
somebody else.

965
00:40:02,480 --> 00:40:04,020
But that's as far as

966
00:40:04,020 --> 00:40:05,700
we will go for attribution.

967
00:40:05,840 --> 00:40:06,500
We don't care.

968
00:40:08,820 --> 00:40:09,900
That said

969
00:40:10,200 --> 00:40:11,520
we understand that's a use case.

970
00:40:11,740 --> 00:40:12,880
It's part of what the community

971
00:40:12,880 --> 00:40:14,440
gets out of CVE.

972
00:40:15,360 --> 00:40:17,900
If you value CVE

973
00:40:17,900 --> 00:40:20,460
if you think it is useful

974
00:40:20,460 --> 00:40:22,300
if you think it helps make

975
00:40:23,020 --> 00:40:25,060
your life as a vulnerability researcher

976
00:40:25,060 --> 00:40:26,370
as a defender better

977
00:40:26,840 --> 00:40:27,900
we need your help.

978
00:40:30,320 --> 00:40:31,680
We're trying the best we can.

979
00:40:33,420 --> 00:40:34,920
Throughput is a big problem.

980
00:40:35,080 --> 00:40:35,900
We're doing much better.

981
00:40:36,000 --> 00:40:37,020
We're actually on track to have

982
00:40:37,020 --> 00:40:38,660
twice as many CVEs this year

983
00:40:40,200 --> 00:40:41,680
that's probably around

984
00:40:42,200 --> 00:40:43,740
what was it last year?

985
00:40:44,040 --> 00:40:46,160
I think almost 20,000

986
00:40:46,160 --> 00:40:47,320
we're on track for.

987
00:40:48,360 --> 00:40:49,140
That's a lot.

988
00:40:49,980 --> 00:40:51,740
And still not all the vulnerabilities

989
00:40:51,740 --> 00:40:52,600
that are out there.

990
00:40:52,780 --> 00:40:54,820
And not all the vulnerabilities that are out there

991
00:40:54,820 --> 00:40:56,220
across all the different

992
00:40:58,000 --> 00:40:58,560
verticals.

993
00:40:58,560 --> 00:41:00,960
If you're really into automotive

994
00:41:00,960 --> 00:41:02,000
vulnerabilities

995
00:41:02,740 --> 00:41:04,460
just send us what you got.

996
00:41:04,620 --> 00:41:06,900
We can start building this into

997
00:41:06,900 --> 00:41:08,920
the vulnerability management disclosure processes

998
00:41:10,200 --> 00:41:12,040
and hopefully this little part

999
00:41:12,040 --> 00:41:13,660
will help jumpstart the culture

1000
00:41:13,660 --> 00:41:16,100
that needs to be out there of disclosing

1001
00:41:16,100 --> 00:41:18,040
responsibly or coordinatedly

1002
00:41:18,040 --> 00:41:18,960
or however you want to disclose

1003
00:41:19,540 --> 00:41:22,140
so that everybody is in a better shape afterwards.

1004
00:41:23,360 --> 00:41:24,560
If you're going to take a picture

1005
00:41:24,560 --> 00:41:25,380
take a picture here.

1006
00:41:25,500 --> 00:41:27,840
It's all the links that we've talked about.

1007
00:41:29,140 --> 00:41:30,240
The CNA rules

1008
00:41:30,240 --> 00:41:32,400
include the counting rules

1009
00:41:32,400 --> 00:41:33,700
if you want to see what the

1010
00:41:34,040 --> 00:41:35,640
gory details of counting look like.

1011
00:41:37,580 --> 00:41:38,020
So

1012
00:41:38,020 --> 00:41:39,840
this is a good slide to grab.

1013
00:41:41,420 --> 00:41:42,860
And we're out of time.

1014
00:41:42,980 --> 00:41:43,700
So I'm going to

1015
00:41:44,200 --> 00:41:45,560
Anthony and I will be out there

1016
00:41:45,920 --> 00:41:48,160
as soon as we clean up our laptops

1017
00:41:48,160 --> 00:41:49,500
and if you have any questions

1018
00:41:49,500 --> 00:41:50,940
please come out and talk to us.

1019
00:41:50,980 --> 00:41:52,760
Otherwise, thank you very much for your attention

1020
00:41:52,760 --> 00:41:53,800
and enjoy your time at DEF CON.

1021
00:41:57,000 --> 00:41:58,340
We have time for questions.

1022
00:41:58,480 --> 00:42:00,260
Feel free to come up to the mic

1023
00:42:00,900 --> 00:42:01,800
and ask away.

1024
00:42:08,780 --> 00:42:09,980
Come on, yeah, come on

1025
00:42:10,200 --> 00:42:10,380
to the mic.

1026
00:42:11,720 --> 00:42:13,460
No? Alright. Hi.

1027
00:42:13,740 --> 00:42:16,320
Hi. Is the DWF

1028
00:42:17,020 --> 00:42:18,740
are they considered a CNA now?

1029
00:42:19,000 --> 00:42:20,400
Yes, the DWF is a CNA.

1030
00:42:20,500 --> 00:42:22,540
Okay, so quick history on the DWF.

1031
00:42:22,620 --> 00:42:23,880
The DWF is the distributed

1032
00:42:23,880 --> 00:42:25,920
weakness filing. It is also

1033
00:42:25,920 --> 00:42:27,220
one letter off from CVE.

1034
00:42:30,320 --> 00:42:32,180
That's all them. I had nothing to do with that.

1035
00:42:32,340 --> 00:42:34,320
So anyway, what is the DWF?

1036
00:42:34,380 --> 00:42:36,160
The DWF is a CNA that covers

1037
00:42:36,160 --> 00:42:38,100
all open source software

1038
00:42:38,100 --> 00:42:40,040
that is not already covered

1039
00:42:40,200 --> 00:42:41,080
by somebody else.

1040
00:42:43,700 --> 00:42:44,800
CNAs all have scope.

1041
00:42:44,980 --> 00:42:46,240
They all have the thing that they cover.

1042
00:42:46,460 --> 00:42:48,180
So the DWF is sort of a blanket.

1043
00:42:48,340 --> 00:42:49,320
They cover everything.

1044
00:42:49,540 --> 00:42:51,780
Now, they're less than a year old

1045
00:42:52,400 --> 00:42:54,120
and they are in the process

1046
00:42:54,120 --> 00:42:55,540
of spinning up their operations.

1047
00:42:55,840 --> 00:42:57,600
It takes a lot to run this thing

1048
00:42:58,360 --> 00:42:59,760
even when you're relying on

1049
00:42:59,760 --> 00:43:02,300
the open source community to help you.

1050
00:43:02,520 --> 00:43:04,400
So when you have

1051
00:43:04,400 --> 00:43:05,320
an open source vulnerability

1052
00:43:05,320 --> 00:43:06,940
you can go to the DWF.

1053
00:43:07,180 --> 00:43:09,980
You can do a pull request. It is based

1054
00:43:10,200 --> 00:43:11,800
of GitHub. You can do a pull request.

1055
00:43:12,000 --> 00:43:14,240
You can add your information in there.

1056
00:43:15,100 --> 00:43:16,000
I think actually

1057
00:43:16,000 --> 00:43:17,760
first you have to request the CVID

1058
00:43:17,760 --> 00:43:19,600
through their web form. They have a separate

1059
00:43:19,600 --> 00:43:22,040
web form. They're working on

1060
00:43:22,040 --> 00:43:23,860
a lot of processes that I know

1061
00:43:23,860 --> 00:43:25,860
some people have had trouble getting CVIDs

1062
00:43:25,860 --> 00:43:26,340
from them.

1063
00:43:27,680 --> 00:43:29,820
You can contact us and we'll make sure

1064
00:43:29,820 --> 00:43:31,960
you get a CVID. Again, all of

1065
00:43:31,960 --> 00:43:33,960
open source is covered. The DWF

1066
00:43:33,960 --> 00:43:35,960
hopefully once they're up to speed operationally

1067
00:43:35,960 --> 00:43:37,520
they'll be covering all of that.

1068
00:43:37,600 --> 00:43:39,880
We can send people that way and they can get it through there.

1069
00:43:40,200 --> 00:43:42,220
Until then, and even after

1070
00:43:42,220 --> 00:43:44,120
then, you can always come to MITRE if you have

1071
00:43:44,120 --> 00:43:45,820
a CVID request and we will

1072
00:43:45,820 --> 00:43:48,060
help you fulfill it. The goal

1073
00:43:48,060 --> 00:43:49,440
was open source is big.

1074
00:43:50,140 --> 00:43:52,140
One way to federate this out, to scale

1075
00:43:52,140 --> 00:43:54,000
it out is to just get another entity that can

1076
00:43:54,000 --> 00:43:55,940
focus on it and use a lot of different

1077
00:43:55,940 --> 00:43:57,640
tools that we may not have access to.

1078
00:43:59,880 --> 00:44:01,160
They're working on it. They're trying.

1079
00:44:02,060 --> 00:44:02,880
Yay open source.

1080
00:44:06,120 --> 00:44:06,940
Yeah, go ahead.

1081
00:44:08,720 --> 00:44:09,700
Can you come up to

1082
00:44:15,600 --> 00:44:16,640
the mic?

1083
00:44:18,900 --> 00:44:19,540
I'll repeat it.

1084
00:44:19,540 --> 00:44:21,960
You can download the entire corpus.

1085
00:44:22,940 --> 00:44:23,880
It's on our website.

1086
00:44:24,440 --> 00:44:25,640
You can always do that.

1087
00:44:26,660 --> 00:44:28,080
Josh Bressers, who

1088
00:44:28,080 --> 00:44:29,500
works at Elastic, he recently

1089
00:44:30,560 --> 00:44:31,980
joined Elastic and he's

1090
00:44:31,980 --> 00:44:34,040
been playing with all sorts of data. He took the

1091
00:44:34,040 --> 00:44:35,980
full data set and he's doing some really cool stuff

1092
00:44:37,240 --> 00:44:38,200
with visualization,

1093
00:44:38,540 --> 00:44:40,180
looking at statistics and things like that.

1094
00:44:40,200 --> 00:44:43,980
If you look for him on Twitter,

1095
00:44:44,100 --> 00:44:46,040
Josh Bressers, talk to me afterwards.

1096
00:44:46,200 --> 00:44:47,060
I can hook you up.

1097
00:44:47,180 --> 00:44:48,460
He actually just posted a few things

1098
00:44:48,460 --> 00:44:49,220
about it recently.

1099
00:44:49,800 --> 00:44:51,640
He'll be posting to his blog about it.

1100
00:44:51,740 --> 00:44:53,420
He also runs the open source security podcast.

1101
00:44:53,800 --> 00:44:54,520
I don't know if you've heard of that.

1102
00:44:55,500 --> 00:44:57,100
That data set is available.

1103
00:44:57,600 --> 00:44:59,480
Speaking of that, we're hoping to have more

1104
00:44:59,480 --> 00:45:01,660
metadata available for people to

1105
00:45:02,020 --> 00:45:04,300
learn more about what we're seeing

1106
00:45:04,300 --> 00:45:05,980
as information is coming in.

1107
00:45:06,080 --> 00:45:07,340
Again, nothing private, but just

1108
00:45:07,340 --> 00:45:09,300
meta information based on

1109
00:45:10,200 --> 00:45:12,280
the type of vulnerabilities that are coming in,

1110
00:45:12,360 --> 00:45:13,620
how they're coming in, things like that.

1111
00:45:13,720 --> 00:45:15,880
We're going to post that to the website and have that

1112
00:45:15,880 --> 00:45:18,100
available in a way that's

1113
00:45:18,100 --> 00:45:18,740
useful, hopefully.

1114
00:45:19,520 --> 00:45:21,800
Keep note. Counting vulnerabilities

1115
00:45:21,800 --> 00:45:23,840
is a black art.

1116
00:45:25,100 --> 00:45:25,940
The number of

1117
00:45:25,940 --> 00:45:27,800
CVEs that a product has says

1118
00:45:27,800 --> 00:45:29,820
nothing about the security of that product.

1119
00:45:30,120 --> 00:45:32,020
The number of CVEs that you have

1120
00:45:32,020 --> 00:45:33,940
squashed says nothing about how good

1121
00:45:33,940 --> 00:45:35,560
you are at being a security person.

1122
00:45:35,700 --> 00:45:37,540
Be very careful when you're counting vulnerabilities.

1123
00:45:37,960 --> 00:45:40,120
We count one way, a different

1124
00:45:40,200 --> 00:45:42,180
vulnerability database counts different.

1125
00:45:43,440 --> 00:45:46,180
You need to remember that when you're

1126
00:45:46,180 --> 00:45:47,620
looking at numbers and statistics.

1127
00:45:47,880 --> 00:45:50,340
You can only really compare us to ourselves.

1128
00:45:50,800 --> 00:45:52,120
You can't compare us to other people.

1129
00:45:52,680 --> 00:45:54,100
If you've ever heard any

1130
00:45:54,100 --> 00:45:55,800
talks from Josh Corman with

1131
00:45:55,800 --> 00:45:57,940
I Am The Calvary, he has a really good visualization

1132
00:45:57,940 --> 00:45:58,580
of this.

1133
00:45:59,520 --> 00:46:01,960
He's talked a lot about it. He's been very critical of

1134
00:46:01,960 --> 00:46:03,560
CVE in a good way.

1135
00:46:03,700 --> 00:46:05,460
We like Josh because he's saying,

1136
00:46:05,560 --> 00:46:07,960
look, vulnerability management is a big deal.

1137
00:46:08,500 --> 00:46:10,180
CVE is trying real hard and they're not

1138
00:46:10,200 --> 00:46:11,440
getting it, and the rest of the community,

1139
00:46:11,560 --> 00:46:13,040
they're not getting it. Hey, Congress,

1140
00:46:13,220 --> 00:46:14,900
let's do some better stuff, especially about

1141
00:46:14,900 --> 00:46:16,120
medical device security.

1142
00:46:17,760 --> 00:46:19,440
There's a lot of good information there from

1143
00:46:19,440 --> 00:46:21,240
the I Am The Calvary folk, and again,

1144
00:46:21,280 --> 00:46:22,460
Josh Pressure is another good source.

1145
00:46:22,820 --> 00:46:24,640
We'll give you some more information too in the future.

1146
00:46:26,820 --> 00:46:27,580
Any other questions?

1147
00:46:33,760 --> 00:46:34,820
For one more time,

1148
00:46:35,420 --> 00:46:36,520
Dan and Anthony, everyone.