1
00:00:00,540 --> 00:00:08,300
From your perspective, what does CVD mean to you?

2
00:00:08,300 --> 00:00:13,200
And kind of what are your motivations when you're working on an issue and you need to disclose something?

3
00:00:14,460 --> 00:00:20,500
So I think the main point of CVD is, of course, to protect users.

4
00:00:23,580 --> 00:00:27,900
And my main thought is a bit historical.

5
00:00:27,900 --> 00:00:36,260
So I've been around in the 90s where disclosure was you find a mailing list where other hackers hang out and you zero-day everybody.

6
00:00:37,200 --> 00:00:39,160
And nobody really cared.

7
00:00:39,940 --> 00:00:46,520
And that became responsible disclosure, which is the predecessor for coordinated vulnerability disclosure.

8
00:00:46,520 --> 00:00:59,080
The problem with responsible disclosure is that it was really made to shame hackers that didn't follow the desired protocol of industry text.

9
00:00:59,080 --> 00:01:13,800
And the current iteration, coordinated vulnerability disclosure, is instead of trying to be coercive, trying to actually work with people that want to submit issues.

10
00:01:14,600 --> 00:01:18,340
And it is a much, much improved over that.

11
00:01:18,380 --> 00:01:35,320
And it does a much better job of making sure that every party to a vulnerability disclosure, that means vendors, customers, and hackers, have their needs met.

12
00:01:35,320 --> 00:01:41,160
So that's what I think coordinated vulnerability disclosure is for me.

13
00:01:41,680 --> 00:01:42,880
Awesome.

14
00:01:42,880 --> 00:01:45,540
Daniel, another researching perspective.

15
00:01:45,540 --> 00:01:48,980
What do you think about coordinated vulnerability disclosure?

16
00:01:49,000 --> 00:01:53,720
So we are interested in understanding how things work.

17
00:01:53,720 --> 00:02:02,260
And sometimes that involves understanding how things work that are not supposed to happen, which we then call vulnerability.

18
00:02:02,420 --> 00:02:06,820
And of course, then we want to figure out the truth behind it.

19
00:02:06,820 --> 00:02:10,880
And that involves talking to the vendor.

20
00:02:11,540 --> 00:02:17,480
Very often, they tell us that we got something slightly wrong.

21
00:02:17,480 --> 00:02:24,580
And that is an opportunity to correct these mistakes before we submit the paper or publish the paper.

22
00:02:24,580 --> 00:02:35,880
But also, from my perspective, it's the only ethical thing to do to protect the customers, to protect people who are using these products.

23
00:02:36,220 --> 00:02:45,340
And for me, I'm not in this game for finding vulnerabilities as a motivation by itself.

24
00:02:45,340 --> 00:02:54,260
My main motivation is finding the truth, finding a better understanding of how things work.

25
00:02:54,900 --> 00:02:56,720
That's awesome.

26
00:02:56,900 --> 00:03:02,520
Katie, you've had a lot of experience monkeying around with CVD.

27
00:03:03,320 --> 00:03:05,620
What does a CVD mean to you?

28
00:03:05,620 --> 00:03:06,480
Hold on a minute.

29
00:03:06,680 --> 00:03:09,480
All right.

30
00:03:09,480 --> 00:03:19,480
So CVD means to me... So when I think about it, I think of it from a risk-based perspective.

31
00:03:19,480 --> 00:03:25,220
And so what Probe is alluding to here is that my background is government in nature.

32
00:03:25,220 --> 00:03:28,200
So I spent about 15 years in the U.S.

33
00:03:28,200 --> 00:03:30,020
government, 12 years of that in the U.S.

34
00:03:30,020 --> 00:03:30,920
Air Force.

35
00:03:31,260 --> 00:03:40,420
And then several other years at the Department of Homeland Security, where I ran the vulnerability disclosure profiles programs that most people are familiar with.

36
00:03:40,420 --> 00:03:45,340
So like the NIST NVD program and the Carnegie Mellon CERT CC program.

37
00:03:45,340 --> 00:03:48,680
And the MITRE CVD program.

38
00:03:48,700 --> 00:03:50,920
So I was a sponsor for those programs.

39
00:03:50,920 --> 00:03:58,580
So in a single year, 2017, we coordinated and disclosed 14,800 vulnerabilities for public disclosure.

40
00:03:58,920 --> 00:04:04,000
So that's 14,000 IT vulnerabilities and about 800 ICS vulnerabilities.

41
00:04:04,500 --> 00:04:10,360
In the following two years, so 18 and 19, we coordinated and disclosed over 20,000 cybersecurity vulnerabilities.

42
00:04:10,620 --> 00:04:13,960
So I've kind of seen things from lots of different perspectives.

43
00:04:14,020 --> 00:04:20,920
And when I think about CVD, I think about balancing risk, right?

44
00:04:20,920 --> 00:04:23,220
So CVD is really a process.

45
00:04:23,400 --> 00:04:33,100
And every time you, when you're going through this process, understand like every organization is going to treat it differently, because there isn't a standardized, like one size fits all kind of thing here.

46
00:04:34,060 --> 00:04:38,300
There's differences that are going to happen across the coordination stack.

47
00:04:38,300 --> 00:04:47,220
So if you're looking at digital services, it's going to be different from software, which is going to be different from open source, which is going to be different from hardware or ICS, like all the differences are going to come into play there.

48
00:04:47,220 --> 00:04:54,820
But the overarching sort of thing, the big takeaway here is that CVD is balancing risk.

49
00:04:54,940 --> 00:05:06,380
It's all about making sure that there is an opportunity for the product vendor to fix the problem before an adversary has the opportunity to take advantage of that.

50
00:05:06,380 --> 00:05:08,780
So like, it's all about protecting the end user.

51
00:05:08,780 --> 00:05:14,900
And I think everybody that I've ever met in the entire ecosystem is all focused on that.

52
00:05:14,900 --> 00:05:20,960
Like we may be speaking different languages, we may talk past each other, but I think that everyone is trying to protect the end user.

53
00:05:20,960 --> 00:05:23,680
So to me, that's what that's what CVD is about.

54
00:05:24,240 --> 00:05:25,200
That's awesome.

55
00:05:25,200 --> 00:05:31,580
And remind you, remind me to circle back to you later in our talk, talk a little about some of the psychology behind some of this.

56
00:05:31,580 --> 00:05:47,860
I would love to hear my dearest friend Lisa's thoughts on kind of coordinating vulnerability disclosure, and kind of what some of your motivations that you've seen, or kind of what's behind your modus operandi?

57
00:05:48,420 --> 00:06:05,420
Yeah, so I think where I've seen a sort of an evolution of what we do now, where I think Heartbleed was sort of the big start of where we paid attention, realized we all need to come together a little bit.

58
00:06:05,420 --> 00:06:08,880
And then Spectre Meltdown sort of brought us even closer.

59
00:06:09,120 --> 00:06:21,880
So my thought is, is that not only do we work with the researchers, but we work across the industry to, you know, make sure that we're all doing our best to protect our customers.

60
00:06:21,940 --> 00:06:36,060
So like Katie was talking about, it's really our end users that are our focus, and making sure that we have the right people involved to be able to solve the issue, and then provide a security update so that our customers could get it and be best protected.

61
00:06:45,180 --> 00:06:55,560
So Omar, you've been doing this for a while, and you work for a very large company, so I'm sure you get the opportunity to see a lot of different vulnerabilities.

62
00:06:55,560 --> 00:06:59,880
So what does the whole process mean, and what are some of the motivations you've seen?

63
00:07:00,300 --> 00:07:06,720
Yeah, I think that pretty much everybody has summarized it very well, is protecting the end consumer at the end of the day.

64
00:07:06,940 --> 00:07:12,800
But I see it as a very, very complex ecosystem, right, that we're trying to actually solve.

65
00:07:13,220 --> 00:07:24,540
And Katie, you know, mentioned in one side hardware, another side software, then if you decompose that, then you have open source, which we're going to probably talk about a lot in here.

66
00:07:24,820 --> 00:07:28,160
And you have things that, you know, perhaps you cannot even control.

67
00:07:28,160 --> 00:07:41,560
I have had, I guess, the pleasure or not the pleasure, in some cases, where we're looking at vulnerabilities and even in some cases, whenever we try to actually solve the issues, the companies don't even exist anymore.

68
00:07:42,380 --> 00:07:47,700
That's another predicament, you know, that in some cases, we actually have to take into consideration.

69
00:07:47,700 --> 00:07:50,280
And it's a fairly complex ecosystem.

70
00:07:50,860 --> 00:08:07,980
And at the end of the day, what we have to actually put our heads together is how can we modernize our practice into the way that not only, yes, we deal with a single vulnerability, we reproduce it, we actually fix it, we find the patch, but how can we accelerate that process,

71
00:08:07,980 --> 00:08:15,380
that one, everybody's talking somewhat the correct vocabulary, or at least they can understand each other.

72
00:08:15,600 --> 00:08:19,000
And then we also understand the overall risk, right?

73
00:08:19,680 --> 00:08:22,700
And how to prioritize things and so on.

74
00:08:22,700 --> 00:08:26,120
So it's fairly complex, so we can actually talk about it for quite some time.

75
00:08:26,120 --> 00:08:27,400
And that's what we're here for.

76
00:08:27,800 --> 00:08:31,040
But those are some of the initial perspectives that I want to share.

77
00:08:31,780 --> 00:08:33,860
Thank you, I appreciate that.

78
00:08:34,520 --> 00:08:38,700
I want to talk about something we touched on in one of our prep calls.

79
00:08:38,740 --> 00:08:46,380
We have Anders and Daniel that come at this research angle from slightly different angles.

80
00:08:46,880 --> 00:08:51,520
So maybe you two gentlemen can figure out who wants to start first.

81
00:08:51,520 --> 00:09:06,480
But thinking about the perspective of academia versus a professional bug hunter, security researcher, maybe you can describe some of the particulars in either area.

82
00:09:06,480 --> 00:09:11,220
And maybe what are some of the differences you might see between those two different types of research?

83
00:09:13,440 --> 00:09:27,620
So from the industry, or rather the bug hunter, be professional or not, there is very often an element of good old-fashioned fun.

84
00:09:28,460 --> 00:09:36,480
I started out hacking things not because I wanted to achieve anything with it, just because it was great fun.

85
00:09:37,800 --> 00:09:42,660
When you become pro with it, there is different motivations.

86
00:09:42,780 --> 00:09:51,280
So if you work for a company and you work on their products, obviously your motivation is to make those products secure.

87
00:09:51,900 --> 00:10:03,640
And sometimes my past job, I did some research and that was sponsored by that company, but essentially me doing what I like to do.

88
00:10:03,920 --> 00:10:10,700
Their end of it was attention to that company's competences.

89
00:10:10,740 --> 00:10:13,620
And my end, of course, was having a bit of fun with it.

90
00:10:14,520 --> 00:10:20,780
So that is probably pretty typical of what hackers get out of hacking.

91
00:10:21,580 --> 00:10:40,220
Yeah, from the more academic side, it's more of advancing the field, the knowledge in the field, and there you try to figure out how things work and what the implications there are, what the security implications of certain understandings are.

92
00:10:40,220 --> 00:10:50,920
For instance, understanding when you can execute a certain piece of code and that does something that is not intended, what are the implications?

93
00:10:51,000 --> 00:10:58,820
And this is interesting from a security perspective than if it enables someone to do something that they shouldn't be allowed to do.

94
00:10:59,700 --> 00:11:04,880
Yes, often this involves disclosing a vulnerability to a vendor.

95
00:11:04,880 --> 00:11:15,060
But I would say that, for instance, bug bounties that might be very relevant for people from the industry.

96
00:11:15,620 --> 00:11:33,560
I think this plays a smaller role in academia, because you have to participate in CVD anyway, if you wouldn't, then you would get a lot of problems in the academic community.

97
00:11:33,560 --> 00:11:42,660
There's a lot of peer pressure that you participate in this, because that's the only ethical approach to handling vulnerabilities.

98
00:11:42,740 --> 00:11:52,080
At the same time, if you keep them secret for too long, then there's also the question on why did you keep them secret for so long?

99
00:11:52,610 --> 00:11:55,880
Maybe also some perspective that I can share.

100
00:11:56,400 --> 00:12:06,690
In academia, you see more and more often the effect that academic publications are easier to publish if you have a CVE.

101
00:12:07,360 --> 00:12:22,700
I don't think that's a good thing, because I think that a CVE does not necessarily describe that a research result really brings you new insights.

102
00:12:22,700 --> 00:12:34,060
A CVE is an identifier for a vulnerability, not an identifier that says this is something with a new insight.

103
00:12:34,450 --> 00:12:48,460
And also, if you participated in CVD, and you mentioned this in the paper, this also gives you bonus points, at least that's my impression, to get the paper published.

104
00:12:50,180 --> 00:12:59,740
Of course, it's good if you participate in CVD, but my feeling is also that this is going a bit too strong.

105
00:13:00,860 --> 00:13:05,640
And yes, now overemphasized in our community a bit.

106
00:13:05,640 --> 00:13:09,160
What about, you know, having an icon or a fun name?

107
00:13:11,580 --> 00:13:14,680
So, there are, of course, there are.

108
00:13:16,740 --> 00:13:22,980
I'm really a big fan of logos and names and anything that is fun.

109
00:13:23,620 --> 00:13:26,480
But of course, there are multiple layers here.

110
00:13:26,660 --> 00:13:34,040
For instance, we had this paper, Hello from the Other Side, which had this name not just by coincidence.

111
00:13:34,740 --> 00:13:43,060
And it was about a covert channel in the cloud where we send data through the cache from one virtual machine to the other.

112
00:13:43,060 --> 00:13:46,840
And we really went crazy on that.

113
00:13:46,840 --> 00:13:56,340
So, we built an entire TCP stack on top of that and tunneled an SSH session through the cache covert channel for whatever reason.

114
00:13:56,340 --> 00:14:01,600
And we then sent a music video through this cache covert channel.

115
00:14:01,600 --> 00:14:04,300
And of course, we couldn't just take any music video.

116
00:14:04,300 --> 00:14:07,860
So, we had to make our own parody of Hello from the Other Side.

117
00:14:07,860 --> 00:14:09,180
And this is just fun.

118
00:14:09,180 --> 00:14:12,520
I mean, we are just a bunch of people and we like to have fun.

119
00:14:12,740 --> 00:14:22,820
And basically, once you finish your project, it's really nice to close this up with something nice and funny.

120
00:14:23,340 --> 00:14:27,920
And logos on the other side also help about...

121
00:14:28,700 --> 00:14:31,860
So, they help you communicating about the issue.

122
00:14:32,660 --> 00:14:44,200
I realized that every time I create slides, my favorite slides are the slides that don't have any words on them, just pictures, logos, and icons.

123
00:14:44,200 --> 00:14:47,360
Because then I can follow the speaker.

124
00:14:47,500 --> 00:14:50,820
I don't have to read because I'm very bad at reading.

125
00:14:50,820 --> 00:14:53,000
I can't read and listen at the same time.

126
00:14:53,000 --> 00:14:55,040
And I bet many people can't do that.

127
00:14:55,040 --> 00:14:59,180
So, there's only icons and symbols and logos.

128
00:14:59,180 --> 00:15:04,880
I can follow what the person is saying and look at the images at the same time.

129
00:15:05,040 --> 00:15:12,740
And I'm always annoyed if I have to speak about the vulnerability and I don't have a logo for it because then I have to put text on the slide.

130
00:15:13,740 --> 00:15:18,040
So, since Daniel brought it up, let me ask the other panelists.

131
00:15:18,420 --> 00:15:22,220
How do you all feel about branded vulnerabilities?

132
00:15:22,220 --> 00:15:23,700
Do you like logos?

133
00:15:23,700 --> 00:15:26,700
Do you find that helpful to you and your constituents?

134
00:15:26,980 --> 00:15:28,980
Oh, I'm sorry, Daniel.

135
00:15:28,980 --> 00:15:29,940
No.

136
00:15:32,200 --> 00:15:38,460
When there's a logo or PR or everything, it grabs the media's attention so quickly.

137
00:15:38,600 --> 00:15:47,660
And sometimes the issue is not that severe or very hard to be able to breach.

138
00:15:47,680 --> 00:16:03,540
And it gives a little bit of extra fear to our customers and also encourages us to maybe reprioritize other issues first, which are more severe to our customers.

139
00:16:03,640 --> 00:16:08,020
So, although I think I get the point, it's easy to talk about it.

140
00:16:08,020 --> 00:16:11,080
They grab the media's attention really quickly.

141
00:16:12,160 --> 00:16:23,480
But if you put a caveat about the real severity of the issue in there, maybe that would help a little bit.

142
00:16:23,840 --> 00:16:24,620
Yeah.

143
00:16:24,620 --> 00:16:27,920
So, in our team, we often have this discussion.

144
00:16:27,920 --> 00:16:29,200
Should we have a logo?

145
00:16:29,200 --> 00:16:31,640
Should we have a website or shouldn't we?

146
00:16:32,140 --> 00:16:33,520
And we have...

147
00:16:33,520 --> 00:16:43,000
So, for most of our papers and most of the vulnerabilities that we discovered, we don't because we just say they are not significant enough.

148
00:16:43,000 --> 00:16:54,940
And I think that's also the responsibility of the researcher to first assess, is this something that significant that I need this additional PR or is it not?

149
00:16:55,120 --> 00:16:55,520
Right.

150
00:16:55,520 --> 00:16:57,400
Well, then you're doing it the right way.

151
00:16:57,400 --> 00:16:57,900
So, yay.

152
00:16:57,900 --> 00:16:58,600
Cheers.

153
00:16:58,880 --> 00:16:59,660
Maybe.

154
00:16:59,660 --> 00:17:07,040
But it's also... I mean, I will have a different view on what is really important than you have, right?

155
00:17:07,040 --> 00:17:08,980
Everyone will have a different view on that.

156
00:17:08,980 --> 00:17:16,320
So, something that I think is really, really important, you might say, well, it's not really exploitable in our use case.

157
00:17:16,320 --> 00:17:16,580
Right.

158
00:17:16,580 --> 00:17:18,960
We don't ever want to downplay a researcher's work.

159
00:17:18,960 --> 00:17:21,320
I mean, it's important, whatever they do.

160
00:17:21,320 --> 00:17:26,360
And we appreciate especially doing CBD with them and not being zero-day.

161
00:17:27,860 --> 00:17:37,900
But yeah, it's a more matter of, especially in a bigger industry when you're trying to protect your customers best, how do you approach it?

162
00:17:38,220 --> 00:17:41,680
There is a fight for customer attention.

163
00:17:42,240 --> 00:17:49,580
And for customers to make the right security decisions, they need to be attentive about the right things.

164
00:17:49,580 --> 00:17:55,960
And logos and hype in the press wants that.

165
00:17:56,360 --> 00:18:01,920
And we have bad security outcomes for our customers due to that.

166
00:18:01,920 --> 00:18:12,620
Of course, there's also the thing that even fixed vulnerabilities sometimes cause suffering for system administrators running around the basement and patching systems and all the like.

167
00:18:14,180 --> 00:18:23,140
And to some extent, I sometimes find overhyped bugs disrespectful to those people.

168
00:18:25,980 --> 00:18:26,300
Yeah.

169
00:18:26,300 --> 00:18:37,840
So, in my case, I don't have an issue with logos or anything, even though in a previous call and a couple of exchanges, Daniel, I alluded into that.

170
00:18:37,980 --> 00:18:44,360
Whether it's a logo, as a matter of fact, even for Cisco, we had the first vulnerability with emojis.

171
00:18:44,360 --> 00:18:46,840
So, we have emojis, logos, and everything else.

172
00:18:46,840 --> 00:18:57,360
So, at the end of the day, if it helps to do an alias for a vulnerability and bring some awareness, I'm perfectly okay with that.

173
00:18:57,360 --> 00:19:12,580
The only challenge that I'm seeing or opportunity, right, is that in some cases, whenever we write information, and even vendors, in my case, we have a research institution at Cisco called Talos, and we find vulnerabilities in other people's products,

174
00:19:12,580 --> 00:19:13,140
right?

175
00:19:13,360 --> 00:19:14,880
And yes, it is.

176
00:19:14,880 --> 00:19:22,500
So, in that case, yes, we also have logos, and we put them up there, and we put up blog posts and so on, right?

177
00:19:22,720 --> 00:19:32,400
But what we need to do collectively is to make sure that the media is not either downplaying it or over-proportioned.

178
00:19:32,400 --> 00:19:34,080
So, we have to have a balance, right?

179
00:19:34,680 --> 00:19:37,140
And we all say the media, the media, the media, right?

180
00:19:37,140 --> 00:19:42,880
But it's our responsibility, right, on how we create our security advisories.

181
00:19:43,140 --> 00:19:44,600
And it's both ways.

182
00:19:44,600 --> 00:19:49,580
Even though we're vendors here and everything, but we have to point fingers to us.

183
00:19:49,580 --> 00:19:51,300
How clear is our advisories?

184
00:19:51,300 --> 00:19:58,360
Can we have some collateral that the end consumer actually knows about, you know, what the implications are?

185
00:19:58,360 --> 00:20:04,260
And we work with the researcher to make sure that we all understand, you know, what the problem is.

186
00:20:04,260 --> 00:20:15,460
And in a previous conversation, I also mentioned that the biggest nerd fights in history is whenever it comes to CVSS scoring, right?

187
00:20:15,560 --> 00:20:18,020
And, you know, whenever it comes to risk.

188
00:20:18,020 --> 00:20:32,840
And at the end of the day, you know, whether it's CVSS, whenever we come up with some new ways and everything, we have to have some type of way of saying, yeah, you have to jump right now and fix this vulnerability versus the other 100 criticals that we're also,

189
00:20:32,840 --> 00:20:34,500
you know, dependent on.

190
00:20:34,500 --> 00:20:42,460
And especially nowadays, since it's not so much of a vulnerability coming from like a Cisco or an Intel or Red Hat or anything else, it's open source, right?

191
00:20:42,460 --> 00:20:51,400
By the time that actually I'm boring you to death in here, probably three more CVEs that are super important has been disclosed and we don't know about, right?

192
00:20:51,400 --> 00:21:01,520
So that's the type of balance and the type of, I guess, for a corny way of saying orchestration that needs to take place, right, in this CBD.

193
00:21:03,640 --> 00:21:09,320
I want to add another point here regarding the overhyping.

194
00:21:09,560 --> 00:21:20,880
Having a logo, having a website for something, for a vulnerability, for a research result does not necessarily mean that you overhype it.

195
00:21:20,880 --> 00:21:37,640
We had experiences where we just put papers on archive without any website, without any logo and anything, and media picked it up and media reported about it and not necessarily correct in all cases.

196
00:21:37,640 --> 00:21:50,620
And we had this, on AMD CPUs, and their media picked it up.

197
00:21:50,620 --> 00:22:06,120
We intentionally didn't make any website or logo, but media picked it up and the reporting, it was not entirely wrong, but it was definitely, I would say, definitely overhyped.

198
00:22:07,280 --> 00:22:28,300
And we discussed this also in our team and the conclusion that we had was that in some cases we should have a website, even if it's not a vulnerability that we want to hype, because it's not that significant, but just to have a very clear message which says how relevant is it.

199
00:22:28,540 --> 00:22:42,760
And I think that's the important part that we also always had on our websites, a short three-sentence summary like what is it, who has to care about this and what can an attacker do.

200
00:22:42,760 --> 00:22:44,140
Yeah, I think that's great.

201
00:22:44,140 --> 00:22:45,280
That's great you do that.

202
00:22:45,280 --> 00:22:46,520
And it's good advice.

203
00:22:46,520 --> 00:22:54,680
I think that I would love to see it be more adopted in the researcher world, for sure.

204
00:22:54,680 --> 00:22:56,240
I think it would help out.

205
00:22:57,380 --> 00:23:11,020
So before I move on to my next question, I'm just going to say when I retire, all of you in vendor world are doomed, because I'm going to offer my services to LREG for free writing up crazy headlines.

206
00:23:11,420 --> 00:23:17,680
But to touch on a point that Daniel made, he invoked the Katie Masuris clause.

207
00:23:17,920 --> 00:23:21,180
Let's talk about bug bounties and CVD.

208
00:23:22,400 --> 00:23:24,700
Oh, Katie, maybe you should take that one.

209
00:23:25,560 --> 00:23:31,680
Because our friend Katie Noble Trimble here actually has a lot to do with that in her organization.

210
00:23:31,680 --> 00:23:38,660
Could you maybe talk about how coordinated disclosure interacts both good and bad with bug bounty?

211
00:23:39,620 --> 00:23:42,460
Yeah, so I love Katie Mo.

212
00:23:42,600 --> 00:23:48,480
I think that first off, let me just say that it is an honor to be confused with Katie Mo.

213
00:23:49,880 --> 00:23:53,300
My hair color is like, you know, getting close to hers too now.

214
00:23:53,300 --> 00:23:54,260
I don't know.

215
00:23:54,340 --> 00:23:58,600
I think I have been accepted to conferences because they thought that I was she.

216
00:23:58,760 --> 00:24:03,360
When I showed up, I don't think they were nearly as excited to see me instead of her.

217
00:24:03,380 --> 00:24:06,520
So, yeah, bug bounties.

218
00:24:06,840 --> 00:24:10,660
So, bug bounties are a tool.

219
00:24:10,660 --> 00:24:13,340
I'm trying to be serious here about bug bounties.

220
00:24:13,980 --> 00:24:15,680
So, bug bounties are a tool, right?

221
00:24:15,680 --> 00:24:17,680
So, they're a tool in a toolkit.

222
00:24:17,680 --> 00:24:20,760
And they are there to incentivize.

223
00:24:20,760 --> 00:24:28,000
But they're a part of a well-structured product security portfolio.

224
00:24:28,140 --> 00:24:30,040
They're not the whole thing.

225
00:24:30,040 --> 00:24:35,160
And there are different motivations that will help people from different perspectives.

226
00:24:35,580 --> 00:24:46,700
So, in the academic world, for instance, bug bounties may not weigh as much because a lot of academic institutions don't allow an individual to accept that cold hard cash, right?

227
00:24:46,860 --> 00:24:51,000
If you are a professional bug hunter, that might be how you pay your mortgage.

228
00:24:51,000 --> 00:24:53,520
And so, there's a lot more tied to that.

229
00:24:54,100 --> 00:24:58,000
But the problem becomes that bug bounties are a wonderful tool.

230
00:24:58,060 --> 00:24:59,580
And they're great.

231
00:24:59,580 --> 00:25:12,440
But you have to have a good program in place already to accept that information, to be able to execute on that information, and figure out, like, how do you actually... there are so many questions about it.

232
00:25:12,440 --> 00:25:13,620
How do you award?

233
00:25:13,720 --> 00:25:15,100
How do you manage it?

234
00:25:15,100 --> 00:25:18,860
Like, are we gonna tie the payments to CVSS scores?

235
00:25:18,860 --> 00:25:22,280
Are we gonna tie the payments to a well thought out proof of concept?

236
00:25:22,280 --> 00:25:24,260
There's so many pieces that go into it.

237
00:25:24,260 --> 00:25:27,760
And so, I would say, like, bug bounty is not a one size fits all.

238
00:25:27,760 --> 00:25:31,280
It's gonna vary from organization to organization.

239
00:25:31,420 --> 00:25:35,420
And some organizations are gonna have different timelines, different pieces.

240
00:25:36,380 --> 00:25:38,220
It's... yeah, it's complex.

241
00:25:38,420 --> 00:25:40,460
So, bug bounty is not the end of the world.

242
00:25:40,460 --> 00:25:41,760
Bug bounty is a tool.

243
00:25:41,820 --> 00:25:43,440
It's a tool in your toolkit.

244
00:25:43,600 --> 00:25:44,940
So, it's a great tool.

245
00:25:44,940 --> 00:25:48,000
I love the tool because I'm the director of it at Intel.

246
00:25:48,000 --> 00:25:52,160
But it's not... it's not everything.

247
00:25:52,320 --> 00:25:56,220
So, vulnerability disclosure is more than bug bounty alone.

248
00:25:56,580 --> 00:26:05,760
Yeah, I think when you think about it, too, like, there's a lot of smaller companies that probably, you know, don't have that or don't have the funding for it.

249
00:26:05,860 --> 00:26:09,580
But that doesn't mean that they're not wanting to work with researchers.

250
00:26:09,580 --> 00:26:12,560
They just can't get the budget to do it.

251
00:26:12,920 --> 00:26:18,820
You know, so... so, you know, do your best to try to figure out how to reach out to those companies.

252
00:26:18,980 --> 00:26:25,280
Hopefully, they have a web page or email address, you know, secure at p-cert at security at.

253
00:26:25,300 --> 00:26:27,040
I know we struggle a little bit.

254
00:26:27,040 --> 00:26:34,720
But, you know, there's plenty of us around that can help find the right contacts.

255
00:26:35,600 --> 00:26:40,020
Because we're, you know, sort of come together from all over the place.

256
00:26:40,020 --> 00:26:43,440
And we're sort of an expanding group here.

257
00:26:43,440 --> 00:26:44,640
I like it.

258
00:26:45,040 --> 00:26:50,000
Bug bounties are one of the wonderful signs of how the industry has changed.

259
00:26:50,000 --> 00:26:54,100
Back in the 90s, researchers were often rewarded with...

260
00:26:55,380 --> 00:26:58,120
their disclosure was rewarded with lawsuits.

261
00:26:58,420 --> 00:27:06,860
And now people in the industry are working with the people and have started actually paying the researchers for their efforts.

262
00:27:07,540 --> 00:27:17,460
So I'm very much thumbs up on bug bounties, especially because it shows how the industry has changed on how they view hackers.

263
00:27:17,680 --> 00:27:20,840
Hackers are nowadays helpers and not the enemy.

264
00:27:21,080 --> 00:27:25,520
Yeah, I thought you were going to say t-shirts for a minute there in the early days.

265
00:27:25,520 --> 00:27:26,400
I like it.

266
00:27:26,840 --> 00:27:28,440
It just won the t-shirt.

267
00:27:31,400 --> 00:27:35,500
Oh, earlier than that, it was lawsuits.

268
00:27:35,500 --> 00:27:39,260
Then it became t-shirts and then maybe stickers.

269
00:27:39,280 --> 00:27:44,880
And then, yeah, and now financial rewards.

270
00:27:44,880 --> 00:27:45,620
Awesome stickers.

271
00:27:48,160 --> 00:27:50,960
So let's... we've touched on it a little bit.

272
00:27:50,960 --> 00:27:54,700
Let's see if we can spend some time now that we're past the top of the hour here.

273
00:27:54,700 --> 00:28:00,660
Let's talk about coordinated vulnerability disclosure inside an open source context.

274
00:28:00,940 --> 00:28:22,400
So I'll go last, but let our esteemed panel here, whether it's our researchers or our vendor friends, kind of describe what works out really well and what are some challenges for you within the open source world, which makes up about 90% of all software now.

275
00:28:23,700 --> 00:28:25,380
Hey, Jerry, open source won.

276
00:28:37,400 --> 00:28:39,280
I guess I start.

277
00:28:39,900 --> 00:28:45,060
So basically, that's top of mind for me.

278
00:28:45,060 --> 00:28:49,480
And it sounds corny whenever people say, hey, what keeps you up at night and everything else?

279
00:28:49,480 --> 00:28:51,000
It's actually open source right now.

280
00:28:51,200 --> 00:28:56,080
And it's not so much of the predicament of using open source.

281
00:28:56,080 --> 00:28:59,380
We have to use and embrace and contribute to open source.

282
00:28:59,380 --> 00:29:01,360
I mean, I'm a super big fan of that.

283
00:29:02,220 --> 00:29:06,860
The challenge when it comes to open source is that it can be anything, right?

284
00:29:06,860 --> 00:29:08,580
It's like IoT can be anything, right?

285
00:29:09,260 --> 00:29:12,740
And it's critical infrastructure for a lot of things.

286
00:29:12,740 --> 00:29:28,180
So to give you somewhat of a real life, I guess, realization that we had a while back, probably about four or five years ago, whenever Heartbleed came, we were looking in the industry, right?

287
00:29:28,180 --> 00:29:30,620
Not only Cisco, but a whole bunch of other companies, right?

288
00:29:30,620 --> 00:29:38,880
So what to prioritize, as far as actually giving funding, probably doing research, and so on.

289
00:29:38,880 --> 00:29:45,960
So we said, okay, so open SSL, and things like that, which are actually super important for us.

290
00:29:46,760 --> 00:29:48,700
There's a lot of people looking at it right now.

291
00:29:48,700 --> 00:29:55,640
So can we look at things that perhaps are actually critical infrastructure, that nobody actually has probably taken a look at it, right?

292
00:29:55,840 --> 00:29:57,620
We're going down the list.

293
00:29:57,760 --> 00:30:01,700
And as a matter of fact, that number two is perfectly fine.

294
00:30:01,700 --> 00:30:05,280
Because there are two guys that work on open SSL that get paid to do it.

295
00:30:05,280 --> 00:30:06,380
Yes, indeed.

296
00:30:06,560 --> 00:30:13,920
And the example that I was going to go is NTP, the Network Time Protocol, so NTPD specifically.

297
00:30:14,120 --> 00:30:18,260
And there's also two guys that don't get paid, which is bad, right?

298
00:30:18,360 --> 00:30:21,400
They're not even the full-time job, right?

299
00:30:21,400 --> 00:30:25,020
Well, I guess now, you fast-forward this year, it's a little bit better now, right?

300
00:30:25,020 --> 00:30:26,120
So it's a lot better.

301
00:30:26,500 --> 00:30:32,660
And it's not that the problem is the poor guys that actually are contributing to the code.

302
00:30:32,660 --> 00:30:34,000
It's a matter of scalability.

303
00:30:34,000 --> 00:30:41,800
It's a matter of actually even running static analysis into the thing that didn't even exist five years ago for these components.

304
00:30:41,920 --> 00:30:44,000
Now, of course, we're modernizing our ways.

305
00:30:44,000 --> 00:30:47,340
Even if you submit things on GitHub, a lot of things are actually happening.

306
00:30:47,500 --> 00:30:50,120
We're getting better, for sure.

307
00:30:50,420 --> 00:30:53,120
It's just that it's getting way more complicated, right?

308
00:30:53,120 --> 00:31:01,640
And more people actually are not only, of course, contributing, but using it more than contributing, which is the other predicament.

309
00:31:01,900 --> 00:31:05,120
And in some cases, we were talking about bug bounties.

310
00:31:07,040 --> 00:31:09,760
In some cases, actually, amazing.

311
00:31:09,760 --> 00:31:11,700
I'm a big fan of bug bounties, right?

312
00:31:11,700 --> 00:31:18,460
The challenge is that in some cases, we also don't think about, does this affect other vendors?

313
00:31:18,900 --> 00:31:30,300
Am I actually finding some type of vulnerabilities that perhaps, yes, it's a SQL injection, cross-site scripting, which is actually pretty common, but I'm doing some fuzzing and I crash an application.

314
00:31:30,480 --> 00:31:32,080
What is the underlying issue, right?

315
00:31:32,080 --> 00:31:37,400
And in some cases, it's actually kind of a commodity, even though it goes back and forth and it hasn't been shared.

316
00:31:37,460 --> 00:31:46,420
And then two or three months down the road, you say, oh, but this CVE was not shared with these vendors that they're also affected.

317
00:31:46,480 --> 00:31:55,880
So we go back and in some cases, you actually see people trying to find the same or found the same CVEs and they reported a different way, but there were no coordination.

318
00:31:55,880 --> 00:32:00,340
So those are the things that are crucial for us to be successful.

319
00:32:00,340 --> 00:32:05,040
And not only among vendors, but actually downstreams and upstreams as well, right?

320
00:32:05,040 --> 00:32:08,820
So especially whenever it comes to IoT is number one.

321
00:32:08,980 --> 00:32:09,300
Yeah.

322
00:32:09,300 --> 00:32:11,860
So Omar, you brought up a few points.

323
00:32:11,860 --> 00:32:14,320
One is who do you bring in, right?

324
00:32:14,320 --> 00:32:15,700
Ahead of time.

325
00:32:15,780 --> 00:32:26,840
And I think that we're seeing that even if you're the competition, we still want to work together in the background here.

326
00:32:27,280 --> 00:32:34,580
Our whole goal is to protect our customers and we often have the same customers, even if we're competitors.

327
00:32:34,580 --> 00:32:43,060
So we like to make sure that we're all working together and ideas is when do you bring someone in and it's not only the researchers.

328
00:32:43,540 --> 00:32:45,440
Okay, Crowe, you got your hand raised.

329
00:32:45,440 --> 00:32:46,460
Go ahead.

330
00:32:46,500 --> 00:32:48,000
Can I tell you a secret?

331
00:32:48,180 --> 00:32:49,220
Yes.

332
00:32:49,720 --> 00:32:55,160
Open source has worked with our competitors for 25 plus years.

333
00:32:56,200 --> 00:32:57,140
Yeah, yeah.

334
00:32:57,140 --> 00:33:06,400
Well, I would have to say with some of those other vendors in the industry, it wasn't always so nice.

335
00:33:08,420 --> 00:33:10,260
We're getting better.

336
00:33:11,200 --> 00:33:23,280
But I think not only the researchers should be thinking who else is affected, but when we are the receiving end, we should be thinking who else is affected and bring them in.

337
00:33:23,280 --> 00:33:35,520
I recently worked on an issue, a very recent one, where we are in and the researchers were in right with the industry people who were fixing the issue.

338
00:33:35,520 --> 00:33:41,580
And there was great coordination amongst the group there.

339
00:33:41,580 --> 00:33:45,400
It was pretty awesome to see of how much we've evolved.

340
00:33:46,240 --> 00:33:51,780
How do you do that, though, with like open source, though, when everything's shown?

341
00:33:51,780 --> 00:33:53,240
It's super easy.

342
00:33:54,680 --> 00:33:59,380
Well, and I guess I'll chime in before we let other folks.

343
00:33:59,380 --> 00:34:11,440
I found it very interesting how some vulnerabilities have been patched wide in the open under the cover of some other patch.

344
00:34:11,520 --> 00:34:15,020
I don't know what you're talking about at all.

345
00:34:17,980 --> 00:34:27,640
So, in general, open source, there is no single definition of what open source is.

346
00:34:27,640 --> 00:34:36,700
You could have two young ladies in Bangladesh that have an amazing idea and they're just trying to get this creativity out there to share with the world.

347
00:34:36,740 --> 00:34:40,340
You could have large corporations like a lot of the folks represented here.

348
00:34:40,340 --> 00:34:41,740
You could have academics.

349
00:34:42,120 --> 00:34:44,000
So, open source is a lot of different things.

350
00:34:44,000 --> 00:34:49,540
And there's really no one definition or moniker that fits, that works with every community.

351
00:34:49,540 --> 00:35:07,660
But if you're thinking about the types of open source that might make up a product or some kind of cloud offering, you're going to have the high end of the spectrum are things like the kernel group, which is very mature and organized, Kubernetes.

352
00:35:07,660 --> 00:35:14,080
And then you start moving down to Apache Foundation, these other large communities.

353
00:35:14,080 --> 00:35:19,300
And then you get down to something where it's a single person, a couple people that are just playing around.

354
00:35:19,300 --> 00:35:34,600
And it's hard to put any kind of structure or process to all these different models because a lot of people that code for open source are doing it for free, for no remuneration from large companies that make a lot of money off of it.

355
00:35:34,920 --> 00:35:40,740
But they do it for free because they love kind of adding value and expressing their creativity.

356
00:35:41,160 --> 00:35:45,580
And as Daniel mentioned, open source is very creative.

357
00:35:45,580 --> 00:35:50,940
And some of the larger projects, we do have ways to privately take in data.

358
00:35:50,940 --> 00:35:56,980
So we take in a private bug report that's well established within the community for a long time.

359
00:35:57,140 --> 00:36:05,120
And when you're looking at a less mature community or package or library, they might not have that capability.

360
00:36:05,120 --> 00:36:18,680
And that's where larger kind of big brothers and big sisters, like a Red Hat or a SUSE or Canonical, kind of step in and try to help mentor these smaller projects to help them get these good practices set up so they can take in a private note.

361
00:36:18,680 --> 00:36:30,020
Because sometimes a lot of, you know, my team, we track between 3,000 and 5,000 vulnerabilities a year out of 450,000 packages.

362
00:36:30,020 --> 00:36:30,920
That's a lot.

363
00:36:30,920 --> 00:36:44,580
Not all of them really kind of bubble up to the level of they need the attention of like a Spectre meltdown or a Heartbleed or a, you know, Blueborn, all these kind of the other big name nonsense things.

364
00:36:44,620 --> 00:36:46,480
But, you know, they still need to get fixed.

365
00:36:46,480 --> 00:36:56,280
And what open source is really good at is you identify a problem, the team attacks it collaboratively, they develop a solution and release that update very quickly.

366
00:36:56,280 --> 00:37:05,120
And they don't like to spend a lot of time making a big deal out of it because they've already moved on to the next feature, the next big thing.

367
00:37:06,420 --> 00:37:09,040
But, yeah, it can be challenging.

368
00:37:09,040 --> 00:37:10,680
But there are definitely ways to do it.

369
00:37:10,680 --> 00:37:11,720
There are methods to do it.

370
00:37:11,720 --> 00:37:14,180
There are groups that allow this.

371
00:37:14,180 --> 00:37:15,960
I hear a lot of bleeping in the background.

372
00:37:15,960 --> 00:37:17,440
What's going on, folks?

373
00:37:17,440 --> 00:37:19,780
I was wondering when you were going to change your hat again.

374
00:37:19,780 --> 00:37:20,600
Sorry.

375
00:37:23,680 --> 00:37:25,600
There we go.

376
00:37:28,420 --> 00:37:29,500
All right.

377
00:37:29,500 --> 00:37:31,500
Let me get this panel back under order.

378
00:37:33,420 --> 00:37:44,110
When you're thinking about doing a CVD, what are some things you might want to prepare for?

379
00:37:44,110 --> 00:37:47,890
What can you do to help make that coordination very successful?

380
00:37:48,150 --> 00:37:54,450
And let me start off with... we'll start off with Lisa because she has a lot of ideas.

381
00:37:54,490 --> 00:38:00,570
So, from your perspective, what will make you successful when you're getting ready to do this multiparty coordination?

382
00:38:01,390 --> 00:38:06,370
So, I guess it depends on how familiar you are with it or not.

383
00:38:06,370 --> 00:38:22,670
Like, if I was a researcher and I wasn't too familiar, I would potentially utilize CERT or something like that if I felt like it was going to cross over more than one company in the industry just to have them help.

384
00:38:23,190 --> 00:38:37,950
Because I think it could be overbearing of figuring out how if one, you know, if industry, you know, company A wants this date to get it fixed and company B wants another date and company C wants another date.

385
00:38:37,950 --> 00:38:43,330
So, I think that's one way that a researcher could use.

386
00:38:43,330 --> 00:38:49,710
But I think the idea is to figure out when you start is what are your rules?

387
00:38:49,710 --> 00:38:51,190
What's your embargo date?

388
00:38:51,190 --> 00:38:58,470
How many days do you actually want to, you know, give the company or vendor to fix the issue?

389
00:38:58,530 --> 00:39:00,510
Typically, it's 90 days.

390
00:39:00,510 --> 00:39:03,810
I would prefer that as a company.

391
00:39:03,810 --> 00:39:08,230
I think it's respectful to do that, especially when things get more difficult.

392
00:39:08,230 --> 00:39:13,510
So, think about your rules and what you want to do and how you want to approach it.

393
00:39:13,510 --> 00:39:17,970
But I'll pause there because I know we're running out of time to get other people in.

394
00:39:18,530 --> 00:39:22,270
Anders, what are your thoughts about how you could make CVD successful?

395
00:39:22,970 --> 00:39:24,130
Three things.

396
00:39:24,610 --> 00:39:25,330
Use it.

397
00:39:25,330 --> 00:39:31,350
It's a tool not only for vendors but also for hackers.

398
00:39:34,010 --> 00:39:50,530
Listen to what vendors are saying or what hackers are saying and try to make the best out of it and be aware that there's often a lot of complexities involved with it.

399
00:39:50,530 --> 00:39:50,830
Right.

400
00:39:50,830 --> 00:39:52,550
Be patient, right?

401
00:39:52,550 --> 00:39:53,230
Yeah.

402
00:39:55,030 --> 00:39:59,030
Omar, what are your thoughts on how you can make CVD successful?

403
00:39:59,330 --> 00:40:05,830
I think that I'm going to capitalize on something that Daniel mentioned in a previous call.

404
00:40:06,410 --> 00:40:17,290
And it's in some cases, even if you provide some data to even a vendor or whoever, whenever you're coordinating, you have to make it first easy to understand.

405
00:40:17,290 --> 00:40:28,630
But at the same time, in some cases, getting the right people at the right place at the right time to make sure that you understand the technical implications of a given vulnerability instead of running around.

406
00:40:28,630 --> 00:40:41,310
Even if you have the 90 days or 60 days or 100 days, getting that streamlined and modernizing the way that we actually exchange information among all the affected parties, that is number one.

407
00:40:42,510 --> 00:40:46,870
And Art Mannion, which is actually a good friend of mine, he leads... We all love Art.

408
00:40:47,850 --> 00:40:51,690
He's the person that will tell you we cannot scale, right?

409
00:40:51,690 --> 00:40:54,350
This is a big ecosystem.

410
00:40:54,350 --> 00:41:00,910
And just having a Rolodex of all the vendors that actually use a component is foolish, right?

411
00:41:00,910 --> 00:41:04,430
We will never, ever, ever be able to actually have a complete thing.

412
00:41:04,530 --> 00:41:11,510
But what we have to think about is what Lisa mentioned is that in some cases, we are working with competitors, right?

413
00:41:11,930 --> 00:41:13,650
Even more than our companies.

414
00:41:13,650 --> 00:41:16,070
For example, you know, I work with Lisa a lot.

415
00:41:16,070 --> 00:41:26,130
I work with Juniper even more, in some cases more than I worked with Cisco, whenever it comes to fixing a vulnerability, saying BGP, OSPF, or whatever the case might be.

416
00:41:26,130 --> 00:41:35,310
So that type of thing of, oh, I'm going to talk to this competitor, is there a problem and everything else is not, you know, that's 20 years ago, a fallacy.

417
00:41:35,590 --> 00:41:51,390
And then the last one is that at the end of the day, I have the reality and I tell my guys at Cisco is that whenever I push a button, I publish a CV out there, the number ones that actually are reading that CV are the bad guys, right?

418
00:41:51,610 --> 00:41:58,890
And probably at the same time that you're committing an open source code, you know, probably that's going to be the case too, right?

419
00:41:58,950 --> 00:42:10,830
So what we have to do is think about how can we exchange information to the consumer, to downstream and upstream providers in a more modern way.

420
00:42:10,830 --> 00:42:17,890
I know that sounds corny, but you know, can we actually make it more and we're trying to do that machine readable, right?

421
00:42:17,890 --> 00:42:30,930
So if in the case that we actually have that Rolodex of people and everything else, you know, get some tool and that's what Art and, you know, the guys in CERT are also doing, you know, we're creating tools that allow us to do that.

422
00:42:31,070 --> 00:42:36,150
And even if you have, I don't know, even two years ago that didn't exist and it wasn't even a thought, right?

423
00:42:36,230 --> 00:42:37,590
So we have to move faster.

424
00:42:37,590 --> 00:42:39,670
That's the number one thing that I want to say.

425
00:42:40,790 --> 00:42:47,330
So as I close, I'll say Art Mannion has the best facial hair in the industry.

426
00:42:50,710 --> 00:42:52,970
But I want to thank our panel here.

427
00:42:52,970 --> 00:42:54,930
This was a great hour together.

428
00:42:54,930 --> 00:42:57,170
I really appreciate your time and expertise.

429
00:42:57,550 --> 00:43:04,290
We're going to be hanging out for a little bit on this awesome Discord channel that my kids made fun of me all week about because I've had it up.

430
00:43:04,910 --> 00:43:12,910
Thank you everybody for coming and your attention here, panelists, and thank you to the audience for having us here at the IoT Village.

431
00:43:13,070 --> 00:43:16,010
Enjoy your day and enjoy the rest of the con.

432
00:43:16,010 --> 00:43:18,330
We've got some great stuff lined up the rest of the weekend.

433
00:43:25,450 --> 00:43:26,490
We're out.