start	end	text
540	10100	From your perspective, what does CVD mean to you? And kind of what are your motivations
10100	13200	when you're working on an issue and you need to disclose something?
14460	20500	So I think the main point of CVD is, of course, to protect users.
23580	31540	And my main thought is a bit historical. So I've been around in the 90s where disclosure was
31540	37760	you find a mailing list where other hackers hang out and you zero-day everybody. And nobody
37760	44200	really cared. And that became responsible disclosure, which is the predecessor for
44200	51840	coordinated vulnerability disclosure. The problem with responsible disclosure is that it was really
51840	59080	made to shame hackers that didn't follow the desired protocol of industry text.
59080	66560	And the current iteration, coordinated vulnerability disclosure, is instead of
66560	73800	trying to be coercive, trying to actually work with people that want to submit issues.
74600	82480	And it is a much, much improved over that. And it does a much better job of making sure that
83400	90040	every party to a vulnerability disclosure, that means vendors, customers, and hackers,
90560	101160	have their needs met. So that's what I think coordinated vulnerability disclosure is for me.
101680	107420	Awesome. Daniel, another researching perspective. What do you think about coordinated
107420	114340	vulnerability disclosure? So we are interested in understanding how things work. And sometimes
114340	120980	that involves understanding how things work that are not supposed to happen, which we then call
120980	127760	vulnerability. And of course, then we want to figure out the truth behind it. And that involves
127760	138100	talking to the vendor. Very often, they tell us that we got something slightly wrong. And that
138100	144580	is an opportunity to correct these mistakes before we submit the paper or publish the paper.
144580	152760	But also, from my perspective, it's the only ethical thing to do to protect the customers,
152760	160320	to protect people who are using these products. And for me, I'm not in this game for
160320	169860	finding vulnerabilities as a motivation by itself. My main motivation is finding the truth,
169860	178920	finding a better understanding of how things work. That's awesome. Katie, you've had a lot
178920	186480	of experience monkeying around with CVD. What does a CVD mean to you? Hold on a minute.
186680	198740	All right. So CVD means to me... So when I think about it, I think of it from a risk-based
198740	205380	perspective. And so what Probe is alluding to here is that my background is government in nature. So
205380	211580	I spent about 15 years in the U.S. government, 12 years of that in the U.S. Air Force. And then
212260	217820	several other years at the Department of Homeland Security, where I ran the vulnerability disclosure
217820	222600	profiles programs that most people are familiar with. So like the NIST NVD program and the
222600	231100	Carnegie Mellon CERT CC program. And the MITRE CVD program. So I was a sponsor for those programs. So
231100	237440	in a single year, 2017, we coordinated and disclosed 14,800 vulnerabilities for public
237440	245220	disclosure. So that's 14,000 IT vulnerabilities and about 800 ICS vulnerabilities. In the following
245220	250360	two years, so 18 and 19, we coordinated and disclosed over 20,000 cybersecurity vulnerabilities.
250620	256900	So I've kind of seen things from lots of different perspectives. And when I think about CVD,
256900	264760	I think about balancing risk, right? So CVD is really a process. And every time you,
265180	268860	when you're going through this process, understand like every organization is going to treat it
268860	273100	differently, because there isn't a standardized, like one size fits all kind of thing here.
274060	278740	There's differences that are going to happen across the coordination stack. So if you're
278740	281440	looking at digital services, it's going to be different from software, which is going to be
281440	285100	different from open source, which is going to be different from hardware or ICS, like all the
285100	290420	differences are going to come into play there. But the overarching sort of thing, the big takeaway
290420	299660	here is that CVD is balancing risk. It's all about making sure that there is an opportunity for the
300220	305860	product vendor to fix the problem before an adversary has the opportunity to take advantage
305860	310720	of that. So like, it's all about protecting the end user. And I think everybody that I've ever met
310720	316480	in the entire ecosystem is all focused on that. Like we may be speaking different languages,
316480	320960	we may talk past each other, but I think that everyone is trying to protect the end user.
320960	326880	So to me, that's what that's what CVD is about. That's awesome. And remind you, remind me to
326880	331580	circle back to you later in our talk, talk a little about some of the psychology behind some of this.
331580	340000	I would love to hear my dearest friend Lisa's thoughts on kind of coordinating
340000	344820	vulnerability disclosure, and kind of what some of your motivations that you've seen,
344820	352860	or kind of what's behind your modus operandi? Yeah, so I think where I've seen a sort of an
352860	361220	evolution of what we do now, where I think Heartbleed was sort of the big start of where
361220	366940	we paid attention, realized we all need to come together a little bit. And then Spectre Meltdown
366940	373620	sort of brought us even closer. So my thought is, is that not only do we work with the researchers,
373620	381020	but we work across the industry to, you know, make sure that we're all doing our best to protect our
381020	386860	customers. So like Katie was talking about, it's really our end users that are our focus,
386860	391980	and making sure that we have the right people involved to be able to solve the issue,
391980	396060	and then provide a security update so that our customers could get it and be best protected.
405180	412000	So Omar, you've been doing this for a while, and you work for a very large company,
412000	416200	so I'm sure you get the opportunity to see a lot of different vulnerabilities. So what does
416200	421060	the whole process mean, and what are some of the motivations you've seen? Yeah, I think that
421060	426720	pretty much everybody has summarized it very well, is protecting the end consumer at the end of the day.
426940	432800	But I see it as a very, very complex ecosystem, right, that we're trying to actually solve.
433220	439860	And Katie, you know, mentioned in one side hardware, another side software, then if you
439860	444540	decompose that, then you have open source, which we're going to probably talk about a lot in here.
444820	451260	And you have things that, you know, perhaps you cannot even control. I have had, I guess,
451260	455860	the pleasure or not the pleasure, in some cases, where we're looking at vulnerabilities and even
456940	461560	in some cases, whenever we try to actually solve the issues, the companies don't even exist anymore.
462380	466920	That's another predicament, you know, that in some cases, we actually have to take into
466920	473300	consideration. And it's a fairly complex ecosystem. And at the end of the day, what we have to
473300	479840	actually put our heads together is how can we modernize our practice into the way that not only,
479840	484860	yes, we deal with a single vulnerability, we reproduce it, we actually fix it, we find the
484860	491020	patch, but how can we accelerate that process, that one, everybody's talking somewhat the
491580	497500	correct vocabulary, or at least they can understand each other. And then we also understand the
497500	504140	overall risk, right? And how to prioritize things and so on. So it's fairly complex, so we can
504140	508760	actually talk about it for quite some time. And that's what we're here for. But those are some of
508760	515580	the initial perspectives that I want to share. Thank you, I appreciate that. I want to talk
515580	522420	about something we touched on in one of our prep calls. We have Anders and Daniel that come at
522420	529700	this research angle from slightly different angles. So maybe you two gentlemen can figure
529700	537120	out who wants to start first. But thinking about the perspective of academia versus a professional
537680	544760	bug hunter, security researcher, maybe you can describe some of the particulars in
545360	548900	either area. And maybe what are some of the differences you might see between those two
548900	561160	different types of research? So from the industry, or rather the bug hunter, be professional or not,
562260	569940	there is very often an element of good old-fashioned fun. I started out hacking
570660	576480	things not because I wanted to achieve anything with it, just because it was great fun.
577800	586080	When you become pro with it, there is different motivations. So if you work for a company and you
586080	593400	work on their products, obviously your motivation is to make those products secure. And sometimes
593400	601340	my past job, I did some research and that was sponsored by that company, but essentially me
601340	610900	doing what I like to do. Their end of it was attention to that company's competences. And
610900	618300	my end, of course, was having a bit of fun with it. So that is probably pretty typical of what
618300	629080	hackers get out of hacking. Yeah, from the more academic side, it's more of advancing the field,
629080	633860	the knowledge in the field, and there you try to figure out how things work and what the
633860	640220	implications there are, what the security implications of certain understandings are.
640220	647720	For instance, understanding when you can execute a certain piece of code and that does something
647720	653540	that is not intended, what are the implications? And this is interesting from a security perspective
653540	658820	than if it enables someone to do something that they shouldn't be allowed to do.
659700	666920	Yes, often this involves disclosing a vulnerability to a vendor. But I would say that,
667520	675060	for instance, bug bounties that might be very relevant for people from the industry.
675620	686920	I think this plays a smaller role in academia, because you have to participate in CVD anyway,
686920	693740	if you wouldn't, then you would get a lot of problems in the academic community. There's
693860	699040	a lot of peer pressure that you participate in this, because that's the only ethical approach
699040	707600	to handling vulnerabilities. At the same time, if you keep them secret for too long,
707600	712080	then there's also the question on why did you keep them secret for so long?
712610	720040	Maybe also some perspective that I can share. In academia, you see more and more often the
720040	726690	effect that academic publications are easier to publish if you have a CVE.
727360	733260	I don't think that's a good thing, because I think that a CVE does not necessarily
734500	745320	describe that a research result really brings you new insights. A CVE is an identifier for
745900	754060	a vulnerability, not an identifier that says this is something with a new insight.
754450	760140	And also, if you participated in CVD, and you mentioned this in the paper,
760140	768460	this also gives you bonus points, at least that's my impression, to get the paper published.
770180	778020	Of course, it's good if you participate in CVD, but my feeling is also that this is going a bit
778020	785640	too strong. And yes, now overemphasized in our community a bit.
785640	789160	What about, you know, having an icon or a fun name?
791580	794680	So, there are, of course, there are.
796740	802980	I'm really a big fan of logos and names and anything that is fun.
803620	809160	But of course, there are multiple layers here. For instance, we had this paper,
809160	814040	Hello from the Other Side, which had this name not just by coincidence.
814740	820630	And it was about a covert channel in the cloud where we send data through the cache from one
821040	828340	virtual machine to the other. And we really went crazy on that. So, we built an entire TCP
828340	833640	stack on top of that and tunneled an SSH session through the cache covert channel
834540	841600	for whatever reason. And we then sent a music video through this cache covert channel.
841600	846340	And of course, we couldn't just take any music video. So, we had to make our own parody of
846340	851340	Hello from the Other Side. And this is just fun. I mean, we are just a bunch of people and we like
851340	859880	to have fun. And basically, once you finish your project, it's really nice to close this up with
859880	867920	something nice and funny. And logos on the other side also help about...
868700	876700	So, they help you communicating about the issue. I realized that every time I create slides,
876700	883400	my favorite slides are the slides that don't have any words on them, just pictures, logos,
883400	890400	and icons. Because then I can follow the speaker. I don't have to read because I'm very bad at
890400	895220	reading. I can't read and listen at the same time. And I bet many people can't do that. So,
895220	902500	there's only icons and symbols and logos. I can follow what the person is saying and look at the
902500	908640	images at the same time. And I'm always annoyed if I have to speak about the vulnerability and I
908640	912740	don't have a logo for it because then I have to put text on the slide.
913740	920720	So, since Daniel brought it up, let me ask the other panelists. How do you all feel about
920720	926700	branded vulnerabilities? Do you like logos? Do you find that helpful to you and your constituents?
926980	936860	Oh, I'm sorry, Daniel. No. When there's a logo or PR or everything, it grabs the media's attention
936860	946240	so quickly. And sometimes the issue is not that severe or very hard to be able to
946240	955360	breach. And it gives a little bit of extra fear to our customers and also encourages us to
956000	964680	maybe reprioritize other issues first, which are more severe to our customers. So, although I
964680	971080	think I get the point, it's easy to talk about it. They grab the media's attention really quickly.
972160	979880	But if you put a caveat about the real severity of the issue in there,
979880	987420	maybe that would help a little bit. Yeah. So, in our team, we often have this
987420	993520	discussion. Should we have a logo? Should we have a website or shouldn't we? And we have...
993520	999440	So, for most of our papers and most of the vulnerabilities that we discovered, we don't
999440	1005300	because we just say they are not significant enough. And I think that's also the responsibility
1005300	1011460	of the researcher to first assess, is this something that significant that I need this
1011460	1018600	additional PR or is it not? Right. Well, then you're doing it the right way. So, yay. Cheers.
1018880	1025920	Maybe. But it's also... I mean, I will have a different view on what is really important than
1025920	1030260	you have, right? Everyone will have a different view on that. So, something that I think is
1030260	1036320	really, really important, you might say, well, it's not really exploitable in our use case.
1036320	1039920	Right. We don't ever want to downplay a researcher's work. I mean, it's important,
1039920	1046360	whatever they do. And we appreciate especially doing CBD with them and not being zero-day.
1047860	1054480	But yeah, it's a more matter of, especially in a bigger industry when you're trying to
1054480	1057900	protect your customers best, how do you approach it?
1058220	1066060	There is a fight for customer attention. And for customers to make the right security decisions,
1066060	1073980	they need to be attentive about the right things. And logos and hype in the press
1073980	1081920	wants that. And we have bad security outcomes for our customers due to that.
1081920	1088040	Of course, there's also the thing that even fixed vulnerabilities sometimes cause suffering
1088040	1092620	for system administrators running around the basement and patching systems and all the like.
1094180	1103140	And to some extent, I sometimes find overhyped bugs disrespectful to those people.
1105980	1110800	Yeah. So, in my case, I don't have an issue with logos or anything,
1110800	1117840	even though in a previous call and a couple of exchanges, Daniel, I alluded into that.
1117980	1123420	Whether it's a logo, as a matter of fact, even for Cisco, we had the first vulnerability with
1123420	1130120	emojis. So, we have emojis, logos, and everything else. So, at the end of the day, if it helps
1130680	1137360	to do an alias for a vulnerability and bring some awareness, I'm perfectly okay with that.
1137360	1142860	The only challenge that I'm seeing or opportunity, right, is that in some cases,
1142860	1149040	whenever we write information, and even vendors, in my case, we have a research
1149040	1153140	institution at Cisco called Talos, and we find vulnerabilities in other people's products, right?
1153360	1159980	And yes, it is. So, in that case, yes, we also have logos, and we put them up there,
1159980	1166700	and we put up blog posts and so on, right? But what we need to do collectively is to make sure
1166700	1173500	that the media is not either downplaying it or over-proportioned. So, we have to have a balance,
1173500	1178700	right? And we all say the media, the media, the media, right? But it's our responsibility, right,
1178700	1186440	on how we create our security advisories. And it's both ways. Even though we're vendors here
1186440	1191640	and everything, but we have to point fingers to us. How clear is our advisories? Can we
1191640	1197340	have some collateral that the end consumer actually knows about, you know, what the
1197340	1202820	implications are? And we work with the researcher to make sure that we all understand, you know,
1202820	1207560	what the problem is. And in a previous conversation, I also mentioned that
1208120	1216620	the biggest nerd fights in history is whenever it comes to CVSS scoring, right? And, you know,
1216620	1220740	whenever it comes to risk. And at the end of the day, you know, whether it's CVSS, whenever we
1220740	1225300	come up with some new ways and everything, we have to have some type of way of saying,
1225300	1231440	yeah, you have to jump right now and fix this vulnerability versus the other 100 criticals
1231440	1236780	that we're also, you know, dependent on. And especially nowadays, since it's not so much
1236780	1241260	of a vulnerability coming from like a Cisco or an Intel or Red Hat or anything else, it's
1241260	1246600	open source, right? By the time that actually I'm boring you to death in here, probably three more
1246600	1252700	CVEs that are super important has been disclosed and we don't know about, right? So that's the type
1252700	1258540	of balance and the type of, I guess, for a corny way of saying orchestration that needs to take
1258540	1265760	place, right, in this CBD. I want to add another point here
1266800	1276340	regarding the overhyping. Having a logo, having a website for something, for a vulnerability,
1276340	1282320	for a research result does not necessarily mean that you overhype it. We had experiences where we
1282320	1292100	just put papers on archive without any website, without any logo and anything, and media picked
1292100	1306320	it up and media reported about it and not necessarily correct in all cases. And we had this,
1306320	1313060	on AMD CPUs, and their media picked it up. We intentionally didn't make any website
1313060	1321780	or logo, but media picked it up and the reporting, it was not entirely wrong,
1321780	1330760	but it was definitely, I would say, definitely overhyped. And we discussed this also in our team
1330760	1337340	and the conclusion that we had was that in some cases we should have a website, even if it's not
1338000	1343940	a vulnerability that we want to hype, because it's not that significant, but just to have a very
1343940	1350980	clear message which says how relevant is it. And I think that's the important part that we also
1350980	1359120	always had on our websites, a short three-sentence summary like what is it, who has to care about this
1359120	1365840	and what can an attacker do. Yeah, I think that's great. That's great you do that. And it's good
1365840	1373660	advice. I think that I would love to see it be more adopted in the researcher world,
1373660	1381420	for sure. I think it would help out. So before I move on to my next question, I'm just going to say
1381420	1387400	when I retire, all of you in vendor world are doomed, because I'm going to offer my services
1387400	1394600	to LREG for free writing up crazy headlines. But to touch on a point that Daniel made, he
1394600	1401180	invoked the Katie Masuris clause. Let's talk about bug bounties and CVD.
1402400	1408900	Oh, Katie, maybe you should take that one. Because our friend Katie Noble Trimble here
1408900	1413320	actually has a lot to do with that in her organization. Could you maybe talk about how
1413320	1418660	coordinated disclosure interacts both good and bad with bug bounty?
1419620	1426480	Yeah, so I love Katie Mo. I think that first off, let me just say that it is an honor to
1426480	1433500	be confused with Katie Mo. My hair color is like, you know, getting close to hers too now. I don't
1433500	1439460	know. I think I have been accepted to conferences because they thought that I was she. When I showed
1439460	1446520	up, I don't think they were nearly as excited to see me instead of her. So, yeah, bug bounties.
1446840	1453340	So, bug bounties are a tool. I'm trying to be serious here about bug bounties.
1453980	1459460	So, bug bounties are a tool, right? So, they're a tool in a toolkit. And they are there to
1459980	1468380	incentivize. But they're a part of a well-structured product security portfolio. They're
1468380	1474040	not the whole thing. And there are different motivations that will help people from different
1474040	1479640	perspectives. So, in the academic world, for instance, bug bounties may not weigh as much
1479640	1485920	because a lot of academic institutions don't allow an individual to accept that cold hard cash,
1485920	1491000	right? If you are a professional bug hunter, that might be how you pay your mortgage.
1491000	1496620	And so, there's a lot more tied to that. But the problem becomes that bug bounties are
1496800	1502480	a wonderful tool. And they're great. But you have to have a good program in place already
1502480	1508240	to accept that information, to be able to execute on that information, and figure out, like, how do
1508240	1515580	you actually... there are so many questions about it. How do you award? How do you manage it? Like,
1515580	1520960	are we gonna tie the payments to CVSS scores? Are we gonna tie the payments to a well thought
1520960	1524900	out proof of concept? There's so many pieces that go into it. And so, I would say, like,
1524900	1531280	bug bounty is not a one size fits all. It's gonna vary from organization to organization.
1531420	1537240	And some organizations are gonna have different timelines, different pieces. It's... yeah,
1537240	1542600	it's complex. So, bug bounty is not the end of the world. Bug bounty is a tool. It's a tool in
1542600	1548000	your toolkit. So, it's a great tool. I love the tool because I'm the director of it at Intel.
1548000	1555140	But it's not... it's not everything. So, vulnerability disclosure is more than bug
1555140	1560140	bounty alone. Yeah, I think when you think about it, too, like, there's a lot of smaller
1560140	1566600	companies that probably, you know, don't have that or don't have the funding for it. But that
1566600	1571900	doesn't mean that they're not wanting to work with researchers. They just can't get the budget to do
1571900	1577860	it. You know, so... so, you know, do your best to try to figure out how to reach out to those
1577860	1584060	companies. Hopefully, they have a web page or email address, you know, secure at p-cert at
1584060	1591600	security at. I know we struggle a little bit. But, you know, there's plenty of us around that
1591600	1599280	can help find the right contacts. Because we're, you know, sort of come together from all over the
1599280	1607180	place. And we're sort of an expanding group here. I like it. Bug bounties are one of the wonderful
1607180	1614100	signs of how the industry has changed. Back in the 90s, researchers were often rewarded with...
1615380	1621500	their disclosure was rewarded with lawsuits. And now people in the industry are working with
1621500	1628940	the people and have started actually paying the researchers for their efforts. So I'm
1629280	1635800	very much thumbs up on bug bounties, especially because it shows how the industry has changed
1635800	1641700	on how they view hackers. Hackers are nowadays helpers and not the enemy. Yeah, I thought you
1641700	1648440	were going to say t-shirts for a minute there in the early days. I like it. It just won the t-shirt.
1651400	1659260	Oh, earlier than that, it was lawsuits. Then it became t-shirts and then maybe stickers.
1659280	1665620	And then, yeah, and now financial rewards. Awesome stickers.
1668160	1672880	So let's... we've touched on it a little bit. Let's see if we can spend some time now that we're
1672880	1677820	past the top of the hour here. Let's talk about coordinated vulnerability disclosure
1677820	1686840	inside an open source context. So I'll go last, but let our esteemed panel here,
1686840	1690800	whether it's our researchers or our vendor friends, kind of describe
1690800	1698120	what works out really well and what are some challenges for you within the open source world,
1698120	1705380	which makes up about 90% of all software now. Hey, Jerry, open source won.
1717400	1727920	I guess I start. So basically, that's top of mind for me. And it sounds corny whenever people say,
1727920	1731000	hey, what keeps you up at night and everything else? It's actually open source right now.
1731200	1737920	And it's not so much of the predicament of using open source. We have to use and embrace and
1737920	1744440	contribute to open source. I mean, I'm a super big fan of that. The challenge when it comes to
1744440	1750040	open source is that it can be anything, right? It's like IoT can be anything, right? And it's
1750040	1759680	critical infrastructure for a lot of things. So to give you somewhat of a real life, I guess,
1759680	1765100	realization that we had a while back, probably about four or five years ago, whenever Heartbleed
1765100	1770420	came, we were looking in the industry, right? Not only Cisco, but a whole bunch of other companies,
1770420	1777840	right? So what to prioritize, as far as actually giving funding, probably doing research,
1777840	1783260	and so on. So we said, okay, so open SSL, and things like that, which are actually
1784180	1790100	super important for us. There's a lot of people looking at it right now. So can we look at things
1790100	1794840	that perhaps are actually critical infrastructure, that nobody actually has probably taken a look at
1794840	1801140	it, right? We're going down the list. And as a matter of fact, that number two is perfectly
1801140	1806380	fine. Because there are two guys that work on open SSL that get paid to do it. Yes, indeed.
1806560	1813920	And the example that I was going to go is NTP, the Network Time Protocol, so NTPD specifically.
1814120	1819960	And there's also two guys that don't get paid, which is bad, right? They're not even the
1819960	1824660	full-time job, right? Well, I guess now, you fast-forward this year, it's a little bit better
1824660	1830940	now, right? So it's a lot better. And it's not that the problem is the poor guys that actually
1830940	1834920	are contributing to the code. It's a matter of scalability. It's a matter of actually even
1836100	1840980	running static analysis into the thing that didn't even exist five years ago for these
1840980	1846020	components. Now, of course, we're modernizing our ways. Even if you submit things on GitHub,
1846080	1850980	a lot of things are actually happening. We're getting better, for sure. It's just that it's
1850980	1857140	getting way more complicated, right? And more people actually are not only, of course,
1857140	1862940	contributing, but using it more than contributing, which is the other predicament. And in some cases,
1862940	1870960	we were talking about bug bounties. In some cases, actually, amazing. I'm a big fan of
1870960	1875700	bug bounties, right? The challenge is that in some cases, we also don't think about,
1875700	1882360	does this affect other vendors? Am I actually finding some type of vulnerabilities that perhaps,
1882360	1887560	yes, it's a SQL injection, cross-site scripting, which is actually pretty common, but I'm doing
1887560	1892960	some fuzzing and I crash an application. What is the underlying issue, right? And in some cases,
1892960	1897400	it's actually kind of a commodity, even though it goes back and forth and it hasn't been shared.
1897460	1903840	And then two or three months down the road, you say, oh, but this CVE was not shared with
1903840	1908940	these vendors that they're also affected. So we go back and in some cases, you actually see
1908940	1913880	people trying to find the same or found the same CVEs and they reported a different way, but
1914420	1920340	there were no coordination. So those are the things that are crucial for us to be successful.
1920340	1925200	And not only among vendors, but actually downstreams and upstreams as well, right? So
1925980	1931860	especially whenever it comes to IoT is number one. Yeah. So Omar, you brought up a few points.
1931860	1939940	One is who do you bring in, right? Ahead of time. And I think that we're seeing that even if you're
1941200	1949180	the competition, we still want to work together in the background here. Our whole goal is to
1949180	1954580	protect our customers and we often have the same customers, even if we're competitors.
1954580	1960840	So we like to make sure that we're all working together and ideas is when do you bring someone
1960840	1966460	in and it's not only the researchers. Okay, Crowe, you got your hand raised. Go ahead.
1966500	1975160	Can I tell you a secret? Yes. Open source has worked with our competitors for 25 plus years.
1976200	1984740	Yeah, yeah. Well, I would have to say with some of those other vendors in the industry,
1984740	1994720	it wasn't always so nice. We're getting better. But I think not only the researchers should be
1994720	2001220	thinking who else is affected, but when we are the receiving end, we should be thinking who else is
2001220	2009000	affected and bring them in. I recently worked on an issue, a very recent one, where we are in
2009480	2015520	and the researchers were in right with the industry people who were fixing the issue.
2015520	2022920	And there was great coordination amongst the group there. It was pretty awesome to see
2022920	2029180	of how much we've evolved. How do you do that, though, with like open source, though, when
2029180	2033240	everything's shown? It's super easy.
2034680	2039380	Well, and I guess I'll chime in before we let other folks.
2039380	2045540	I found it very interesting how some vulnerabilities have been patched wide in the open
2046080	2055020	under the cover of some other patch. I don't know what you're talking about at all.
2057980	2061180	So, in general,
2063240	2069280	open source, there is no single definition of what open source is. You could have two young
2069280	2074660	ladies in Bangladesh that have an amazing idea and they're just trying to get this creativity
2074660	2079360	out there to share with the world. You could have large corporations like a lot of the folks
2079360	2084080	represented here. You could have academics. So, open source is a lot of different things. And
2084080	2090020	there's really no one definition or moniker that fits, that works with every community. But if
2090020	2096540	you're thinking about the types of open source that might make up a product or some kind of
2096540	2104120	cloud offering, you're going to have the high end of the spectrum are things like the kernel group,
2104120	2110400	which is very mature and organized, Kubernetes. And then you start moving down to Apache
2110400	2115780	Foundation, these other large communities. And then you get down to something where it's
2116420	2121760	a single person, a couple people that are just playing around. And it's hard to put any kind of
2122500	2126720	structure or process to all these different models because a lot of people that code for
2126720	2133520	open source are doing it for free, for no remuneration from large companies that make a
2133520	2139260	lot of money off of it. But they do it for free because they love kind of adding value and
2139260	2145660	expressing their creativity. And as Daniel mentioned, open source is very creative. And
2145660	2152300	some of the larger projects, we do have ways to privately take in data. So we take in a private
2152300	2157660	bug report that's well established within the community for a long time. And when you're
2157660	2165200	looking at a less mature community or package or library, they might not have that capability. And
2165200	2170740	that's where larger kind of big brothers and big sisters, like a Red Hat or a SUSE or Canonical,
2170740	2175380	kind of step in and try to help mentor these smaller projects to help them get these good
2175380	2181640	practices set up so they can take in a private note. Because sometimes a lot of, you know,
2181640	2190020	my team, we track between 3,000 and 5,000 vulnerabilities a year out of 450,000 packages.
2190020	2195580	That's a lot. Not all of them really kind of bubble up to the level of they need the
2195580	2202020	attention of like a Spectre meltdown or a Heartbleed or a, you know, Blueborn, all these
2202020	2206680	kind of the other big name nonsense things. But, you know, they still need to get fixed. And what
2206680	2213080	open source is really good at is you identify a problem, the team attacks it collaboratively,
2213080	2218220	they develop a solution and release that update very quickly. And they don't like to spend a lot
2218220	2223840	of time making a big deal out of it because they've already moved on to the next feature,
2223840	2230460	the next big thing. But, yeah, it can be challenging. But there are definitely ways to
2230460	2235340	do it. There are methods to do it. There are groups that allow this. I hear a lot of bleeping
2235340	2239020	in the background. What's going on, folks? I was wondering when you were going to change
2239020	2245600	your hat again. Sorry. There we go.
2248420	2258960	All right. Let me get this panel back under order. When you're thinking about doing a CVD,
2261910	2265890	what are some things you might want to prepare for? What can you do to help make that
2265890	2272630	coordination very successful? And let me start off with... we'll start off with Lisa because
2272630	2278170	she has a lot of ideas. So, from your perspective, what will make you successful when you're getting
2278170	2285270	ready to do this multiparty coordination? So, I guess it depends on how familiar you
2285270	2290210	are with it or not. Like, if I was a researcher and I wasn't too familiar, I would potentially
2290210	2297250	utilize CERT or something like that if I felt like it was going to cross over more than one
2297250	2305410	company in the industry just to have them help. Because I think it could be overbearing of
2305410	2313770	figuring out how if one, you know, if industry, you know, company A wants this date to get it
2313770	2319730	fixed and company B wants another date and company C wants another date. So, I think that's one way
2320210	2328930	that a researcher could use. But I think the idea is to figure out when you start is what are your
2328930	2336130	rules? What's your embargo date? How many days do you actually want to, you know, give the company
2336130	2344290	or vendor to fix the issue? Typically, it's 90 days. I would prefer that as a company. I think
2344290	2350110	it's respectful to do that, especially when things get more difficult. So, think about your rules and
2350210	2355610	what you want to do and how you want to approach it. But I'll pause there because I know we're
2355610	2360170	running out of time to get other people in. Anders, what are your thoughts about how you
2360170	2369330	could make CVD successful? Three things. Use it. It's a tool not only for vendors but also for
2370230	2379650	hackers. Listen to what vendors are saying or what hackers are saying
2381470	2386550	and try to make the best out of it and be aware that there's
2387890	2393230	often a lot of complexities involved with it. Right. Be patient, right? Yeah.
2395030	2399030	Omar, what are your thoughts on how you can make CVD successful?
2399330	2403710	I think that I'm going to capitalize on something that Daniel mentioned in a previous
2404790	2412030	call. And it's in some cases, even if you provide some data to even a vendor or whoever,
2412030	2418130	whenever you're coordinating, you have to make it first easy to understand. But at the same time,
2418130	2424610	in some cases, getting the right people at the right place at the right time to make sure that
2424610	2428630	you understand the technical implications of a given vulnerability instead of running around.
2428630	2433810	Even if you have the 90 days or 60 days or 100 days, getting that streamlined and modernizing
2433810	2440470	the way that we actually exchange information among all the affected parties, that is number
2440470	2446870	one. And Art Mannion, which is actually a good friend of mine, he leads... We all love Art.
2447850	2454350	He's the person that will tell you we cannot scale, right? This is a big ecosystem.
2454350	2461050	And just having a Rolodex of all the vendors that actually use a component is foolish, right? We
2461050	2465910	will never, ever, ever be able to actually have a complete thing. But what we have to think about
2465910	2472670	is what Lisa mentioned is that in some cases, we are working with competitors, right? Even more
2472670	2478930	than our companies. For example, you know, I work with Lisa a lot. I work with Juniper even more,
2478930	2483250	in some cases more than I worked with Cisco, whenever it comes to fixing a vulnerability,
2483250	2488970	saying BGP, OSPF, or whatever the case might be. So that type of thing of, oh, I'm going to talk
2488970	2492970	to this competitor, is there a problem and everything else is not, you know, that's 20
2492970	2501990	years ago, a fallacy. And then the last one is that at the end of the day, I have the reality
2501990	2507490	and I tell my guys at Cisco is that whenever I push a button, I publish a CV out there,
2507490	2512810	the number ones that actually are reading that CV are the bad guys, right? And probably
2513250	2517630	at the same time that you're committing an open source code, you know, probably that's going to be
2517630	2524610	the case too, right? So what we have to do is think about how can we exchange information to
2524610	2532270	the consumer, to downstream and upstream providers in a more modern way. I know that sounds corny,
2532270	2537890	but you know, can we actually make it more and we're trying to do that machine readable, right?
2537890	2542430	So if in the case that we actually have that Rolodex of people and everything else, you know,
2542430	2548050	get some tool and that's what Art and, you know, the guys in CERT are also doing, you know,
2548050	2553470	we're creating tools that allow us to do that. And even if you have, I don't know, even two years ago
2553470	2558010	that didn't exist and it wasn't even a thought, right? So we have to move faster. That's the number
2558010	2565930	one thing that I want to say. So as I close, I'll say Art Mannion has the best facial hair
2565930	2575150	in the industry. But I want to thank our panel here. This was a great hour together. I really
2575150	2580230	appreciate your time and expertise. We're going to be hanging out for a little bit on this awesome
2580230	2585530	Discord channel that my kids made fun of me all week about because I've had it up. Thank you
2585530	2590250	everybody for coming and your attention here, panelists, and thank you to the audience for
2590250	2596010	having us here at the IoT Village. Enjoy your day and enjoy the rest of the con.
2596010	2598330	We've got some great stuff lined up the rest of the weekend.
2605450	2606490	We're out.