We've got Stuart Scott talking about preserving supply side vulnerability disclosure.
Please give him a warm welcome.
Hey, everybody. Can you guys hear me fine? Awesome.
My name is Stuart Scott. I'm an assistant director with the Atlantic Council's Cyber Statecraft Initiative.
We work kind of at the intersection of tech policy and cybersecurity.
In the past couple of years, we've had a really great opportunity to work a lot on software supply chain security
and do a lot of kind of translation from the policy community to the tech community and the other direction.
And part of how we're trying to frame our thinking is looking at sources of risk and sources of security.
And one that we've recently kind of honed in on as a source of security has been the kind of pipeline of external vulnerability research
from researchers and researching entities to private companies
and open source companies.
Open source libraries.
So as I'm sure many of you know, the bug bounty platform market has been growing
and is projected to hit some pretty crazy numbers in the next couple of years.
It's a pretty direct manifestation of the many eyes theory of security, right?
More people looking at code, better chance of finding and fixing vulnerabilities before someone exploits them.
And we've also seen some increasing government adoption.
Sorry.
Okay.
So government's setting up their own kind of programs and giving some more clarity to the kind of legal protections
and guardrails around researchers.
And the whole process is generally called something like CVD, coordinated vulnerability disclosure.
And we'll kind of refer to it by that throughout.
But it varies between organizations.
And we had a really interesting opportunity to look at some anonymized bug bounty data.
It came up in congressional testimony in February.
And it got us thinking about kind of concentrations of this source of security.
And the big question we ask, you know, part of what we do as policy folks is try to figure out good policy.
But another part of what we do is try to figure out bad policy and how to avoid it.
So we started asking ourselves, what kinds of policies could disrupt this supply of security,
this supply of research coming from a community?
And we had a really excellent opportunity to dive into that question and look at it with some data
after the Log4j disclosure and kind of the saga around that.
So at the front end, and this is a quote from Carnegie Mellon's CVD All You Need to Know Guide.
It kind of describes CVD as picking somewhere in the world,
somewhere between finding a vulnerability and never telling anyone
and finding a vulnerability and immediately telling the world.
So trying to find that optimal path of information sharing that gets you to fix bugs as quickly as possible
without kind of revealing exploits before patches are ready.
So the general kind of timeline around the Log4j disclosures is a useful example of how that kind of information management happens.
So research at Alibaba Cloud found the vulnerability and just directly emailed the Apache kind of maintainers of that library
in November 24, 2021.
And then followed up as he noticed that the vulnerability was starting to make its way more publicly into some cybersecurity and fora,
which kind of prompts the Apache need to patch more quickly and to get people in a position to mitigate and to remediate.
And so within a couple of days of that second follow-up email, Apache starts pushing out their patches
and different kind of government organizations start notifying everyone.
So that's a good management process, right?
You tell people who need to know and can fix the problem.
And then once that information starts becoming more public, you tell everyone so they can kind of prepare and do their side.
But then we saw something interesting.
On the 22nd of December, the MIIT, one of China's kind of ministries of, this is industry and information technology,
sanctioned Alibaba for how they handled that disclosure process.
They were sanctioned for not telling the MIIT quick enough about the vulnerability.
And there are two kind of legal mechanisms that might have been part of that sanction,
which involved removing Alibaba from an information sharing forum.
One.
One was that as part of the information sharing forum, they had to share information and they didn't do that.
And the other that we wanted to look at was there is a law on the books in China,
in effect since September 2021, requiring companies that find vulnerabilities to immediately tell product owners
and then within two days to also tell the MIIT.
So we talked to a couple of sources and it's unclear which mechanism was kind of the force behind the sanction.
But pretty clearly the letter of that law is violated, right?
So you see the little orange boxes.
Network providers have to, within two days, tell the MIIT of vulnerabilities they find.
And this, to our mind, is a really convenient case study for testing this idea of security supply from vulnerability research.
So a lot of the reporting focuses on what happens to vulnerabilities given to government early,
but that's not what we're focused on here.
China is a huge producer of this type of research, so it's very easy in theory for us to tell if some kind of macro movement happens there.
The law is pretty clear and we've got some good dates to test beforehand.
Before and after.
And the kind of mechanism for a chilling effect is also pretty clear, right?
If you're a private company operating in China, you have to kind of consider your internal policies receiving that information.
And if you're a researching entity, you might want to wait on disclosing and even on performing research
until there's more legal clarity because the law itself is pretty vague.
It definitely applies to product providers.
It's unclear to what extent it covers researchers who might be vaguely affiliated with a company,
might be sponsored by a company, might work for a company, but are working on their own free time.
And to be clear, this is not just focusing on that.
This is focusing on China, right?
We see similar issues going on in other countries.
It's just not as clear of a case study for us to analyze, right?
So in Germany, a researcher found a vulnerability in an application for election polling
and was threatened with legal action even though they followed the right channels.
And something similar happened in Missouri.
I've seen a couple of those F12 is not a crime t-shirts around, right?
Guy looked at a website, figured out he could pull social security numbers out of it,
and was threatened by the kind of state government.
But we've also seen government kind of embrace this approach to security more.
They prosecute good faith research, whatever that might be.
And ANISA and CISA each pushing for government agencies and member states in the EU
to develop their own explicit processes, which is a better idea of what to do.
But this is the main question, right?
Did the RMSV have a supply shock effect in the Chinese research contributions?
And if so, what did that look like?
Did it trickle into the broader ecosystem?
And are there other shocks that are worth kind of looking at?
Along the way, we got to look at just some really cool information about how that ecosystem works.
Where different companies are kind of pulling in research.
But the basic selection of data we looked at were the acknowledgements in CVEs and security updates
from Microsoft, Apple, F5, VMware, and Red Hat.
Gives us a nice selection of different products, right?
Some consumer facing things.
Some very in the racks type things.
And with Microsoft, kind of a broad swath of all that.
As well as some proprietary and open source code bases.
And a nice spread of different kind of internal processes for how companies report this kind of information.
And so we're going to take that data, match it against a pretty simple timeline, right?
The first RMSV kind of public mention is July 2020.
Laws passed in July 2021.
And then takes effect September 2021.
And if we have recent enough data, we can even look at kind of post first maybe enforcement.
Even though we can't really confirm that it was enforced at that point.
They don't exist in a vacuum as events, of course, right?
So the summers of 2020 and 2021 both have a lot of other cyber activity going on in China.
They have increased sanctioning of Chinese companies by U.S. companies.
And they also have a couple other regulations being passed that might interfere.
So even if we see an effect that lines up with our time, we do have to be careful about kind of assigning causation where it might just be correlation.
But our basic process is pretty simple.
Pull out all that data, either some scraping text or direct downloads.
And huge thanks to those companies that make that easy to do.
And then parse out a CVE, a relevant date for when that kind of disclosure happened.
And then the entity credited.
And then we had a kind of list of companies and a list of individuals.
We decided to just focus on the companies both for kind of concern for the individual's privacy and also just for ease of access.
It's much harder to tag an individual, it might be a Twitter handle or a GitHub account to a location.
But for a company, it's pretty straightforward.
And it gives us a good proxy for the legal context they operate in.
And then we just batch those by month, clean out the data, and analyze it.
And these are kind of the five companies, again, that went into that process.
And there are a lot of issues with that.
We eventually got it to work.
But there's tons of irregularity in how that data is stored, how that's organized.
So Apple, for example, will tell you about CVEs that it remediates and updates.
And those updates might predate the date that the CVE is kind of publicly acknowledged.
So there's some back work going on.
Other companies organized by CVE.
Some companies organized by a little bit of a blend of like affected product lines.
And there's tons of miscodings and typos and different organizational practices.
But in general, we get a good sense of where things are happening.
So we're going to talk a bit about the Microsoft and the Red Hat data where we had some interesting findings.
In the Apple and VMware data, there was no supply shock effect.
But we'll show the Apple data a little bit just to give a sense of what that absence, that null finding looks like.
And for the F5 data, we didn't find any even Chinese-affiliated entities participating in that contribution ecosystem.
So here's a quick look at the kind of proportional breakdown by month of contributions to vulnerability disclosure in Microsoft.
So looking at the left part of the graph, we get a pretty good idea of a general story throughout our data set.
Early on, the U.S. is kind of the dominant contributor.
Then as other countries' IT sectors mature, as other kind of relationships are built between the kind of security center at Microsoft and all those researchers,
and as they develop, we see more participation internationally.
So this orange line shows increasing Chinese contributions.
But it's true of pretty much all the other countries.
But what's really interesting is the steep drop.
It lines up, as you can see, on July 2020, so one of our key dates, and interrupts a pretty clear trend.
So to get a bit more of a close look at that, we look not just at portions but the raw counts of contributions.
And the green line at the top is just the total number of contributions.
It's a little more of a nuanced story there.
So as you can see, the drop is a bit of a return to the mean for the Chinese contribution count.
But it takes place in the context of overall more contributions happening at that time.
And not a corresponding drop in that total amount.
So the trend lines are a little less dramatic.
But that steep, steep decline is still there at one of our key dates.
It is kind of our most compelling key date.
It's when a lot more sanctioned activity happened between the U.S. and China.
And it's also when a lot of those kind of new Chinese cybersecurity regulations were drafted and started circulating.
But it is also a little harder to assign causality to any single one of those laws because of that kind of flurry of activity.
The next set we wanted to show off was the Red Hat data.
And you'll have to forgive the very spiky stuff on the left here.
But it went a lot farther back to about 1996.
And early on, again, very U.S. dominated.
And then more countries started to participate.
We see kind of more general representation.
But, again, we find an interesting decline here.
But not at a date we immediately had a good explanation for.
So this is between February and April 2017.
Steep decline in the proportion of Chinese contributions to this research ecosystem.
And, again, kind of a disruption of a trend line that's kind of then resumed afterwards but at a decreased rate.
And if you look at the contribution counts rather than the portions, same story again.
It is worth noting, too, that because as an open source ecosystem there's a lot more individual work going on,
there's a much bigger gap between the total contributions and the by country contributions.
But, again, same story.
You can see that drop off and then kind of maintained throughout as a decreased rate.
So we had the chance to talk to some other folks.
And they pointed out that around this timeframe,
we had entered an information sharing agreement, I believe, with Huawei.
So there's something going on there.
We don't know the exact mechanism or if that's even the relevant event, if there is one.
But it is worth kind of looking into.
And then just for final context, here are the Apple data sets.
And, you know, there was no drop off here.
But it is nice to note that kind of, again, very prevalent U.S. presence in the ecosystem.
And then as other countries kind of tap in, more broad representation.
And here, again, are the kind of total counts,
rather than the proportions.
It's very spiky data, too.
There's some seasonality.
I think every quarter and then every other month,
there'd be kind of an increased amount of security updates going out.
But to take away from those, you know, very crowded graphs,
our main finding was that, on the one hand,
there isn't a clear kind of ecosystem-wide supply shock occurring because of the RMSV yet,
although it's really hard to judge the exact timelines going on.
But there's definitely potential for one given that kind of very steep decline in Microsoft
that seems easily explained.
And also that less well-explained but still present drop in the Red Hat data.
So our main kind of recommendation to government and policymakers
is to not take the kind of supply of external research for granted
and to find ways to kind of invest in that community,
either proactively avoiding supply shocks or, in general,
just to improve kind of the volume and the spread of research they're getting.
So we figured out three kind of different avenues to approach that for government and industry.
One is just for governments.
And partners of governments to better harmonize their kind of CBD processes and guardrails, right?
Providing clarity for researchers and not having to force them to worry about jurisdiction, right?
A researcher in Europe working on an American product
but perhaps finding a vulnerability while they're somewhere else.
Just providing kind of clarity and clear legal protections for them is useful.
And it doesn't have to be too explicit that it kind of limits the scope of researchers
and their ability to look at things in an interesting and different way.
Another great tool is just supporting kind of vulnerability discovery tools.
Takes a lot of the burden off of researchers
and gives them kind of new angles to plug into that work
without having to start from scratch.
And then, finally, we wanted to take a pass at trying to incentivize work
in areas where either some kind of supply shock might have happened
or where there's simply not a lot of research going on.
In general, if there's, you know, some product type or ecosystem
where there are not a lot of vulnerabilities being found,
any vulnerabilities found there have more security value.
And government and industry can shape incentives and, you know, bug bounty rewards
to reflect that kind of increased vulnerability.
But to quickly sum up our work, you know,
we wanted to start thinking about supply side vulnerability research,
start thinking about it as a community that contributes to overall security,
and figure out can bad policy kind of shock that system,
and if so, how to avoid it.
And we found that, you know, our best case study so far
doesn't provide an overwhelming abundance of evidence for that supply shock,
but it is kind of compelling and a little bit troubling.
And fortunately, there are things that governments and companies can do
to kind of protect that community.
And that doesn't depend on there being necessarily a clear case for a supply shock
because it's a good investment to make in security.
And finally, this is still very much a working project for us.
We've got some open-ended research questions
about which companies stopped contributing to the Microsoft ecosystem, for example.
So we'd love kind of any feedback, any kind of experience you all have had
with, you know, contributing to work from companies
either in bug bounty programs or directly.
Happy to answer some questions.
And our emails are up here if you'd like to reach out.
And you can check out the full kind of draft report on our website.
But thanks so much for having us.
Yeah, sure.
We have a few minutes if you have any questions.
If not, happy to give everyone some time back.
I also cannot see out there, so...
Awesome. Thanks so much.
Thanks.