Hey, everyone. Thank you so much for that. So I am Perry Adams. I am from DARPA. And I have
with me here today Michael Salito, who is the head of geopolitics and security policy at
Anthropic. I have Heather Atkins, the VP of security engineering at Google. Vijay
Bolina, the CISO and head of security research at Google DeepMind. Dave Weston, who is the VP of
enterprise and operating system security at Microsoft. Matt Knight, who is the head of
security at OpenAI. And Omkar Arasara. I'm so sorry, Omkar. We should have practiced this
beforehand. Omkar Arasara. You can, you can, you can, you can, you can, you can, you can, you
can. Yes, please give him a, a round of applause for my putting up with me. He is the general
manager of the open source security foundation project of the Linux foundation. So now we got
through that. I wonder if I have the largest panel at DEF CON this year. I think it took
us a minute and a half just to get through names. So before we let them talk, I, I have
I'll give a little background on what we're here to talk about today. So I'm Perry Adams.
I'm at DARPA where I run a number of research efforts that try to push the bounds of what's
possible in computer security defense. And while I'm not doing that, when I'm not doing
that, I organize DEF CON CTF and the finals are currently ongoing. So to put a plug for
that, you should absolutely visit the CTF floor. But one thing that I'm doing right now that I
just announced at Black Hat is called the AI Cyber Challenge. And so if you don't know
anything about it, I encourage you to visit AICyberChallenge.com, which has quite a bit of
detail of what this challenge will be. And so I'm here at Black Hat and DEF CON to
announce it for the first time. This is a White House backed initiative that has been
put up to nearly $20 million on the line to bring together the best and the brightest in AI and
cyber security to create AI driven systems that can secure the software that all Americans rely on.
So this is a competition to develop automatic systems that can find and fix vulnerabilities in
code. And so a little bit about the structure of this. We are going to be working with the
Department of Data Science. We are going to put on a qualification event in May of this year,
and 20 teams will advance to the semifinals at DEF CON in 2024. The top five teams there will win
$2 million each and will advance to the finals, which will occur at DEF CON in 2025. DARPA is
also funding up to seven small businesses up to $1 million each to participate. And so the
application process for those awards will open on August 17th. And the application process for
the open registration, the open track, will start in November. And so this is a competition that is
open to anyone. And I encourage you all to visit AICyberChallenge.com to learn more about this
effort. Because the reason I want to announce it to this audience is that Black Hat and DEF CON bring
together some of the greatest minds in this space. And I would love to see participation from you
all in something like this. Now, I'm also incredibly gratified to have some really fantastic
collaborators on this effort. So Anthropic, Google, DeepMind, Microsoft, and OpenAI have come
together to collaborate with DARPA on this effort to make their cutting-edge technology available to
participants to build on top of for this effort. And so I want to thank you all for joining me today. I'm going to
give you a little bit of an overview of what we're doing. Similarly, we're collaborating with the
Open Source Security Foundation to help participants develop systems that can solve real-world
challenges, especially those, for instance, seen in the open source community such as supply chain,
security, and the security of our critical infrastructure systems, which are built on top of so
much open source code. And so today, we're really here to not hear me talk, because I've been
talking about this quite a lot lately, but to talk about some of the things that we're working on. And I'm going to
hear my panelists talk about what they're looking forward to in this competition, how they see it fit
into the broader context of the work that they do every day. And so I'm going to start all the way at
the end of the table with Omkar. And so, Omkar, help us set the stage. What challenges does the
open source community face in software security? And what do you see is wrong with the status quo?
Is the microphone on?
I could talk really loud.
I think it is.
All right. Thank you. Thank you, Perry. And let me first say, as a guy who first went to DEF CON at
Alexis Park, just seeing the amount of security professionals that are here, I think this was probably bigger
than all of the attendees at Alexis Park the last time I was at DEF CON, so it's an honor to be here. As most of you in the room know, open source is everywhere.
Open source is in safety-critical systems from planes to trains to mobile phones, routers, switches, computers. And we believe that it is our duty as
custodians of security within the Linux Foundation, home of things like the kernel, you may have heard of it, as well as other projects like
CNCF and, of course, the Open SSF, we believe that it's our public duty to ensure that our open source supply chain is secured by construction.
That is not an easy thing to do. This is not enterprise software. We can't just jam a secure SDLC in and be done with it.
So we're working with our community as best we can, and we're really looking forward to efforts like this one where we're able to use cutting-edge technologies
in order to address vast classes of problem across open source in a way that is complementary to our communities.
Simply as well.
opening thousands of PRs that don't follow the coding style of the project that you're trying
to fix or aren't conformant with the culture of that project is not the way you fix security.
And we look forward to working with DARPA as well as our partners on this challenge. We're
looking forward to some very innovative solutions to these systemic challenges that we've faced
in open source that underpin some of our most safety critical systems.
Thank you. And so my next question goes to my three friends from Google and Microsoft.
So Heather, Vijay, and Dave. You all between Google and Microsoft maintain some of the
largest code bases in the world. How do you deal with those challenges, some of the
challenges that Amkar talked about at that scale? And how do you foresee the use of AI
as applied to that problem? So do you want to start with Dave? Sure. Yeah. So I think the
challenge of finding vulnerabilities is really a challenge of scale. At Microsoft we use many
different techniques. So, for example, fuzzing and dynamic analysis techniques are a great way
to add scale, find vulnerabilities, especially in large code bases. We use compiler changes,
platform mitigations, safer languages. I'm sure there's some Rust fans out there. Shout out to my
team. But obviously we have a large code base that is, you know, often impacted by memory safety
issues. So I'm really excited about the scope and the opportunity of this competition. I'm
optimistic, but I also, having worked in this space a bit, I know there's some real challenges
here. So I think there's going to have to be some breakthroughs here to really deliver. And I'm
excited and I think we could do this. You know, one of the areas that I think is really exciting
is augmenting static analysis. You know, I mentioned that fuzzing is a fundamental, a
technique, but it has some limitations. Building tests and drivers, getting code coverage, a lot
of the sort of block and tackling is not, you know, particularly beautiful and it's difficult to
do. So getting that coverage is sometimes hard. Static analysis, on the other hand, particularly
on native languages, can be challenging as well, just reasoning at that scale. And so being able
to do things like automatically generate annotations that can then feed into existing static
analysis systems. And augmenting static analysis can be challenging as well. So being able to do
that and augmenting those systems with some AI approaches are, I think, something that I think
will show up in this competition, if I had to guess. And it's something that we're working on
internally and it's been pretty fruitful in the early stages. So I'm actually just really
excited as a security geek to see where folks go. And I think there will be huge benefit to
Microsoft and many other folks here if we can really tackle this challenge. So it's great. Great
to see. And before I hand it back to Google, just to build on what Dave said is I think how he
describes the use of AI, not simply taking large language models and throwing them at code and
saying, oh, it's going to LLM a vulnerability, but thinking about the ways in which not just
generative AI but other machine learning-based approaches can augment traditional program
analysis tools like static and dynamic analysis, can take the hard computational problems in that
space and the complexity issues and reduce the search space, guide it, and provide augmentation
in the spaces that traditional computer science approaches haven't been able to tackle. That's
something I'm really excited to see in this challenge. So jumping off of that, I'll hand it to
Google. Do you want me to go first? All right. I think it's important to think a little bit
about what the outcome we're trying to achieve is. I mean, imagine a world where most of the new
code on the planet is mostly free of bugs. And all the code that you maintain when you make new
changes is also mostly free of bugs. So I think it's important to think a little bit about what the
outcome is. And when you find vulnerabilities, they're patched and fixed very quickly. That's the
outcome here. And we've been trying to do that as an industry for a long time. Dave mentioned a
bunch of things we've been doing from static analysis, manual audit, fuzzing. DARPA ran a
cyber grand challenge. What was it? 2015, 2016 and the winning teams there found vulnerabilities
mostly through fuzzing. I think we are now in a new generation of software development where we
we are about to get a real leg up on this problem.
And fixing the whole software development life cycle
to give developers a high amount of assurance.
And so I'm really excited about the partnership with DARPA
because I think we're gonna really challenge
in a very open collaborative community kind of way
what those solutions might be,
whether you're an open source developer
or a commercial developer.
And I think we're probably going to invent some new things.
I look forward to kind of all the innovations we see there.
BJ, do you want to jump in?
Yeah, great.
I think one thing that's quite exciting
about the software security space
is that there's a large group of AI researchers
that are also deeply interested in this space.
The problem is quite tractable,
both on the static analysis side
and on the data analysis side.
Google has one of the largest fuzzing clusters in the world
across our code base and we care about
that space quite a bit.
You've seen some tools that we've released
over the past few years, especially in this space.
And there's, from a research standpoint,
some very specific problem spaces
that a lot of folks across Google
have started to focus on.
For example, using reinforcement learning
to do deeper and better code coverage
across our code base, across our integration
and build pipelines, and optimizing for that space.
specific outcome as heather mentioned so that's pretty exciting we've seen some pretty pretty
impressive results in that space as well and then you know again to kind of build on on top of some
of the things we're doing around fuzzing using the latest and greatest state-of-the-art models
that we have to do things like fuzz target identification and general work that we have
around our ability to identify and make sense and things that are our clusters find in a scalable
way thank you I and so I talked a little bit about taking different AI approaches and combining them
with program analysis techniques but we have also seen just really interesting potential
significant potential in a range of areas from the large language models that have been put out
recently
and so to both Michael and Matt Anthropic and OpenAI how do you think LLM specifically can assist
with finding and fixing vulnerabilities in software yeah thanks Perry I couldn't be more
excited to be supporting the AI cyber challenge and hopefully many of you as participants it's
a great vision which very much aligns with our own at OpenAI we believe that large language
models are going to be transformational for cyber defense and we're extremely
excited to see what we can do to make sure that we're able to do that in the future as well so
after our test run I'm definitely excited to have the next generation of security tooling built on our models
we've been working with the Community on Defensive Security Research Programs through our Cybergrant confusion
program we've had some just really amazing submissions from the community we're giving away a million
dollars and to start plus GPT for access and api credits to get people started and we're excited to give the same treatment to AI cyber challenge participants too look at
access to our models you'll be able to use credits and so on to do your to do your work continuing by our staff and that's in the platform that we want to connect with and
So I would just invite you to think about ways in which you are limited as security engineers.
These are areas where language models can augment your own work
and help fill in areas where maybe you don't have time,
parts of your workflow that are sort of drudgery or maybe where you have scale limitations.
These are areas where language models are going to be able to fill in.
And I just want to add a personal note here.
I competed in DARPA's Spectrum Collaboration Challenge a couple years ago in a personal context.
It was some of the most fun that I've had writing software.
It's a great way to get access to some really cool technology,
work on some really challenging problems, and also be eligible to earn some money too.
So I hope you'll consider competing.
I look forward to supporting you, and thanks to Perry and DARPA for making this happen.
Yeah, thanks.
So large language models like this,
Cloud can complement the wide range of tools already used by cybersecurity experts.
In some cases, they can automate tasks that require human judgments
so that vulnerabilities can be found and fixed in an automated and more cost-effective fashion.
But I think the most important thing to think about is how quickly the technology is improving.
You know, two years ago, nobody was using LLMs to help with writing code.
Last year, a study came out that found that GitHub co-pilot users were 50% more productive
than people that didn't use it.
And today, LLMs are useful assistance for researchers to find and fix bugs.
They can generate, interpret, explain, suggest alternatives to code.
But by next year's DEF CON, and certainly within two years when the finals are held,
the base models are just going to be dramatically more powerful than they are today,
in the same way that we've seen even in the last year, things change quite dramatically.
So we're really excited to support the Cyber Challenge
and see what everybody's able to build on our model.
This could be a really great opportunity to do something
that makes a huge contribution to the open source community
and to the security of open source that underpins so much of our economy and society.
Thank you.
And I have to say, this is one of the parts of this challenge,
just to put a fine point out that I am so excited for,
is that participants will have access to resources from our collaborators to build on top of.
So we'll have access to very, very cutting-edge technology from across multiple companies
and be able to show really what's possible when that's applied
to a very important challenge like cybersecurity.
And I also want to stress the prize money that DARPA is putting on the table.
The top prize for the final winner in 2025 will be $4 million.
The second place will be $3 million.
And the third place will be $1.5 million.
And that's in addition to the $2 million each those top five finalists will get.
And that's because DARPA sees this as an incredibly important initiative
to push software security forward.
So in our last three minutes, does everyone want to go down
and give a 20-second snippet of advice for prospective AICC participants?
Sure, I'll start.
So language models are super sensitive to the prompt that you put in.
And so if the first prompt doesn't work and the second and the hundredth don't work,
just keep trying.
It could be as simple as finding a statement like think step by step
and you might just unlock a huge set of new capabilities
that you didn't know existed in the model.
Yeah, I think also in addition to thinking about how the LLM is working,
think a little bit about how you as a developer code
and where it would be most helpful.
We call that kind of the user journey of software development.
Do you want it kind of there as you're coding?
Do you want it when you check in your code?
Do you want it running after?
And just think about all the opportunities that you might be able to target.
I think everybody here will agree, but two years is an eternity in AI.
So there's going to be quite a bit of improvement in capabilities.
So keep that in mind when you're thinking about your final solution
throughout the competition.
I think that's going to be important.
You probably shouldn't be over-indexing on current capabilities
because there could be some substantial step changes
in those capabilities too.
Yeah, this is obviously going to be a significant challenge.
I've looked a little bit at the rule set and the things that have to happen here.
And so I think creativity is going to be a huge part of this,
thinking about how to nest together different solutions to get to this outcome
and ultimately get a high-precision outcome.
I think traditionally the problem with static analysis is not that it can't find bugs,
but can it find bugs without the noise?
And so I think thinking even beyond just the specific challenges that are there,
but how this is going to be deployed at scale will be key to getting the results.
I would encourage you to think about places and opportunities
where current tools are constrained or where you are constrained in your own bandwidth.
That is where the action is at.
Solve for that and you'll have interesting results.
And remember that the versions of these models that we have today
are the worst versions that we'll ever have at any point going forward.
These things are going to get better.
Do not bet on today's foundation model capabilities
because the next ones may supplant it.
So get ready for that.
I have the luck of going last and saying something profound
after all those wonderful suggestions.
As I mentioned earlier, open source underpins the most critical infrastructure that we use today.
The work that you're going to embark on using some of these leading edge tools
will have profound impact.
This isn't a term paper.
This isn't a release.
This isn't a new product.
This is everything.
This is all the marbles.
So we really look forward to seeing
the contributions that y'all as participants in the challenge will be contributing.
And most of all, we look forward to leveling up open source supply chain security.
Thank you all.
And I would second all of that advice.
Actually, and I'll add one last thing on Dave's,
which is that we will also be looking at false positives that come out
because to create good tooling,
you really need to be thinking about usability and the amount of noise.
But to end with the obligatory point,
please visit AICyberChallenge.com
where there is all the information you need to understand how to participate.
And more information will be coming out this fall.
Thank you all so much.