start	end	text
0	8100	Hopefully you guys will see this as a nice talk for setting the stage.
8460	15240	The original idea was to have it as a very in-depth, very long two-panel talk. I think that
15920	24900	was very optimistic. So this will be hopefully a cool talk in kind of the history and the
24900	31280	uh, the field of program analysis. How we got where we got and where we came from.
32940	37640	Let's see, we're almost there. Is it not? Can we have a countdown?
39920	46960	New set? Alright! The stage is mine. Psych! I'm not talking about program analysis. We're
46960	54400	gonna look at cats all day. But in all seriousness, when I saw that it was the 25th anniversary of
54400	64580	DEF CON, I had this thought because just the week before I read the CFP, I was reading some
64580	70600	paper about program analysis that was published in the 90s. And I was thinking that, you know,
70600	77220	wow, these guys had like a lot of the same ideas that we're having now. And they were talking
77220	83740	about this in the 90s. I wonder how much other such history has kind of fallen by the wayside.
84900	91140	And so I started looking a little bit, digging through, and I wrote this proposal for the CFP
91820	96900	to kind of go through the last 25 years of program analysis. And then when I was putting together
96900	104280	the slides, I realized that program analysis goes way farther back. So where does, you know,
104280	112020	the 25 years number come from? And I thought about some more. I realized that 25 years
112020	119180	is actually my involvement with program analysis. 25 years ago, my grandma gifted me a book called
119180	124840	Professor Fortran's Encyclopedia, which had like a cat named X and a professor named Fortran
124840	131420	and a caterpillar named Caterpillar. And it went through their adventures in computer land
131420	137320	and taught kids all about computer. I was crazy about this book. I ditched classes in first grade
137320	143340	to hide in the stairwell and reread this book over and over. And I learned basic from it. And
143340	152800	then later on, moved on to C and so forth. And I realized very early on that if I, you know,
152800	157500	make a standard program that asks me for my name, and my name is too long, my last name, by the way,
157500	165220	is Shorshutashvili. So I actually triggered bugs with this last name, then the program would crash.
166320	170940	That turns out to be a buffer overflow and so forth. And then I started realizing that
171980	177040	it's incredible to look at a computer program, try to understand how it really works. I mean,
177040	183040	that's a whole different world in there, right? A world of ones and zeros and assembly instructions
183040	190340	and logic gates. And somewhere along the line, it becomes alive and it starts behaving in ways
190340	196920	that you don't quite understand ahead of time. And that's manual program analysis,
196920	204100	understanding how this works. But then you can take that and make it automated. You can create
204100	211680	programs that can understand programs. And it's like a whole level of craziness. That's like
211680	217240	creating some sort of intelligence almost. And that's what fascinates me about program analysis.
217240	221960	That's what's kept me going down the path I've been going for the last 25 years.
222300	230300	And so this talk is about, you know, the history of program analysis. And it's about, you know,
230300	234700	why we want to do program analysis in terms of kind of the dry things like we want to,
234700	242180	you know, hack programs, we want to make sure programs are safe. But try to approach this
242690	250120	topic from the sort of kid's perspective of discovering a whole new world of how programs
250120	256380	work, and how programs might understand other programs. Program analysis is useful,
256380	265840	but it's also kind of magical in a way. So with that said, let's move on to technical issues.
267180	274120	Hmm. Hold on. Boom. Success. Let's move on to program analysis.
276320	284780	Program analysis is a field that has basically three parts to it, right? You want to analyze
284940	290740	a program, but you always want to analyze a program to figure something out about it. So
290740	296140	for example, you might want to make sure that a certain specification of the program holds. The
296140	302800	program doesn't crash, the program doesn't allow random people to unlock your smart key for your
302800	308580	house, the program doesn't allow, you know, for your brakes to be remotely controlled, and so on.
309440	315940	And you have a goal that you want to achieve with that specification. So for example,
315940	323200	if your specification is, hey, I don't want this to crash, you could say, I want to make sure that
323200	329620	that specification holds always. Or you might want to say, I want to make sure that that
329620	334420	specification does not hold. I want to find the crash, right? So kind of, are you looking for bugs?
334420	338920	Are you trying to make sure things are safe? And then, of course, once you decide what you want to
338920	344100	do, you need to decide how you want to do it. What is the technique that you're going to use to
344100	354560	reason about a specification on your program? And with these three kind of categories in mind,
354560	360540	I'm going to run you through the history of program analysis and where along the line you
360540	369460	collected different items, kind of different instances of these three categories. And so
369460	376400	we're going to commence on a brief history of computing, starting in the 1830s with Charles
376400	383520	Babbage. Charles Babbage created the analytical engine. It was a mechanical computer, super far
383520	389760	ahead of its time. And he wrote a bunch of programs for it. By all accounts, these programs were
389760	393860	actually really buggy. So that's really interesting. Some of the first computer programs written
393860	399100	were already buggy. We didn't have that term yet. I'll talk about that later. And about a decade
399100	409260	later, Ada Lovelace published a series of notes and more reasoning about this analytical engine,
409260	416780	including the first complex program for one of these machines. So this is Ada Lovelace's
417580	426760	description of the execution trace of this program. And the program computed Bernoulli numbers.
426760	433080	And so she goes through instruction by instruction and says how variables change,
433080	443760	and so forth. And this is in the 1840s, right? And so that's where things started. In 1947,
443760	453620	about 105 years later, we got the term bug. Grace Hopper found, or people working with her,
454440	461120	a actual moth inside a computer causing a computer error. So 100 years roughly after the
461120	468040	first instance of kind of a computer in the semi-modern era, we have kind of this official
468040	477660	term for the word bug. And this very quickly started getting people thinking about how do we
477660	482820	kind of prevent actual software bugs. The moth, I guess, technically was kind of a wetware bug.
483620	491140	But how do we prevent bugs in our software? And one of the kind of early visionaries here was
491140	498680	Alan Turing, who's known for, of course, the Turing machine, but also cracking Enigma and
498680	507340	onwards and onwards. But he also had the first paper that I could find that talked about,
507340	516720	maybe we should try to make sure that the program is correct. And he published this in 1949. So two
516720	524360	years after the kind of invention of the term bug, people start thinking about how do we make sure
524360	530320	that there are no bugs in a program. And this creates the field of program verification.
530360	537020	So program verification says, hey, given a program and the specification, we want to make sure that
537020	543580	there are no bugs in this program, that this program carries out the computation needs to
543580	554300	carry out and so forth, and doesn't go astray. So as an example, here's a simple program that
554300	560940	has a crash, but that crash is not reachable, right? So if you give this program
563800	570800	the somehow three numbers that add up, where a plus b equals c, but then c minus b is not
570800	576900	equal to a, it'll crash. And you want to see if this program is safe. You can run a program
576900	585460	verification on it, which will identify, you know, hopefully that this condition can never
585460	590960	hold, and will produce a proof that tells you that the program is safe according to your
590960	598540	specification. So that's great. And if everything worked like that in the real world, then we might
598540	604500	not have bugs. And in certain cases, it does work like this in the real world. So there is formally
604500	608540	verified hardware, according to a specific specification, of course. So things that
608540	613220	the specification doesn't reason about, you know, maybe still can go wrong with that hardware,
613220	620760	but things that are covered by the specification don't. So there's, you know, hardware that can
620760	627840	provably not contain kind of timing side channel attacks, for example. And there's verified
627840	633460	software. I heard recently that Google, for example, is switching over to a formally verified
633460	642620	crypto system. And so the problems, of course, come when the software kind of gets bigger and
642620	647400	complexity starts getting introduced, and verification stops being able to reason about
647400	653260	things very effectively. So for example, here we have a different type of program, right? And
653260	661400	this program also doesn't crash. That A cubed plus B cubed equals C cubed, that's Fermat's last
661400	668520	theorem, right? That is not solvable. There's no solution. But it took humans hundreds of years
668520	675500	to figure this out. And the computer, you know, verification tools, unless they're special coded
675500	683400	for this, will also be unable to handle it. So it will kind of see this condition, it'll say, okay,
683400	690880	well, if this condition is true, then the program will crash. Can I prove that the condition is
690880	694940	always false? And if it can't prove that the condition is always false, it has no choice
694940	706580	but to alert that the program cannot be verified. So what happens then, right? So then
707110	713760	you have to go in and you have to start looking at the program manually and finding bugs. And
713760	721020	if anyone's used kind of commercially available static analysis tools, you're very well familiar
721020	727680	with the false positive problem, right? You open up a binary or a piece of software and the tool
727680	734520	pops up alerts all over the place. And some of those are actual bugs. But you spend a lot of time
734520	743100	chasing down the alerts. So when we look at, you know, software, open source software, for example,
743840	751860	we often see, you know, a security check done twice, the same check. If A is not null,
751860	758720	if A is not null. Why is that check there twice? Because for whatever reason, a program verification
758720	766600	tool alerted that there could be a bug there and the programmer figured out some hack to make it
766600	776920	shut up. So program verification, while it's a very useful field, like I said, there is verified
776920	784420	software and hardware out there. It's not kind of the final answer on its own. So we also at some
784420	790160	point need to have kind of the flip side. So program verification can say that is safe. But,
790160	795600	you know, after that it says, okay, I can't really reason about it. So the flip side is you have a
795600	801020	technique that says that is not safe. I can prove it. But I can't really prove, you know, if it is
801020	808560	safe. But if it is not safe, I'll give you a counterexample. And the way this works is by,
808560	813880	you know, finding counterexamples to whatever technique. So in this case, with a slightly
813880	823480	different condition, that is A squared plus B squared equals C squared, a technique might find
823480	828640	A, B, and C of three, four, and five as a counterexample that will reproduce a crash.
829320	836840	And so this is, there are many techniques that, you know, we'll mention later that can be used
836840	844500	to do program testing. But one of the very early ones, the first instance of program testing of
844500	850260	this style that I found was in the 1950s, which is about four decades before I thought that this
851700	861600	kind of idea was created. Programmers in the 1950s would program in punch cards. And to test
861600	867880	their code, it was common practice to dig punch cards out of the trash and feed them as input to
867880	875020	your program. This is unexpected input, just random stuff. And of course, by random stuff,
875020	882960	program shouldn't crash. So this is kind of an early example of this sort of
883960	891400	kind of institutionalized testing that we're, you know, only seeing come back recently.
892200	898060	And so, you know, these early developments, verification, testing, the creation of those
898060	906420	two ideas, and then the idea of what you might check for. So the, you know, trash decks check
906420	913020	for crashes. Turing reasoned about ensuring logical properties. But this is all done manually.
913020	918500	So this is kind of our first set of tools in our tool bag, and they're manual tools. We have to
918500	925500	dig them out. Okay, let's get our trash, you know, cards. This one got eaten by something and,
925500	931520	you know, put in the ones that won't jam your machine and try them. But none of this was
931520	939760	automated. And so as computers became more and more ubiquitous, the need for automation started
939760	949800	showing itself. So again, we come on a mention of Grace Hopper here, because she not only found the
949800	957780	first bug, but she also invented the compiler. So Grace Hopper in 1952 published a paper saying,
957780	962120	hey, what if we wrote in kind of higher level languages and it got, you know, squished down to
962120	970660	computer code. And this took a little while to catch on. And the reason it took a while to catch
970660	977880	on, one of the reasons is that the code that was generated by this compiler was very slow.
977880	984920	It couldn't be, you know, optimized as well as handwritten assembly could at the time.
985220	991940	And so this identified another need, another goal of program analysis, transformation of binaries
991940	997480	into something different, something faster in this case. And in the modern era, you know,
997480	1003640	you can transform binaries, for example, or transform programs to ensure that they're safe.
1003640	1008000	Transfer programs to meet any sort of specification around on a different set of hardware.
1010000	1015420	And so now we needed to optimize code because, you know, people write slow code when they're
1015420	1024660	writing source for whatever reason. And another thing happened, people started realizing that
1024660	1031820	we have all of these architectures with all of this data flowing around in them. And if you're
1031820	1038120	not very, very, very careful, some data could be read from or written to a different location than
1038120	1047280	intended, and the memory could be corrupted. So I just played, you know, two days of a CTF full of
1047280	1057280	these errors, of course, memory corruption bugs. And this was first described in 1968. In 1968,
1057280	1063260	there was a paper saying, hey, you know, we noticed that if the kernel, they called it a
1063260	1070380	monitor at the time, blindly trusts everything that, you know, the user space sends it, bad
1070380	1075720	things can start to happen. And so this paper introduced that concept. It's also the paper that,
1075720	1082280	you know, one of the early works in memory protection and virtual memory and so forth,
1082280	1088300	which is pretty interesting that immediately they realized this was a problem. And a little while
1088300	1093780	later, there was, you know, Ken Thompson's reflecting on trusting trust, where he reasoned
1093780	1099060	that you could build in a backdoor into a compiler that could build in backdoors into sensitive code.
1099060	1106290	And so suddenly, we have two more things that we need to worry about in terms of specifications,
1108070	1113170	or even more, but, you know, including memory safety, information disclosure,
1113170	1119810	and authentication, right? So now you might want to verify that a program doesn't have a backdoor.
1120810	1126910	Or you might verify that a program will not leak your sensitive private information. That is,
1126910	1132750	that one is very relevant today with all of our phones having all of our information and
1132750	1140710	very little control over it. And so the need for automation was pretty well established
1141790	1150450	by the kind of turn of the century, for sure. But even before that, those papers were like
1150450	1158690	in the 1960s or something, late 1960s. And very quickly, oops, sorry, very quickly,
1158690	1163510	people started to meet this need. The research community started pumping out
1163510	1168670	automated approaches. And so I'm going to introduce a couple of these automated approaches.
1170610	1177310	And before I do so, I want to make you guys experts in program analysis, so that we can
1177310	1184330	reason about what's going on here. So we'll talk about a couple of prerequisites, right?
1184330	1190830	So let's say you have a program. So on the top right, there's like a little Python program
1192790	1199030	that's already a vulnerable program, by the way. And so you have this program,
1199950	1203950	and you want to analyze it. So you have to reason about a program in a slightly different way
1203950	1210710	as a computer than as a human. So for example, a program is viewed by a computer as a set of
1210710	1217410	basic blocks. A basic block is a piece of code that is executed all together before passing on
1217410	1224570	control flow somewhere else. So in our case, we'll pretend that input is part of this basic block. So
1224570	1230250	this X equals input is a single basic block, and then it might branch depending on the input,
1230250	1237010	right? So if you put in 42, it'll do one thing. If you put in something else, it'll do another.
1237010	1242350	And so this is the end of the basic block, and then the result of that if statement is two other
1242350	1251150	basic blocks, the true case and the false case. And each of these has a constraint that has to
1251150	1256870	be met, right? In order to go down one path, you have to put in 42. In order to go down another,
1256870	1263110	you have to put in not 42. And so these are called constraints. If you then look at all of the basic
1263110	1268710	blocks of a program, this is called a control flow graph because that's, you know, these are
1268710	1276570	basic blocks and control flows between them. And if you look at a path down this graph, now in this
1276570	1282050	case the graph is nice and simple, but in reality it could have loops, a path might hit the same node
1282050	1289490	multiple times, and so on. And then you look at the constraints all along this path. These are
1289490	1296290	the path predicates that uniquely identify this path. So if you collect all of these constraints
1296290	1307770	and you say that x is not 42 but x is 1337, that input will... any input that matches those
1307770	1314170	constraints will take the same path through the program. So these are kind of the prerequisites
1314170	1320150	of our program analysis techniques. So now we'll go through three of them. Actually for time reasons
1320150	1325670	we'll go through two of them and mention a third. But we'll start with symbolic execution. And
1325670	1331130	symbolic execution got a lot of hype recently, especially with stuff like the cyber grant
1331130	1341570	challenge and so on, as a way for machines to reason about program code. And you think that
1341570	1348430	this is something new, but it turns out that it's not. It's quite old actually. So 40 years ago,
1348830	1353890	a little bit more, the first symbolic execution paper that I could find was proposed.
1355290	1359630	And there were several actually, right in one year basically. So this idea
1360370	1366730	had some prerequisites that were met and then it exploded. And so the symbolic execution engines
1366730	1372710	were developed that could then symbolically execute Fortran code. Now we have symbolic
1372710	1378210	execution engines that you can download off GitHub that can execute binaries, Java,
1378910	1388530	Android, whatever you want. And this is kind of a fairly powerful way of reasoning about programs.
1388530	1393090	And let's run through an introduction of it with a slightly more complicated
1394550	1399530	example than what we learned about program analysis with.
1400970	1409570	Hold on. So here's a program that has several bugs, right? One is it,
1410370	1416870	well, it definitely has one bug and that is if you enter a username of service
1416870	1423070	and a command code of seven, it'll crash, right? You can imagine that instead of this crash,
1423090	1427910	it does some system relevant stuff. So it could be a backdoor instant memory
1427910	1431910	corruption, but basically we have a program with a bug and we would like to find this bug.
1431910	1436210	So let's see how we might approach this problem with symbolic execution, right?
1437290	1442930	So first let's look at the control flow graph of this program, starting from the beginning. Again,
1442930	1448570	the control flow graph is made up of basic blocks and the control flow transitions between them.
1448570	1454750	So we start at this input. If the input is service, it branches. If the username is service,
1454750	1463690	it branches into one side of the equation of the program where it checks for the command code
1463690	1470950	being seven. And if it is, it crashes. If not, it prints an on command and then it exits.
1471750	1479970	On the other side of the branch, it asks for a passcode. If the passcode is invalid,
1479970	1484970	it prints invalid passcode. Otherwise, it calls the authentication function,
1484970	1489250	then it prints exit and exits. So this is our control flow graph. And how would
1489250	1496350	we use symbolic execution to try to find that crash, try to find the ability to reach this crash?
1496350	1504590	So we would start executing the program in an emulator. And in this emulator, instead of ones
1504590	1514210	and zeros, we are working on X, Y, and other symbols. So, for example, username is a symbol
1514210	1522010	in this emulator. So when we pull in an input, it will produce an unconstrained variable called
1522010	1527090	username. And we don't know what username is. Username can be anything. Because, you know,
1527090	1536670	that is how we implement our emulator. And as we check for values of username, we
1536670	1542330	create what are called constraints, right? So these are the exact constraints that the basic
1542330	1549270	block checks when it pushes execution forward. And we collect two constraints in this case.
1549270	1553630	On the one side, username equals service. On the other side, it does not equal service. These are
1553630	1560810	constraints. And we continue executing both sides of this statement with symbolic execution.
1561530	1568090	But then we hit something that is very bad for symbolic execution. We hit, for example,
1568090	1574870	string processing functions. So a string processing function like A2I might check a
1574870	1580910	given byte, see if it is a number. If it is a number, it'll do one thing. If it's not, it'll
1580910	1586390	do another, right? Or if it's a certain number, it might do one. If it's not a certain number,
1586390	1591950	it'll do another. So it'll keep branching the symbolic emulator at every...
1593810	1602050	I'm sorry? Oh, awesome. It'll keep branching the symbolic emulator at every if statement,
1602050	1610390	every check on this number until we get into a situation where there are just too many checks,
1610390	1616230	right? So it'll try to check for a command code of, you know,
1617330	1624910	10 bytes with arbitrary values of 9 bytes and so on. And there's an exponential path explosion
1624910	1631730	here. And there are some approaches in the symbolic execution world to try to deal with
1631730	1636730	this sort of path explosion. But in general, these approaches boil down to the requirement
1636730	1643210	to remove paths, right? And lose this information. And so in the end, the symbolic execution engine
1643210	1651990	might only find certain paths and not others and miss the bug. Because we have to simplify
1652450	1658850	to keep things tractable. So that's symbolic execution. It might find the bug if you're lucky,
1659090	1667190	but if you're not, and it has to simplify its state space essentially as it's executing,
1667190	1673890	there are certain things that it might miss. Symbolic execution was proposed as you saw in
1673890	1685590	1975. But the interesting thing is that in Ada Lovelace's notes from 1942, we can see
1686310	1694610	that she did a symbolic trace of the execution of her program. So where she has this sort of
1694610	1703750	program execution log in her notes, she writes down the symbol, the equation for each variable
1703750	1711470	as the program progresses. And this is the first symbolic trace of a program, done in 1842 before
1711470	1719290	computers were invented. So that was a pretty interesting thing to find. And kind of a nice
1719290	1729190	point of history that we can put as the start of program analysis in this sense. 1842, not 1975 or
1729190	1738570	1949 or any of those much, much later dates. And so symbolic execution is a kind of tool in our
1738570	1746430	tool set that we can use to achieve either verification of a specification, the testing of a
1746430	1754650	specification, or to support the transformation of code. Of course, how you verify a program, for
1754650	1760910	example, using symbolic execution is a pretty complex thing with the symbolic execution style
1760910	1765930	that I described for program testing. It generates inputs to find vulnerabilities, but it can't prove
1765930	1775630	that there aren't any. But that's kind of a, we could go down that route for an entire semester
1775630	1782590	course, and not just a 45 minute introductory talk. So I'll leave that as future research for
1782590	1790910	you guys. And we'll go on to static analysis, because static analysis can show that, for example,
1791250	1800850	a program is immune to a certain class of vulnerabilities or properly implements a certain
1800850	1809410	specification. Static analysis, or at least one of the very common ways to do static analysis,
1809410	1815950	which is abstract interpretation, was proposed in 1977, shortly after symbolic execution.
1815950	1830610	And abstract interpretation looks at the control flow graph, the program as a whole, essentially.
1830610	1840010	It doesn't have to figure out how to reach a certain basic block to reason about properties
1840010	1845770	of the program at that basic block, which is the problem that symbolic execution had, and the reason
1845770	1853010	that symbolic execution could not find the bug in our example. And so static analysis with
1853010	1858070	abstract interpretation might be able to find that bug, probably will be able to find that bug if it
1858070	1865610	is a simple bug, or actually, I should say, will be able to find that bug. Because if it cannot
1865610	1872350	prove that a piece of code is safe, it will raise an alert. The problem is that, as we talked about,
1872350	1877570	there are false positives. So in our program, it might think that that function has a bug, even if
1877570	1885350	it doesn't, alert on that, and eat up time for a human to go through and verify all these results.
1886670	1893350	And so through abstract interpretation, we got a way to perform verification or specification
1893350	1904150	on a program in 1977, 40 years ago. And then we moved on to fuzzing. And so fuzzing is kind of one
1904150	1910890	of the biggest ways to find vulnerabilities in modern software. And it's actually surprisingly
1910890	1918670	simple. Proposed in 1981, initially, in its most basic form, which was, hey, let's throw random
1918670	1924950	input at a program and see if it crashes. It's evolved quite a bit since then. And so the
1925710	1932490	specific implementation of fuzzing that I'll describe here is implemented by American Fuzzy
1932490	1940250	Lab. It's a fuzzer that mutates its inputs when it detects differences in program execution.
1940970	1946630	And so let's see how it would run on our program here. We'll start with
1947470	1954850	some randomly generated test case or human-seeded test case. So a human might input a username of
1954850	1964070	ASDF and a password of 111. And that input will trigger certain basic blocks to be executed,
1964070	1969770	right? And so we executed the username input, of course. We executed the check for the
1970470	1977190	kind of service username that could lead to the crash. We did not satisfy it, so we took the else
1977190	1986690	branch, read in the passcode, and errored out on an invalid passcode and exited. So we triggered
1986690	1993690	those basic blocks. And so the fuzzer starts mutating this input, right? It might make a
1993690	2006730	capital D or something. Or it might mutate input in a way that passes another check. So it mutated
2006730	2014230	the input in a way that passes that passcode check. It actually doesn't, because there's
2014230	2019070	another bug in the program that I didn't notice before. But let's pretend that it
2020790	2029630	creates a valid size of a passcode and triggers that authentication function, right? So that
2029630	2036690	is a new basic block that it triggers. Now with fuzzing, we've triggered a block that symbolic
2036690	2045470	execution before us wasn't able to trigger just through randomly mutating inputs. But then we get
2045470	2053610	stuck. Because through randomly mutating inputs, we are unable to satisfy complex checks, right?
2053610	2060210	We need a username of service, and we have randomly mutated usernames. The chances are
2060210	2065330	that we'll create the string service are very slim, and we'll keep guessing and guessing and
2065330	2070770	guessing. Of course, in this specific case, you could seed the fuzzer with the string service,
2070770	2077370	and you could scan the program for all of its strings and automatically
2078050	2083350	make the fuzzer pass this check. But string checks aren't the only complex checks
2083350	2092010	in programs. There are hashes, there are complex input formats, dependencies between
2092010	2100230	different input bytes. And so in the general case, we cannot guarantee, of course, with a fuzzer
2100230	2108850	that we will find code that is protected by these complex checks. And so fuzzing
2109970	2117990	with this random data, of course, is a throwback to this trash deck concept. So fuzzing
2117990	2128090	was invented in 1950, not 1981. Or at least manual fuzzing. So again, we find that
2128090	2134810	program analysis techniques harken back to the middle of the century, and sometimes even
2134810	2141970	farther to the 1800s. So now we have this kind of toolbox of a bunch of specifications that we
2141970	2150710	can enforce, some goals that our analysis could try to achieve, and the technique through which
2150710	2159490	we can achieve them. And next, we should evaluate them, understand how these different
2163410	2169430	goals or how these different techniques help us achieve program analysis goals. And
2171310	2180670	as a scientist, if you start thinking up an approach, and you have a cool idea,
2180670	2186870	have a new program analysis technique, you want to evaluate it. So you want to take
2188090	2193430	programs and specifications, feed them into your technique, and see how well it does. This is my
2193430	2202510	clipart slide, by the way. So hopefully my clipart game is on point. So you run your analysis,
2202510	2207910	you get your results, and you're happy, right? Well, there are a number of problems with this.
2207910	2215350	One is that the analysis is more complex than you might think, because any program analysis
2215350	2221390	has to deal with the environment of the program it's analyzing. And environments like Windows,
2221670	2231250	macOS, Linux, mobile devices, cyber-physical devices, they have really complex environments.
2231730	2240090	Having to model this environment, even before understanding how much better your program
2240090	2247870	analysis technique is than previous techniques, is quite a big investment that people often don't
2247870	2257290	make, or make in a very fast manner. Another problem is finding a good data set to evaluate
2257290	2263130	on, right? There's not really a good data set for... or there wasn't a great data set for
2263130	2266990	vulnerability research. This problem is actually being worked on, and new data sets are being
2266990	2274390	created, as I'll talk about. But you need a good data set with ideally known vulnerabilities,
2274390	2280810	so you can reason about how many of them your bug finds, and with variable vulnerabilities
2280810	2291030	in different programs, so your technique isn't just specifically tailored for one style of
2291030	2295890	vulnerability that you happen to detect, or one specific program that you happen to be very good
2295890	2300810	at analyzing. Another problem is these specifications. Specifications in the real
2300810	2309790	world are very complex, whereas specifications that you might want to use for your analysis tool
2310470	2315970	initially in the early stages of development need to be fairly simple and ideally pre-provided. I
2315970	2323710	mean, if you're looking at analyzing something like Chrome, and your specification is that Chrome
2323710	2331130	should not leak user data to someone that should not get this user data, that's impossibly broad
2331130	2338730	and impossible to enforce. Is your bank allowed to have your various cookies or something along
2338730	2345650	these lines? So you need a very good data set, standard data set, with a very good
2345650	2352850	standard implementation, standard set of specifications, so that you can analyze
2352850	2361230	how well your technique runs compared to different work, and compare those results
2361230	2367230	to prior work, because that's a very important part of science and technical development,
2367230	2377870	understanding if you're making an improvement or not. And so we had problems as an analysis
2377870	2383090	community in meeting all of these goals, and having a good data set of applications,
2383090	2391600	with a good set of specifications, on a good environment that wasn't too complex to implement,
2392090	2396310	and that could produce results that we could reason about, that we could say,
2396310	2404090	we have ground truth data for the vulnerabilities that are actually in this data set, and we detect
2404790	2413230	x of them, or we manage to rule out vulnerabilities in code that has no vulnerabilities,
2413230	2422910	and so our program verification is correct. And so, like I mentioned, there have been
2422910	2429790	several projects to solve this problem, to create these data sets, and one of these
2430750	2434970	projects essentially was the Cyber Grand Challenge. If you guys were here last year,
2434970	2444170	you probably saw all of the Cyber Grand Challenge, you know, hoopla, where for the first time in
2444170	2451070	history, seven fully automated program analysis machines fought each other in a game of capture
2451070	2458450	the flag. So the same game that I spent the entire weekend playing and only got four hours of sleep
2459050	2466270	is the game that machines who didn't need to sleep played yesterday, last year rather. And
2469170	2473510	these machines, of course, suffered from all of the problems that I just talked about,
2473510	2477870	the environment problem, the specification problem, and so the Cyber Grand Challenge
2477870	2487630	created a custom operating system with very specific vulnerability specifications to
2489470	2494250	evaluate these systems. Now, for the actual competition, of course, the source code wasn't
2494250	2500610	available, since then they released the source code, so you can really look at the source code,
2500610	2506830	understand the vulnerability, and then reason about how well your tool finds certain
2506830	2516590	vulnerabilities across 249 binaries. So it's a 249 program, so it's a data set that has a
2516590	2524910	lot of variants, there are a lot of different types and different difficulties of vulnerabilities,
2524910	2533970	and all of this is freely available for people to analyze their tool and understand how well it does.
2533970	2541750	That's my other piece of clip art. So the Cyber Grand Challenge and the programs that
2541750	2547530	were written for this, these 249 programs making up the Cyber Grand Challenge data set,
2548790	2556390	allowed us to do what I call, to perform what I call the program analysis nursery experiments.
2556390	2560310	So I see the Cyber Grand Challenge data set as the nursery of program analysis, where you can
2560310	2566170	reason about different improvements for different techniques. So for example, Symbolic Execution on
2566170	2571550	the entire data set finds nine vulnerabilities. This Symbolic Execution has implemented in
2571550	2576910	Anger, which I'll talk about in a sec, which is an open source binary analysis program available
2576910	2585090	on GitHub. But Symbolic Execution on this entire data set finds nine vulnerabilities. That's not a
2585090	2590550	lot, right? There are vulnerabilities in every program of these 249 programs. It only finds
2590550	2596210	vulnerabilities in nine of them. Of course, we can optimize Symbolic Execution using various
2596210	2602250	tricks. So Anger supports, for example, concrete execution when there is no uncertainty in the
2602250	2608610	state, when all the data is known, or when there are constraints on all the data. And so by
2608610	2618450	utilizing all of these optimizations, we can get 26 crashes, crashing 26 binaries.
2618950	2625730	Utilizing static analysis, so combining Symbolic Execution with Veritesting, which is a technique
2625730	2632230	created by Carnegie Mellon University that utilizes parts of abstract interpretation to help
2632230	2637470	with Symbolic Execution, we can push that up to 31. So I should say here that this is our
2637470	2644050	implementation of Veritesting, and it might be suboptimal. So their Veritesting might be
2644050	2650970	considerably better. But that all pales in comparison with fuzzing. This seemingly simple
2650970	2658810	technique of throwing random data and seeing what happens finds 106 crashing binaries in the data
2658810	2666350	set. So this gives us an idea of where program analysis is, and this is a good showcase for
2666350	2673470	the simplicity of evaluating all of these different analyses on this data set and reasoning about
2673470	2680230	the results. Because now we can look at them in terms of, you know, summation numbers. But
2680230	2686570	there's also some subtleties there. They find bugs, different types of bugs in different binaries. So
2686570	2695570	we can look into that and make inferences and try to create new approaches. So for example,
2695570	2700650	we looked at the difference in the types of bugs that symbolic execution found and that fuzzing
2700650	2707950	found, and we realized that they have very different coverage. So the types of the basic
2707950	2712150	blocks that fuzzing can trigger are very different from the basic blocks that symbolic execution can
2712150	2716830	trigger. And sometimes one finds a crash, sometimes another finds a crash, sometimes neither of them
2716830	2724770	find a crash. In fact, very frequently. And so we decided to combine them in a very
2726690	2733410	straightforward but careful way. And the idea is as follows. We begin by fuzzing the binary. This is
2733410	2738410	nice and fast and achieves decent code coverage, but as we discussed, eventually gets stuck
2738410	2745050	on complex checks. And then we use symbolic execution to find the constraint that can get
2745050	2750890	through that check, but possibly suffer from other limitations of symbolic execution and not
2750890	2755590	find the actual bug. And then we synchronize that back into the fuzzer. We synchronize the
2755590	2762890	knowledge of how to bypass the complex check back into the fuzzer, and it continues to mutate that
2762890	2770390	input until it finds the bug. And we do this iteratively, back and forth between the fuzzer
2770390	2778910	and symbolic execution. So by implementing symbolic assistance for the fuzzer in this way,
2778910	2787990	we achieve on this entire dataset 118 crashes. And an interesting thing here is that we have
2787990	2795970	this 118 crashes in the dataset, but if you take the union of all of the different techniques,
2795970	2801170	all the different techniques have crashed something like this from memory, 150 to 160
2802010	2809010	binaries. So there's a lot of uniqueness between the different techniques used here.
2810490	2816030	Driller, the applicability of it varies by program, whether or not there are complex checks.
2816390	2821810	We can run through a quick run here. So this is a control flow graph of an example program,
2821810	2831890	and driller starts in the top left corner there. And it first starts by fuzzing a portion of the
2831890	2837990	program before it gets stuck in a complex check. And this is the basic block coverage on the right
2837990	2845070	in this graph here. You can see just kind of flat lines, right? So then if we run the symbolic
2845070	2853910	execution engine to help the fuzzer reach extra code paths, it allows us to find just a little
2853910	2860310	tiny bit more basic blocks in the executor, some more basic blocks in that control flow graph of
2860310	2867210	the program. And if we keep invoking the symbol execution engine to help the fuzzer get unstuck
2867210	2874450	every time that the coverage flat lines like that, we eventually drill into the program deep
2874450	2879930	enough to find the bug. And so that's how driller works. There's still a lot to do. So there's this
2879930	2889230	whole red area that however many programs, about 100 programs that we have never exploited before,
2890050	2896230	that we've never crashed before using any automated techniques, which represent missing
2897830	2902870	tools basically that humanity has yet to develop, or at least we have yet to develop.
2903730	2909010	And so if this all sounds interesting to you, I encourage you to join in. There are several ways
2909010	2913610	that you can participate in program analysis research. One is you can contribute to open
2913610	2920250	source frameworks. So we run one of these frameworks in our research lab. It's called
2920250	2927790	anger. Go to anger.io. There's a lot that we could use help with documentation, environment support,
2927790	2935430	better techniques, and so on. And the other is that I'm always looking for students. I'm starting
2935430	2945650	as a professor at Arizona State University. I just finished a PhD at UC Santa Barbara, and both of
2945650	2951610	those places are incredible places to go to if you want to do research in program analysis or in
2951610	2960090	general security. So if you're thinking about graduate school, or you are curious and want to
2960090	2966850	do an internship, reach out to me. I'll put you in touch with the right people, and you could explore
2966850	2973690	that. The presentation is available online. All my contact info is right here, and I guess we'll
2973690	2977590	questions in the hall? Awesome. Thank you, guys.