start	end	text
0	3380	Thanks everyone for attending my talk.
3400	7360	So those of you who don't know me,
7360	10880	I am a PhD candidate at Arizona State University.
11180	14380	I am specializing in use of artificial intelligence
14380	18560	and machine learning in the field of cybersecurity.
18560	22200	I'm also a security consultant at Bishop Fox.
22340	24160	I also authored a book
24160	27820	known as Software Defined Virtual Network Security,
27820	31620	dealing with security in software-defined systems.
31840	34720	I'm also co-founder of DevilSec,
34720	37760	which is the hacking club that we have at ASU.
37760	40820	I've worked for BlackBerry and Public Services
40820	43460	and Computer Sciences Corporation in the past,
43460	46920	and this is my contact information.
46980	51280	So let's dive into the overview of the talk.
51280	53940	What is the motivation?
53940	58180	What would be the overview of our system, ASAP?
58260	62500	There are three main modules in the system.
62500	69040	Stringer, which is used for the discovery of information,
69040	72100	both services and vulnerability in the network.
72100	76300	Americano, which is used for analysis of different paths
76300	78960	an attacker can take in a network.
78980	81800	And Cappuccino, which is our AI-based
81800	85180	autonomous attack plan generator.
85900	89120	And finally, we do the validation of these attack plans,
89120	91500	and we will jump into the demo
91500	96260	after the end of the presentation.
96920	100880	So let's see what is machine learning.
100880	104740	It's a statistical way of learning from the information
105520	106940	present in your network,
106940	109420	what kind of network traffic you have,
109420	111520	the logs on your system.
111520	114820	So using some pattern recognition
114820	118880	to identify some attack patterns in a network,
118880	121820	we use machine learning techniques for that.
121820	124900	And it's already been used successfully
124900	128080	in things like spam detection on your email.
128080	132360	You don't get those Nigerian Prince emails these days
132360	136360	because of amazing job done by spam detection systems.
137080	139300	Artificial intelligence, on the other hand,
139300	145120	it perceives the network traffic,
145120	147120	what kind of activities are going on
147120	148920	within the environment,
148920	152760	and uses that information to take some decisions.
152760	156360	So it basically acts on the information
156360	159560	provided by different agents in the environment.
159560	163620	You can think of smart ideas
163620	167800	that we can design in a system,
167800	169620	which basically collects information
169620	172060	from different parts of the system,
172060	173740	updates its beliefs,
173740	178100	and then takes some intrusion prevention system measure.
178100	182080	So that is something that we can take the help
182080	183680	of AI in designing.
185740	188080	AI and machine learning have found
188080	191860	some useful applications in cyber security,
191860	194180	both in industry and research.
194180	197680	We use the attack patterns for detection
197680	200420	to basically identify malicious actors
201040	202880	within our network.
202880	207420	And we try to see if the attacks
207420	209080	are very stealthy in nature.
209080	211640	There are attacks like advanced persistent threat,
211640	215340	which are basically slow and low kind of attacks,
215340	216860	which are hard to detect.
216860	221020	They are carried over like multiple days.
221620	223480	A good example is Sony hack,
223480	227360	which was carried out over a period of several months.
227380	229680	So identifying some valuable patterns
229680	232960	from those kinds of attacks is some place
232960	236780	where we can definitely utilize machine learning.
237500	242880	And there are recent investigation
244500	249860	in the use of AI to design deception based system,
249860	253740	moving target defense and cyber deception in general
253740	258500	are two ways or two fields of research
258500	263440	that explore how to identify the attack patterns
263440	266740	in a network and basically use that information
266740	270800	to present a fake view of the network to an attacker
271180	274380	and deceive him into honeypots
274380	278360	where they can do further analysis of his attack intentions.
280640	285120	So why do we need AI and machine learning in cybersecurity?
285120	287780	And I did some background research
287780	292120	and it's estimated that there will be 25 billion IoT devices
292440	296000	in US by 2021.
296000	299380	And the investment in cybersecurity
299380	302760	will be up to a trillion dollars.
302980	306000	With penetration testing, if we look at it,
307000	310580	market size would be 3.2 billion US dollars.
310580	314880	So the number of devices are growing perhaps
315890	317680	at a quadratic scale,
317680	322400	but we have a shortage of cybersecurity workforce.
322400	326220	It's estimated that 65% of the organizations
326220	330340	feel that their staff is not very well equipped
330990	334500	in cybersecurity.
335110	338560	And 36% of the organization reported that
338560	342400	there is a lack of training
343470	349160	or skills in existing cybersecurity workforce.
349340	352700	So that is where we plan to use AI
352700	355470	to kind of bridge this gap.
359090	361720	So if you look at a very practical example
361720	366620	of application of artificial intelligence,
366620	369440	DARPA Cyber Grand Challenge was one place
369440	373020	where AI was successfully applied.
373020	376660	There were seven participating schools, seven or eight,
376660	380540	who took part in a hacking competition
380540	384680	where each school was trying to
385840	388780	target the infrastructure of everybody else
388780	392020	while keeping its own infrastructure secure.
392040	394420	So the important catch was that
394420	397060	all of these participants were AI,
398020	400980	not AI, but autonomous systems.
400980	407240	And there was no human involved in this competition.
407240	413200	So Mayhem, which was a company based out of CMU,
413200	416240	they automated what white hat hackers could do.
416240	418840	So they found and exploited the weaknesses
419900	422380	present in these system.
422380	425460	What they did was they created a mathematical model
425460	429080	of the paths that attacker can take.
429080	430820	And then they used two techniques,
430820	433220	symbolic execution and fuzzing.
433220	437460	So symbolic execution was the way
437940	441780	to point out interesting code paths.
441780	445400	And fuzzing was kind of a hammer,
445400	447760	which was hammering through those code paths
447760	451120	to exploit the vulnerabilities pointed out
451120	453740	using symbolic execution.
453780	456220	And they won this championship
456220	461560	and they managed to find 14,000 vulnerabilities
461560	464440	on Debian system as well.
464440	467060	And 250 of these vulnerabilities were new.
467060	472420	So imagine a human attacker trying to do all this,
472420	474940	it's kind of difficult.
474940	477800	And this shows a successful motivation
477800	481420	to use machine learning and AI
481420	483320	in the field of cybersecurity.
484520	487980	So if you look at our system, ASAP,
487980	490320	there are four main modules.
490320	494280	Stinger, which is, S stands for scanning.
494280	497180	So we use Stinger for scanning and Recon.
498060	501640	The information from Stinger is fed into Americano.
501640	503940	So A stands for attack analysis.
503940	509740	And we use this information from Americano
509740	514500	to identify the attack states in the network.
514500	519580	Latte, which is a module, L stands for log here.
519580	522740	So it's a module which identifies network and host logs
522740	526240	to gather the threat evidence.
526240	530660	And Cappuccino, which is kind of the network controller,
530660	534700	it takes all this information from Americano and Latte
536920	544440	to encode in form of AI model, Markov decision process.
544720	547880	And based on that model, it identifies some attack plans,
547880	552060	like if a penetration tester were to test
552060	554420	or attack this network,
554420	559720	what kind of plan would yield him maximum output?
560660	564280	And eventually we can execute these attack plans
564280	566100	on a cloud or web application
567560	571440	and update the risk score and attack graph,
571780	573780	which is basically Americano.
575760	580340	So I am addicted to caffeine.
580340	583040	That is why I chose the name of these modules
583040	586460	based on different kinds of coffee flavors.
587360	590460	So let's dive deeper into Stinger.
590460	593360	So Stinger basically scans the network topology
593360	598040	for service information and discovers the vulnerability.
598040	603240	So we have automated Nessus and OpenVS APIs
603240	608060	to identify this attack information.
608820	612300	And this attack information is then fed into Americano,
612300	615000	which is attack graph generation tool.
616540	621040	So let's look at one of the known vulnerabilities.
621180	624420	This is a shell shock vulnerability,
624420	626980	and there are different parameters
626980	629120	from common vulnerability scoring system
629120	630980	for this particular vulnerability.
630980	635320	Like you need just a network access to execute this attack
635320	638260	and it has a low access complexity.
638260	642660	So you don't need to do a lot of investment
642660	646100	as an attacker to exploit this vulnerability.
646100	650860	So this is a example of kind of vulnerability
650860	654140	where we can implement some sort of automation
654140	659820	once we are able to identify this vulnerability.
661440	664580	And the reason of providing this information
664580	666920	is that we will see later
666920	669720	that how we can use these CVSS parameters
669720	673120	like access complexity and possibly CVSS score
673910	678630	to encode the information in our AI solver.
680270	681760	These are some other parameters
681760	685420	like the impact on confidentiality,
685420	688920	integrity and availability is very high
688920	692900	and attacker can take full control of that system
692900	696280	if he were to exploit this particular vulnerability.
697380	701060	So let's take a look at a motivating example
701320	703520	where attacker is located on internet
703520	707420	and his goal is to reach this database server
707420	713000	and exfiltrate the information out of database server
713000	715660	to his command and control center.
715660	717840	And there are some publicly known vulnerabilities
717840	721120	on these machines.
723640	727320	So basically attacker is trying to exfiltrate
727320	729400	the information from database server
729400	731980	but there is a firewall on his way
731980	735660	so he cannot directly access this database server.
735660	739180	So he either needs to go through the web server
739180	743300	or wait for internal user
743300	745900	to download some of his malicious code
745900	751160	and use that as a pivot to go to the database server.
752460	755960	And the web server is the only publicly available service
755960	757120	in this network.
758500	764300	So attacker can try to exploit the web server
764300	765500	using unknown vulnerability
765500	769620	or he can have some malicious script on a popular website
770310	771660	that a user downloads
771660	775540	and that way he can gain access into his workstation.
776280	781160	And using this, he can then take advantage
781160	782640	of the access control list
782990	785520	which basically allows any network traffic
785520	787480	from web server to go to database server
787480	792420	or any workstation traffic to go to database server.
792420	796820	And that way the attacker can exploit
796820	798900	the SQL injection vulnerability
799940	802000	that is present on the database server
802840	806580	and then use it to gain persistent access
808060	812300	to his command and control center.
813220	816380	So you will see that there are two attack paths
816380	821280	in this small network to achieve the same goal.
821300	824000	So imagine a very giant network
824000	827080	with tens of thousands of instances
827080	831580	and you are asked to perform a penetration test
831580	835380	for that network in a limited period of time.
835720	840160	So you need some kind of autonomy or automation
840860	847420	in that particular case to be able to have a good coverage
848120	850120	in your penetration test.
852400	855040	So we saw this example, but what about it?
855040	858920	Like, how do we basically use this information?
859000	862600	We can do some initial attack analysis based on this example
862600	864780	and see that attack is multi-stage
865660	868200	and attacker had specific attack vectors
868200	869200	for this vulnerability
869200	871160	and he went through multiple hosts
872640	875520	and he circumvented some of the defenses
875520	877280	that were present on the systems
878240	882040	to achieve his goal of data exfiltration.
884200	891360	So let's discuss on kind of a philosophical level
891360	895560	why AI can be used to hack faster.
895560	899520	So imagine you are going home on a particular day
900520	903780	and you decide to take a turn
903780	907340	on Arizona Avenue and Main Street
908440	912860	and you have been taking this route forever
912860	914680	to reach your home,
914680	919920	but you encounter a traffic jam on the way.
921480	924260	So you went by your intuition
925020	928020	and this got you into a traffic jam.
928020	931800	But if you had a GPS to help you navigate,
931800	934020	you could have avoided that jam.
934860	937160	So similarly, as penetration testers,
937160	940780	when we try to go after certain vulnerabilities,
940780	948040	we have kind of a preset methodology.
948040	951360	So we will go through some authentication issues,
951360	952640	authorization issues.
952640	957140	We will see if we can use the user management in some way.
957180	960060	We can try to see if we can get
960060	962580	horizontal privilege escalation.
962640	965000	We go after data stores.
965380	967480	We go after application logic.
967480	971300	If it involves the code review,
971300	973200	we go through the procedure of code review
974220	977340	and use all of that to see like
978000	980220	what's the maximum we can get
981180	983940	in this penetration test.
984360	987560	But here is the challenge.
987560	992900	Like if you have say 20 hours for a particular assessment,
992900	994180	do you think on
996340	1000780	environment where you have to do pen test on application
1000780	1008040	as well as the cloud part of the backend components,
1008360	1011200	it's very challenging to get a good coverage
1011700	1013920	in that scenario.
1014640	1020680	So AI and machine learning can act as kind of
1022040	1026300	navigator for us on these assessments.
1026300	1032380	So we can think of ASAP as kind of AI based GPS
1032380	1036280	to navigate the attack surface.
1036280	1040460	And it may not work on all kind of unknown vulnerabilities
1040460	1043380	like say data encryption issues,
1043380	1046320	which you identify which is a vulnerability,
1046320	1050080	but it can help us in semi-automating
1050080	1054820	some of the tasks that we may miss out.
1054820	1056740	So the worst thing would be that
1056740	1061360	there is a very low complexity vulnerability
1061360	1063060	that was present on the system,
1063060	1066560	but you just ran out of time on your pen test
1066560	1070520	and you couldn't exploit that vulnerability.
1070520	1072920	And later the client finds out,
1072920	1074360	hey, why did you miss it?
1074360	1077300	So then you are in a tough situation.
1077540	1079460	So that is another kind of motivation
1079460	1082160	to develop this kind of a system.
1083900	1088120	So with Americano, we get the information from Stinger
1088900	1092440	and we use these vulnerabilities and software configuration
1092440	1099940	to pass to a first order logic based framework.
1100160	1102160	And that framework basically generates
1102680	1105680	a multi-stage multi-hop attack graph.
1105680	1107860	And attack graph basically shows that
1108430	1111400	different paths an attacker can take in a network
1111400	1116060	to be able to reach his desired goal.
1117980	1120720	So if you look at the definition of attack graph,
1120720	1122440	we have some nodes and edges
1122440	1126000	which are a property of a given graph.
1126040	1129080	There are some fact nodes, NF.
1129400	1131240	Fact nodes will be something like
1131240	1132880	the existence of vulnerability
1132880	1137800	or the existence of network connections.
1138260	1144280	And conjunct node are denoted by NC,
1144280	1146340	the disjunct node are denoted by ND
1146340	1149080	and root node, which is basically goal of attack
1149080	1151260	is denoted by NR.
1153020	1157520	So conjunct node can be something that you can achieve
1159450	1164380	based on your initial exploitation of certain vulnerability.
1164380	1167400	So you have some fact nodes that you combine
1167400	1168980	with these interaction rules
1168980	1171540	that we provide in first order logic
1172010	1177360	to achieve some other conjunct nodes like exact code.
1177360	1181220	So suppose there is a vulnerability buffer overflow
1181220	1186060	on web server and the attacker can access the web server.
1186060	1188600	So if attacker is located on internet,
1188600	1193120	then that can lead to execution of code on web server.
1194640	1197140	And based on that example,
1197140	1198840	the root node in our case would be
1198840	1202000	to gain a root privilege on database server.
1204290	1207770	So there are two kinds of edges.
1207770	1212130	E pre denotes the precondition edge
1212130	1215670	and E post denotes the post condition edge.
1215870	1218450	So a precondition edge basically combines
1218450	1221350	the fact nodes and conjunct node to
1224070	1227170	show that the next possible state
1227170	1229850	that an attacker can achieve.
1230030	1233990	And E post means the edges that are triggered
1234590	1237310	if some preconditions are satisfied.
1237770	1240310	And we have some base initial condition nodes
1240310	1245530	in this attack graph that we can denote using NI.
1248900	1251900	So to simplify this, we have some advisories
1252960	1257300	that we identify based on the scanning of the network.
1257600	1259800	We have host configuration information,
1259800	1262740	we have network configuration information.
1263080	1266160	The principle indicate like
1266560	1270040	who has ownership on which machine.
1270600	1273500	And we use interaction rules and policies
1273500	1276760	to provide input to this attack graph
1276760	1279180	with reasoning engine,
1279180	1283940	which then generates attack graph for us.
1287040	1289980	So before going any further,
1289980	1300020	let's look at some information of these Mulwell rules.
1300020	1302460	Mulwell is basically a reasoning system
1302920	1305040	which encodes this information.
1305040	1307440	And it's a work by University of Kansas,
1307440	1314100	which we kind of used in our development of the ASAP system.
1314100	1317940	So advisories show that what kind of vulnerability exists
1318520	1320820	in the network, vulnerability property,
1320820	1323360	host configuration shows that the web server
1324140	1328500	has Apache software, it's running on port 80.
1328960	1330720	This is the daemon.
1330780	1334920	Network configuration is basically the access control list,
1334920	1338960	which says that from internet to web server,
1338960	1344100	there is a TCP connection that can be established on port 80.
1345360	1350980	And principle show that a user has a user account on this PC
1350980	1353880	and there is another system admin,
1353880	1357040	which is kind of a root level account on the web server.
1357040	1360380	So all these information we can obtain
1361620	1363460	by scanning the network
1363460	1367620	and by obtaining the host configuration information,
1367620	1370020	the network rules.
1371060	1376640	And then they go through these first order logic rules.
1376640	1378100	So this is one of the rules,
1378100	1380420	which says that if there is a vulnerability existing
1380420	1382360	on host with vulnerability ID
1382360	1385900	and the vulnerability has a property
1385900	1388700	that it's a remote exploit,
1388700	1393360	and there is a network service corresponding to this host
1393360	1396760	and the attacker has a network access on this host
1396760	1399260	and attacker is malicious,
1399680	1402300	then this will lead to a code execution.
1403380	1406000	So these are basically predicates of this rule
1406000	1409060	and this is executed code by attacker on host
1409060	1414020	and gaining of privilege is basically the host condition
1414020	1415800	that is obtained
1416200	1419460	when all these preconditions are satisfied.
1420880	1424020	And we also use the policies
1425320	1429460	which show the user access on different resources
1429460	1434020	in the system to encode into these interaction rules.
1435860	1438480	So this will be a logical attack graph
1438480	1443340	of our system that we saw.
1443340	1446780	You can think of attacker located on internet
1446780	1448560	as this node zero.
1449140	1453080	Then the node zero interacts with different nodes
1453510	1456440	and these ovals represent the rules.
1456800	1460940	So like one of this oval will be interaction rule.
1462040	1462960	And based on that,
1462960	1466300	the attacker progresses to the next privilege node.
1466300	1471720	So we can think of this as like a root exploit
1471720	1473700	on say Apache web server,
1473700	1478220	then attacker probably gains some other network level access
1478960	1482200	using another host condition of that graph
1482200	1484180	and eventually reaches his goal
1485420	1489520	of gaining root access on a database server
1489520	1491900	by exploiting SQL injection.
1493580	1500080	So the main brain or AI in this work is Cappuccino.
1500240	1501780	So what Cappuccino does
1501780	1504640	is it takes the information from attack graph
1504640	1509320	and information about different configurations
1509320	1513420	and vulnerability from this CV search database
1513900	1516480	and log information from the Latte module
1517680	1520580	to create a MDP graph.
1522020	1526660	So MDP graph can then be used to derive attack plan
1527260	1529080	that we as penetration testers
1529080	1533840	will implement on the network.
1534820	1537640	So let's see how the states can be extracted
1537640	1539200	from attack graph.
1539200	1541080	So there were fact nodes which shows
1541080	1544600	that attacker was at internet initially
1545480	1547960	and the next privilege node that he gained
1547960	1552540	was a network access on say another machine FTP.
1554160	1556820	So there are two things that attacker can do
1556820	1558740	when he is located on internet,
1558740	1560280	either he can take no action
1560280	1565140	or he can exploit this vulnerability.
1566420	1568640	Let's say the CV ID of this vulnerability
1568640	1573100	is CV 2013 4124.
1573100	1578300	So these will be the states that we can extract
1579020	1583300	from attack graph to be used in our Markov game
1585220	1587660	or our Markov decision process.
1588180	1590300	So let's revisit the attack graph
1590300	1593920	and let's see that for this another example,
1596300	1600840	if there are two paths that attacker needs to take
1600840	1604600	to be able to exploit this FTP machine.
1604600	1608620	So one way he can go about is that he goes from SSH
1609070	1610960	and then tries to exploit the FTP
1610960	1616340	or another way is that he first exploit this web server
1616340	1617940	and goes to FTP.
1619740	1623180	So the corresponding attack graph for this network
1623180	1626120	will be that attacker has SSH access,
1626120	1628600	there is a SSH vulnerability,
1628600	1631240	he exploits SSH, he gains a root on SSH
1632460	1636920	or the attacker can also go through
1636920	1639120	the exploitation of web server.
1639120	1643860	So basically he goes through the web server
1643860	1646320	to exploit some vulnerability on web server
1646320	1649620	and then he reaches the FTP server.
1649620	1652220	So in this graph, there are basically two paths
1652220	1654000	that attacker can take.
1655480	1659980	So if we are to obtain a Markov decision process from this,
1659980	1664080	basically MDP has some components,
1664080	1666820	state, action, transition and reward.
1667580	1671200	So state represents the access that an attacker
1671200	1673900	can obtain at any point in the network.
1673900	1676200	So he can be a user in SSH,
1676200	1677700	if he exploits the vulnerability,
1677700	1680540	he can be a root user on SSH.
1680700	1682320	If he exploits the web server,
1682320	1685320	he can obtain a root on the web server
1685320	1689120	and eventually he can also obtain a root on FTP.
1690460	1693800	There are two actions that we use
1693800	1698500	to simplify this Markov decision process.
1698500	1701120	So in each state, attacker can either choose
1701120	1704080	to take no action or he can choose
1704080	1707500	to exploit the next vulnerability.
1707960	1710520	And there are some probability values
1710520	1713020	that are associated with these actions.
1713020	1715720	And these probability values we will explain
1715720	1719160	like how we derive meaningful probability values
1719160	1721120	for the MDP.
1722580	1724140	And we kind of relate these
1724140	1727180	to the access complexity of the vulnerability.
1727180	1730360	So if the access complexity is low,
1730360	1733660	then probably it's easier to exploit that vulnerability
1733660	1735680	and there is a higher probability
1735680	1738540	of transitioning to the next state.
1741500	1745280	And the rewards are the values attacker obtains
1745800	1747880	by being in a particular state.
1747880	1752100	Say attacker does not want to exploit that vulnerability.
1752100	1755240	So he will have kind of a low reward.
1755240	1758000	The reward is that basically he is not detected
1758000	1761760	by say an intrusion detection system.
1761760	1764740	So that's kind of a reward for him
1764740	1767420	but it's not very big reward.
1768380	1771280	As compared to if he is able to obtain a root account
1771280	1773060	on one of the external services,
1773060	1775760	that's a high positive reward.
1776660	1779840	So we put this value like a plus five
1779840	1782920	and we use the CVSS score of the vulnerabilities
1782920	1785700	to derive the reward.
1786400	1791320	So the reward can at any point be between zero and 10.
1792580	1796120	And if there are uncertainty in the attacker action,
1796120	1797200	this can be considered
1797200	1801020	as a partially observable Markov decision process.
1801980	1806860	But in this work, we are using the simple Markov decision
1806860	1811020	process to show how we encode this attack information.
1812100	1817840	And when we solve this Markov decision process
1817840	1822840	using a value iteration solver, we obtain some policies.
1822840	1826960	So policies are different paths that attacker can take
1827580	1830640	to obtain some of his goals.
1831740	1834120	And the reward for each path.
1834120	1838600	So value iteration tries to maximize the value
1838600	1846060	that an attacker can gain by following a particular policy.
1846060	1847680	So there are two policies,
1847680	1852720	like he can either exploit SSH, then exploit FTP,
1852720	1855640	or he can exploit SSH, then go to web server
1855640	1858900	and then exploit FTP.
1858900	1862140	So obviously the reward in second policy is higher
1862140	1864500	compared to first policy.
1864800	1866560	But this is for a simplified network.
1866560	1868760	We can hand encode these values
1868760	1872420	and solve this Markov decision process.
1872420	1873900	But as a penetration tester,
1873900	1878440	if you are dealing with a very gigantic network,
1878440	1880480	we would want to encode this information
1880480	1883380	using some of the MDP solvers.
1885180	1891040	So, as I mentioned, the states are the privilege level
1891040	1892400	of the attacker.
1893440	1897340	The value for the transition metric,
1897340	1901040	which is basically the probability of transitioning
1901040	1904700	from one state to next state using an action.
1904700	1910760	So suppose S0 was a state of user access on SSH
1910760	1915660	and S1 is the state of root access on SSH.
1917420	1924580	And the access complexity of this vulnerability is low.
1924580	1926420	That means that there is a high probability
1926420	1929960	of transitioning to state S1.
1929960	1936380	So we encode the value 0.9 for low access complexity,
1936380	1939180	vulnerability 0.6 for medium,
1939180	1944880	and 0.2 for a high access complexity
1944880	1948120	because the vulnerabilities
1948120	1950120	for which access complexity is high,
1950120	1952420	they are obviously difficult to exploit.
1952420	1954560	So there is a good chance that attacker will stay
1954560	1958120	in state S0 if he tries to take an action.
1959140	1963660	And the reward value basically for transitioning
1963660	1969020	to state S1, if the CVSS score of the vulnerability is 6.4,
1969020	1971180	that will be the reward that attacker will gain
1971180	1973680	by transitioning to that state.
1977580	1982120	So as we discussed, the states represent
1982120	1983920	the current privilege level of attacker
1984500	1988700	and the actions are the actions that attacker will take.
1989040	1993300	And the transition probabilities are the values
1993300	1995440	we derive from the access complexity
1995440	1998640	of each CVSS vulnerability.
2000540	2006280	And if we encode this information for action exploit SSH
2009150	2012650	in a form of a transition metric,
2012650	2015370	so by taking action exploit SSH,
2015370	2019070	there is a 0.9% probability that attacker will go
2019070	2021410	from state S0 to S1.
2021410	2026010	So let's consider the rows 0, 1, and 2
2026010	2027630	and columns 0, 1, and 2.
2027630	2034110	So S0, S1, which is 0, 1 row and column,
2034110	2037870	will show that there is a 0.9% probability
2037870	2040530	that attacker will transition from S0 to S1
2040530	2042150	by taking that action.
2042150	2046130	But that action has no implication on other states.
2046130	2048950	So if he was in state S1,
2048950	2053590	then exploit SSH doesn't do a whole lot
2053590	2056750	for going to state of S2.
2056830	2061270	So attacker will remain in state S1 if he takes that action.
2063930	2067690	So there is a high probability of being in state S1 and S2
2068250	2072730	if he takes action exploit SSH in S1 and S2.
2073670	2077030	And the reward, as we discussed,
2077030	2083150	is the values he obtained by the CVSS score
2083150	2084250	of those vulnerabilities.
2084250	2087530	So if he takes action A in state S,
2087530	2090850	it maps to a real number between 0 and 10.
2093730	2097310	So as we discussed that taking no action
2097310	2098970	will have a low reward,
2098970	2103270	and exploiting SSH will have a positive reward,
2103270	2105210	which, like if the CVSS score
2105210	2107470	for this vulnerability was 6.4,
2107470	2110430	that will be the reward of being in state S1.
2110430	2113770	And similarly, the reward of being in state S2
2113770	2116570	will correspond to the FTP vulnerability.
2120290	2122770	So bridging it all together,
2122770	2127430	how we arrive at Markov decision process from ATT&CK graph.
2127430	2130750	So this is the algorithm that I designed.
2130750	2134510	Basically, you pass the ATT&CK graph to get the nodes,
2134510	2137910	which show the privilege of the attacker in step one.
2138270	2140990	You find the CVIDs of these vulnerability
2140990	2147150	that lead to this exploitation in step two.
2147170	2149030	In step three from ATT&CK graph,
2149030	2152170	you fetch the CVSS score of the vulnerabilities.
2153110	2158630	In step four A, you use the CVSS score
2158630	2161110	to create a reward metric,
2161110	2165710	which show the actions and state transition mapping
2165710	2166850	to a real value.
2166850	2169510	So basically it will be say a column matrix
2169510	2173850	for different CVSS score.
2173930	2177430	In state four B,
2177430	2179370	basically you use the access complexity
2179370	2182910	to derive the transition metric.
2183530	2188890	And you provide all this information to MDP solver.
2188990	2192670	So the reward metric, the transition metric,
2192670	2195950	the different states and the action.
2195950	2197890	And we use the PyMDP solver
2197890	2203070	as a solver with the value iteration function
2204430	2206670	to generate an ATT&CK plan,
2206670	2210610	which then we provide to this module PyMetaExploit
2211270	2212850	that executes the ATT&CK plan
2212850	2217590	and shows that how fast is this ATT&CK plan
2217590	2219250	that we obtained.
2221010	2223390	So for the validation of ATT&CK plan,
2223390	2224910	like if the ATT&CK plan says
2224910	2228350	that first exploit SSH and then FTP,
2228350	2233210	we have MSRPC, which is a daemon of Metasploit,
2233670	2237590	which is running and basically it uses the Python scripts
2237590	2240030	to execute different plans.
2240630	2245310	And tells us like what is the cost
2245310	2247010	of running different ATT&CK plans,
2247010	2250310	like how much time it took for different ATT&CK plans.
2251490	2254010	So for MSRPC, you will need to have
2254790	2259390	one of the MSRPC session running on one window
2259390	2261770	and then on another window,
2261770	2266430	you will execute your ATT&CK plan and see how it goes.
2268250	2270670	So let's look.
2272290	2275290	Yeah, so let's look at the demo.
2275290	2278190	So we'll first go to here.
2279070	2283670	So we can perform port scan
2286440	2291280	to identify the services that are running on the network.
2291280	2294640	What are the software versions of those services
2295680	2297480	that are present?
2297480	2300100	It will take some time.
2300100	2303010	So in that meantime, we will go and
2305140	2308150	check out the vulnerability scan
2308150	2310290	that we have in our network.
2310290	2313540	Basically, we have like a
2320130	2325270	and we can see like this scan obtained
2325270	2331230	a lot of vulnerability on a machine that we set up.
2331230	2337990	There were about 71 vulnerabilities,
2339530	2341470	which were a mixture of critical, high
2341470	2348740	vulnerability and low vulnerability.
2348740	2351880	So our ports can reveal some information
2351880	2356420	that we basically need from the Stinger module.
2357050	2362240	We have the CVE search API, basically,
2362240	2365920	which provides us information on different
2368360	2371460	vulnerability, like if you provide the CVE ID,
2371460	2374630	it tells us the access complexity and CVSS score,
2374630	2377250	which we will later use in Cappuccino.
2381750	2385290	We have Nessus scanning APIs,
2385290	2389830	which can be used for connecting with the backend
2393360	2397220	of the Nessus on the provided port
2397220	2400380	and basically check the
2402620	2405140	state of the network scan,
2405500	2407440	basically create policies
2407440	2411240	and see like if the scan is running or not.
2411780	2416400	We can basically export the scan using these APIs.
2416400	2419320	One thing is that in latest version of Nessus,
2419320	2421580	I think they have disabled some of these APIs,
2421580	2425160	but if you are using the old version of Nessus,
2425980	2427920	you can use these APIs.
2427920	2431700	So the scan that we obtained from Nessus,
2431700	2435180	we need to provide as input to the attack graph,
2436180	2439260	which is our Americano module.
2443720	2448460	And the Nessus scan file we obtained
2452340	2457600	is this ASAP underscore something dot Nessus.
2457600	2463220	This will serve as input to the attack graph.
2465680	2471040	So Malware is a tool which uses this information.
2471040	2476300	Basically translate the dot Nessus file
2478760	2482990	using Nessus vulnerability translate.
2484670	2488080	And this will generate a list of interaction rules
2488930	2492340	and another script, which is a graphgen.sh
2492340	2496400	can be used for obtaining the attack graph
2496400	2498700	from this translated information.
2498700	2501900	So I already have these files.
2501900	2504660	So you will need to set up Nessus and XSB
2504660	2508240	if you want to use the Americano module
2508240	2511460	and I have the details in the readme file.
2514330	2517310	So we run the malware module
2517310	2520970	and the attack graph that we obtained,
2522050	2524490	it has some nodes edges,
2524490	2531560	which show the information about the network services,
2531560	2533000	the different vulnerabilities
2533000	2535820	that are present on this network,
2535820	2537540	the interaction rules.
2539080	2543000	If we check the visual representation,
2543000	2544460	it will look something like this,
2544460	2547440	like the attacker was initially on internet.
2547500	2551980	Then he used some rules of access control list,
2551980	2555000	which allowed the access to a particular port
2555940	2560460	to have direct access on another machine.
2561560	2564040	In the network, so he can network access.
2564040	2566440	Then he used some other vulnerability
2566440	2570920	to mount a remote exploit on one of the server programs.
2570920	2579500	So this information is represented using a dot file.
2579500	2582740	And we can obviously use some D3 libraries
2582740	2585700	to improve the visualization,
2585700	2588780	but this is how the attack graph will look like.
2589690	2591720	So now we use this attack graph
2594540	2600640	and we provide this information to the MDP solver,
2603860	2607040	which is present in our Cappuccino module.
2618390	2621690	So the attack graph parser basically parses the attack graph,
2621690	2624950	obtains the information about the access complexity
2624950	2628370	from the CV search.
2628370	2634090	It checks the predecessor and successor nodes
2634090	2636990	of the network, obtains the attributes
2636990	2644290	that I described in the state transition.
2644470	2648290	And basically it encodes this information
2648290	2654730	in form of a Markov decision process.
2655650	2658090	So if you run this code,
2660110	2665130	basically the edge information will tell us
2665130	2669070	if you are transitioning between two edges.
2669070	2672510	So let's see, between 1007 and 994,
2673270	2679230	the edge has labeled CV 2011-04-11.
2679230	2680770	And then we will use this information
2680770	2684090	to obtain the access complexity
2684090	2686630	of that particular vulnerability
2689110	2692530	from the attack graph and encode it
2692530	2695950	in form of like a reward metric.
2695950	2697950	Then the value iteration algorithm
2697950	2702230	will learn the attack policy
2702230	2706830	that is beneficial for the pentester in this case.
2706830	2710870	I took a subset of the entire graph
2710870	2713590	to run the value iteration algorithm
2713590	2718650	and it tells us that taking action of exploit,
2718650	2719950	so zero represents no action
2719950	2722610	and one represents the exploit action.
2722710	2725670	So in this state, taking a particular action
2725670	2729130	will be beneficial for the attacker.
2730470	2734010	So it will take some time to basically run
2734010	2737150	the value iteration solver
2737150	2739910	and it will give you a attack plan.
2746930	2749470	We use the attack plan validator
2749470	2753270	to basically validate the plan of attack.
2753270	2757790	So we use the attack plan to exploit FTP
2759610	2762830	and another SSH vulnerability.
2762830	2765930	And basically you can see we obtained two shells
2766190	2767350	in this case.
2768610	2773890	And it took 0.35 seconds to complete.
2774130	2778670	And basically using this attack plan validation,
2778670	2782550	we can validate how much time it took
2782550	2788690	for our attack plan to be finalized.
2791430	2794610	So basically for the attack plan validation to work,
2794610	2797910	you should have the MSF RPC daemon
2797910	2799710	running in one shell, one session
2799710	2804030	and you need to execute your attack plan in another session.
2806090	2808190	So in summary, the threat landscape
2808190	2810450	is very complex and ever-growing
2810450	2817010	and the autonomous pen testing solution,
2817010	2820290	like the one that we have used in ASAP,
2821230	2826550	can help us navigate through this complex attack surface.
2827270	2830490	And it is effort towards the generation
2830490	2833790	of autonomous attack plans and their validation.
2835170	2837970	So that is another module that we still have in progress
2837970	2842770	and in future our plan is to use the log information
2842770	2844350	to validate these attack plans
2844350	2847650	and possibly also work for some kind
2847650	2849530	of report generation tool,
2849530	2852610	which help us generate the report
2852610	2855390	on how these validation was performed
2855390	2857910	for individual exploits.
2863060	2866440	So thank you everyone for attending my talk.
2866440	2870520	The source code is posted on the GitHub repo,
2870520	2872640	the link which you can see.
2872640	2876460	And if you want to contact me,
2876460	2879320	here is my contact information.
2879460	2883060	I appreciate the organizers of Red Team Village
2883060	2886080	to give me a chance to speak
2886080	2888960	at this year's Def Con Red Team Village
2888960	2891980	and have fun, everyone.