start	end	text
0	4600	Welcome back. And thank you, Tim. Thank you for supporting DEF CON. Thank you for supporting the
4600	10840	Red Team Village. The floor is yours. Take it away. Hi, folks. Good morning, good afternoon,
10840	15640	good evening, wherever you are in the world. Welcome to my session, All of the Threats,
15640	19320	Intelligence, Modeling, Simulation, and Hunting through an Attacker's Lens.
20860	25300	So just to put a little bit of background behind this at a very high level,
25300	29980	ATT&CK rocks. But what happens when ATT&CK doesn't give you the information you need?
32140	39760	So introduction. TLDR, who I am, what the plan for this presentation actually looks like.
40240	48200	So I'm not a data scientist. I'm not aligned to either blue or red. I do various activities on
48200	54060	both sides of the fence. And I fully acknowledge that this isn't a solved problem. This is my stab
54060	61140	in the dark trying to get somewhere with it. So who I am. So I work at Cisco. I've done various
61140	67440	roles. I've done assessment work. I've done threat intelligence. I occasionally help out
67440	71840	with incident response when we get weird situations. And indeed, I've also done my
71840	81000	fair share of spreadsheet filling as an auditor. I am a technical person. I like bugs. I like Unix
81000	85320	in particular. And I like exploiting stuff. Probably the most recent stuff you can go and
85320	92880	find is Mimikatz and Unix. What happens in an active directory on a Unix estate. So the plan
92880	96300	for this presentation, I'm going to give you a little bit of background. I'm not presuming that
96300	100280	everybody knows the difference between threat intelligence, threat hunting, threat simulation,
100280	107260	etc. I'm going to talk a little bit about building bespoke threat models. I'm going to talk about
107260	113660	how we can use data we already have. I'm going to talk about how that translates into the real
113660	116840	world. I'm going to look at some of the stuff that we do from an offensive standpoint and how that
116840	123200	maps into things like attack. I'm going to talk about how we can be using this kind of information
123200	126260	to improve our threat models. And hopefully I'm going to finish up some recommendations and
126260	131980	conclusions and it'll all make sense and you'll go in and everything will be great. So background.
133200	137620	The concept was we need to bring all of these different disciplines together. We have people
137620	140760	that do threat intelligence, we have people that do threat modeling, we have people that do simulation,
140760	145700	we have people that do hunting. And attack gives us a common language for all of those things.
145780	151480	But does it go far enough and does it help us with, I guess, the business system? So that's
151480	157480	kind of where we are. So as I said, bring the five disciplines together. Most of the people
157480	164960	guessing on this will be involved in at least one of these. Maybe you're doing recovery work
164960	168760	when breaches happen. Maybe you're doing detective work, you're sitting in a sock.
168960	172820	Maybe you're the person that's meant to be responsible for keeping everything safe.
173420	180440	You know, we're all in this together, right? So an ideal approach. Ideally, we'd all have a good
180440	186140	understanding of targeting hypotheses and how we validate. You know, we'd understand actors,
186140	193400	TTPs, assets. We'd be able to draw out hypotheses about how actors get into networks, what they do
193400	198640	once they're in networks, and what they're looking for in terms of actions on targets,
198640	203900	what they're looking to get away with. And hopefully, we'd have a common way to talk
203900	209000	about that from a validation standpoint, either in terms of posture or telemetry,
209000	212580	or indeed any of the other mechanisms that may or may not exist.
213320	219120	So threat intelligence. That's all about identification, understanding emerging TTPs,
219120	224840	understanding what malicious behavior looks like, collecting and enriching that, and providing
225660	231680	situational awareness to your organization, to those people that you work with.
232160	237960	Threat modeling. The mission there, I guess, is to describe the assets. So take that information
237960	244840	that threat intelligence gives you, augment it, build kill chains, map attack surfaces,
245120	251580	talk about vulnerabilities and weaknesses and how they inform that threat model,
251580	256380	understand the motivation, understand the impacts of attacks.
257180	261860	Simulators. Yeah, there's various different classes within that. You've got simulation,
261860	265500	you've got emulation, you've got traditional penetration testing. But essentially,
265500	269020	that's looking to take that information about what the threat model looks like,
269020	275100	that information about what threat actors are out there, and recreate it so that your blue team,
275100	282080	your defenders can understand, can test their telemetry, can test that they can pick it up,
282080	288120	and hopefully so they can improve either the identification or the automation,
288120	294940	the response to those threats. Hunting. That's, I guess, the final part of the jigsaw from my
294940	302740	perspective. That's about understanding how threat intelligence, how threat models,
302740	308220	what they look like on the ground. When an attacker has got in, what are they doing?
308220	312400	What does it look like? And how can you go and find them and hopefully kick them out?
312400	316220	So it's about understanding posture, it's about understanding telemetry,
316220	321300	it's about reading that information, identifying where attackers are actually active,
321300	328320	taking steps to remediate that problem. So that's kind of a high-level view of the people
328320	336020	I'm talking to. What of it? What should we be doing? I talked about the fact attack is great.
336300	340700	It's really good if you're talking about heterogeneous Windows networks, active directory,
342260	349700	phishing through to crypto mining, phishing through to crypto lock-in. The traditional
349700	354160	stuff that everybody worries about on their Windows desktops. But where it doesn't, in my
354160	359240	opinion, particularly help is when you step beyond that. Most businesses, well, they actually have
359240	365040	data, they have applications that essentially drive their business. And that's the purpose of
365040	370240	this talk, is to look at how we take attack and leverage in those kinds of situations.
371520	377940	So building bespoke threat models. I'm going to go through what I think requirements look like,
377940	383460	what I think the workflow looks like. Some ideas about how to use a workflow effectively.
384760	392020	How you can apply hypotheses. Some of the use cases that maybe we don't really tackle today,
392020	399640	but perhaps we ought to. Some of the capability gaps that certainly exist. And why in my head,
399640	405420	and this is just my head, I think threat intelligence collection is really just a backstop.
405420	411080	There are far more useful sets of information about our platforms, about our assets, about
411080	416180	our data, about our users, that should give us a far better understanding of where threats are
416180	423340	likely to raise their heads. So in terms of requirements, broadly speaking, we need to
423340	430420	define a mission. And I will say the value of any system, well, that's the data. Secondly,
430420	436080	we need to have an understanding of what threat looks like from one perspective or another.
436920	441460	In terms of building hypotheses, well, we need to understand what the organization actually cares
441460	446900	about. And we probably need access to the design. It's very difficult to build threat models
447460	451520	purely based on attack once you step into business systems, because you're
452100	459040	really not matching like for like. A Windows desktop, the users, the use cases,
459040	463620	are significantly different from, you know, an enterprise resource planning system,
464820	469320	a data warehouse. So we need to understand what the designs look like, you know,
469320	473260	three-tier architectures, are they using containers, are they using microservices,
473260	478640	that kind of thing. And then from a validation standpoint, one way or another, we need some
478640	484180	visibility of those targets. From an intelligence standpoint, I guess, third-party sourced evidence
484180	487960	is kind of useful. If you're talking about simulation, well, you need to be in a position
487960	493100	to go and actually test the systems that you're talking about. From a hunting standpoint,
493100	499760	you need configuration, you need access to the logs. Ideally, you'd need dynamic telemetry,
499760	505080	that is to say, telemetry that's feeding back in real time. But at the very least,
505080	511320	the ability to see the events, the log events that are occurring in SAP, in a database environment,
511780	519780	is pretty key to getting this right. So in terms of workflow, from my perspective,
519780	526260	I've sat in all those different nodes at one stage or another. I sat there and looked at TTPs.
526320	533120	I've gone and tried to simulate vulnerabilities and weaknesses to exercise attack surfaces.
533980	540800	I've considered the impact from a governance standpoint. And in all and amongst that,
540800	545940	you've got to understand the motivation. But hopefully, no matter what part of the security
545940	550580	lifecycle you live in, you'll map into one of those areas fairly cleanly.
551680	557560	And then, of course, the challenge is to hand off to the next node. And indeed, in practice,
557960	563780	there's nothing to stop you handing across in other ways. Understanding impact, well,
563780	568260	that feeds vulnerabilities and weaknesses. Understanding motivation, well, that might
568260	575000	actually tell you something about TTPs. So iterating effectively through it.
576440	580660	Understanding a platform. I talked about the fact attack is great. But if you understand
581220	586460	the platform, maybe you can take attack on its own and filter out those TTPs that you're most
586460	592680	interested in. Some of the work I've been doing recently has been around, for example,
592680	600960	taking attack and extracting verticals, extracting regional differences,
601900	608560	extracting particular vulnerability classes. In terms of threat modeling in a practical sense,
608560	611820	you've got to be able to go and speak to the people, you've got to understand the roles,
611820	617680	you've got to understand the processes. Tooling wise, there is nothing better than pen and paper
617680	623400	or a whiteboard. But if you want to record your information, Microsoft Threat Modeling Tool,
623400	628820	others of its, ILK, Visio and Excel, well, they can all help. And then fundamentally,
630060	634760	producing a worksheet. And it doesn't really matter if that worksheet is built from the
634760	641120	perspective of an adversary, a red team, whether it's built from the perspective of a threat
641120	645240	intelligence person, whether it's built from the perspective of a threat hunter.
645240	650680	Once you have that worksheet, actually, you can pivot those questions to answer whichever
650680	659850	question you're actually asked. So applying hypothesis to real world platforms and applications.
661030	667530	First of all, at a very high level, familiarizing yourself with ATT&CK. It is still useful,
667530	670650	and you'll see this a little bit later in some of the data I draw out.
671650	677050	Mapping out those ATT&CK surfaces, you might recognize the four I've iterated through.
677050	683870	They're actually straight out of CVSS. It's a starting point. Vulnerabilities and weaknesses.
684730	692350	ATT&CK TTPs are great, but actually CAPH, CWE, perhaps give you value that you're not necessarily
692350	697810	recognizing today. If you're fortunate enough to have good threat intelligence, understanding
697810	703630	motivation, understanding threat groups. If you're fortunate enough to have access to business data,
703630	709990	understanding system value. And then, of course, impact. It's a pretty lazy way to do it,
709990	714530	but stride still helps in that regard. Fundamentally, attackers are still spoofing,
714530	723180	tampering, causing information to be disclosed, etc. So use cases.
724500	731120	In an enterprise, fundamentally, ATT&CK gets used for manual design verification.
731500	736780	It gets used for sourcing IOCs. It gets used for telemetry configuration.
737660	742180	And it gets used for response prioritization. There will be other use cases and there will
742180	747600	be sub-use cases to each of these. But fundamentally, the threat model
748160	753840	concept is about understanding a design. It's about understanding what's likely to be attacking
753840	759400	it. It's about making sure you've got the right security controls. And these days with consider
759400	767700	breach, telemetry is a key aspect. And it's about being able to respond when breaches occur, because
767700	779270	again, consider breach. So capability gaps. I've been into a fair few SOCs, either working in them
779270	784970	or talking to them, evaluating their current capabilities. And that's what one's a pretty
784970	792630	key one in my mind. SOCs are great at dealing with enterprise. Most people these days are
792630	799350	familiar with ATT&CK. They do understand enterprise risks. But actually, once you start to grow them
799350	804830	beyond that, and start to ask them about their SAP infrastructure, their banking enterprise
804830	813590	applications, their data warehouses, we're kind of blind to a degree. And I think that's probably one
813590	818470	of those things that we need to tackle. And I hope that some of what I'm talking today about
819080	824170	is about informing how we can tackle some of this. We're clearly not going to teach every
824170	830150	SOC analyst every application that a business cares about. So making better use of the knowledge
830150	838550	they have, giving them the tools to translate generic enterprise risks into key business risks,
838550	847710	that's pretty critical. Collection and routing, logs, audit events, telemetry. I guess depending
847710	853710	on where you work, who you work for, you might say, we're pretty good at this. You might say,
853710	858110	I'm not quite so sure. You might even just put your hand up and say, no, we're terrible.
858770	863850	I think every incident response engagement I've turned up to over the last five years
865690	871550	hasn't had as much information as we would require to evaluate the system. And if you talk about
871550	877810	enterprise, that's a really good example. We did a response engagement recently, where it turned
877810	882410	out that actually it was a message queue that was being tampered with. And guess what? Their message
882410	888570	queue had no useful logs at all. Nothing that told you that much about who was accessing the
888570	894410	message queue, where they were accessing it from. And indeed, when we eventually managed to find the
894410	899770	application logs, the development data essentially about how the message queue was functioning,
899770	905430	and we found that it was logging the messages, we were so excited. But even with that information,
905430	910550	trying to take that back to a SOC and say, actually, which of these are the legitimate events?
910550	915010	Which of these are? It was a challenge, because the security organization just didn't understand
915010	925490	that. Orchestration and enrichment. Key to those two previous points is getting SOCs to a point
925490	933070	where they know what data is useful, and they know how to use it. It's easier to do with Windows,
933070	938850	it's easier to do with endpoint devices. But actually, if you want to kick someone out of
938850	945630	to go back to that point, a message bus, where do you start? What data is useful? We had to tell them
945630	952510	every step of the way, because they simply didn't know. And then, of course, you look at things like
953470	959630	the... what is bad? What does bad look like in an enterprise resource planning application
959630	966650	in a Unix estate for microservices? So those capability gaps, we need to start to fill them.
969010	974290	But all of that data exists, which is kind of why I argue that threat intelligence really
974290	979590	should be a backstop. If we're not consuming the data that the organization is already generating
979590	983910	in one form or another and using it effectively, we're going to be forced to use threat intelligence
983910	994170	as our only means to identify threat actors, identify attacks. But in terms of threat intelligence,
994170	1000510	even there, we could be using it more effectively, right? We know the vulnerabilities exist, for
1000510	1006390	example, yet we don't necessarily use that in an effective way from a threat intelligence standpoint.
1007190	1011930	We don't use it in a predictive fashion as much as we might like.
1012650	1016690	We certainly don't construct the same degree of hypotheses around
1017790	1026410	application vulnerabilities as we do around password cracking, phishing, etc. We need to
1026410	1032770	get better at putting together hypotheses, straw men, that we can start to buy off in pieces.
1033050	1039310	And ultimately, from a threat intelligence standpoint, we need to get better at tracking
1039310	1045310	that kind of threat intelligence in just the same way that we track TTPs for enterprise threats.
1046290	1052530	A SAP system will still have that kind of information. We should still be able to work
1052530	1058650	out the validity of the information. We should still be able to understand where it might affect
1058650	1062870	us as an organization. We should still be able to understand why an attacker might wish to do it.
1063030	1068330	And we need to start to ask threat intelligence teams to give us that information so that we can
1068330	1074930	make the right decisions. So that's some of the problems. Let's have perhaps a little bit of a
1074930	1081230	solutions. And here, what I was really looking to do was to map some of the more traditional
1081230	1088410	concepts of threat modeling, Stride, Microsoft's tooling, and equivalent into attack language,
1088410	1096110	into kill chains, into the attack matrix. So working in Cisco, I have a pretty good
1096890	1100950	source of information about all of the assessment work that we do.
1101510	1106450	The vulnerabilities, the weaknesses. I have the ability to extend that tooling.
1106570	1110970	So I kind of wanted to say, well, actually, if we've got vulnerability data from an assessment,
1110970	1115770	how do we translate that into a threat model that a SOC can use?
1115770	1120470	So I want to label our findings. I want to start to analyze our data a bit more depth.
1121690	1126790	So that's our vulnerability model, fundamentally. We take all of the things you would find in a
1126790	1134230	pen test, the idea of scoring criticality, defining the weakness in generic terms,
1134230	1138350	describing the vulnerability, describing the impact, talking about recommendations,
1138930	1147350	industry references, etc. But on top of that, we have our not inconsequential VDB. It's internal,
1147350	1151310	but it imports from Nessus, from MITRE, and from other sources. And our reporting engine
1151310	1156770	is really driven off that. So when we report a finding, if it's a finding we've seen before,
1156790	1163010	it will come from our VDB. If it's a new finding, it will end up in our VDB. So we have quite a
1163010	1170010	rich historical view. I won't say it goes back 20 years. We have to clear down a lot of customer
1170010	1176590	data, as you can imagine. But certainly in terms of the VDB portion, we have an understanding about
1176590	1181230	how vulnerability trends have changed over time. And we should be able to leverage that to build
1181230	1189590	more interesting analysis. So what have we been doing? So we've been automating some of our
1189590	1197110	scenario generation. We've been taking ATT&CK using PyTAC, which is a Python framework,
1197110	1203990	to start to filter the STIX output from the ATT&CK matrix. We've been starting to look at
1203990	1210010	how we label vulnerabilities in our VDB, vulnerabilities in our customer reports,
1210010	1215390	with ATT&CK TTP, so that we can say, we found this vulnerability on a traditional pen test,
1215390	1220250	but actually it maps here into your kill chain. And these are the kinds of controls you're likely
1220250	1226990	to need. Not just the tactical fixes of yet you should go and patch, or yet you should go and
1226990	1233670	harden this particular service. But actually, have you got monitoring here? Is this system
1233670	1241230	being protected by behavioral analysis engines, etc? And then of course, we can take that
1241230	1246690	information, we can export it out, and we're using STIX for that. And then we can do cross data
1246690	1251850	sharing. So for example, I can take out vulnerability data from our reporting engine,
1251850	1257190	and I can share it with TALOS, or with PISA, or with any of the other security teams that would
1257190	1262890	sit within Cisco. The idea being that we can get a better understanding of what our customers'
1262890	1268950	environments look like, and we can drive some of the features that we think are critical into,
1268950	1276450	obviously, Cisco's products. And ultimately, we're taking some of that knowledge and using it to make
1276450	1282010	more effective business risk analysis. And we're starting to go to customers and actually use these
1282010	1288090	techniques on their data, not in our VDB, but on their risk management systems.
1290010	1296230	So I talked about labeling findings. This is probably a pretty trivial example,
1296730	1302730	certainly with the data that we have. But broadly speaking, we're scoring our findings, we're
1302730	1309110	comparing those scores against ATT&CK TTP definitions, and we're labeling findings.
1309410	1314150	And then we're identifying similar findings elsewhere, so that we can validate that the
1314150	1321510	scorings we're applying are appropriate. And then applying labels where relevant, so that we start to
1321510	1328610	build up a contextual map of how individual vulnerabilities, individual weaknesses found in
1328610	1335470	an incident response engagement, incident instances of technical vulnerabilities in a red team,
1335470	1343870	how they all map together. To do this, it's not necessarily trivial, certainly if you don't have
1343870	1349330	the data to begin with. But in our instance, we've been developing plugins that give us customer views,
1349330	1355750	platform views, ATT&CK surface views of our data. Like I said, we've been extending our findings to
1355750	1362470	capture this information. But then we've been able to develop plugins that do that automation for us
1362470	1368550	using the mechanism shown on the previous page. And then we can start to render that information
1368550	1372750	in kind of useful fashion. So I think a couple of slides time, I'll show you an example where I've
1372750	1379610	taken a bunch of vulnerabilities from our VDB and using a tool called Gephi, I've actually mapped
1379610	1385610	them into a full keychain. And you can start to see where the critical nodes are.
1385610	1392350	And this is generic vulnerability data, not red team ATT&CK TTPs we're talking about here.
1394910	1400570	Let's stop and pause for a second. CVSS is not a shoe size contest.
1401270	1407830	How many organizations out there look at CVSS as a number? They look at a 10 and they go,
1407830	1412810	we need to patch that immediately. They look at it as a six and they go, we can patch that in a
1412810	1418170	little bit of time. They look at it as a three and they go, actually, we don't need to patch it at all.
1418170	1425630	So frustrating. So what we've been doing there, we've been attempting to map CVSS
1425630	1431590	into the cyber keychain. One of the reasons that CVSS is kind of quite interesting
1431590	1437910	is because nobody feels fearful about sharing that data in isolation. If I go to a customer,
1437910	1442730	I say, I don't want anything else about your estate. I'd like to know your CVSS scores.
1443610	1448270	Most organizations feel comfortable with that. And what that means is that they can then give
1448270	1453090	me some information, which actually, when you start to break it down, becomes slightly more useful.
1455770	1463110	So this is a rough mapping of our model. And what I've essentially got on there on the left-hand
1463110	1469790	side is all of the key kill chain phases. And then as you progress through, I've got the aspects of
1469790	1477670	CVSS that are likely to impact or likely to be interesting in each of those stages. And then
1477670	1484350	I've rated them in terms of how useful they are. So I'm able to say that, you know, from a
1484350	1492030	confidentiality standpoint, if the confidentiality impact is high, that's probably going to be useful
1492030	1496370	from a reconnaissance standpoint. It's also probably going to be useful from an actions
1496370	1502310	and objective standpoint. And similarly, with some of those other nodes in that graph.
1503110	1509950	And where you get to is this. If you take all of our BDB data, and indeed, if you take Cisco's
1509950	1515350	publicly disclosed vulnerabilities, and you break down and you score in the way that I've described,
1515350	1520630	you can see where individual vulnerabilities that we report to customers, individual vulnerabilities
1520630	1527790	that get reported to us, well, how they potentially, and it's my model, so only potentially,
1527790	1533230	how these potentially map into the kill chain phases. And given that the kill chain phase is
1533230	1538870	inspired attack, we're then in a position where maybe we can start to look at actually which parts
1538870	1546490	of the kill chain are we exercising effectively, which parts we're exercising badly. Yeah, we never
1546490	1555490	report almost anything that helps us with weaponization. We rarely report anything that
1555490	1563850	talks to installation. And command and control, well, actually, if you think about it from a
1563850	1568490	pentest standpoint, that kind of makes sense. We're unlikely to really look at command and
1568490	1572310	control. We're unlikely to look at installation. We're unlikely to look at weaponization.
1572690	1578450	So it's almost reflective of what you expect to see. But what it means is actually,
1579330	1583450	if we were doing a pentest tomorrow, if we were doing a threat hunt tomorrow,
1583450	1587810	where perhaps should we focus our efforts? Installation and command and control might
1587810	1592090	be kind of interesting if you're an organization that has regular pentests,
1592090	1596270	simply for the fact that pentests are unlikely to produce useful findings in that space.
1598750	1603570	And then I said we talked to some of our customers and started to use this kind of information on
1603570	1610710	their risk databases, on their vulnerability databases. For that, we've been going into
1610710	1615430	organizations, perhaps a couple of steps up the tree. We've been talking to CISOs, executives,
1615430	1622730	etc. And we've been using this concept of FAIR, which is essentially a risk framework
1623290	1629830	that allows you to talk about risk in quite an interesting way. It allows you to talk about
1629830	1636890	things like resistive strength, threat capability, probability of action, primary loss, secondary loss,
1636890	1643030	etc. And it allows you to put proper numeric scores. Indeed, FAIR encourages you to score
1643030	1651590	in ultimately a dollar, a pound size value. But what it means is that we can go and talk
1651590	1657290	to organizations at levels that the executive cares about. The executive doesn't necessarily
1657290	1664590	care about CVSS version 10. What they care about is that they're able to trade, that their IP
1664590	1670390	isn't being stolen, that if they're a bank, someone hasn't got access to the treasury and
1670390	1678570	stolen all the money. So starting to take vulnerability data and turn it into business
1678570	1685890	impact, business risk, etc. Yeah, that's kind of quite compelling to a lot of C-suite.
1688960	1696500	So why do the labeling? This statement, it's a great statement and it's very true.
1696940	1702260	Defenders thinking lists, attackers thinking graphs. If you think about it logically that
1702260	1706480	makes perfect sense. An attacker doesn't care how they get into an organization.
1707200	1713060	Yeah, if they can't get in one way, they'll get in another. That's a graph. If you think about how
1713060	1717540	risk management businesses work, how risk management organizations work,
1717540	1724580	guess what? For most part, they're operating on spreadsheets or the equivalent. So getting to a
1724580	1730020	place where we can help those risk management functions think more effectively in graphs,
1730020	1735940	that's kind of going to be useful. So when we do the labeling, it allows us to produce
1737200	1744660	documents such as this. So this I think was from recollection was AWS, but essentially was taking
1744660	1752960	all of the TTPs, all of the threat groups, all of the parts of the attack matrix and map them,
1753680	1762100	filtered as I say through the view of AWS rather than the whole STIX object collection.
1762740	1769500	But we're now starting to do the same thing with penetration tests. We're starting to do the same
1769500	1773900	thing with red teaming. And hopefully we'll get to a point where we can deliver it right across
1773900	1780380	the board. Every report in my view, this visualization is pretty key to helping people
1780380	1789160	understand what matters to them. So that's all of the theory. That's all of the data science
1789160	1794920	from a person that isn't a data scientist. What I really wanted to do is have a look at how our
1794920	1800580	data maps to the real world. And I talked about the fact that my background is Unix and big systems
1800580	1806600	and that attack doesn't really help. So I kind of constructed three hypotheses, which I thought
1806600	1814580	were probably pretty short to evaluate. I was probably going to be challenged to find useful
1814580	1823600	data to support me. But it was a starting point. So I chose my targets. As I said, Unix. I built
1823600	1829520	hypotheses. I validated those hypotheses. And I'm hoping to start to feed some of that learning back
1829520	1835560	into our reporting. And I'm certainly more comfortable to talk to some of the missed
1835560	1842480	opportunities that I've seen along the way. So in terms of targeting, if you're familiar with
1842480	1847120	Portcullis, obviously we're Cisco these days. But if you're familiar with Portcullis, you'll know we
1847120	1852060	spent years doing lots of interesting stuff in Unix. It seemed a good place to start. We've
1852060	1858080	written quite a lot of things that you could consider to be TTPs, if you were that way inclined.
1858480	1865420	And as I said, we had quite a lot of access to our customer data. So we were able to do things like,
1865420	1872060	actually, what have we reported over the last 10 years? And the way we have our data segmented
1872060	1879000	means that our VDB keeps track of how many times a particular issue gets reported.
1879000	1882600	Even if we clear down the customer data, we still have an understanding of trends
1883280	1890280	going back not many decades, but certainly the last 5-10 years kind of thing.
1891420	1898420	So the hypotheses I constructed. Attackers are using our tools to target environments.
1899060	1905240	Attackers are using techniques from attack to target Unix environments.
1905660	1912480	And attack is not representative of the TTPs that we, that is the traditional pen testing arm of
1912480	1919080	Cisco, find success with when we go and do professional service engagements for our customers.
1923620	1931460	So, hypothesis validation. I used a small selection of our TTPs, in particular, two tools.
1932200	1940700	Pentest, Monkeys, Unix check, and my Linux implementation for attacking AD on Unix.
1945320	1953240	And, as I said, there's a lack of information out there. So I was kind of hoping we'd go and have a
1953240	1958300	look at a bunch of incident response reports. And we'd kind of say, okay, there's loads of
1958300	1964260	useful information in there. Guess what? When you look at enterprise systems,
1964260	1970680	that might be the case. When you go and look at big box servers, far less often does it occur.
1971240	1978280	So we went back to the drawing board. Detonations, bits of attack that might be useful,
1978500	1986300	a lot of Googling furiously, and to the point about VDBs, many of our other data sources.
1989650	1996310	So, hypothesis one. Attackers are using our tools to target Unix environments.
1996310	2005750	So, guess what? It's almost impossible to determine whether, if an attacker was to
2005750	2013110	compromise a Unix box, and they would use off-the-shelf, publicly available tools,
2013790	2017630	it's almost impossible to tell whether they'd actually be picked up.
2017770	2024010	Unix privilege check was undetected in pretty much all of the cases. And even
2025130	2029170	where it was detected, it wasn't classified as malicious.
2030570	2034710	Kind of cool. I kind of like the fact that I could rock up, I could use this tool
2034710	2040450	when I do an assessment. But actually, you probably want to know whether it's on your
2040450	2046570	network, even if ultimately you go, it's the pen testers, and they're meant to be using it.
2050290	2054370	Again, looking for detonations. I went out and had a look to see if anyone had spotted
2054370	2062450	in the wild. And guess what? They haven't. Is that because it's not malicious? Is that
2062450	2066830	because people have looked at it and kind of gone, we don't mind, it's only being used by
2066830	2073630	pen testers. Actually, it's probably because if an attacker gets you onto a system,
2073630	2077890	they're probably not going to pass it through a Windows box unless they absolutely have to,
2077890	2082150	or at least not in a way that a Windows AV is like to pick it up.
2082810	2088490	Which means that unless you have good telemetry on your Unix estate, both those tools, you're
2088490	2094590	unlikely to see them up until the point that the breach occurs and you start going back through
2094590	2101250	your historical data. And even then, you're probably not going to see it unless you've got
2101250	2108690	an attacker that's left it in the history, or an attacker who has left it on the file system,
2108690	2116970	or an attacker who is ultimately pretty incompetent. If I wanted to get those tools
2116970	2123210	onto a Unix box right now, I could do so. The fact that we don't even have the telemetry to allow us
2123210	2131090	to spot the sloppy amongst us is probably a challenge that we ought to address.
2133730	2140110	So, science of life and attack. Well, neither of those tools are mentioned. Kind of understand why
2140110	2147810	it's not generic enterprise attacks tools. But guess what? It's used by almost every
2147810	2153350	penetration tester out there. It's in lots and lots of tutorials. And it's even being
2153350	2158890	used by people that we would probably consider to be threat actors. Phineas Fisher literally
2158890	2168530	mentions it in his paper around how to breach organizations. Now, you might argue that
2168530	2174370	Phineas Fisher isn't an adversary you care about. It didn't really matter. The fact that
2175050	2180770	we haven't even identified these tools exist when our own community knows they exist and is using
2180770	2186990	them. You know, we talk about offensive security tools. These are offensive security tools that
2186990	2190410	will be being used by people and we'll just never see it.
2192310	2198610	So, the second hypothesis. Attackers are using techniques from attack to target Unix environments.
2199310	2205310	Like I said, that's pretty hard in a practical sense. If you look at the stuff that comes up
2205310	2213850	in attack from a Linux standpoint, it's mostly IOT or front end centric. It's a bit of a chicken
2213850	2218830	and egg situation. If you can't monitor your estate effectively, you're probably not going
2218830	2224670	to spot it. And from an incident response standpoint, well, organizations, perhaps they're
2224670	2229910	not going to be particularly honest, certainly publicly, about just how deep breaches went.
2230370	2234030	But anecdotally, and I'll call back to the Phineas Fisher point,
2234510	2239570	they certainly are doing things that would fall into the attack categories.
2240270	2246470	Unix backend breaches do occur. There is almost always some level of application
2246470	2252350	level interaction. Remember I said the value of any asset is the data that it holds.
2254450	2259270	Some access are truly incompetent if it's not a Linux host. We did an incident response
2259270	2264970	engagement for a mainframe platform that had a Unix-like shell. And the attacker had gone
2264970	2270650	to that Unix shell without really realizing what they'd done and poked around a bit,
2270650	2276010	not had a huge amount of success, and then attempted to wipe their evidence.
2278590	2284290	If I'm looking at this critically, the TTPs from attacker are actually probably still
2284450	2293790	a reasonable place to start with Unix, with enterprise business systems, with servers, etc.
2294470	2298310	But perhaps we don't have the telemetry to support it at this point in time.
2301420	2307500	If anybody wants to know what a good breach report looks like, this is probably the best
2307500	2315620	one I've found. It's looking at fast cache on AIX. And it's not perfect, but it does support
2315620	2322640	the fact that a lot of this stuff is stuff that attack knows about. Okay, it's AIX, not Linux,
2322640	2328780	but question marks against that. But it certainly calls out to the same techniques that the fast
2328780	2339080	cache malware was using in terms of persistence, privilege escalation, evasion, persistence,
2339640	2346380	impact, etc. There's a little bit of speculation on application and entry point,
2346380	2353820	probably not enough. But this is the kind of thing, in my mind, should be going into the
2353820	2359240	attack matrix. Or if it's not going to the attack matrix, should be going into the equivalent matrix
2359240	2363900	for business systems. If we don't capture this kind of information, we're never going to get
2363900	2372340	our socks to start looking for them. So hypothesis three. So this was taken from our VDB.
2373060	2379480	I took the top 10 or so issues that we regularly report to customers when we go and look at Unix
2379480	2387140	systems. I went and I had a look at whether they were captured inside ATT&CK as it stands today.
2388520	2393960	Security patches, password reuse, etc. They're all covered.
2395440	2401600	Role accounts for interactive logins. Semi covered, but perhaps not the way I would have
2401600	2409660	viewed it from my standpoint. That third, fourth one down, that's not captured at all in any useful
2409660	2420240	sense. It gets us into almost every enterprise Unix box we go and look at. You get your
2420240	2425640	role account, you get your low privilege user. Almost every single one of them will get to
2425640	2432960	read through that. So we ought to make sure that ATT&CK factors that in, describes it,
2432960	2438300	and gives the information that allows a sock to start to detect, to start to respond
2438840	2446100	when those kinds of attacks are used. So broadly speaking, I guess where we're at,
2446100	2451700	I think we don't have the right level of platform and application awareness today,
2451700	2457700	which inevitably means there's an intelligence gap. Which means that no matter whether you're
2458580	2463140	doing intelligence, whether you're modeling, whether you're simulating, or whether you're
2463140	2468940	hunting, you're going to be working, I think, from an incomplete set of hypotheses.
2469140	2474480	Which means that the models that you're working to probably don't relate to the systems that you're
2474480	2479780	actually caring about. And you're probably not going to be as accurate as you might want to be
2480620	2490120	when you start to validate them. How do we improve all of this? It's the million dollar question.
2493260	2498800	Threat modeling certainly plays a part. If you were simply to take the approach I did with
2498800	2505860	the vulnerabilities, and use that to drive the threat models for systems you don't know about,
2506620	2510600	you would certainly get to a place where your blue team would probably have a better understanding of
2510600	2517780	what bad is likely to look like. Once your blue team have an understanding,
2519300	2525640	then they can drive the visibility. And then you can start to refocus your offensive services to
2525640	2534280	validate that visibility is appropriate. Ultimately, I would posit that at least some
2534280	2540580	of the problem is there aren't enough bums on blue team seats. I guess what I mean by that,
2540580	2544380	we need to find a way of making it attractive to have those people that understand those
2544380	2549420	applications, the security models behind them, sitting directly in the blue teams,
2549420	2554240	or certainly sitting adjacent to those blue teams, so that the blue teams can start to
2554760	2560480	understand what some of the applications the businesses care about actually look like.
2563440	2568040	Yeah, we can't fix architectural alignment or mission tomorrow.
2568840	2575280	We probably can work on threat visibility and on target visibility, if we know what we're
2575280	2584140	aiming for. If we understand the threats, we can better protect our customers and our data.
2584860	2591020	More importantly, if you can start to talk about individual vulnerabilities,
2591020	2597260	in terms of their impact or threat model on the kill chain, not only can you make sure that your
2597260	2604180	SDLC is looking for the right things, but you can also make sure that the people that are
2604180	2608740	doing the code, you know, the people that are sitting there writing the web applications,
2608740	2614780	writing the code that interfaces with your message bus, your database, etc. So they understand
2614780	2620680	why and how an attacker might attack them. And hopefully we'll get into a situation where less
2621020	2627680	vulnerabilities of note will make it into business applications in G-course. Ultimately,
2628300	2634500	organizations will have profit motivated. And if we can talk in language that
2635540	2643820	makes it apparent to the C-suite why it matters, what the cost of not doing X is,
2643820	2646740	I think perhaps we'll be slightly more successful in that regard.
2649240	2652100	So, offensive services.
2653920	2660980	I think there's life beyond Nessus and MITRE. But that data is still useful,
2660980	2666660	even from a red team standpoint. I guess we need to figure out how to
2667540	2673260	use it more effectively. You know, we need to be in a position where we can
2673980	2679780	far more quickly, far more comfortably talk to the kinds of bespoke briefings that organizations
2679780	2689180	want. Craft war games that are actually appropriate to the things that matter to a business.
2689740	2696560	We need to be able to articulate all of that. And obviously, better metadata,
2696560	2700560	visual representation certainly helps with that communication piece.
2700560	2707440	And we need to refine our assessment methodologies. I think too much of pen testing
2707440	2717420	today is focused on the break in. And I think the data I showed around CVSS shows that actually,
2717420	2723980	given that pen tests are still the lowest bar most organizations will jump through,
2723980	2728460	they're actually getting testers to a place where they feel comfortable to talk about
2729260	2734880	the other side of the line. The system has been compromised. How do we get back out
2735480	2739500	is something we probably ought to focus more time on.
2740980	2747560	So some conclusions. What have we learned? How do we do this better? Next steps.
2749380	2757240	We can automate extraction of hypotheses. We can build threat models in an automated fashion.
2759460	2766780	Vulnerability data, no matter how poor you may think it is, could still be useful
2767820	2777740	with the right labeling applied. Actually, from a tester standpoint, it'd be kind of great if we
2777740	2782940	rocked up organizations and they had a threat model based on their existing understanding of
2782940	2789580	their vulnerability state. Extract your quality information, build that threat model, give it to
2790140	2795320	the tester at the start and say, this is kind of where we think the threats live. Help us validate
2795320	2803780	that. Again, if you can get to a place where you can take your vulnerability data and start to
2803780	2809280	think about it in attack terms, then you can start to think about the control sets that you need.
2809280	2815120	And potentially, you can start to think about automatic generation and validation.
2817160	2824180	Ultimately, it's a communication problem, right? Attack has given us a wonderful tool to help us
2824180	2832000	bridge the gap between red and blue, between operational and management. We just need to
2832000	2841060	make sure we take the best advantage of it. So doing this better. Better consideration.
2841380	2848120	I know that from a red team standpoint, we would argue that perhaps we're there. But actually,
2848120	2857920	I don't think a VA, a generic pen test necessarily needs to consider the scope they're given to be
2857920	2865360	an impedance to giving better outcomes to customers. We do a really good job of collecting
2865360	2870340	awesome information about threats and vulnerabilities, whether it's Reddit, whether
2870340	2878180	it's peer list, unfortunately closed down, or anywhere else. But even that data, we don't
2878180	2885400	particularly do a good job of enriching it. Having a platform that allows proper automated enrichment
2885400	2891400	of that kind of information. Maybe you're lucky and you're sitting in an organization that has
2891400	2898820	it. It's a bit of a forlorn dream for me. Personally, getting our database to a point
2898820	2903960	it's fully labeled and being able to run queries against those labels will be pretty kick ass.
2905260	2911720	Taking that the next step and making it actionable. I mentioned that we use sticks.
2911720	2919500	Sticks is good at talking about the types of threats. It doesn't really capture the level
2919500	2925680	of information that's required for machine action ability. There are some solutions to that,
2925680	2931600	the Sigma and others. But we're really not at a point where we define vulnerabilities in a way
2931600	2938520	that's particularly helpful for scale. And ultimately, if we don't generate for scale,
2938520	2945100	improving EDR and workload protection products is going to be pretty forlorn. If every vulnerability
2945100	2952580	report that every pen test or security researcher turned out had essentially a cool chain graph
2953200	2959880	that an EDR solution or workload protection product could import and then look for,
2959880	2967320	rather than being reliant upon hashes, IP addresses, host names, that would take us a huge
2967320	2977840	way forwards. So next steps, call to action, I guess. Just because we don't have the capabilities
2977840	2984780	to look for bad guys doesn't mean they're not there. Attackers, we use the easiest TTPs that
2984780	2989560	get them to the root prompt. Those things I was talking about are pretty damn easy.
2990940	2998580	Behavior, don't rely upon agents that are merely relying upon hashes and IP addresses.
2999980	3006820	If you're doing Unix stuff, make sure you're exercising those things, because the bad guys
3006820	3015720	certainly are. And Linux apps, Unix check, I kind of look forward to the day when they appear
3015720	3023000	in a virus title, and they're being detonated and being marked as malicious. It'll make my job a
3023000	3031680	little bit harder, but actually it'll make Blue Team's jobs a hell of a lot easier.
3031680	3038640	Finally, a few thanks, my old colleagues, Cisco's wider team, MITRE,
3038640	3046660	ATT&CK wouldn't exist without them. Swimlane, PyTAC is still the most comfortable interface I've found
3046660	3052640	rather than writing in Perl. And Blue Team's everywhere, because you do a thankless job,
3052640	3059600	and for the most part, you do it as well as you can do. And with that, questions?