1 00:00:00,000 --> 00:00:06,440 So, yeah, I'm here presenting today on Cloud Native Attack Detection and Simulation. 2 00:00:07,280 --> 00:00:13,140 It's an expansion of a talk I did at Forward CloudSec, and we've spoken previously on some of this at some other conferences, too. 3 00:00:13,140 --> 00:00:18,020 But this is the first time I've had the opportunity, really, to gather it all together into one big presentation. 4 00:00:18,200 --> 00:00:19,540 So I'm really happy to be here. 5 00:00:19,540 --> 00:00:21,260 Hopefully, everyone will get something out of it. 6 00:00:22,580 --> 00:00:25,480 So there's a few key things to go through today, really. 7 00:00:25,780 --> 00:00:43,640 The first thing is that I want to spend some time kind of comparing and contrasting on-premise detection and cloud detection, what the key differences are between them, and some critical architectural differences, but also just in terms of what an attacker's likely to try versus what they did on-premise, 8 00:00:43,640 --> 00:00:45,280 how a lot of that varies. 9 00:00:46,860 --> 00:00:58,280 Yeah, exactly, digging into the likely attacker activity, we're going to see what that actually comes out as on the cloud side of things, what you should be looking for, where you should be looking for it, and so on. 10 00:00:58,280 --> 00:01:04,480 And lastly, the simulation piece, sort of buzzword is this is the idea of bringing DevOps to detection. 11 00:01:04,660 --> 00:01:12,620 We've seen a real strong uptake in a lot of the DevOps tooling in the cloud space, because it makes everything easier if you can automate stuff. 12 00:01:12,660 --> 00:01:21,680 So we've spent quite a lot of time building out some tooling that we've open-sourced to do just that, and we'll talk through that in some more detail. 13 00:01:22,840 --> 00:01:28,580 But just before we get started, I'm sure pretty much everyone's seen some sort of threat actor pyramid. 14 00:01:28,600 --> 00:01:40,320 I've got a slightly different take on it to what you often see, in that rather than putting sort of China and Russia at the top and a bunch of script kiddies at the bottom, we're talking more about capabilities than we are about individual groups. 15 00:01:40,440 --> 00:01:49,640 So what we've got at the top is bespoke tactics, techniques, and procedures, things that attackers are developing and designing specifically for whatever campaign they're on. 16 00:01:49,680 --> 00:02:00,420 And the middle bracket, custom tooling, is where attackers are in a position where perhaps they're not developing their own ODAs or doing their own security research, but they are building their own implants, their own tools, and so on. 17 00:02:00,540 --> 00:02:07,200 And sort of the bottom end of the pile is people grabbing tools off GitHub and firing them off willy-nilly across the world. 18 00:02:07,660 --> 00:02:16,320 And the reason I bring this up is because, to me, the interesting part of detection sits at the top end there, at the bespoke TTPs, the custom tooling. 19 00:02:16,820 --> 00:02:21,900 And realistically, that's also a very expensive place to sit if you're trying to build detective capability. 20 00:02:22,100 --> 00:02:37,260 And so a lot of what I do is around driving the costs down or making it easier for organizations and for people to pick up the top end, the advanced end of detection, and use it within their organization. 21 00:02:37,280 --> 00:02:40,320 And that will come up quite a few times through today's talk. 22 00:02:41,880 --> 00:02:45,260 So first off, on-premise versus cloud. 23 00:02:45,260 --> 00:02:49,380 So there's quite a few key differences here, really, when it comes to attack detection. 24 00:02:49,520 --> 00:02:52,460 And one of the biggest is just in terms of the telemetry that we've got. 25 00:02:52,660 --> 00:03:03,260 So on-premise, historically, the vast majority of the useful telemetry and data that we'd be threat hunting in sat around the endpoint. 26 00:03:03,260 --> 00:03:10,020 So we're running endpoint detection response products, you know, pick your favorite from CrowdStrike, Tanium, Cyberese, and those sorts of things. 27 00:03:10,500 --> 00:03:25,320 And so we'd be running those on all of our endpoints, probably our servers too, and gathering information like process execution trees, you know, what's happening in memory, looking for specific syscalls and Windows API calls that might be malicious. 28 00:03:25,580 --> 00:03:27,520 And we do a lot of our detection on that. 29 00:03:27,520 --> 00:03:33,480 If you've paid dark trace or vector a lot of money, you might also have some network telemetry, but it's usually a secondary source. 30 00:03:33,480 --> 00:03:36,820 Likewise, with data you might be getting out of your applications. 31 00:03:37,120 --> 00:03:43,540 The one key exception there is I'm going to class Active Directory as an application in this little diagram here. 32 00:03:44,040 --> 00:03:52,880 Active Directory, obviously, is one of the big points we can get a lot of useful detection information out of on-premise, especially when it comes to sort of lateral movement and privilege escalation. 33 00:03:52,880 --> 00:03:56,660 But by and large, still, you know, we're focused in really on the endpoints of the servers. 34 00:03:57,440 --> 00:04:09,600 Now, when we go to cloud, we've still got all of that to some degree, especially if you're still running virtual machines in a lot of your cloud environments, you know, you're running containers, we've got container security monitoring solutions now that work with that. 35 00:04:09,600 --> 00:04:15,740 But actually, we've now got this box that sits around it all, which I'm going to call control plane telemetry. 36 00:04:15,780 --> 00:04:31,320 So that is data that we're getting out of the cloud itself and from a given cloud provider about what's being done from an administrative perspective inside the cloud account that you're looking at, or the Azure subscription or Google Cloud projects about who's creating, 37 00:04:31,320 --> 00:04:36,280 modifying, destroying what resources, where, when, and how they're doing that. 38 00:04:36,840 --> 00:04:43,060 And so that really actually forms then the core of your detective capability in the cloud. 39 00:04:43,100 --> 00:04:48,760 Attackers are still going to do some of what they used to on the endpoints and at the network level. 40 00:04:48,760 --> 00:05:03,240 But really, a lot of the threats we've seen, and in fact, a lot of the successful attacks, Capital One springs to mind, you know, happened against the control plane where an attacker's got hold of some credentials for the AWS account, the Azure subscription, 41 00:05:03,240 --> 00:05:04,660 and they're targeting that. 42 00:05:05,860 --> 00:05:12,700 Now, what comes with that is just a dearth of knowledge, really, in terms of what that actually looks like. 43 00:05:12,700 --> 00:05:17,720 We've had sort of 20, 30 years to build up an understanding of what an attack on premise looks like. 44 00:05:17,900 --> 00:05:25,940 We know what an attacker is going to try on a Windows network full of, you know, Windows laptops connected to an Active Directory infrastructure. 45 00:05:25,940 --> 00:05:27,160 It's all pretty well understood. 46 00:05:27,160 --> 00:05:28,980 And new techniques come out all the time. 47 00:05:29,020 --> 00:05:41,320 But at the end of the day, the broad attack vectors and a lot of the stuff we should be looking for are all pretty consistent, or at least reasonably well known, even if not always that easy to spot. 48 00:05:41,320 --> 00:05:43,880 The problem then comes when we move to the cloud. 49 00:05:44,360 --> 00:05:50,800 Not only are we dealing with more platforms than we used to, but also we just don't really have an idea of what a lot of this stuff looks like now. 50 00:05:50,940 --> 00:05:54,420 Attackers aren't targeting workstations and virtual machines. 51 00:05:54,420 --> 00:05:58,120 We're targeting serverless functions, Kubernetes clusters, the control plane. 52 00:05:58,600 --> 00:06:05,340 And so, what that looks like. 53 00:06:05,560 --> 00:06:07,200 What an attacker is likely to try. 54 00:06:07,400 --> 00:06:09,560 And it's just nowhere near as well understood. 55 00:06:10,200 --> 00:06:16,220 That said, from my experiences working in this space, I think there's a few key things to call out, really. 56 00:06:16,460 --> 00:06:22,180 One of which is that there's this concept of uncertainty of malicious intent. 57 00:06:22,660 --> 00:06:27,500 On premise, there's a lot of stuff that's very malicious and it's easy to signature off. 58 00:06:27,500 --> 00:06:28,940 That doesn't really exist in the cloud. 59 00:06:28,940 --> 00:06:31,440 We'll get into that in a bit more detail in a second. 60 00:06:32,020 --> 00:06:35,680 But one of the other things that comes with it is that context is key. 61 00:06:36,320 --> 00:06:42,960 Understanding what an attacker is doing in a specific environment rather than just what they're doing in general in a Windows estate. 62 00:06:44,020 --> 00:06:48,980 These things vary an awful lot in the cloud, whereas they didn't historically on premise to anything like the same degree. 63 00:06:48,980 --> 00:06:50,860 And again, get into that more in a second. 64 00:06:51,020 --> 00:06:55,740 And one of the big wins from a detection perspective in the cloud is that visibility tends to be a lot easier. 65 00:06:55,940 --> 00:07:01,480 And so, if you look at Google Cloud, for instance, we can turn on audit logs. 66 00:07:01,480 --> 00:07:03,200 In fact, they're turned on by default. 67 00:07:03,280 --> 00:07:16,280 And we can feed those into a Google Cloud organization-wide log sync and get all of the logs for all of our projects at the control plane layer fed straight into a single location and ship it from there off into our scene. 68 00:07:16,440 --> 00:07:24,540 And there's no more sort of, we've got our EDR on 90% of our laptops, but there's these 10% of legacy Windows 2000 boxes that we can't do anything with. 69 00:07:24,540 --> 00:07:25,740 There's nothing like that. 70 00:07:26,340 --> 00:07:38,800 The problem comes, actually, that now shadow IT is even more of a problem than it used to be, because it used to be that if you had shadow IT in your on-premise environments, you could probably still walk around to a switch and have a look at what was plugged into it. 71 00:07:38,900 --> 00:07:48,020 And now you've got pretty much zero visibility of marketing walking out to one of their third-party suppliers and saying, hey, build us a website for this marketing campaign. 72 00:07:48,420 --> 00:07:51,720 Here's an AWS account that we registered yesterday with one of our credit cards. 73 00:07:51,980 --> 00:07:53,800 Keeping track of that's really, really hard. 74 00:07:53,800 --> 00:08:03,260 And actually, one of the tricks we've seen a few organizations use is to get really friendly with the finance department and ask finance to essentially report all transactions for the cloud providers. 75 00:08:03,400 --> 00:08:06,100 And then you can chase down which teams are doing what that way. 76 00:08:06,220 --> 00:08:09,980 It's a bit of a low-tech solution by our usual approaches, but it works pretty well. 77 00:08:10,880 --> 00:08:23,160 And lastly, while we've got this fantastic ability to automate lots of things in the cloud now with all the APIs they expose, the problem now is that we're not the only ones leveraging that, right? 78 00:08:23,160 --> 00:08:26,680 Attackers are also automating a lot of what they do. 79 00:08:26,800 --> 00:08:35,400 The classic example here, again, with AWS is attackers leveraging access keys that are linked to GitHub for Bitcoin mining. 80 00:08:35,740 --> 00:08:51,220 You can subscribe to a public feed of commits to public GitHub repositories, and attackers tend to feed that straight into essentially their own continuous delivery platforms, which monitor the commits to public repositories, look for things that look like AWS access keys, 81 00:08:51,220 --> 00:08:57,560 and feed those straight into a system that will then deploy Bitcoin miners in the account of the compromised creds. 82 00:08:57,560 --> 00:09:11,700 It's gotten bad enough now that the meantime to compromise between keys hitting GitHub and Bitcoin mining startings, about 20 to 30 seconds, and Amazon themselves started monitoring that same feed and revoking keys when they find them. 83 00:09:11,700 --> 00:09:30,920 So while obviously that's pretty low-tech and in the grand scheme of things not that interesting, the fact that we're seeing that automated makes me think that in the long run we're going to start seeing attackers building out tooling and capabilities that focus on getting in and doing as much damage or as much of what they want to do as rapidly as possible before the defenders have woken up and realize there's a problem. 84 00:09:32,600 --> 00:09:34,740 And so I mentioned mindset shift here. 85 00:09:35,840 --> 00:09:38,380 We have this idea of uncertainty of malicious intent. 86 00:09:38,380 --> 00:09:40,480 So there are some things that we know are obviously bad. 87 00:09:40,480 --> 00:09:44,600 If you've got Mimikatz running on a system, that's probably not supposed to be happening. 88 00:09:44,600 --> 00:09:52,780 Likewise, someone accessing LSAS memory or, you know, there's a variety of fairly common things that are obviously bad and we can signature on them, right? 89 00:09:52,780 --> 00:09:56,540 That's how antivirus has been working for the last 20 years. 90 00:09:56,940 --> 00:09:59,540 Some things are usually bad. 91 00:09:59,940 --> 00:10:05,280 HTA spawning PowerShell is a classic example there, but actually that's not always bad. 92 00:10:05,320 --> 00:10:16,600 We were doing some threat hunting on one organization's estates and found that when we ran a query for PowerShell being spawned by MSHTA files, we got like 15,000 hits or something. 93 00:10:17,120 --> 00:10:20,440 So we immediately thought there'd been a major compromise and started digging into it. 94 00:10:20,440 --> 00:10:31,780 And it turns out that actually Hewlett-Packard enterprise printer drivers for a certain set of models work by, or the driver installations work by using an MSHTA file to spawn PowerShell to do the installation. 95 00:10:32,180 --> 00:10:35,680 So it's probably bad, it might not be. 96 00:10:35,800 --> 00:10:50,760 And then you've also got things like users requesting Kerberos tickets for services they've not spoken to before, you know, network comms that isn't normal, but doesn't immediately stick out as malicious and things that bear further investigation rather than things that you immediately signature on. 97 00:10:50,760 --> 00:10:55,960 The problem in cloud is that there are very few things that are obviously or usually bad. 98 00:10:56,300 --> 00:10:58,680 There's a few, but there's not many. 99 00:10:59,360 --> 00:11:17,100 The vast majority of what I'd expect an attacker to do and what I see people doing when they're doing offensive security testing revolves around abusing legitimate functionality, API calls that the cloud providers expose to, you know, install persistence mechanisms, 100 00:11:17,100 --> 00:11:22,160 escalate their privileges, access data they shouldn't be, all of these kinds of things. 101 00:11:22,160 --> 00:11:24,480 And so everything becomes about context. 102 00:11:25,780 --> 00:11:33,900 So let's say we've got a, we're in AWS, we've got an IAM user, we've had some access keys added to that IAM user. 103 00:11:34,060 --> 00:11:36,720 In principle, that might be okay, that might not. 104 00:11:36,720 --> 00:11:38,100 So we dig into it. 105 00:11:38,160 --> 00:11:47,620 It turns out that was made, or that change was made by a user account or a set of access keys associated with some kind of continuous delivery pipeline that we're running on premise. 106 00:11:48,580 --> 00:11:50,240 Okay, that's probably fine. 107 00:11:50,900 --> 00:11:53,700 You know, actually that might be expected in this environment. 108 00:11:54,240 --> 00:12:03,980 But if we've got a change made by an administrator from a weird network location, maybe without his 2FA that you'd usually be using or something else, that actually might not be. 109 00:12:03,980 --> 00:12:05,680 And so we might want to further investigate. 110 00:12:05,920 --> 00:12:09,280 But that might well be entirely dependent on the environment. 111 00:12:09,780 --> 00:12:11,740 And there are two things that will come out of this. 112 00:12:11,780 --> 00:12:16,700 One of which is that over time attackers are going to become more and more context aware. 113 00:12:16,700 --> 00:12:22,760 I'm already seeing that sort of continuous integration, continuous delivery systems are favorite targets of a lot of attackers. 114 00:12:22,760 --> 00:12:24,480 I'll get into that in a bit. 115 00:12:24,940 --> 00:12:30,520 And so being able to masquerade as sort of known trusted entities will become more of a thing over time. 116 00:12:31,880 --> 00:12:42,120 But equally, that then presents a real problem for managed detection and response providers or MSSPs who are trying to monitor a wide variety of client estates. 117 00:12:42,220 --> 00:12:46,780 Because there's no such thing anymore as, you know, we detect known bad and that's great. 118 00:12:47,400 --> 00:12:52,980 We've got to be able to monitor on a per environment basis, not even per customer or client. 119 00:12:53,480 --> 00:12:57,260 And so that's where things like user behavior analytics starts becoming really useful. 120 00:12:57,360 --> 00:13:00,860 You know, is this behavior normal for this environment, for this specific entity? 121 00:13:00,980 --> 00:13:03,120 Or is this something that we've never seen before? 122 00:13:03,380 --> 00:13:09,300 Even just things like least frequency analysis, you know, for this environment, how often do we see this particular API call made? 123 00:13:09,440 --> 00:13:13,080 You know, if it's never been called before, then that might warrant some investigation. 124 00:13:13,140 --> 00:13:16,280 If it happens 500 times a day, then well, it's probably fine. 125 00:13:17,480 --> 00:13:26,380 Equally, the other thing that comes out of this is that there are some things that just don't exist from on premise in terms of actions and attackers likely to perform. 126 00:13:26,580 --> 00:13:41,020 And so let's say, again, we're working in AWS, we've got an attacker who comes in and they run IAM create policy version to create a new version of the policy document for one of the policies that's attached to them. 127 00:13:41,020 --> 00:13:47,460 And so this is changing the permissions they've got by altering the permissions definitions, basically. 128 00:13:47,720 --> 00:13:56,560 And actually, if you can do create policy version on any resource, and you have policies attached to you, then that essentially gives you admin access. 129 00:13:56,560 --> 00:13:59,500 There's a concept that doesn't really exist to the same degree on premise. 130 00:13:59,500 --> 00:14:03,880 So there's a number of these sorts of actions where we don't really have an on premise equivalent. 131 00:14:03,880 --> 00:14:07,560 And these are kind of new techniques that defenders are going to have to get their heads around. 132 00:14:08,780 --> 00:14:17,380 This is then massively complicated by the fact that your average enterprise really isn't one cloud provider with a few cloud accounts these days, right? 133 00:14:17,380 --> 00:14:24,880 I can't remember the last time I went to a client and they said, we're just in AWS or we're just in Azure, and it turned out to be true. 134 00:14:25,820 --> 00:14:27,360 F-Secure is a great example for that. 135 00:14:27,360 --> 00:14:33,760 In fact, we use AWS very heavily, but we have got bits of Google Cloud and Azure lying around as well for various reasons. 136 00:14:34,220 --> 00:14:41,340 But even if you're just looking at that, that's a fairly complex thing to deal with in and of itself. 137 00:14:41,340 --> 00:14:46,880 Multiple cloud providers and completely different technology stacks and different log formats, all of that kind of thing. 138 00:14:46,880 --> 00:14:58,780 When you then expand that to include everything else that exists in most organizations, cloud estates these days, you know, we're probably pulling container images down from Docker Hub, storing our source code in GitHub or something like that. 139 00:14:58,780 --> 00:15:08,440 We're probably using Slack for comms and we've got Office 365 for our emails and all of these are fairly juicy targets for an attacker in one respect or another. 140 00:15:08,700 --> 00:15:14,940 And they all need monitoring to some degree, which presents a pretty big problem for a lot of organizations really, just getting their heads around all of this. 141 00:15:16,300 --> 00:15:33,480 So when it comes to actually designing how you do detection in the cloud, I think it would be worth just pausing for a second and talking through a few key observations that we've made about how to successfully try and integrate sort of that much of a disparate estate, 142 00:15:33,480 --> 00:15:35,860 you know, a wide variety of technology platforms. 143 00:15:36,040 --> 00:15:40,600 And the big one really is to centralize everything. 144 00:15:41,000 --> 00:15:51,040 Have a single scene that handles your on-premise, your cloud, your everything, ship all of your logs into it from everywhere, you know, all into the same single place. 145 00:15:51,060 --> 00:16:09,740 One thing that we've definitely noticed is that if you have a security operations center or a threat hunting team, what have you, who are monitoring quite a disparate estate, the more things you make them go look at, the more likely it is that some things are going to get missed or people aren't really paying as much attention to one thing as the others. 146 00:16:10,040 --> 00:16:11,700 So put it all in one place. 147 00:16:11,860 --> 00:16:19,340 And the other thing you'll note here with this diagram is that for each of the major cloud providers, there's like one account that we're shipping logs off to before they go into the scene. 148 00:16:19,720 --> 00:16:22,120 And there's a few big benefits to this, really. 149 00:16:22,120 --> 00:16:39,420 One of which is that if we're shipping all of our logs to a centralized location, we can lock that down to some degree so that only our security people can access it, which means that all the logs, even when they're in the cloud still, are in a place that an attacker probably doesn't immediately have access to if they've compromised one of the project's accounts. 150 00:16:39,600 --> 00:16:50,400 And it means that it's then relatively straightforward as well for defenders if, let's say, there's some issues with the scene or they need more data than that's actually being shipped off to the scene for various reasons. 151 00:16:50,740 --> 00:16:55,720 Spunk charging per gigabyte is usually one of the examples there behind why that doesn't happen. 152 00:16:55,720 --> 00:16:56,880 You might need more data. 153 00:16:56,880 --> 00:17:02,060 So you go into this one location for each cloud provider, and that's where that data lives. 154 00:17:03,380 --> 00:17:06,900 And there's also a few key data sources to be looking at here. 155 00:17:06,900 --> 00:17:09,960 And I'm going to pick a couple out to dig into in detail. 156 00:17:10,060 --> 00:17:14,440 But you'll notice from this table, the control plane audit logs have bolded at the top there. 157 00:17:15,080 --> 00:17:16,620 And there's a very good reason for that. 158 00:17:16,620 --> 00:17:25,280 And that is that, in my experience, something like probably 70-80% of attacker activity is going to show up in those control plane logs when we're looking at the cloud. 159 00:17:25,340 --> 00:17:39,560 Just because even if an attacker is targeting virtual machines or what have you, they're eventually likely to want to pivot out into the underlying control plane layer and start messing around with other resources in the environment, data stores, or escalate their privileges or anything like that. 160 00:17:39,560 --> 00:17:41,780 That's all going to come down through the control plane. 161 00:17:43,840 --> 00:17:52,960 So for AWS, you've got CloudTrail, you've got audit log in both Azure and GCP, and actually Kubernetes has that for the Kubernetes API too. 162 00:17:53,600 --> 00:17:59,560 And as we say here, this logs almost every control plane level event. 163 00:17:59,560 --> 00:18:05,960 I say almost because people keep proving me wrong with that with AWS CloudTrail. 164 00:18:06,560 --> 00:18:11,900 We've seen a few people now digging around trying to find API calls that aren't correctly logged, and there are a few here and there. 165 00:18:12,000 --> 00:18:16,960 AWS do their best to catch them, but I'm sure that same issue will exist in Azure and GCP. 166 00:18:17,300 --> 00:18:20,520 But by and large, you're going to get logging for pretty much everything that happens. 167 00:18:21,540 --> 00:18:26,200 The key thing to remember here is to make sure that you have it all turned on properly. 168 00:18:26,620 --> 00:18:35,280 AWS is particularly bad for this in that CloudTrail has a number of different knobs and settings you can tweak that vary what it logs and how it logs it. 169 00:18:35,660 --> 00:18:45,280 So often we find that either multi-region logging hasn't been turned on, which means you're only getting data for US East 1 or wherever it is you've got the trail deployed. 170 00:18:45,400 --> 00:18:50,780 And equally, if you don't have global events enabled, then you don't get anything for IAM or a couple of other critical services. 171 00:18:51,080 --> 00:18:57,860 So it's well worth digging into your provider's documentation and working out exactly what you should have turned on there and making sure you've got that. 172 00:18:59,040 --> 00:19:02,440 Now, Cloud Native Detection Services. 173 00:19:02,440 --> 00:19:12,800 So GuardDuty is probably the big name here, but you've also got Advanced Threat Protection from Azure and Security Command Center from Google has some of this built into its tool. 174 00:19:13,000 --> 00:19:19,000 And what we've got here is automatic detection of lower sophistication attacker activity. 175 00:19:19,620 --> 00:19:22,860 And what I mean by that is this is essentially sort of antivirus for cloud. 176 00:19:22,860 --> 00:19:29,700 We're spotting known bad things like network communications to known malware command and control addresses. 177 00:19:30,220 --> 00:19:45,560 In the case of GuardDuty, it'll detect when someone tries to run commands from AWS API commands from Kali Linux if they haven't changed the user agent on the basis that obviously everyone using Kali Linux must be bad, so therefore we should be firing alerts for it. 178 00:19:46,700 --> 00:19:54,980 And while they have their flaws, actually they're still a pretty cost effective way of detecting sort of low complexity, low sophistication attacks. 179 00:19:55,140 --> 00:19:58,260 So I would always turn it on. 180 00:19:58,380 --> 00:20:00,000 I just wouldn't rely on it. 181 00:20:00,120 --> 00:20:03,040 You know, you need more than this to be able to spot more advanced attackers. 182 00:20:04,580 --> 00:20:07,580 I'm also leaving out Sentinel when we're talking about this. 183 00:20:07,680 --> 00:20:12,640 So Sentinel is Microsoft's scene that they go with or they pair with Azure. 184 00:20:12,900 --> 00:20:17,160 And actually Sentinel's pretty capable by and of itself. 185 00:20:17,220 --> 00:20:27,420 And there's a pretty great team of people building some pretty serious capability into that that goes well beyond what we've seen in GuardDuty and what they've got in the advanced threat protection package. 186 00:20:27,720 --> 00:20:33,200 And so if you're in Azure and you've not got anything like this set up yet, I'd definitely be looking quite hard at Sentinel. 187 00:20:33,640 --> 00:20:36,820 We've been pretty impressed with it where we've been running into it so far. 188 00:20:40,180 --> 00:20:44,500 Service specific telemetry is the last one I want to dig into here when we're talking about log sources. 189 00:20:44,500 --> 00:21:04,100 So here we're talking about the kind of logs you get out of things like storage accounts, access and object logs, executions of serverless functions, audit logs from your key management system keys, this sort of thing where we've got specific telemetry about what's happening inside a specific service or set of resources. 190 00:21:05,000 --> 00:21:10,120 This is a bit of an interesting one because everything here is going to be case by case. 191 00:21:10,520 --> 00:21:12,700 Sometimes this data could be extremely valuable. 192 00:21:12,700 --> 00:21:14,680 Sometimes it could be basically useless. 193 00:21:15,280 --> 00:21:25,400 So you're going to end up needing to tune this on a per environment basis and choose what to turn on and where, you know, as relevant depending on what use you think you're going to get out of the data. 194 00:21:25,400 --> 00:21:30,400 So one good example here is S3 access logs in AWS. 195 00:21:31,320 --> 00:21:40,540 If we've got an S3 bucket that's doing nothing but hosting images and CSS files for a static website, we really don't care who's accessing it. 196 00:21:40,540 --> 00:21:50,860 On the other hand, if you have your organization's most important data in some S3 buckets, it's probably well worth having those logs turned on so that you can spot who's accessing them where, when, and how. 197 00:21:51,700 --> 00:21:54,020 So, yeah, this one takes a bit more thought. 198 00:21:54,040 --> 00:22:02,320 But once you've got, like, the key main sources turned on, the audit logs, that sort of thing, this is well worth looking into, I think. 199 00:22:04,180 --> 00:22:16,620 So next up, one of the real problems we've got at the moment is having talked now about what data you can get to detect attacks with, we've got to work out what an attack actually looks like, right? 200 00:22:16,620 --> 00:22:24,860 I've spoken a little bit about that to some degree, but we've got a real problem actually at the moment just in terms of getting hold of data to show what attackers are actually doing in the cloud. 201 00:22:25,460 --> 00:22:37,740 So I've been talking on and off to MITRE's cloud team about this for the MITRE attack framework, and they're doing their best, but they're really struggling to get their hands on decent threat intel, and I think the rest of the industry's in the same boat. 202 00:22:37,740 --> 00:22:53,280 So this is the on-premise attack framework on the left, and the cloud one on the right, for those who aren't familiar with MITRE attack, it's a taxonomy of likely attacker tools, techniques, and procedures, TTPs, things that attackers likely to try in a given environment. 203 00:22:53,280 --> 00:22:55,340 So the one on the left, as I say, is Windows. 204 00:22:55,340 --> 00:22:59,640 As you can see, there's a lot of things in there that we've noted attackers doing over time. 205 00:23:00,220 --> 00:23:04,200 For cloud, much less so, as you can see on the right there. 206 00:23:04,760 --> 00:23:14,760 And so if we were to purely rely on what we know attackers are doing actively, we're barely going to spot anything, because there's just not very much information out there about this. 207 00:23:14,760 --> 00:23:21,800 And where we see reports of, oh, we think an attacker might be doing X, Y, and Z, it's actually often really hard to validate that. 208 00:23:21,800 --> 00:23:31,200 You can't just sort of jump onto Google and dig out an old Mandiant or CrowdStrike PDF, you know, from the last APT29 report or whatever, and there's a lot less information sharing on this. 209 00:23:31,200 --> 00:23:33,720 And part of that, I think, is actually the cloud provider's fault. 210 00:23:33,860 --> 00:23:38,280 I have to say, we've not had great experiences working with some of them on this. 211 00:23:38,820 --> 00:23:47,180 They're always very keen to hide any evidence of sort of real malicious activity, just because it's potentially damaging to their reputation. 212 00:23:47,180 --> 00:23:49,680 But that doesn't help us from a defensive perspective. 213 00:23:50,660 --> 00:23:58,040 So that said, you know, there are a few places you can draw inspiration from, I think, for working out what an attacker is likely to do in your environment. 214 00:23:58,840 --> 00:24:08,160 Certainly Scott Piper at Summit Root is going to be one of the big ones for AWS, and the guys over at Rhino Security have been putting out some great research for years in this space. 215 00:24:08,160 --> 00:24:13,640 If you're a fan of Google Cloud, then GitLab's internal red team have done some pretty fantastic stuff there. 216 00:24:13,640 --> 00:24:17,380 And there's a variety of tools, too, that are pretty useful for working this out. 217 00:24:17,380 --> 00:24:25,460 AWS especially, Paku from Rhino, is full of modules of things that an attacker might want to do. 218 00:24:25,900 --> 00:24:32,620 And so given how powerful it is, you know, we use it pretty frequently when we're doing offensive security testing of one kind or another. 219 00:24:32,740 --> 00:24:41,180 It's pretty likely that a lot of the lower skilled attackers or people using public tooling are going to be taking Paku and trying to do stuff with it. 220 00:24:41,180 --> 00:24:51,440 So if you can signature on most of what Paku's doing, or at least be in a position where you've got the right data to be able to spot it if it does happen, that puts you in a pretty decent place. 221 00:24:51,440 --> 00:25:06,700 So go have a look at a lot of the existing tooling that's out there, and see what penetration testers say they're doing, you know, read around on the blog posts they publish, and base a lot of your detective activity on what looks like it's most relevant from what you see that way. 222 00:25:06,740 --> 00:25:13,400 And that's a pretty great way, in my experience, to expand beyond sort of MITRE ATT&CK and the other common industry frameworks. 223 00:25:14,420 --> 00:25:20,020 But even going beyond that, we can take a look at a bit of a high level as to what an attacker's actually likely to do. 224 00:25:20,520 --> 00:25:31,360 So there's four key areas, really, that I think I see us, other pen testing firms, red teams, and so on, exploiting on engagements. 225 00:25:31,560 --> 00:25:42,080 The big ones really are identity management, pivoting around between environments, source code management, continuous delivery, and application vulnerabilities. 226 00:25:42,740 --> 00:25:45,100 You know, we'll dig into those one after the other. 227 00:25:45,460 --> 00:25:48,460 So first up, identity management exploitation. 228 00:25:48,460 --> 00:26:03,780 So here what we've got is where we've got an attacker who's managed to get hold of some credentials in some fashion or another, either from access keys they've stolen out of source code or what have you, service account details that way, or they're targeting a single sign-on mechanism, 229 00:26:03,780 --> 00:26:05,980 they found some credentials somewhere and they're doing that. 230 00:26:06,520 --> 00:26:12,480 What we end up with is an attacker leveraging credentials to get into the environment and start doing things from there. 231 00:26:12,480 --> 00:26:14,920 So this is pretty common. 232 00:26:15,340 --> 00:26:26,220 Verizon's 2020 data breach investigation report reckons that somewhere in the region of about 80% of cloud breaches involved lost or stolen credentials. 233 00:26:26,480 --> 00:26:29,400 So this really is the big ones we're paying attention to. 234 00:26:31,020 --> 00:26:44,220 This also comes up in another context, which I'll get into in a minute, but actually making sure that you've got sensible procedures around how you handle credentials, enforcing multi-factor authentication and so on is really important. 235 00:26:44,680 --> 00:26:51,040 The other thing to note here is that these credentials are often found in what seem like fairly stupid locations. 236 00:26:52,520 --> 00:27:01,360 Performing penetration tests and stuff, we've often found them in file shares, SharePoint sites, source code repositories to a degree too. 237 00:27:01,580 --> 00:27:03,360 People leave them lying around. 238 00:27:03,760 --> 00:27:05,840 And so that can be pretty problematic. 239 00:27:06,140 --> 00:27:21,500 But even in cases where people haven't left the credentials lying around, actually, a lot of the single sign-on tools that are used, a lot of the ways we do identity management in the cloud aren't as well understood as some of the legacy on-premise equivalents. 240 00:27:21,500 --> 00:27:25,320 So people make stupid mistakes, or say stupid mistakes. 241 00:27:25,400 --> 00:27:29,360 People make mistakes through lack of education, lack of understanding of what's going on. 242 00:27:29,360 --> 00:27:37,940 In this particular case, we had an organization deploy AWS Cognito, but pretty much by default and with the default configurations enabled. 243 00:27:38,580 --> 00:27:45,520 For those who aren't familiar, Cognito is a single sign-on user management tool that's designed to be integrated into web applications. 244 00:27:45,720 --> 00:27:52,280 So the idea is that you can use this to do your user management, so you don't have to implement that in your own web app, and there's an AWS service that takes care of it for you. 245 00:27:52,280 --> 00:27:56,080 You can also plug it into a few AWS services natively. 246 00:27:56,120 --> 00:28:01,460 So in this particular case, they'd used it as the front end for a fairly large data store. 247 00:28:02,300 --> 00:28:09,320 And because everything had been left at default, you had open registration enabled, so anyone could come along and register an account. 248 00:28:09,320 --> 00:28:16,240 It's what you want if you've got a public-facing web application, probably not if it's the authentication mechanism for a sensitive data store. 249 00:28:16,380 --> 00:28:28,540 And then it turns out that by default, if you don't configure multiple user account groups, then everyone goes into the default group, and the default group is administrator in the data storage thing that we were looking at. 250 00:28:28,540 --> 00:28:35,240 So we essentially just registered ourselves an account coming in from the internet and had admin access to this big data store. 251 00:28:35,620 --> 00:28:39,660 And so the important thing to note here is they'd not deliberately misconfigured it. 252 00:28:39,660 --> 00:28:41,300 They'd not even accidentally misconfigured it. 253 00:28:41,300 --> 00:28:47,780 They'd left it as default, and it turns out the defaults had really bitten quite badly in the back side in this case. 254 00:28:49,440 --> 00:28:51,880 Next up, pivoting around between environments. 255 00:28:51,880 --> 00:28:54,780 So this is quite a common one we see. 256 00:28:54,780 --> 00:29:09,020 Sorry, and we find that it's really common, obviously, for attackers to go in with phishing campaigns, either for phishing for credentials or getting a foothold on a network with, you know, pick your favorite impromptu, Cobalt Strike, Covenant, whatever, 257 00:29:09,020 --> 00:29:11,640 and pivot around from there, move up into the cloud. 258 00:29:11,900 --> 00:29:16,580 And so actually, that can be a pretty effective route. 259 00:29:16,860 --> 00:29:19,320 We're seeing it an awful lot that way around. 260 00:29:19,320 --> 00:29:28,400 We have also seen a few cases where attackers have somehow breached the cloud environment, either for an application vulnerability or something else. 261 00:29:28,460 --> 00:29:36,620 And they've been able to pivot around inside the cloud until they get to, say, an S3 bucket or some other data store that's got something sensitive in it. 262 00:29:36,800 --> 00:29:54,780 And in a couple of cases, we found either sort of SSH keys that allowed us to pivot back on premise over the VPNs they had set up in the cloud, or credentials to sort of third party management solutions that they were using, that we were then able to go in and manipulate what was happening there. 263 00:29:55,280 --> 00:29:59,000 So actually, the danger here is sort of in both directions, as much as not in the diagram. 264 00:29:59,220 --> 00:30:03,680 You can either start on premise and pivot into the cloud, or you can start on cloud and pivot on premise. 265 00:30:03,700 --> 00:30:07,200 And you'd probably pivot between cloud environments as well. 266 00:30:07,360 --> 00:30:10,340 Not something I've ever done personally, but I'm sure it'd be feasible. 267 00:30:10,560 --> 00:30:25,800 And so as an example, one thing I've seen pulled off on a number of occasions now, why I generally recommend not using Active Directory for your cloud single sign-on, is that we've seen cases where an organization's on-premise Active Directory has been compromised, 268 00:30:25,800 --> 00:30:27,980 or we'd compromised it as part of a red team. 269 00:30:28,720 --> 00:30:38,660 We've got ourselves in, we've added some users that we have access to, to the right groups in Active Directory to get cloud access, AWS, Azure, what have you. 270 00:30:38,840 --> 00:30:45,060 And then by virtue of having that, we then become administrators in the cloud, at which point we've owned the cloud as well. 271 00:30:45,700 --> 00:30:54,200 So either it's well worth investing in hardening your Active Directory, if you are going to do Active Directory single sign-on to the cloud, and especially Azure AD. 272 00:30:54,620 --> 00:31:05,200 Or equally, we've seen a number of organizations now go out and buy Ping, Okta, some other third-party single sign-on, and use that for their cloud access, completely separate to all their on-prem stuff. 273 00:31:05,400 --> 00:31:15,520 But it's definitely worth segregating or separating your Active Directory environment and your cloud environments as much as possible, given how weak most legacy AD environments are. 274 00:31:16,900 --> 00:31:19,920 Next up, source code management and continuous delivery. 275 00:31:20,160 --> 00:31:27,900 And so here, this is where we've got an attacker who's targeting code repositories or the pipelines that are deploying these things into the cloud. 276 00:31:28,500 --> 00:31:35,360 We're manipulating code being deployed as applications, or we're manipulating infrastructure as code that controls the cloud environment itself. 277 00:31:35,440 --> 00:31:38,600 And if we can commit code, we can have the pipeline push it out. 278 00:31:38,600 --> 00:31:42,840 If we can manipulate the pipeline, we can modify the code as it goes through the pipeline. 279 00:31:43,800 --> 00:31:48,080 Essentially, your cloud is as secure as the things deploying the applications and infrastructure. 280 00:31:48,820 --> 00:31:50,520 And that's something I think often gets overlooked. 281 00:31:50,520 --> 00:31:55,400 It's really important to also take a look at your source code repositories and pipelines. 282 00:31:56,660 --> 00:31:58,760 Next up, application exploitation. 283 00:31:58,760 --> 00:32:09,520 So we've got an attacker coming in from the web, and we're going back to sort of the early days of application security, where SQL injection and remote code execution was something you found on a regular basis. 284 00:32:10,020 --> 00:32:12,000 Obviously, it's a lot less common now. 285 00:32:12,000 --> 00:32:22,140 But we've seen new classes of vulnerabilities crop up, or we've seen things like server-side request forgery become more relevant now than they used to be on premise, thanks to instance metadata services and so on. 286 00:32:22,700 --> 00:32:33,620 But being able to pivot into the cloud via an application attack is something that is obviously very useful to an attacker, but it's something that gets exploited a fair bit, too, in my experience. 287 00:32:33,620 --> 00:32:43,220 It's usually vulnerabilities in commercial off-the-shelf components, or open source apps that are being deployed that have been misconfigured, in my experience, and we see a lot of that. 288 00:32:44,680 --> 00:32:52,560 But generally speaking, obviously, this is still a significant threat, and that's why companies still keep investing in AppSec teams. 289 00:32:52,920 --> 00:33:03,240 On the plus side, though, when it comes to application exploitation, it's generally a lot easier to spot this stuff in the cloud than it was on premise, or at least a lot easier to gather the right data for it. 290 00:33:03,500 --> 00:33:18,480 In this case, we're running an Azure Kubernetes cluster, we've got some applications running in, and by the time the traffic hits the application, we actually exploit it, it's passed through probably a web application firewall on the front, we've got some kind of API gateway, 291 00:33:18,480 --> 00:33:33,280 we've got the ingress controller going into the Kubernetes cluster, we've got the logs from the pod, from the container that the app's running in, we've got whatever application logs are being generated, and we can ship all of those out into our scene, 292 00:33:33,280 --> 00:33:35,120 and we can monitor and react to that. 293 00:33:35,440 --> 00:33:39,880 So there's quite a lot more we can do here than on premise. 294 00:33:39,960 --> 00:33:50,160 Also worth noting, if you do decide to go down the route of detecting against things happening in your applications, you should probably focus in on looking at authenticated traffic. 295 00:33:50,360 --> 00:33:57,280 I'm sure we're all aware that most applications on the internet get bombarded with SQL map scans and all that sort of stuff. 296 00:33:58,120 --> 00:34:03,980 The easiest way to discard all that and focus in on the interesting data is by looking at what your authenticated users are doing. 297 00:34:08,140 --> 00:34:17,120 So we talked about a variety of different things that an attacker is likely to try, and this is essentially the paths we go down when we're looking to work out what we should be spotting in a given environment. 298 00:34:17,320 --> 00:34:30,340 And threat model your environment, and work out what an attacker is likely to try, how they're going to do it, what the likely attack paths are, prioritize those based on what you think is most likely to be exploited, either because it's easy or because it's the most obvious or whatever else. 299 00:34:31,080 --> 00:34:37,440 Then define those attack paths, ideally in some sort of code format so it's easy to share around or machine read. 300 00:34:38,360 --> 00:34:48,920 Verify that the right logs, telemetry is available to defenders, and then go in and do some actual testing to see whether if we fire these test cases off, we get the right log data, we get something that we can action. 301 00:34:51,650 --> 00:35:05,390 But if you're not familiar enough, really, with your cloud environments to be able to jump in and do this, or you don't have the ability to do it at the scale your organization is working at, you've got to pick somewhere to start. 302 00:35:05,390 --> 00:35:19,750 And one of the easiest places, some of the biggest quick wins, are around either the objective end of the kill chain, where an attacker is looking to get access to key data sources or to start ransomware-ing everything or whatever else, and look at what those are, 303 00:35:19,750 --> 00:35:28,450 put key monitoring around that, but also look at infrequent actions that are generally highly privileged or affect your security model. 304 00:35:28,450 --> 00:35:46,290 Things like anything that involves turning off or altering the telemetry you're getting, modifying cloud trail, turning guard duty off in AWS, changing your audit log configs in Azure, or anything around manipulating your permissions model, your IAM policies in AWS, 305 00:35:46,290 --> 00:35:50,290 RBAC in Azure, anything that's modifying any of that. 306 00:35:50,290 --> 00:35:54,490 Those will make really good points to build out some initial detection around. 307 00:35:57,890 --> 00:36:05,050 So let's say now we've got some detection that we've built, and we think we've got some use cases implemented to spot some stuff. 308 00:36:05,650 --> 00:36:07,330 How do we validate those? 309 00:36:07,330 --> 00:36:14,790 As I say, by this point, we should have a set of likely attack paths identified, so we know what we're testing for, we've got some use cases for that. 310 00:36:14,930 --> 00:36:30,570 We then go and execute those attack paths, usually manually, and we then review whether we actually fired off some alerts or not, and then we work out what we need to do next based on how much of our detection actually worked as expected. 311 00:36:30,790 --> 00:36:43,550 Now, some of this, actually, we can quite easily automate, in theory at least, certainly executing the attack paths where it's targeting the control plane layer or we're targeting things like Kubernetes that are designed to be automated. 312 00:36:44,170 --> 00:36:47,070 It's generally fairly easy to automate the attack paths as well. 313 00:36:47,070 --> 00:37:00,010 One of the more interesting ones is doing an automated gap analysis, essentially spotting what fired that should have done, what didn't fire, and so on and so forth, so that we can essentially track over time in an automated way our improvements, what's working, 314 00:37:00,010 --> 00:37:00,970 what's not. 315 00:37:02,370 --> 00:37:09,470 And so these are all great ideas, but actually the industry is still mostly working on a manual basis for all of this, especially in the cloud. 316 00:37:09,730 --> 00:37:18,410 So one thing I think, one really important lesson to take away from the rest of the cloud community is treat everything as code. 317 00:37:18,410 --> 00:37:25,210 DevOps guys have really got this down, and we're starting to see the security industry get on board as well in the detection space. 318 00:37:25,410 --> 00:37:37,190 And the big reason for that, really, is that if all of your detection is defined as code, it makes it much easier both to share knowledge internally, because anyone can just take a look at your rules, and if they're in a common format that they already understand, 319 00:37:37,190 --> 00:37:41,410 you know, some people have written some new ones, they can just go in and they can start reading it and get their heads around it. 320 00:37:41,470 --> 00:37:46,210 And if we have reasonably common code formats, it also makes it much easier to share. 321 00:37:46,410 --> 00:38:05,110 And so if we can share information on how to spot, you know, what given threat groups are up to, specific TTPs, and we can either post those on GitHub or share them between sort of some of the closed threat intel sharing groups, it means that everyone's detection gets better for comparatively minimal effort on the part of an individual organization. 322 00:38:05,390 --> 00:38:10,630 A few big things to pick out in this space, Sigma by Florian Roth. 323 00:38:10,630 --> 00:38:23,690 So we've got a set of seam agnostic rules, essentially, that there's then a compiler for that produces your Splunk or your QRadar or your Elasticsearch queries off the back of this common rule set. 324 00:38:23,690 --> 00:38:31,510 So someone writes a Sigma rule for a particular TTP, and then you have the ability to use that in every platform going pretty much. 325 00:38:31,790 --> 00:38:33,550 There's a lot of compilers in there. 326 00:38:33,910 --> 00:38:36,490 The other interesting one in this space is Jupyter Notebooks. 327 00:38:36,490 --> 00:38:45,630 So this is more sort of threat hunting, you know, proactive investigation rather than reactive alerting based on known TTPs. 328 00:38:45,770 --> 00:38:52,250 But the idea of a Jupyter Notebook, for those who aren't familiar, is it's essentially a way of having a document with Python embedded in it. 329 00:38:52,270 --> 00:39:09,130 And so you can run code inside your documents, basically, and tie explanations of what's happening and the data that an analyst is seeing, and directly to the executing code, which makes it quite a nice way to share information, share threat hunts. 330 00:39:09,790 --> 00:39:12,790 And the Spectrops guys in particular have been doing some fun stuff with that. 331 00:39:12,790 --> 00:39:14,610 But it's starting to take off now. 332 00:39:14,610 --> 00:39:18,010 It's a data science thing, but it's been quite nicely repurposed now. 333 00:39:18,610 --> 00:39:26,830 And if you're looking for another conference talk to watch, John Lambert's the GitHubification of InfoSec from T2 last year. 334 00:39:26,830 --> 00:39:40,290 It was a really interesting presentation on how the industry needs to work better together to start making it more expensive for attackers to compromise organizations by sharing knowledge better than we do right now. 335 00:39:42,920 --> 00:39:54,560 Now, one of the interesting problems we've had historically in the cloud is that a lot of the existing solutions for automatically testing attacker activity don't extend to the cloud. 336 00:39:54,740 --> 00:39:57,160 They're entirely focused on on-premise at the moment. 337 00:39:57,160 --> 00:40:07,020 So things like MITRE Caldera, Atomic Red Team, there's a number of tools out there that have been very successful at automating attack simulation on-premise estates. 338 00:40:07,180 --> 00:40:09,540 But actually, we didn't have that for the cloud. 339 00:40:09,540 --> 00:40:15,200 So I went away and built one called Leonidas, which we've open-sourced. 340 00:40:15,620 --> 00:40:28,760 So Leonidas is essentially a web API where we have a variety of different attack definitions defined within it, and each API endpoint is a test case you can execute. 341 00:40:29,220 --> 00:40:47,120 So under the hood, actually, the way it works, we took a slightly different approach to how a lot of these tools work in that it's been designed to make it as easy as possible for you to share the ability to execute these test cases and to detect them between your team members internally and within the wider community, 342 00:40:47,120 --> 00:40:47,540 too. 343 00:40:47,540 --> 00:40:57,300 So let's say we've got a security team, we've got some analysts, we've got some red teams, purple teams, whatever, and one of them's read a new blog post about some new TTP in the cloud that they're interested in. 344 00:40:57,800 --> 00:41:00,760 So they sit down and they define this in our format. 345 00:41:00,760 --> 00:41:02,880 We've got like a YAML-based definitions format. 346 00:41:03,740 --> 00:41:07,720 And you write the TTP, it gets checked into a Git repository. 347 00:41:07,900 --> 00:41:16,100 We then have a CICD pipeline that essentially builds out a copy of the API based on the definitions that are in the Git repository. 348 00:41:16,640 --> 00:41:23,800 And that runs as a serverless function at the moment, a Lambda function AWS, but we're building out support for Azure and GCP, too. 349 00:41:24,180 --> 00:41:30,740 And so you've then got this API exposed, and we can have some target resources that we're looking to attack. 350 00:41:31,280 --> 00:41:39,940 Your purple team comes in or your red team comes in, executes some attacks against these target resources, and we have our logs shipped out to our scene. 351 00:41:40,500 --> 00:41:49,940 And so we can then verify whether when we triggered this new TTP, whether we have the right logs in place, whether we spotted it or not, and whether we need to tweak things further. 352 00:41:50,860 --> 00:41:57,580 So from these definitions, we define the actual test case itself. 353 00:41:57,580 --> 00:42:00,700 In this particular case, this is an AWS test case. 354 00:42:00,700 --> 00:42:06,280 We're looking at enumerating cloud trails, which as an attacker you probably want to do to see if you're being monitored or not. 355 00:42:06,280 --> 00:42:17,440 And so we've got a single line of code here, Python calling the describe trails API function on the cloud trail APIs in AWS. 356 00:42:17,540 --> 00:42:27,860 And behind the scenes, the framework abstracts away a lot of things around what identity we're executing this as, which region are we targeting, all these kinds of things. 357 00:42:27,860 --> 00:42:40,460 And so the actual code that's written for these test cases is pretty minimal, which obviously then makes it easier to write a lot of these quite fast, or make it easy for less skilled analysts to get involved in developing these. 358 00:42:41,800 --> 00:42:47,480 We also define what you need to detect the use case as part of the same definition file, right? 359 00:42:47,480 --> 00:42:51,480 So we've got both the test case and the detection case defined together. 360 00:42:51,480 --> 00:43:03,820 Here we've got a detection case that says we've got a event name of describe trails with a particular event source, and we're likely to find that in cloud trail audit logs. 361 00:43:03,820 --> 00:43:12,420 So we can then compile that, well, we usually compile it down to Sigma and then rely on Sigma's compilers to compile it out for the different SIEM solutions. 362 00:43:12,420 --> 00:43:18,420 But we can also easily compile Lucene queries for Elasticsearch or pretty much whatever you want off it. 363 00:43:18,600 --> 00:43:22,300 You know, the data's there, it's just a case of writing something to generate the right queries. 364 00:43:22,660 --> 00:43:30,060 And also because we try and be responsible about these things, we define the permissions that are necessary for Leonidas to execute a specific test case. 365 00:43:30,160 --> 00:43:44,380 And so what we actually do here then is generate the right IAM permissions for AWS, or the necessary permissions for your service accounts in Azure or what have you, in order to allow us to execute the test case as safely as possible. 366 00:43:44,380 --> 00:43:52,760 And we can also set restrictions on which resource it can target and so on to make sure you're only, you can only leverage it against test environments rather than your production accounts. 367 00:43:53,240 --> 00:43:55,880 Or we can also generate some documentation off the back of it. 368 00:43:55,880 --> 00:44:01,740 All of this is defined in a reasonably human-readable but not necessarily friendly file format. 369 00:44:02,000 --> 00:44:16,980 And so instead, what we do here is we then generate markdown files off the back of that and use mkdocs, for those who are familiar with it, to generate a pretty website, which makes it a lot easier for those who are less familiar with some of this stuff to understand what's actually happening, 370 00:44:16,980 --> 00:44:20,880 what this test case is, how it works, and ties it back out to the MITRE framework. 371 00:44:20,880 --> 00:44:25,760 So you can click on the MITRE links there and it takes you out to the relevant TTP there. 372 00:44:26,700 --> 00:44:28,980 So we can do quite a lot really with it. 373 00:44:28,980 --> 00:44:36,860 So in the spirit of this being both a presentation and sort of a tool demo, it's demo time. 374 00:44:38,140 --> 00:44:39,600 Let's hope this works. 375 00:44:39,640 --> 00:44:51,220 So what we have here is a case configuration file for Leo, which is essentially a Python script that allows you to easily talk to that API and comes with it. 376 00:44:51,220 --> 00:45:05,780 So we can just define some fairly simple test cases here that say we're going to do an AWS attack path that's going to work out who we are, have a look at some defensive measures that might be in place, set up some persistence, and then steal some secrets. 377 00:45:07,280 --> 00:45:12,420 So what we're going to do now is actually run that. 378 00:45:25,660 --> 00:45:27,660 Fingers crossed this actually works. 379 00:45:27,840 --> 00:45:30,140 Let's hope the demo gods don't bite me in the backside. 380 00:45:39,540 --> 00:45:40,360 Here we go. 381 00:45:40,360 --> 00:45:48,740 So we're hitting the endpoint and we've got here, we've got Leonidas hosted up as a lambda function behind an API gateway. 382 00:45:48,740 --> 00:45:54,240 So we've run a test case to work out who we are, and this has come back with us being Leonidas. 383 00:45:55,700 --> 00:45:59,700 So we've now had a look and see if we've got any guard duty detectors running. 384 00:45:59,920 --> 00:46:00,680 We have. 385 00:46:00,680 --> 00:46:02,600 It's not ideal, but we're going to carry on anyway. 386 00:46:03,460 --> 00:46:07,380 We just hope that we can move faster than the defenders can. 387 00:46:07,760 --> 00:46:11,480 So next up, we're going to enumerate the cloud trails that we've got running. 388 00:46:16,650 --> 00:46:27,770 And so this was with the idea of spotting whether we've got anyone who's actually monitoring us, who's going to be hunting down our attacker activity within that. 389 00:46:28,790 --> 00:46:30,070 There we go. 390 00:46:30,090 --> 00:46:36,130 So we've got two cloud trails running here, both of which are probably going to pick us up, but that's fine. 391 00:46:36,130 --> 00:46:37,190 We'll carry on anyway. 392 00:46:37,270 --> 00:46:42,410 So we've now added an IAM user in this particular case, Defcon user here. 393 00:46:42,590 --> 00:46:45,390 And that's to allow us to come back later on if we wanted to. 394 00:46:45,430 --> 00:46:46,910 Let's add an API key to it. 395 00:46:46,910 --> 00:46:48,750 And so we've got programmatic access. 396 00:46:48,750 --> 00:46:53,810 So that's come back with some access keys and things, which I'll be deleting straight after this talk. 397 00:46:54,410 --> 00:46:57,550 We then have a look and see what secrets we've got running in secrets manager. 398 00:46:57,810 --> 00:47:03,530 And so in this particular case, we've got a secret in here already that I created another time. 399 00:47:03,710 --> 00:47:11,090 And we're then actually going to execute the access commands to then actually dig out the secret value here. 400 00:47:11,090 --> 00:47:15,450 And you can see we've got a test API key lying around inside the secret string. 401 00:47:16,230 --> 00:47:26,450 So we just run a essentially seven-step kill chain here with a short YAML file defining what test cases we want to execute. 402 00:47:26,450 --> 00:47:30,850 And that's all just hitting an API that's running up in AWS. 403 00:47:33,070 --> 00:47:43,010 So now on top of that, we can actually just interact with it through the Swagger interface if we wanted to. 404 00:47:43,110 --> 00:47:48,710 So you can test out the test cases here if you wanted to as you're developing new ones. 405 00:47:48,770 --> 00:47:51,230 And you can run all this locally as well as in Lambda, obviously. 406 00:47:51,950 --> 00:47:55,090 And this is just really to help with development. 407 00:47:55,090 --> 00:48:03,530 But you've also got Swagger definitions here if you wanted to import that into something like Postman or Insomnia or build it into other tooling. 408 00:48:03,830 --> 00:48:06,090 And the API is pretty easy to work with. 409 00:48:06,410 --> 00:48:12,790 And the other thing that is worth me calling out quickly here is that Leonidas actually generates its own logs. 410 00:48:12,850 --> 00:48:14,870 So I'm just going to take a quick look. 411 00:48:14,870 --> 00:48:18,830 Let's load up the CloudWatch log streams from what we've just done. 412 00:48:26,430 --> 00:48:28,810 No, in fact, we won't be doing that. 413 00:48:29,590 --> 00:48:32,290 So we'll get into that in a second. 414 00:48:34,570 --> 00:48:37,730 But one of the really powerful things with this, then, we've got an API. 415 00:48:38,070 --> 00:48:43,650 We've got a set of things we can execute automatically. 416 00:48:43,690 --> 00:48:51,630 And we can then build that into pretty much any other security testing that we've got going on, especially some of the on-premise testing that we're already doing. 417 00:48:51,630 --> 00:48:56,630 Let's say we're using something like Caldera or Atomic Red Team to do on-premise simulation already. 418 00:48:56,630 --> 00:49:02,430 Actually, we can tie this together, and we can have simulations kicking off on-premise. 419 00:49:02,430 --> 00:49:18,950 Let's say simulate some phishing, a bit of lateral movement, and some Active Directory privilege escalation, and then pivot into the cloud from there, and then start using Leonidas to trigger cloud test cases off the back of that with the credentials that we've just stolen in the simulated attack on-premise. 420 00:49:19,130 --> 00:49:20,910 And to me, that's really exciting. 421 00:49:20,910 --> 00:49:32,990 We've been able to do on-premise automation for a while, and we've done a fair bit of manual stuff in the cloud like this, but now we can tie all that together and execute hybrid kill chains in a completely automated fashion. 422 00:49:35,590 --> 00:49:55,950 So with the log data that I mentioned, Leonidas generates its own log telemetry on I executed this test case, this identity, these parameters were passed in, this was the resulting data that came back, and that all feeds into a CloudWatch log stream in AWS with Stackdriver or Azure Monitor for the Azure ones. 423 00:49:56,610 --> 00:49:59,670 And so we can ship that out somewhere ourselves and monitor that separately. 424 00:49:59,730 --> 00:50:16,990 And one thing that strikes me as being really exciting here, or some potential for some really exciting work, is then integrating your seam alerts and your tagged notables, whatever, that are being generated by these test cases, and essentially performing automatic gap analysis, 425 00:50:17,690 --> 00:50:25,710 working out for every Leonidas test case that's fired, as denoted by the Leonidas logs that are coming in, did we get what we expected out the other side of it? 426 00:50:25,710 --> 00:50:39,870 And we can then track over time as we write new detection cases and tweak things, how much better we're getting, which gives us some nice data both to make us feel good about ourselves, but also it helps justify to whoever it is that's signing the paychecks, 427 00:50:39,870 --> 00:50:44,950 and that what we're doing is worthwhile, and that we're improving over time, and we can demonstrate that, and then so on and so forth. 428 00:50:47,170 --> 00:50:55,850 So one of the reasons I bring up that sort of continuous integration aspect specifically is that what we've got here is the ability to support us on a continuous journey. 429 00:50:55,850 --> 00:51:10,830 We can consistently improve our detection capability, write new use cases, write new test cases, embed them in Leonidas, immediately start simulating them alongside all the other ones we've already got, and it supports us on that journey. 430 00:51:10,830 --> 00:51:20,970 Detections are moving target, attackers are consistently coming up with new techniques, new mechanisms for targeting you, new tooling, so on and so forth. 431 00:51:20,970 --> 00:51:32,250 So we need to be able to move with the times as well, and being able to define all this stuff as code and automate as much of it as possible makes it easier to get better faster with fewer people. 432 00:51:32,750 --> 00:51:39,090 And so it's worth actually treating your detection improvement projects as a software dev project. 433 00:51:39,450 --> 00:51:41,510 I find that that works pretty well. 434 00:51:42,910 --> 00:51:54,090 You want to be identifying new threats and risks on a continuous basis, or at least taking the time to do that regularly and feeding that into some sort of design process to come up with new use cases for you to detect with. 435 00:51:54,570 --> 00:52:02,410 We then simulate that, we evaluate whether what we've changed as a result of that actually made some improvements or not, and we cycle round and round. 436 00:52:02,410 --> 00:52:04,510 And that's how you get better at this stuff over time. 437 00:52:06,110 --> 00:52:08,930 So there's a few things to conclude on there, I think, really. 438 00:52:09,790 --> 00:52:13,390 First off, actually, like the cloud brings a lot of benefits from a security perspective. 439 00:52:14,030 --> 00:52:18,190 And loads of people have been talking about that in the prevention space for a while. 440 00:52:19,410 --> 00:52:21,590 There's a lot from the detection side as well. 441 00:52:21,590 --> 00:52:24,530 We've got visibility like nothing we've ever had on premise before. 442 00:52:24,530 --> 00:52:30,270 If you've got a list of all your cloud accounts, actually, you can get all of your log data comparatively easily. 443 00:52:30,550 --> 00:52:34,770 And we don't end up in this situation where we've got some visibility of some things and not of others. 444 00:52:34,810 --> 00:52:45,390 And we lose attackers because they've pivoted off the box that they landed on and they persisted inside some legacy printer with some old dodgy firmware and we've got no monitoring for that. 445 00:52:45,390 --> 00:52:47,950 We don't have those kinds of problems in the cloud anymore. 446 00:52:49,290 --> 00:52:59,770 The problem that comes with that is all of this, the benefits that the cloud bring, all of the APIs and the automatability, means that our attackers are already automating this stuff. 447 00:52:59,790 --> 00:53:04,190 We were talking about the crypto mining earlier, but that's going to become more and more of a problem over time. 448 00:53:04,190 --> 00:53:12,830 We're going to start seeing attackers automating more complex, more interesting kill chains to go after potentially more damaging things. 449 00:53:12,830 --> 00:53:15,470 At the end of the day, Bitcoin mining is not great. 450 00:53:15,470 --> 00:53:20,050 You don't really want someone having you pay for their cryptocurrency. 451 00:53:20,430 --> 00:53:30,250 But it pales in comparison to attackers automating, like stealing loads of really important data or defacing someone's front page of their website or whatever else. 452 00:53:30,310 --> 00:53:33,550 And we're going to start seeing attackers doing that in the long term, I think. 453 00:53:33,550 --> 00:53:36,110 So we need to get better at this now. 454 00:53:36,230 --> 00:53:39,890 We need to be in front of this whole cloud detection thing from the go. 455 00:53:40,990 --> 00:53:43,070 And we need to be getting better at this as an industry. 456 00:53:43,770 --> 00:53:56,610 And so to go along with that, we released Leonidas, which is a tool for automating a lot of this stuff, both the attack simulation and the attack detection piece and write the detection cases. 457 00:53:57,050 --> 00:53:59,130 Right now, AWS only. 458 00:53:59,350 --> 00:54:02,050 We are not actually that far off Azure and GCP now. 459 00:54:02,050 --> 00:54:04,310 We're starting to test some of those out. 460 00:54:04,310 --> 00:54:08,150 There's currently 40 test cases in there, 41 for AWS. 461 00:54:08,430 --> 00:54:10,450 But we're pushing more out all the time. 462 00:54:10,590 --> 00:54:14,910 And you can grab it over our GitHub now, fsecurelabs.leonidas. 463 00:54:17,830 --> 00:54:19,430 So thanks for listening. 464 00:54:19,430 --> 00:54:20,930 And at that point, I'll take questions. 465 00:54:20,930 --> 00:54:23,150 Just one question from the audience. 466 00:54:23,370 --> 00:54:29,550 What do you recommend they use for an identity store other than cloud SSL? 467 00:54:29,910 --> 00:54:36,590 So basically, I'd recommend anything that's not Active Directory. 468 00:54:36,590 --> 00:54:45,350 And so we've seen a lot of organizations pick up things like Ping, or Okta, or Auth0, Authy. 469 00:54:45,350 --> 00:54:53,250 And there's a number of providers that offer single sign-on platforms that are designed to essentially do single sign-on for large cloud estates. 470 00:54:53,730 --> 00:55:02,090 And the reason that I recommend that is purely that it breaks that link between your on-premise legacy identity store and the cloud. 471 00:55:02,110 --> 00:55:10,570 And while that brings in a bunch of extra overhead from a management perspective, all of your sort of HR joiners, leavers, processes, all of that kind of thing, you know, it's extra steps to add to that. 472 00:55:10,570 --> 00:55:25,690 It also then means that if your Active Directory is breached, you don't get immediately owned in the cloud as well, which is something that, you know, in my experience working with Active Directory over the past few years, is something that would happen to quite a few organizations, 473 00:55:25,690 --> 00:55:26,170 I think.