[00:01.700 --> 00:02.940] Hello, DEF CON. [00:02.940 --> 00:05.720] I speak to you today from my own version of the Batcave, [00:05.720 --> 00:07.880] the ultimate in social distancing. [00:08.160 --> 00:11.160] This is Hacking the Hybrid Cloud, and I am Shah Metcalf. [00:11.540 --> 00:13.960] I'm the founder of Trimark, a professional services company [00:13.960 --> 00:15.500] that helps organizations better secure [00:15.500 --> 00:16.760] their Microsoft platform, [00:16.760 --> 00:19.180] including the Microsoft Cloud and VMware infrastructure. [00:19.360 --> 00:21.360] I'm a Microsoft certified master in Active Directory [00:21.360 --> 00:22.980] and a Microsoft MVP. [00:23.080 --> 00:24.540] I've spoken at a number of conferences [00:24.540 --> 00:26.780] and I'm excited to be back speaking at DEF CON, [00:26.780 --> 00:29.360] even if I'm not physically at DEF CON. [00:29.360 --> 00:31.500] Last year, I keynoted the DEF CON Cloud Village [00:31.500 --> 00:32.780] and three years ago gave a talk [00:32.780 --> 00:34.840] on Hacking the Cloud at DEF CON. [00:35.280 --> 00:38.120] This talk will take a journey through the hybrid cloud, [00:38.120 --> 00:40.520] starting with the comparison between cloud and virtualization [00:40.520 --> 00:42.780] discussing how on-prem Active Directory [00:42.780 --> 00:44.700] domain controllers can be compromised. [00:45.020 --> 00:46.880] I will also walk through the cloud hosted [00:47.640 --> 00:49.760] managed Active Directory deployments [00:49.760 --> 00:53.320] from AWS, Azure, and GCP. [00:53.640 --> 00:55.180] Since this is hybrid talk, [00:55.180 --> 00:57.520] I will walk through some hybrid components [00:57.520 --> 00:59.660] and see how they can be attacked. [00:59.760 --> 01:01.340] I'll touch on cloud administration [01:01.340 --> 01:04.640] and identity access management, or IAM concepts, [01:04.640 --> 01:06.620] and finally walk through escalation scenarios [01:06.620 --> 01:09.800] for compromising cloud-hosted on-prem domain controllers, [01:09.800 --> 01:11.400] AWS, and Azure. [01:12.220 --> 01:14.880] First, we need to talk about what hybrid cloud is, [01:14.880 --> 01:16.700] at least in the context of this talk. [01:16.700 --> 01:19.460] There are likely lots of definitions of hybrid cloud. [01:19.580 --> 01:21.040] Due to this, I'm defining hybrid cloud [01:21.040 --> 01:23.440] as something on-prem and something in the cloud. [01:23.440 --> 01:26.380] This typically means infrastructure on-prem and in the cloud, [01:26.380 --> 01:28.480] more IaaS than strict SaaS. [01:28.960 --> 01:30.500] Now that we've defined hybrid cloud, [01:30.500 --> 01:32.400] let's look at three common scenarios. [01:32.480 --> 01:34.580] The first is the on-prem Active Directory environment [01:34.580 --> 01:37.480] with Office 365 SaaS services. [01:37.540 --> 01:39.700] SaaS stands for software as a service. [01:39.740 --> 01:41.360] Many organizations are leveraging the cloud [01:41.360 --> 01:43.800] for infrastructure as a service, or IaaS, [01:43.800 --> 01:45.560] as their cloud data center. [01:45.560 --> 01:47.820] The third scenario here is an on-prem Active Directory [01:47.820 --> 01:51.880] with a cloud-managed AD as an additional or resource forest. [01:51.940 --> 01:53.660] Since cloud can be used in many ways, [01:53.660 --> 01:55.000] it's likely that most organizations [01:55.000 --> 01:57.440] use a combination of these, among others. [01:58.380 --> 02:02.120] This section covers virtualization concepts and controls [02:02.120 --> 02:04.020] as foundational information to better set up [02:04.020 --> 02:06.060] and describe potential attacks. [02:07.020 --> 02:09.260] Conceptually, the cloud is virtualization, [02:09.260 --> 02:11.240] certainly from an IaaS perspective. [02:11.240 --> 02:13.960] Amazon has used Xen and a switch to Nitro. [02:14.300 --> 02:16.740] Azure uses a custom version of Hyper-V Core, [02:17.040 --> 02:20.000] and Google Cloud Platform uses KVM. [02:21.520 --> 02:23.160] Now, there is a cloud fabric [02:23.160 --> 02:24.720] that ties the virtualization elements [02:24.720 --> 02:26.340] together with orchestration, [02:26.340 --> 02:28.780] which is what makes the cloud the cloud. [02:29.300 --> 02:31.760] So, this diagram shows Azure IaaS [02:31.760 --> 02:33.440] with a VM and related components. [02:33.440 --> 02:35.740] This is similar to on-prem virtualization. [02:37.500 --> 02:39.300] And Amazon's Nitro is pretty cool. [02:39.300 --> 02:40.900] The Nitro hypervisor can take advantage [02:40.900 --> 02:42.860] of Nitro-optimized hardware, [02:42.860 --> 02:44.800] including a Nitro security chip. [02:44.800 --> 02:47.660] Any instance created prior to Nitro is still running Xen. [02:50.400 --> 02:53.680] AWS also has this concept of connectivity [02:53.680 --> 02:56.240] between the AWS-managed Active Directory [02:56.240 --> 02:57.820] that they host themselves, [02:57.820 --> 03:01.820] and EC2, which is their IaaS environment [03:01.820 --> 03:03.700] within the AWS cloud. [03:03.700 --> 03:06.480] And then, of course, since you can host Windows servers [03:06.480 --> 03:08.320] and Windows systems and other types of systems, [03:08.320 --> 03:09.620] you can also integrate and connect [03:09.620 --> 03:11.380] to other cloud environments. [03:11.380 --> 03:14.820] This demonstrates how cloud can get complex pretty fast. [03:15.840 --> 03:17.640] And you can run VMware on-prem [03:18.020 --> 03:21.800] and in AWS and Azure and GCP. [03:23.480 --> 03:25.700] Domain controllers control the keys to the kingdom [03:25.700 --> 03:27.540] and require special security protections [03:27.540 --> 03:30.300] that are often ignored on virtual platforms. [03:30.780 --> 03:32.840] Compromising physical domain controllers [03:32.840 --> 03:34.960] boils down to either gaining physical access [03:34.960 --> 03:38.480] or compromising an out-of-band management system like iLO. [03:38.540 --> 03:41.300] iLO hosts a web server on port 2381, [03:41.300 --> 03:43.640] and we can scan for this using PowerShell. [03:43.840 --> 03:47.340] So, if iLO is hosted on the same network [03:47.340 --> 03:52.240] as production systems and clients, such as workstations, [03:52.240 --> 03:54.300] we can scan and identify iLO. [03:54.420 --> 03:56.300] iLO has had a number of security issues [03:56.300 --> 03:57.780] with firmware patches available, [03:57.780 --> 03:59.400] but the issue is that pretty much [03:59.400 --> 04:01.920] no one updates firmware on physical servers. [04:03.080 --> 04:04.320] Airbus Security has identified [04:04.320 --> 04:06.580] many of these security issues with iLO, [04:06.580 --> 04:08.260] from accessing memory to an RCE [04:08.260 --> 04:10.040] to get read-write memory access [04:10.040 --> 04:13.140] to the ability to execute arbitrary commands via DMA [04:13.140 --> 04:15.560] to gain host access through iLO [04:15.560 --> 04:17.360] even when iLO is disabled. [04:17.440 --> 04:18.440] And I have a link at the bottom [04:18.440 --> 04:22.460] that includes a list even beyond what I have up here [04:22.460 --> 04:24.280] and a Go scanner to discover [04:24.280 --> 04:26.180] vulnerable servers running iLO. [04:27.280 --> 04:29.980] When VMware is hosting virtual DCs, [04:29.980 --> 04:32.680] which is probably around 95% of the organizations are better, [04:32.680 --> 04:34.460] certainly the larger ones, [04:34.460 --> 04:37.640] there's different ways that you could compromise VMware [04:37.640 --> 04:40.340] to get access to a virtual domain controller. [04:40.460 --> 04:42.440] This could involve compromising an account [04:42.440 --> 04:44.280] with rights to VMware. [04:44.280 --> 04:45.720] Often this is a regular user account [04:45.720 --> 04:49.620] that is a member of an 80 group called VMware Admins, [04:49.620 --> 04:51.560] or we just need to compromise an account [04:51.560 --> 04:53.880] that has VMware access to, [04:53.880 --> 04:58.720] or at least one virtual domain controller, [04:58.720 --> 04:59.840] potentially more. [04:59.860 --> 05:02.600] If we can compromise a system which runs vCenter, [05:02.600 --> 05:04.860] we can ultimately compromise VMware. [05:05.220 --> 05:08.700] Also, every ESXi server has a root account. [05:08.700 --> 05:11.040] And most of the time, every ESXi server [05:11.040 --> 05:12.560] has the same root password, [05:12.560 --> 05:14.840] which may be the organization default password, [05:14.840 --> 05:17.640] which could be guest or found in a password file. [05:18.260 --> 05:21.900] We can compromise the vCenter system, [05:21.900 --> 05:24.580] we can compromise the vSphere environment. [05:25.740 --> 05:28.800] And the VMware VIX API has functionality [05:28.800 --> 05:31.900] that allows for direct access to guest OSs, [05:31.900 --> 05:33.080] which is used by solutions [05:33.080 --> 05:35.440] such as VMware Site Recovery Manager. [05:35.440 --> 05:37.600] This requires VMware tools to be installed [05:37.600 --> 05:39.500] on the guest OS or VM. [05:39.500 --> 05:42.540] The VIX API allows guest operations [05:42.540 --> 05:44.480] such as starting, stopping the application, [05:44.480 --> 05:46.020] file directory manipulation, [05:46.020 --> 05:47.440] uploading, downloading files, [05:47.440 --> 05:48.800] all within the guest OS [05:48.800 --> 05:50.540] without requiring any network connectivity [05:50.540 --> 05:52.120] to the virtual machine. [05:52.120 --> 05:53.840] This functionality may also be used [05:53.840 --> 05:56.520] by vSphere accounts with limited privileges [05:56.880 --> 05:59.840] to access a guest OS without the need to authenticate. [06:00.120 --> 06:01.540] There is a scenario where an account [06:01.540 --> 06:04.080] with vCenter privileges can use the vSphere API [06:04.080 --> 06:05.980] to interact with the local OS installation [06:05.980 --> 06:08.900] of VMware tools to allow a pass-through session [06:08.900 --> 06:10.160] for an unprivileged account [06:10.160 --> 06:11.860] to connect to that virtual machine [06:11.860 --> 06:14.140] and execute local commands. [06:14.140 --> 06:17.660] Since this applies to virtual domain controllers, [06:17.660 --> 06:19.420] Active Directory could be compromised in this manner [06:19.420 --> 06:21.600] by allowing execution of commands locally [06:21.600 --> 06:23.760] on the virtual DC with accounts [06:23.760 --> 06:27.500] not normally allowed local logon access to the DCs. [06:29.260 --> 06:31.900] For Hyper-V, the attacks are similar to VMware. [06:31.900 --> 06:33.340] We compromise an account with rights [06:33.340 --> 06:35.520] or find a way to compromise a server, [06:35.520 --> 06:37.460] either through a server admin account, [06:37.460 --> 06:39.440] discover the local admin password, [06:39.440 --> 06:42.600] or a modified group policy to gain access to the server. [06:44.520 --> 06:46.000] So I decided I would take a look [06:46.000 --> 06:48.540] at the so-called big three cloud hosting providers, [06:48.540 --> 06:50.120] Managed Active Directory Environments, [06:50.120 --> 06:51.640] and then focus on what this means [06:51.640 --> 06:53.660] to Pentesters and Red Teamers. [06:54.120 --> 06:55.660] The Managed Active Directory Environment [06:55.660 --> 06:58.760] does not provide AD admin rights to the AD environment, [06:58.760 --> 07:01.200] so the customer will never have domain admin rights [07:01.200 --> 07:03.140] or even domain controller access. [07:03.340 --> 07:06.180] The cloud provider controls AD infrastructure management [07:06.180 --> 07:10.540] and delegates rights to OUs and AD components to the customer. [07:11.400 --> 07:13.940] Let's start with Amazon AWS Directory Services [07:13.940 --> 07:15.880] from Microsoft Active Directory. [07:15.880 --> 07:17.560] That's quite the mouthful. [07:18.680 --> 07:22.340] The AWS Managed AD has an AWS Delegated Groups [07:22.340 --> 07:24.200] OU that contains the delegation groups [07:24.200 --> 07:27.740] which the customer is provided membership in and access to. [07:27.740 --> 07:30.560] The AWS Reserved OU is controlled exclusively [07:30.560 --> 07:32.600] by the AWS system. [07:32.600 --> 07:35.340] There is a customer OU created in the Managed AD domain [07:35.340 --> 07:37.480] that is typically named after the AD subdomain [07:37.480 --> 07:40.660] or the first part of the fully qualified domain name [07:40.660 --> 07:44.060] or the domain's NetBIOS name. [07:44.060 --> 07:47.900] This customer OU contains OUs for computers and users. [07:48.860 --> 07:51.400] AWS Managed AD spins up two domain controllers [07:51.400 --> 07:53.140] which are in different network segments [07:53.140 --> 07:55.480] but in the same Active Directory site. [07:55.480 --> 07:57.180] The default domain admin account [07:57.180 --> 07:59.340] retains the default name administrator, [07:59.340 --> 08:01.920] though this is moved to the AWS Reserved OU. [08:02.280 --> 08:05.240] When a customer creates an AWS Managed AD, [08:05.240 --> 08:07.600] their first account in the AD environment is called admin [08:07.940 --> 08:10.340] and is granted full delegated rights to the AD environment [08:10.740 --> 08:13.420] for that customer, not to the domain controllers [08:13.420 --> 08:17.980] or AD admin rights to the full AD. [08:18.840 --> 08:21.400] The domain password is default, still seven characters, [08:21.400 --> 08:24.440] but there are five AWS-created fine-grained password policies [08:24.440 --> 08:26.840] which are delegated to the customer to modify. [08:27.140 --> 08:29.540] Based on the group policy linked to the domain controller's OU, [08:29.540 --> 08:32.100] I was able to identify a pretty decent audit policy, [08:32.100 --> 08:33.860] though it was missing Kerberos auditing, [08:33.860 --> 08:36.340] which means Kerberos activity would be missed. [08:37.760 --> 08:40.760] The first customer account, admin, is created with the description [08:40.760 --> 08:43.420] that notes the customer OU distinguished name, [08:43.420 --> 08:45.240] so it's very nice that they provide this description [08:45.600 --> 08:49.080] so that way you can quickly and easily identify where this OU is. [08:50.020 --> 08:53.800] There are several delegation groups in the AWS Managed AD environment. [08:53.800 --> 08:56.260] All these are named with the prefix AWS-delegated [08:56.260 --> 08:59.600] and are contained in the AWS Delegated Groups OU. [08:59.860 --> 09:02.000] The AWS Delegated Administrators Group [09:02.000 --> 09:04.140] is the primary customer admin group. [09:04.200 --> 09:07.000] There are also delegation groups for managed service accounts, [09:07.000 --> 09:09.360] adding workstations to the main Kerberos delegation, [09:09.360 --> 09:11.300] DNS, and server administration. [09:12.720 --> 09:16.080] Let's take a look at Azure Active Directory Domain Services. [09:16.180 --> 09:19.080] While Azure Active Directory is not exactly Active Directory, [09:19.080 --> 09:21.160] since it's a cloud directory with no Kerberos [09:21.160 --> 09:24.120] or NTLM authentication, group policy, or LDAP, [09:24.120 --> 09:26.600] Azure AD Domain Services is Active Directory, [09:26.600 --> 09:28.660] just like you would have on a corporate network, [09:28.660 --> 09:30.840] and is Microsoft's managed Active Directory. [09:32.440 --> 09:35.700] So this environment, Azure AD DS, [09:35.700 --> 09:38.000] has several top-level OUs that are created. [09:38.000 --> 09:40.280] These are prefixed with AADD, [09:40.280 --> 09:45.820] and the AADDC computers and AADDC users OUs [09:45.820 --> 09:47.740] are the primary customer OUs. [09:48.720 --> 09:52.520] The Azure AD DS domain controllers are in the same network, [09:52.520 --> 09:55.140] and at least in this instance that I created, [09:55.140 --> 09:57.860] had sequential IPs, .4 and .5. [09:57.860 --> 10:02.840] The default domain admin account is called DCAASAdmin, [10:02.840 --> 10:05.120] and remains in the default user's OU. [10:05.180 --> 10:08.020] There is one fine-grained password policy that's pre-created. [10:08.020 --> 10:09.880] I'm not going to try to read the name. [10:09.980 --> 10:13.500] As it is, it's a bit tough with all the AA and DDs, [10:13.500 --> 10:15.720] but the customer has delegated modifier rights [10:15.720 --> 10:17.840] to this fine-grained password policy object. [10:18.060 --> 10:20.600] I can't see any auditing configuration set in GPO, [10:20.600 --> 10:22.840] so I have to assume that they are set via local policy, [10:22.840 --> 10:25.220] which I can't see without domain controller access. [10:27.860 --> 10:30.940] Azure AD DS has a few delegation groups [10:30.940 --> 10:37.860] which are prefixed with AADDC, and are in the AADDC users OU. [10:37.860 --> 10:42.600] The primary customer group is AADDC administrators, [10:42.600 --> 10:45.980] which is delegated full control to many OUs and AD components. [10:46.140 --> 10:48.160] I'm pretty sure when Microsoft came up with the names [10:48.160 --> 10:49.540] for these groups, [10:49.540 --> 10:52.920] they were not thinking about having to say this out loud in a row. [10:53.040 --> 10:55.260] So thank you, Microsoft, for that. [10:55.900 --> 10:58.480] Moving on to the third and final managed AD environment [10:58.480 --> 11:00.260] that I reviewed recently, [11:00.260 --> 11:03.340] this is the Google Cloud Platform Managed Active Directory. [11:07.090 --> 11:09.150] It's interesting to note that the GCP-managed AD [11:09.150 --> 11:11.690] seems to cost about twice what the Amazon AWS [11:11.690 --> 11:14.930] and Microsoft Azure-managed AD environments cost, [11:14.930 --> 11:16.990] because they're about $100 each. [11:18.430 --> 11:21.590] The GCP-managed AD environment has a single root OU [11:21.590 --> 11:23.250] for the customer called Cloud, [11:23.250 --> 11:25.650] with a single child OU called Computers. [11:25.650 --> 11:29.170] The Cloud Service Objects OU contains delegation groups. [11:30.350 --> 11:33.850] GCP-managed AD is different from the others [11:33.850 --> 11:35.970] that I just talked about in that domain controllers [11:35.970 --> 11:38.190] are running Windows Server 2019, [11:38.190 --> 11:40.630] though the Active Directory forest functional level [11:40.630 --> 11:42.630] is still set to 2012 R2. [11:43.210 --> 11:45.110] Also, the Active Directory Recycle Bin, [11:45.110 --> 11:46.750] which enables rapid undelete capability, [11:46.750 --> 11:48.650] is not enabled, which is interesting. [11:48.770 --> 11:50.730] The default domain administrator account [11:50.730 --> 11:52.610] keeps the default name Administrator, [11:52.610 --> 11:54.250] and it's disabled. [11:54.250 --> 11:57.730] So, GCP spins up a second domain admin account [11:57.730 --> 12:02.150] called CloudSVCAdmin, and this account is enabled. [12:02.610 --> 12:04.750] The domain password policy is default, [12:04.750 --> 12:07.450] and the customer can create fine-grained password policies. [12:07.730 --> 12:10.010] And with this, the DC event auditing [12:10.010 --> 12:11.470] is not configured through group policy, [12:11.470 --> 12:14.090] so again, I couldn't identify how it may be configured. [12:15.130 --> 12:18.030] GCP-managed AD has several delegation groups. [12:18.030 --> 12:19.770] They start with Cloud Service, [12:19.770 --> 12:22.650] and these are stored in the Cloud Service Objects OU. [12:22.650 --> 12:24.470] The Cloud Service All Administrators [12:24.470 --> 12:25.890] and the Cloud Service Administrators [12:25.890 --> 12:28.090] are the primary customer admin groups. [12:28.270 --> 12:30.850] GCP delegation groups include computer admins, [12:30.850 --> 12:33.330] managed service account admins, and others. [12:33.850 --> 12:35.750] This slide captures some of the common themes [12:35.750 --> 12:38.850] of the three managed AD environments I reviewed. [12:38.850 --> 12:40.970] I used a version of the Trimark Active Directory [12:40.970 --> 12:42.950] assessment tool that I wrote to perform the review [12:42.950 --> 12:44.670] of the different AD environments, [12:44.670 --> 12:46.550] which enabled me to capture data for all three [12:46.550 --> 12:48.650] and identify the key security features [12:48.650 --> 12:51.810] and delegation in about a day to put this talk together, [12:51.810 --> 12:53.030] or at least this part of it. [12:53.030 --> 12:54.410] The bottom of the slide will provide a link [12:54.410 --> 12:57.170] to an Active Directory security review PowerShell script [12:57.170 --> 12:59.470] that I wrote and published in June. [12:59.650 --> 13:00.950] The link points to the article [13:00.950 --> 13:03.410] where I highlight key AD security items [13:03.410 --> 13:04.890] to check and what to check for, [13:04.890 --> 13:06.570] and the PowerShell script simplifies [13:06.570 --> 13:10.450] the data collection part, so analysis is much simpler. [13:10.570 --> 13:12.550] So it's good to check that out. It would be helpful [13:12.950 --> 13:15.470] if you're looking at an environment such as these. [13:16.850 --> 13:20.090] So since the customer does not actually have AD admin rights [13:20.090 --> 13:21.370] to a managed AD environment, [13:21.370 --> 13:24.210] there's not much point in attempting to escalate to DA. [13:24.630 --> 13:27.450] Better to identify which managed AD environment it is [13:27.450 --> 13:29.450] and enumerate the managed AD privilege groups [13:29.450 --> 13:32.230] to determine which accounts are highly privileged. [13:32.550 --> 13:36.150] Interesting note about Azure Active Directory domain services, [13:36.150 --> 13:38.770] Microsoft's managed AD. [13:38.770 --> 13:41.330] It can be managed by Azure AD accounts, [13:41.330 --> 13:43.730] and these are synchronized into Azure AD DS [13:43.730 --> 13:46.250] or even by on-prem AD accounts [13:46.250 --> 13:49.810] that are synced in from the on-prem AD environment. [13:49.810 --> 13:52.010] And these are synced using the on-prem Azure AD [13:52.010 --> 13:57.330] Connect system through Azure AD to Azure AD domain services. [13:57.330 --> 14:00.210] And if password hash sync is enabled, which it usually is, [14:00.210 --> 14:03.720] then the on-prem AD account password hash is included. [14:03.990 --> 14:07.130] Normally, password hash sync does a hash of a hash, [14:07.130 --> 14:11.350] but when syncing through to Azure AD domain services, [14:11.350 --> 14:16.050] that original AD password hash is retained and synchronized, [14:16.050 --> 14:18.770] so that way the password can actually be used [14:18.770 --> 14:21.470] in Azure Active Directory domain services. [14:23.690 --> 14:25.470] So, there are several cloud components [14:25.470 --> 14:27.310] that connect on-prem to cloud. [14:27.310 --> 14:30.150] This connectivity has some very interesting implications. [14:31.250 --> 14:33.070] Amazon's AD Connector is effectively [14:33.070 --> 14:35.850] an on-prem AD authentication proxy. [14:36.210 --> 14:38.250] AD Connector is designed to provide an easy way [14:38.250 --> 14:39.970] to establish a trusted relationship [14:39.970 --> 14:42.290] between your Active Directory and AWS. [14:42.310 --> 14:44.310] So, when AD Connector is configured, [14:44.310 --> 14:47.270] it enables the ability to sign into AWS applications [14:47.270 --> 14:49.350] with on-prem AD credentials, [14:49.350 --> 14:54.050] join AWS Windows instances to the on-prem AD domain, [14:54.050 --> 14:57.230] and provides federated sign-in to the AWS Management Console [14:57.590 --> 15:03.170] by mapping these on-prem identities to AWS IAM roles. [15:04.810 --> 15:07.390] And AD Connector can't be used with custom applications [15:07.390 --> 15:09.310] since it's just for AWS. [15:09.750 --> 15:12.690] Basically, a user opens the secure custom sign-in page, [15:12.690 --> 15:14.890] supplies their Active Directory username and password. [15:14.890 --> 15:17.930] This authentication request is sent over SSL to the AD Connector, [15:17.930 --> 15:20.570] and then the AD Connector performs LDAP authentication [15:20.570 --> 15:22.170] to Active Directory. [15:22.170 --> 15:25.660] The interesting part happens at this point. [15:25.910 --> 15:27.830] Once the user has been authenticated, [15:27.830 --> 15:30.610] the AD Connector calls the STS assume role method [15:30.610 --> 15:33.580] to get temporary security credentials for that user. [15:33.850 --> 15:35.710] Using those temporary security credentials, [15:35.710 --> 15:37.350] AD Connector constructs a sign-in URL [15:37.780 --> 15:40.560] that users use to access the console. [15:40.870 --> 15:42.590] And if a user is mapped to multiple roles, [15:42.590 --> 15:45.190] the user will be presented with a choice at the sign-in time [15:45.190 --> 15:47.110] as to which role they want to assume. [15:47.110 --> 15:49.050] User session is valid for an hour, [15:49.050 --> 15:51.130] which is pretty standard for cloud apps. [15:51.730 --> 15:55.050] Since the AD Connector environment is hosted in AWS, [15:55.050 --> 15:58.110] this would need to be targeted in order to potentially impersonate the users [15:58.110 --> 16:03.410] and be able to leverage that STS assume role method. [16:05.770 --> 16:07.770] Microsoft's pass-through authentication is similar [16:08.350 --> 16:11.670] because with Microsoft pass-through authentication, or PTA, [16:11.670 --> 16:15.630] it provides users the ability to authenticate to Azure Active Directory, [16:15.630 --> 16:17.690] which is passed through to the on-prem AD [16:17.690 --> 16:23.190] and validated in Azure AD by getting the status back [16:23.190 --> 16:26.230] from the PTA agent in the on-prem environment. [16:27.030 --> 16:30.010] PTA is enabled via Azure AD Connect. [16:30.010 --> 16:31.890] The users use the same passwords [16:31.890 --> 16:35.130] to sign into both on-prem and cloud-based applications. [16:35.310 --> 16:37.470] The communication between an agent and Azure AD [16:37.470 --> 16:40.130] is secured using certificate-based authentication, [16:40.130 --> 16:41.910] and these certificates are automatically renewed [16:41.910 --> 16:44.050] every few months by Azure AD. [16:44.230 --> 16:47.350] I couldn't figure out what happens if you get one of these certificates, [16:47.350 --> 16:49.050] but that could be pretty interesting. [16:50.370 --> 16:53.130] But ultimately, this supports user sign-in [16:53.130 --> 16:55.530] into all web browser-based applications [16:55.530 --> 16:57.710] and into Microsoft Office client applications [16:57.710 --> 16:59.370] that use modern authentication. [17:02.860 --> 17:05.960] Adam Chester, aka XPN, has done a great blog post [17:05.960 --> 17:09.180] on how to leverage a compromised Azure AD Connect server [17:09.180 --> 17:12.320] with PTA enabled in order to gather usernames [17:12.320 --> 17:16.800] and cleartext passwords that were used to authenticate via PTA. [17:16.800 --> 17:20.100] So since password authentication is managed by Azure AD Connect, [17:20.100 --> 17:21.940] you can compromise that server. [17:21.940 --> 17:25.420] And then during this authentication process, [17:25.420 --> 17:27.520] Azure AD sends the cleartext password, [17:27.520 --> 17:29.500] not hashed, to authenticate the user. [17:29.500 --> 17:31.380] So it's sent to Azure AD Connect. [17:31.380 --> 17:35.140] Azure AD Connect PTA component is going to check [17:35.140 --> 17:38.460] and validate the username and password with Active Directory, [17:38.460 --> 17:42.180] then send back the status to Azure AD for that. [17:42.340 --> 17:45.560] So Adam identified that you could inject a DLL [17:45.560 --> 17:47.760] to compromise the credentials that were used for PTA [17:48.160 --> 17:52.680] and then have those credentials for anyone that was using PTA. [17:53.780 --> 17:56.260] Azure Active Directory Seamless Single Sign-On, [17:56.260 --> 17:59.760] or Azure AD Seamless SSO, automatically signs users [17:59.760 --> 18:01.440] in when they are on their corporate devices [18:01.440 --> 18:03.320] connected to the corporate network. [18:04.040 --> 18:06.220] This graphic is interesting and nice, [18:06.220 --> 18:08.200] but I prefer this one because it has a lot more [18:08.200 --> 18:09.300] of the flow. [18:09.380 --> 18:10.700] But basically what ends up happening [18:10.700 --> 18:13.240] is that users don't need to type in their passwords [18:13.240 --> 18:17.100] to sign into Azure AD or even really type in their usernames. [18:17.100 --> 18:19.280] It provides the users easy access [18:19.280 --> 18:20.800] to their cloud-based applications [18:20.800 --> 18:24.320] without any additional on-prem components like federation. [18:24.320 --> 18:26.320] This is done by converting the on-prem authentication [18:26.320 --> 18:29.040] to cloud authentication through a computer account [18:29.040 --> 18:30.360] in the Active Directory. [18:30.800 --> 18:33.500] So Michael Grafner from DS Internals [18:34.090 --> 18:36.660] posted this article a few years ago, [18:36.660 --> 18:38.940] or a couple of years ago, where he was talking about [18:38.940 --> 18:43.300] how the Azure AD has a publicly available endpoint [18:43.300 --> 18:44.880] that accepts these Kerberos tickets [18:44.880 --> 18:47.320] and translates them into SAML and JWT tokens [18:47.320 --> 18:49.860] that can be leveraged for cloud apps. [18:51.320 --> 18:53.640] Effectively, if you can compromise the Azure AD [18:53.640 --> 18:55.780] Seamless Single Sign-On computer account, [18:55.780 --> 18:57.480] the password has to be associated with it. [18:57.480 --> 19:00.920] The computer account is called Azure AD SSO ACC [19:01.740 --> 19:04.860] and can be identified as in the computer's container. [19:04.860 --> 19:07.320] If you're able to compromise this and get this password hash, [19:07.320 --> 19:09.880] you can generate a silver ticket for any user [19:09.880 --> 19:11.460] you want to impersonate, [19:11.460 --> 19:13.960] and the service that you use in the silver ticket [19:13.960 --> 19:18.140] is going to be aadg.windows.net.nsatc.net. [19:18.140 --> 19:21.560] You inject this ticket into the local Kerberos cache, [19:21.560 --> 19:24.340] and Azure AD Seamless Single Sign-On, [19:24.340 --> 19:27.440] the computer account password doesn't actually change. [19:27.440 --> 19:29.280] Microsoft does recommend that it gets rolled [19:29.280 --> 19:31.700] and changed at least every 30 days. [19:31.700 --> 19:34.700] What's interesting is because it's a computer account, [19:35.210 --> 19:38.760] there is no actual computer that is behind the scenes [19:38.760 --> 19:40.840] that is using that computer account [19:40.840 --> 19:43.070] other than the Azure AD Connect client. [19:43.480 --> 19:46.620] So normally a computer in Active Directory [19:46.620 --> 19:48.640] would update the computer account password [19:48.640 --> 19:50.400] that's associated with that computer. [19:50.400 --> 19:52.400] Since there isn't really a computer here, [19:52.400 --> 19:54.160] that password isn't getting changed. [19:54.160 --> 19:57.020] So maybe in the future Microsoft will code into Azure AD Connect [19:57.020 --> 20:00.460] to automatically update the computer account password for this. [20:02.000 --> 20:04.660] Three years ago at DEF CON 25 in Las Vegas, [20:04.660 --> 20:06.920] I co-presented a talk called Hacking the Cloud. [20:06.920 --> 20:09.420] I talked about how Azure AD Connect's password [20:09.420 --> 20:12.980] hash sync permissions provided Mimikatz DC-sync rights, [20:12.980 --> 20:16.860] and that the author of Mimikatz DC-sync, Vincent Leteau, [20:16.860 --> 20:19.460] was inspired by password hash sync to create DC-sync [20:19.460 --> 20:21.820] because PHS requests password hashes [20:21.820 --> 20:24.620] for sync users on a regular basis. [20:25.520 --> 20:27.640] We can scan Active Directory domain permissions [20:27.640 --> 20:31.020] looking for potential Azure AD Connect service counts. [20:31.720 --> 20:34.900] Or if the Azure AD Connect install [20:34.900 --> 20:37.380] would use the express install option, [20:37.380 --> 20:38.960] then there's a helpful description on the account [20:38.960 --> 20:42.080] noting on which server Azure AD Connect is installed. [20:42.180 --> 20:43.780] We can then check in Active Directory [20:43.780 --> 20:46.620] to figure out where that computer is stored. [20:46.620 --> 20:50.040] Very often this is in the server's OU or somewhere similar. [20:50.260 --> 20:54.100] Once we know the server name, we can check for, [20:54.100 --> 20:55.320] and its location, [20:55.320 --> 20:58.160] the actual distinguished name of the OU that it's in. [20:58.160 --> 20:59.840] We can check AD for any group policies [20:59.840 --> 21:02.960] that add AD groups to the local administrators group. [21:03.080 --> 21:04.200] Compromising any account [21:04.200 --> 21:05.980] that is a member of the local administrators group [21:05.980 --> 21:07.560] on the Azure AD Connect server [21:07.560 --> 21:10.800] would result in compromise of the Azure AD Connect server [21:10.800 --> 21:12.540] and ultimately Active Directory, [21:12.540 --> 21:16.460] assuming that it has password hash synchronization [21:16.460 --> 21:19.320] rights configured in the environment. [21:20.100 --> 21:21.820] Also, if we can modify a group policy [21:21.820 --> 21:24.360] that links to the Azure AD Connect server OU, [21:24.360 --> 21:26.860] then we can compromise the server that way as well. [21:28.240 --> 21:30.540] When talking about cloud environments, [21:30.540 --> 21:33.440] it's important to mention how administration is performed. [21:33.440 --> 21:36.040] This is commonly referred to as Identity Access Management, [21:36.040 --> 21:36.920] or IAM. [21:37.480 --> 21:40.220] So we've gone from groups in our on-prem environment [21:40.220 --> 21:42.020] with Active Directory to roles, [21:42.020 --> 21:43.660] which is effectively these rights [21:43.660 --> 21:45.380] that are packaged together. [21:45.740 --> 21:48.580] Azure and AWS and GCP have their own methods, [21:48.580 --> 21:51.220] but a lot of the concepts are effectively the same. [21:51.860 --> 21:54.020] Azure IAM uses the concept of an owner, [21:54.120 --> 21:55.380] a contributor, and a reader. [21:56.860 --> 21:59.700] The owner has full access, the contributor has the ability [21:59.700 --> 22:01.740] to create and manage the resources [22:01.740 --> 22:03.400] but can't grant access to others, [22:03.400 --> 22:05.800] and then a reader can just view things. [22:06.900 --> 22:10.280] And so Azure has the tenant, [22:10.280 --> 22:13.920] which is the top-level container of the Azure environment, [22:13.920 --> 22:15.940] and this contains subscriptions. [22:17.660 --> 22:21.140] The tenant admins are effectively the owner role on the tenant, [22:21.140 --> 22:23.620] and subscription admin would be effectively the owner role [22:23.620 --> 22:25.160] on the subscription. [22:28.050 --> 22:30.550] What's interesting is that Microsoft notes [22:30.550 --> 22:32.630] that Azure roles and Azure AD roles [22:32.630 --> 22:35.990] are separate and different, mostly, [22:35.990 --> 22:38.130] and I'll cover that in a little bit. [22:41.620 --> 22:44.040] AWS has a root account, often called [22:44.040 --> 22:45.680] or referred to as a payer account, [22:45.680 --> 22:47.360] which is the primary account, [22:47.360 --> 22:49.720] though it really shouldn't be used regularly. [22:49.720 --> 22:51.560] Then there are account admins. [22:51.560 --> 22:55.160] So in AWS, the account is similar to the Azure tenant. [22:55.160 --> 22:57.720] It's the top-level container for the environment [22:57.720 --> 22:58.980] for that customer. [22:58.980 --> 23:01.940] The account admins have full control over the account. [23:01.960 --> 23:04.780] If an account is a root account and an admin account, [23:04.780 --> 23:07.720] then that account has full organizational control. [23:07.940 --> 23:10.300] It's important to note here that any overscoped roles [23:10.300 --> 23:12.440] provide escalation capability. [23:12.920 --> 23:15.020] And Rhino Security Labs has an extensive article [23:15.020 --> 23:18.880] covering 21 AWS IAM privilege escalation methods, [23:18.880 --> 23:20.620] which allow an attacker to escalate [23:20.620 --> 23:22.820] from a compromised low-privilege account [23:22.820 --> 23:24.640] to full admin privileges. [23:24.640 --> 23:26.020] So I've listed a bunch of them here, [23:26.020 --> 23:27.800] I apologize for the small text, [23:27.800 --> 23:29.980] but really what you want to do is you want to read [23:30.440 --> 23:32.940] the blog article that they wrote, [23:32.940 --> 23:35.100] which is linked at the bottom and in the slides. [23:35.160 --> 23:36.500] They also released a scanning tool, [23:36.500 --> 23:38.460] which is on GitHub, to identify these vulnerabilities [23:39.180 --> 23:42.800] in an AWS user account. [23:42.860 --> 23:45.280] And if you have an account with IAM read access for all users, [23:45.280 --> 23:47.280] the script can be run against every user in the account [23:47.280 --> 23:49.820] to detect these vulnerabilities account-wide. [23:49.820 --> 23:52.180] So if you're asked to pen test AWS, [23:52.180 --> 23:56.500] or you're providing security consulting services around AWS, [23:56.500 --> 23:58.080] or even you work in an organization [23:58.080 --> 23:59.060] and you're moving to AWS [23:59.060 --> 24:01.160] and you have some of the IAM roles configured, [24:01.160 --> 24:02.520] you definitely want to scan them [24:02.520 --> 24:04.820] for potential escalation methods. [24:06.020 --> 24:09.240] API keys, I'm just going to cover this really quickly. [24:09.240 --> 24:11.580] They provide permanent access to an environment, [24:11.580 --> 24:13.400] often with extensive privileges. [24:13.500 --> 24:15.920] These keys can often bypass the typical username [24:15.920 --> 24:17.840] and password used to authenticate as an account [24:17.840 --> 24:20.280] if it's configured as such. [24:21.680 --> 24:23.540] Well, I've covered a number of scenarios [24:23.540 --> 24:26.780] on how to attack on-prem and hybrid cloud components. [24:26.780 --> 24:29.920] Let's focus on on-prem Active Directory domain controllers [24:29.920 --> 24:34.120] hosted in cloud infrastructure as a service. [24:34.120 --> 24:37.420] So first up is Amazon AWS EC2. [24:38.160 --> 24:41.920] Many organizations leverage Amazon AWS EC2 [24:41.920 --> 24:44.460] or IaaS as a cloud data center. [24:44.460 --> 24:49.220] Some of these host on-prem AD domain controllers in AWS. [24:49.820 --> 24:52.180] So we start with Acme Corporation, [24:52.180 --> 24:54.900] which has the on-prem AD environment. [24:54.900 --> 24:57.680] They have stood up an Amazon AWS account [24:57.680 --> 25:01.100] to use EC2 as a cloud data center. [25:01.620 --> 25:04.940] Then Acme configures an IAM role for the server admins [25:04.940 --> 25:08.000] and they stand up a couple of on-prem Active Directory [25:08.000 --> 25:09.960] domain controllers in AWS. [25:11.160 --> 25:12.780] Acme then configures Federation [25:12.780 --> 25:14.400] so that users in the on-prem AD [25:14.400 --> 25:18.100] can authenticate to AWS using their AD credentials. [25:18.540 --> 25:20.640] To simplify access for server admins, [25:20.640 --> 25:23.700] the on-prem AD group AWS EC2 admins is created [25:23.700 --> 25:27.180] and then that's associated with AWS IAM role. [25:27.280 --> 25:29.100] The server team accounts are then added [25:29.100 --> 25:33.440] to the on-prem AD group called AWS EC2 admins [25:33.440 --> 25:37.480] to enable the team to easily administer the AWS environment. [25:39.200 --> 25:41.280] But if an attacker is able to compromise [25:41.280 --> 25:42.460] one of the user accounts [25:42.460 --> 25:45.800] that's a member of the AD group AWS admins, [25:47.460 --> 25:50.700] then they are able to connect to the AWS console [25:50.700 --> 25:52.720] as an AWS admin. [25:53.260 --> 25:55.320] And then now the attacker can configure [25:55.320 --> 25:57.760] the virtual domain controller instance in AWS [25:57.760 --> 25:59.620] and run commands on it. [25:59.640 --> 26:03.680] Once the command runs to modify the domain controller's copy of AD, [26:03.680 --> 26:06.240] the changes replicate back to the on-prem domain controllers [26:06.240 --> 26:09.040] and then we end up with the on-prem Active Directory [26:09.040 --> 26:10.300] being compromised. [26:10.520 --> 26:12.260] So effectively in this scenario, [26:13.280 --> 26:15.400] the attacker can compromise an account [26:15.580 --> 26:16.440] a regular user account [26:17.280 --> 26:19.780] that doesn't have any Active Directory rights. [26:19.780 --> 26:22.400] It's just a member of a group in Active Directory. [26:22.400 --> 26:26.300] This group is associated with the AWS IAM role [26:26.300 --> 26:28.080] through federation. [26:28.120 --> 26:30.540] And so once the attacker compromises this account, [26:30.540 --> 26:35.260] which has these rights granted in the group in Active Directory, [26:35.260 --> 26:39.440] but then are relayed and related to this AWS IAM role, [26:39.440 --> 26:41.540] then they can leverage that in order to compromise [26:42.060 --> 26:43.600] any of the systems that are in there [26:43.600 --> 26:45.060] that they have rights to. [26:45.720 --> 26:50.560] So I've got a summary slide here that covers this very briefly. [26:50.860 --> 26:52.820] But keep in mind that Amazon SSM, [26:52.820 --> 26:54.860] which is effectively the Amazon agent, [26:54.860 --> 26:57.440] is installed by default on most Amazon-provided instances [26:57.440 --> 27:00.240] or templates. You do need the role to execute, [27:00.240 --> 27:02.640] but once you have the rights over that environment [27:02.640 --> 27:05.880] as a server admin would, you can normally go ahead and get that. [27:06.020 --> 27:08.440] Hopefully you're logging this and looking at the logs. [27:08.440 --> 27:10.960] Amazon CloudTrail can pull these logs. [27:10.960 --> 27:14.400] And the logs need to be configured to not be deleted. [27:14.640 --> 27:17.580] So very often an attacker is just going to delete the logs [27:17.580 --> 27:19.420] so you can't see what they did. [27:20.540 --> 27:23.240] For most of 2019, I was digging in Office 365 [27:23.240 --> 27:25.700] and Azure Active Directory looking at features. [27:25.700 --> 27:27.420] As I went through each of them, I found one [27:27.420 --> 27:29.020] that was very interesting. [27:30.160 --> 27:32.180] Remember earlier how I noted that Azure roles [27:32.180 --> 27:33.920] and Azure AD roles are separate? [27:33.920 --> 27:37.020] Well, not exactly. Here it describes a way [27:37.020 --> 27:39.720] that global administrators could actually elevate their access [27:39.720 --> 27:42.320] and have some control over Azure. [27:42.960 --> 27:45.040] Surely the global administrator role description [27:45.040 --> 27:46.500] notes this, right? [27:47.020 --> 27:50.680] Well, no, it didn't, at least not as of May. [27:51.120 --> 27:53.580] The global administrator role provides full admin rights [27:53.580 --> 27:57.520] to Azure AD and ultimately all the Office 365 services. [27:57.520 --> 28:00.680] The Microsoft Online document provides key information [28:00.680 --> 28:04.300] about this, and there's nothing here showing Azure capability. [28:04.300 --> 28:06.020] I took this screenshot in May. [28:06.560 --> 28:08.680] After contacting Microsoft and providing information [28:08.680 --> 28:10.780] about this escalation path, the document's been updated [28:10.780 --> 28:13.800] in the past month or so, stating that, yes, [28:13.800 --> 28:15.560] there is a way for a global administrator [28:15.560 --> 28:17.760] to have Azure rights, [28:17.760 --> 28:20.400] which is a bit different than what it said before. [28:21.740 --> 28:23.900] Once we have access to the Azure AD portal, [28:23.900 --> 28:26.380] which is typically all Azure AD users by default, [28:26.380 --> 28:29.020] we can view several different configuration settings [28:29.020 --> 28:30.380] for Azure Active Directory, [28:30.380 --> 28:32.940] which controls many aspects of Office 365. [28:33.080 --> 28:34.960] This page shows the directory properties, [28:34.960 --> 28:37.640] which now includes the new managed security defaults. [28:37.640 --> 28:39.240] Towards the bottom is access management [28:39.240 --> 28:41.800] for Azure resources toggle. [28:41.800 --> 28:43.620] So we can see what that is. [28:43.980 --> 28:45.600] According to Microsoft documentation, [28:45.600 --> 28:47.240] toggling this option from no to yes [28:47.240 --> 28:49.660] adds the global administrator account [28:49.660 --> 28:51.640] to the user access administrator role [28:51.640 --> 28:54.480] in Azure RBAC at the root scope. [28:54.760 --> 28:57.420] This has control over all subscriptions in the tenant. [28:57.420 --> 28:59.100] And this option is only available to accounts [28:59.100 --> 29:01.600] that are members of the global administrator role. [29:01.600 --> 29:03.020] While this option is configured [29:03.020 --> 29:04.480] in the directory properties section, [29:04.480 --> 29:06.880] this is actually a per account configuration option. [29:06.880 --> 29:10.500] So this magic button provides ability to manage Azure roles, [29:10.500 --> 29:13.960] but no direct Azure rights as into VMs. [29:13.960 --> 29:16.360] Further in the documents documentation, [29:16.360 --> 29:19.340] there's additional context on what elevate access is [29:19.340 --> 29:20.420] and how it works. [29:20.420 --> 29:22.620] When developing Azure Active Directory in Azure, [29:22.620 --> 29:24.620] Microsoft had to decide if the global admin [29:24.620 --> 29:26.800] should have Azure rights initially, [29:26.800 --> 29:28.860] and opted for no, they shouldn't. [29:28.860 --> 29:32.120] But they provided this capability to gain Azure access. [29:32.120 --> 29:34.500] This is that backdoor of sorts, [29:34.500 --> 29:36.060] which has been somewhat highlighted now [29:36.060 --> 29:37.700] that the managed security default option [29:37.700 --> 29:39.140] is right below it. [29:40.320 --> 29:42.400] This graphic shows what happens in the connection point [29:42.400 --> 29:44.000] between Azure AD and Azure. [29:44.000 --> 29:46.440] My biggest concern here is that for many organizations, [29:46.440 --> 29:48.180] the group that manages Azure Active Directory [29:48.180 --> 29:50.320] and Office 365 are often a different group [29:50.320 --> 29:52.240] from those that manage Azure. [29:52.240 --> 29:54.300] This means that someone could elevate access, [29:54.300 --> 29:57.720] think a rogue admin, and no one probably would notice. [29:57.720 --> 29:59.120] This is potentially a serious threat [29:59.120 --> 30:00.980] from an insider threat perspective, [30:00.980 --> 30:02.800] especially around the detection part of this issue, [30:02.800 --> 30:04.180] which I'll cover in a bit. [30:05.040 --> 30:06.660] This note in the Microsoft document [30:06.660 --> 30:09.000] highlights an interesting phenomena with this option. [30:09.000 --> 30:10.200] Only while the account is a member [30:10.200 --> 30:11.400] of the global administrator role [30:11.400 --> 30:14.860] can the account toggle this option to yes or no. [30:14.860 --> 30:16.720] This means that if an account is no longer [30:17.220 --> 30:19.260] in the global administrator role, [30:19.260 --> 30:21.020] and the option has been set to yes, [30:21.020 --> 30:22.640] they cannot change it back to no, [30:22.640 --> 30:24.420] and no one else can change it either. [30:25.180 --> 30:27.460] I also found an API that seems related, [30:27.460 --> 30:29.520] which means that an attacker would not need to visit [30:29.520 --> 30:31.760] the Azure AD portal to perform this action. [30:31.760 --> 30:35.100] They just have to run this API call. [30:35.900 --> 30:39.820] And Ryan added this REST API elevation method to PowerAzure, [30:39.820 --> 30:41.000] which you should check out. [30:41.000 --> 30:43.140] I have a link at the bottom of the slide. [30:44.960 --> 30:47.860] Now, what I'm going to do is walk through a scenario [30:47.860 --> 30:53.100] where we have Acme, and Acme has stood up. [30:53.260 --> 30:56.080] They have a new Office 365 environment. [30:56.080 --> 30:58.100] They've had Azure for a very long time. [30:58.100 --> 31:00.620] They're using Azure as their cloud data center. [31:00.620 --> 31:01.740] They've got domain controllers there. [31:01.740 --> 31:03.880] They've got a bunch of different servers in Azure. [31:04.100 --> 31:06.480] But they're just now spinning up Office 365. [31:06.480 --> 31:07.560] They're going to use it for email. [31:07.560 --> 31:09.260] They're going to use it for a number of different things. [31:09.260 --> 31:12.980] So they have members of the Active Directory team, [31:12.980 --> 31:16.000] Exchange, SharePoint, a bunch of different people, [31:16.000 --> 31:17.880] probably 30 to 50 people [31:17.880 --> 31:20.280] are members of the global administrator role. [31:20.280 --> 31:23.400] Not everyone has had their account configured for MFA, [31:23.400 --> 31:26.400] which is realistic since many admins, [31:26.400 --> 31:28.080] this is from last year's Black Hat talk [31:28.080 --> 31:29.840] that I gave with Mark Morrow, [31:29.840 --> 31:31.300] and it was only about 8%. [31:31.860 --> 31:33.400] Hopefully, it's much higher than that, [31:33.400 --> 31:36.660] but it's still probably less than 15% if I had to guess. [31:36.820 --> 31:39.080] And so this means that most global admin accounts [31:39.080 --> 31:40.640] don't have MFA configured. [31:40.640 --> 31:42.620] And so if an attacker is able to password spray [31:42.620 --> 31:44.940] and guess the account password, [31:44.940 --> 31:46.800] then they're able to take over this account. [31:48.320 --> 31:50.440] So by default, this option is set to no. [31:50.440 --> 31:53.080] So if an attacker compromises a global admin account, [31:53.080 --> 31:55.840] they can go in and actually toggle this from no to yes, [31:55.840 --> 31:59.820] which gives them Azure User Access Administrator role [32:00.320 --> 32:02.220] rights at the root of Azure, [32:02.220 --> 32:08.020] that Azure tenant that's configured on the Azure side. [32:08.160 --> 32:10.240] So once that toggle option is set to yes, [32:10.240 --> 32:12.740] we can see a new account appears in the User Access [32:12.740 --> 32:15.460] Administrator Azure RBAC role. [32:15.460 --> 32:17.240] I call this attack account hacker, [32:17.240 --> 32:18.820] so I don't forget which one I'm using. [32:18.820 --> 32:21.220] But you'll notice that I also added a couple others [32:21.220 --> 32:23.020] that look like they belong. [32:23.080 --> 32:24.940] There's really nothing that's supposed to be in this role. [32:32.100 --> 32:33.520] Monitoring for this can be a challenge [32:33.520 --> 32:35.380] since the primary method is running an Azure [32:35.380 --> 32:37.300] CLI command at the root scope. [32:37.540 --> 32:41.640] I can't find root level Azure RBAC roles listed in the portal, [32:41.640 --> 32:44.500] so I can't go there to see what the membership is. [32:44.620 --> 32:46.680] So something that could be monitored through an API [32:46.680 --> 32:50.040] or if you're pulling logs into your sim, [32:50.040 --> 32:54.560] you could alert on RBAC role or role membership modification. [32:54.920 --> 32:58.520] But when I run that command, I get this output, [32:58.520 --> 33:00.360] which means I would have to parse it. [33:00.900 --> 33:02.720] And so this is the message that appears [33:02.720 --> 33:04.360] when attempting to remove root level role [33:04.360 --> 33:05.920] assignments from a subscription. [33:05.920 --> 33:08.520] Removing an account from a root level Azure RBAC role [33:08.520 --> 33:11.220] requires running an Azure PowerShell command. [33:11.220 --> 33:13.860] When an account toggles elevate access from yes to no, [33:13.860 --> 33:17.080] it is automatically removed from the User Access Administrator. [33:17.760 --> 33:20.120] Once the attacker has User Access Administrator [33:20.120 --> 33:21.300] rights at the root level, [33:21.300 --> 33:24.860] this account can modify roles for any subscription in Azure. [33:24.900 --> 33:27.300] Owner's obvious and probably monitored, maybe. [33:27.300 --> 33:29.180] What about something more innocuous? [33:29.660 --> 33:31.760] So I found Virtual Machine Contributor, [33:31.760 --> 33:34.160] and this is pretty interesting because the documentation states [33:34.160 --> 33:36.300] it lets you manage virtual machines, [33:36.300 --> 33:38.720] but not access to them and not the virtual network [33:38.720 --> 33:41.540] or storage account that they're connected to. [33:41.820 --> 33:44.260] And it includes this ability, [33:44.260 --> 33:47.080] which is Microsoft.compute slash virtual machine [33:47.080 --> 33:48.480] slash run command. [33:48.480 --> 33:50.840] This provides the ability to run commands on the VM [33:50.840 --> 33:53.460] as system such as PowerShell. [33:54.200 --> 33:56.640] And so we can click on run PowerShell script [33:56.640 --> 33:57.860] and we can do that. [33:57.860 --> 33:59.080] The other thing that's interesting here [33:59.080 --> 34:00.660] is that it provides the ability [34:00.660 --> 34:02.680] to re-enable the administrator account, [34:02.680 --> 34:04.420] which on a domain controller in Azure [34:04.420 --> 34:06.780] would be the RID 500 account for the domain. [34:06.800 --> 34:08.360] So we have seen some environments [34:08.360 --> 34:10.220] when we look at them from an Active Directory [34:10.220 --> 34:11.760] assessment perspective, [34:11.760 --> 34:14.780] they have had the default domain administrator account [34:14.780 --> 34:17.880] disabled, but very often it has an old password, [34:17.880 --> 34:19.980] say 10 years old or more. [34:19.980 --> 34:22.720] And it may actually have a Kerberos service [34:22.720 --> 34:24.480] principal name or spin associated with it, [34:24.480 --> 34:26.200] which means that we can Kerberos it. [34:26.200 --> 34:29.200] So if an attacker could just go ahead and Kerberos it, [34:29.680 --> 34:31.140] then re-enable it through here, [34:31.140 --> 34:34.160] they could potentially leverage that default admin account, [34:34.160 --> 34:35.860] which was originally disabled. [34:37.560 --> 34:40.320] So we're going to add the attacker controlled account [34:40.320 --> 34:42.160] to the virtual machine contributor. [34:43.800 --> 34:46.380] And now that our attacker has the ability to run commands, [34:46.380 --> 34:48.320] we can update the administrators group. [34:48.320 --> 34:50.380] This will happen on the Azure based domain controller [34:50.380 --> 34:52.980] then replicate to the domain controllers that are on-prem. [34:53.060 --> 34:55.400] Note that being able to run commands on an Azure VM [34:55.400 --> 34:57.540] is not specific to customer on-prem [34:57.540 --> 34:59.040] DCs hosted on Azure, [34:59.040 --> 35:02.080] but also other systems hosted there as well. [35:02.620 --> 35:04.500] So we take a look to see what would happen. [35:04.500 --> 35:06.240] We can go switch to a computer on-prem [35:06.240 --> 35:10.100] or run another command on the DC to see what the result is. [35:10.100 --> 35:12.660] And we've now compromised the on-prem active directory [35:12.660 --> 35:14.200] from Azure. [35:14.980 --> 35:17.100] If PowerShell logging is configured appropriately [35:17.100 --> 35:18.800] and sent to a sim and monitored, [35:18.800 --> 35:21.020] this event should identify unusual activity [35:21.620 --> 35:23.560] showing up in the PowerShell log. [35:24.460 --> 35:26.700] Another option an attacker has with these rights [35:26.700 --> 35:28.820] is running invoke mimikatz from the internet [35:28.820 --> 35:31.160] or even a compromise system on the corporate network [35:31.160 --> 35:33.100] or within the Azure tenant. [35:33.420 --> 35:34.860] So here I'm running mimikatz [35:34.860 --> 35:36.920] just to get the curb CD password hash [35:36.920 --> 35:39.240] because with that I can create golden tickets, [35:39.240 --> 35:40.660] which means I can persist forever [35:40.660 --> 35:42.660] in the on-prem active directory [35:43.140 --> 35:44.600] even though I've never touched that [35:44.600 --> 35:47.220] or had access to the corporate network. [35:47.220 --> 35:48.700] So that means once I get a foothold, [35:48.700 --> 35:51.600] I can have domain admin possibly forever. [35:52.200 --> 35:54.200] But what if instead of running invoke mimikatz [35:54.200 --> 35:57.720] directly from GitHub, we loaded it into Azure Blob Storage? [35:57.720 --> 36:00.040] So effectively, we're going to use a PowerShell script. [36:00.040 --> 36:01.060] We're going to upload. [36:01.060 --> 36:02.840] I'm not going to call it invoke mimikatz. [36:02.840 --> 36:05.680] I'll call it invmmk.txt. [36:05.760 --> 36:07.740] And Azure Blob Storage is very nice [36:07.740 --> 36:11.280] because it gives me a URL in which I can interact [36:11.280 --> 36:13.800] and connect to this file. [36:14.420 --> 36:16.080] So that would work as well. [36:16.700 --> 36:18.440] Let's take a moment to recap. [36:18.440 --> 36:21.620] From global admin to Azure user access administrator [36:21.620 --> 36:25.460] to Azure admin or virtual machine contributor. [36:25.500 --> 36:26.760] So if an attacker compromises [36:26.760 --> 36:28.740] an organization's global administrator account, [36:28.740 --> 36:30.760] either because they're just starting with Office 365 [36:30.760 --> 36:34.320] or didn't realize the risks around protecting global admins, [36:34.320 --> 36:35.860] either way, the global admin accounts [36:35.860 --> 36:38.700] weren't locked down with PIM, conditional access, or MFA. [36:38.700 --> 36:41.360] Or a global admin session token was stolen [36:41.360 --> 36:45.080] because this admin was using their regular web browser [36:45.080 --> 36:47.760] on the regular user workstation, which was compromised. [36:47.760 --> 36:49.680] This would mean that the attacker was about an hour [36:49.680 --> 36:52.140] to perform these actions, compromise the account, [36:52.140 --> 36:54.760] elevate access to Azure, get access to get Azure rights [36:54.760 --> 36:58.640] through Azure role membership, remove elevate access rights, [36:58.640 --> 37:01.800] perform malicious actions on any or all Azure VMs [37:01.800 --> 37:02.880] and all subscriptions, [37:02.880 --> 37:05.620] then remove role membership in Azure or not. [37:05.620 --> 37:07.620] The total time required only a few minutes, [37:07.620 --> 37:09.900] perhaps as much as 15 minutes total. [37:10.900 --> 37:12.880] In many organizations, there are two different groups [37:12.880 --> 37:15.840] that manage Azure AD, Office 365, and Azure. [37:15.840 --> 37:18.160] As I mentioned earlier, they have no expectations [37:18.160 --> 37:20.360] that the other team has the ability [37:20.360 --> 37:22.640] to affect their environment. [37:23.820 --> 37:26.340] So why is this issue important? [37:26.340 --> 37:28.040] Well, customers don't usually expect [37:28.040 --> 37:29.360] that the global administrator role [37:29.360 --> 37:31.760] that says it has Office 365 admin rights [37:31.760 --> 37:34.020] can also control Azure subscriptions. [37:35.380 --> 37:37.200] We need to be very careful what accounts [37:37.200 --> 37:39.300] are members of roles in Azure at the root level, [37:39.300 --> 37:41.500] as well as on the subscriptions, [37:41.500 --> 37:44.020] since this impacts the security posture. [37:44.020 --> 37:46.120] It's not obvious that removing the account [37:46.120 --> 37:47.220] from global administrator role [37:47.220 --> 37:48.280] does not remove the account [37:48.280 --> 37:51.620] from the Azure RBAC role, user access administrator, [37:51.620 --> 37:52.560] when the access management [37:52.560 --> 37:56.240] for Azure resources toggle option is still set to yes. [37:59.100 --> 38:00.580] And the key points around detection [38:00.580 --> 38:02.820] is there's no way to detect this configuration [38:02.820 --> 38:04.300] in Azure Active Directory at all. [38:04.300 --> 38:06.380] There's no property to query on accounts. [38:06.380 --> 38:08.040] You have to run an Azure CLI command [38:08.040 --> 38:09.320] to check the role group membership, [38:09.320 --> 38:11.200] but that's on the Azure side. [38:11.320 --> 38:12.700] When I walked through the Azure AD [38:12.700 --> 38:14.600] to Azure access elevation, [38:14.600 --> 38:16.400] I attempted to identify a clear event [38:16.400 --> 38:18.500] that I could alert on and was unable to. [38:18.660 --> 38:21.020] The core directory, directory management set company [38:21.020 --> 38:23.140] information log notes that something has changed, [38:23.140 --> 38:24.180] but not what. [38:24.180 --> 38:26.400] The capability for an Azure AD global admin [38:26.400 --> 38:28.440] is quote unquote by design. [38:28.440 --> 38:31.120] However, not expected, not well documented by Microsoft [38:31.120 --> 38:34.560] when reading about what Azure AD global admins can do. [38:34.560 --> 38:36.420] Certainly now the documentation has been updated [38:36.970 --> 38:39.360] to mention this, but I feel like there should be better [38:40.260 --> 38:43.420] logging and alerting around this sort of configuration. [38:44.620 --> 38:47.400] I'm concerned that this is a scenario [38:47.400 --> 38:49.920] where the Microsoft cloud customer can't detect, [38:49.920 --> 38:51.900] can't remediate and ultimately can't prevent [38:51.900 --> 38:53.800] since there's no real gate to lock [38:53.800 --> 38:56.400] if an Azure AD global admin account is exposed. [38:56.420 --> 38:58.560] And then they can jump over on the Azure side. [38:58.600 --> 39:01.220] So the only real good answer I can find for this [39:01.220 --> 39:04.700] is to ensure that global administrator groups [39:04.700 --> 39:08.380] are managed or their roles are managed with Azure AD PIM. [39:08.420 --> 39:10.340] But even this doesn't stop a rogue admin [39:10.340 --> 39:12.280] from doing whatever they want. [39:12.680 --> 39:14.540] The best mitigation is to prevent compromise [39:14.540 --> 39:15.960] of global admin accounts. [39:15.960 --> 39:18.200] Using PIM reduces this risk tremendously. [39:18.220 --> 39:20.460] Outside of that, monitor global admin role membership [39:20.460 --> 39:22.820] in Azure AD and the user access administrator group [39:22.820 --> 39:24.500] at the Azure root level. [39:24.620 --> 39:25.960] Regarding sensitive systems, [39:25.960 --> 39:27.740] severely restrict who has access to them. [39:27.740 --> 39:29.060] Placing them in a separate subscription [39:29.480 --> 39:31.800] will protect against common over-permissioning, [39:31.800 --> 39:33.580] but not against a sort of attack. [39:33.660 --> 39:36.020] So leveraging a separate Azure tenant is the best method [39:36.020 --> 39:38.380] for isolating and securing sensitive systems, [39:38.380 --> 39:39.880] including domain controllers. [39:41.040 --> 39:43.560] I reported this issue to Microsoft security [39:43.560 --> 39:45.640] in September of 2019, [39:45.640 --> 39:46.900] continued corresponding with them, [39:46.900 --> 39:49.560] providing additional details through the rest of the year. [39:49.560 --> 39:50.900] Again, my biggest concern with this [39:50.900 --> 39:52.740] is that on the Azure AD side, [39:52.740 --> 39:54.660] there is no way to detect, remediate, [39:54.660 --> 39:56.720] or prevent a global admin from flipping the switch [39:56.720 --> 39:58.800] and gaining full Azure rights. [40:00.080 --> 40:01.940] Instead of an external attacker, [40:01.940 --> 40:03.360] what if there was a malicious insider, [40:03.360 --> 40:04.460] someone who did not benefit [40:04.460 --> 40:07.440] from the company's spared no expense mantra? [40:08.200 --> 40:10.420] And we know what happened in that scenario. [40:11.280 --> 40:14.140] But how bad could it really get? [40:14.200 --> 40:15.760] Let's play what if. [40:15.760 --> 40:19.640] What if an attacker takes control of Azure resources? [40:19.980 --> 40:21.760] What if the attacker then removes [40:21.760 --> 40:23.900] all the accounts from all the roles? [40:24.160 --> 40:27.040] And what if they decide to ransom the Azure environment? [40:27.040 --> 40:29.420] Azure ransomware? Azureware? [40:30.020 --> 40:31.720] Yeah, that could be pretty bad. [40:32.680 --> 40:34.900] So let's take the cloud architecture concepts [40:34.900 --> 40:35.760] to the next level, [40:35.760 --> 40:37.920] which is pretty much where organizations are headed. [40:38.200 --> 40:38.880] Let's say you start [40:38.880 --> 40:40.780] with the typical Microsoft server architecture [40:40.780 --> 40:42.620] with on-prem Active Directory, [40:42.620 --> 40:45.220] and then we're going to add Azure to it. [40:45.680 --> 40:48.640] And then we're going to add AWS to the mix. [40:48.860 --> 40:51.460] And then we're going to add Google Cloud Platform. [40:51.820 --> 40:54.760] And so I'm just going to provide a simplified view of this [40:54.760 --> 40:57.540] because really the architecture diagrams get pretty messy. [40:58.400 --> 41:00.400] We have AWS, we have Azure, [41:00.400 --> 41:02.100] and we have Google Cloud Platform. [41:02.460 --> 41:04.240] And I simplify this, like I said, [41:04.240 --> 41:06.060] showing the on-prem data center [41:06.060 --> 41:08.840] and federation to the three different cloud environments. [41:08.840 --> 41:11.100] But ultimately this federation just means [41:11.100 --> 41:12.640] that we have this sort of configuration [41:12.640 --> 41:14.560] where everything can talk to everything else [41:15.130 --> 41:16.880] and that they're interconnected. [41:17.580 --> 41:19.300] And this means a compromise in one [41:19.910 --> 41:22.220] will likely lead to a compromise in all cloud environments. [41:22.220 --> 41:23.820] Since the IAM story is quite different in each [41:23.820 --> 41:26.540] and difficult to track roles and privileges across them, [41:26.540 --> 41:28.060] it's likely the accounts are over-permissioned [41:28.060 --> 41:29.180] in this scenario. [41:29.340 --> 41:32.080] And the use of federation provides the ability [41:32.080 --> 41:36.120] to jump across between these different environments. [41:37.000 --> 41:38.620] So a compromise of all three cloud environments [41:38.620 --> 41:40.320] will likely result in on-prem compromise [41:40.320 --> 41:42.600] since there's likely on-prem AD domain controllers [41:42.600 --> 41:44.460] in one or more of the cloud environments. [41:44.460 --> 41:47.040] Federation or any type of authentication flow, [41:47.040 --> 41:48.620] really, between these environments [41:48.620 --> 41:51.020] enables a new type of lateral movement. [41:51.320 --> 41:53.580] The attacker could leverage over-privileged Azure [41:53.580 --> 41:55.780] and AWS roles and the rights provided with them [41:55.780 --> 41:59.340] and showing which ones can provide system access to VMs, [41:59.340 --> 42:00.660] despite how they're described, [42:00.660 --> 42:03.120] is one way that we can identify [42:03.570 --> 42:07.080] and lock down those controls or those roles. [42:08.020 --> 42:10.760] But the attacker could jump across the federation [42:10.760 --> 42:12.800] to gain privileged access to cloud-hosted services [42:12.800 --> 42:15.080] and systems once they compromise a single system [42:15.080 --> 42:16.840] or even an account. [42:17.280 --> 42:18.980] Because ultimately, gaining privileged access [42:18.980 --> 42:23.360] to the IaaS service provider, so Azure, Amazon, AWS, GCP, [42:23.360 --> 42:24.960] means that the attacker could do anything [42:24.960 --> 42:26.600] with the VMs hosted in the environment, [42:26.600 --> 42:29.200] including accessing and modifying data on the systems. [42:29.200 --> 42:31.520] So it's very important that administrative roles [42:31.520 --> 42:33.480] need to be limited and gated. [42:34.800 --> 42:36.660] I recently heard a customer tell me [42:36.660 --> 42:37.940] that an executive was concerned [42:37.940 --> 42:40.520] about putting all their eggs in one basket. [42:40.940 --> 42:44.660] So they signed up for Azure and AWS and Google Cloud Platform, [42:44.660 --> 42:48.040] so now their eggs are in all the baskets. [42:48.360 --> 42:49.540] And the security capabilities [42:49.540 --> 42:52.380] and controls across these environments is quite different. [42:52.560 --> 42:54.420] And it's up to the operation and security teams [42:54.420 --> 42:55.940] to figure that part out. [42:58.060 --> 43:00.720] So the cloud provides many benefits and power, [43:00.720 --> 43:03.080] though cloud integration often complicates security. [43:03.120 --> 43:05.200] Organizations will have to do better to map security [43:05.200 --> 43:07.600] between on-prem and cloud systems and authentication. [43:07.880 --> 43:09.940] It's important as a security professional [43:09.940 --> 43:13.060] or security hobbyist to learn about these things. [43:13.060 --> 43:15.620] The cloud is the Wild West at this point. [43:15.620 --> 43:17.560] It is the whole brave new world. [43:17.720 --> 43:19.440] If you want to go far in your career, [43:19.440 --> 43:22.380] focus on cloud and cloud security because it is wide open. [43:22.380 --> 43:25.120] There's a lot of areas that haven't been explored, [43:25.120 --> 43:30.140] and especially in those areas across on-prem [43:30.140 --> 43:32.000] and the hybrid environment. [43:32.080 --> 43:34.500] So there's a lot of work that needs to be done there, [43:34.500 --> 43:37.180] and there's a lot that we need to do moving forward [43:37.180 --> 43:39.840] to improve the security in this environment. [43:40.200 --> 43:41.400] That's been my time. [43:41.400 --> 43:42.780] Thank you very much for yours.