Hello, DEF CON.
I speak to you today from my own version of the Batcave,
the ultimate in social distancing.
This is Hacking the Hybrid Cloud, and I am Shah Metcalf.
I'm the founder of Trimark, a professional services company
that helps organizations better secure
their Microsoft platform,
including the Microsoft Cloud and VMware infrastructure.
I'm a Microsoft certified master in Active Directory
and a Microsoft MVP.
I've spoken at a number of conferences
and I'm excited to be back speaking at DEF CON,
even if I'm not physically at DEF CON.
Last year, I keynoted the DEF CON Cloud Village
and three years ago gave a talk
on Hacking the Cloud at DEF CON.
This talk will take a journey through the hybrid cloud,
starting with the comparison between cloud and virtualization
discussing how on-prem Active Directory
domain controllers can be compromised.
I will also walk through the cloud hosted
managed Active Directory deployments
from AWS, Azure, and GCP.
Since this is hybrid talk,
I will walk through some hybrid components
and see how they can be attacked.
I'll touch on cloud administration
and identity access management, or IAM concepts,
and finally walk through escalation scenarios
for compromising cloud-hosted on-prem domain controllers,
AWS, and Azure.
First, we need to talk about what hybrid cloud is,
at least in the context of this talk.
There are likely lots of definitions of hybrid cloud.
Due to this, I'm defining hybrid cloud
as something on-prem and something in the cloud.
This typically means infrastructure on-prem and in the cloud,
more IaaS than strict SaaS.
Now that we've defined hybrid cloud,
let's look at three common scenarios.
The first is the on-prem Active Directory environment
with Office 365 SaaS services.
SaaS stands for software as a service.
Many organizations are leveraging the cloud
for infrastructure as a service, or IaaS,
as their cloud data center.
The third scenario here is an on-prem Active Directory
with a cloud-managed AD as an additional or resource forest.
Since cloud can be used in many ways,
it's likely that most organizations
use a combination of these, among others.
This section covers virtualization concepts and controls
as foundational information to better set up
and describe potential attacks.
Conceptually, the cloud is virtualization,
certainly from an IaaS perspective.
Amazon has used Xen and a switch to Nitro.
Azure uses a custom version of Hyper-V Core,
and Google Cloud Platform uses KVM.
Now, there is a cloud fabric
that ties the virtualization elements
together with orchestration,
which is what makes the cloud the cloud.
So, this diagram shows Azure IaaS
with a VM and related components.
This is similar to on-prem virtualization.
And Amazon's Nitro is pretty cool.
The Nitro hypervisor can take advantage
of Nitro-optimized hardware,
including a Nitro security chip.
Any instance created prior to Nitro is still running Xen.
AWS also has this concept of connectivity
between the AWS-managed Active Directory
that they host themselves,
and EC2, which is their IaaS environment
within the AWS cloud.
And then, of course, since you can host Windows servers
and Windows systems and other types of systems,
you can also integrate and connect
to other cloud environments.
This demonstrates how cloud can get complex pretty fast.
And you can run VMware on-prem
and in AWS and Azure and GCP.
Domain controllers control the keys to the kingdom
and require special security protections
that are often ignored on virtual platforms.
Compromising physical domain controllers
boils down to either gaining physical access
or compromising an out-of-band management system like iLO.
iLO hosts a web server on port 2381,
and we can scan for this using PowerShell.
So, if iLO is hosted on the same network
as production systems and clients, such as workstations,
we can scan and identify iLO.
iLO has had a number of security issues
with firmware patches available,
but the issue is that pretty much
no one updates firmware on physical servers.
Airbus Security has identified
many of these security issues with iLO,
from accessing memory to an RCE
to get read-write memory access
to the ability to execute arbitrary commands via DMA
to gain host access through iLO
even when iLO is disabled.
And I have a link at the bottom
that includes a list even beyond what I have up here
and a Go scanner to discover
vulnerable servers running iLO.
When VMware is hosting virtual DCs,
which is probably around 95% of the organizations are better,
certainly the larger ones,
there's different ways that you could compromise VMware
to get access to a virtual domain controller.
This could involve compromising an account
with rights to VMware.
Often this is a regular user account
that is a member of an 80 group called VMware Admins,
or we just need to compromise an account
that has VMware access to,
or at least one virtual domain controller,
potentially more.
If we can compromise a system which runs vCenter,
we can ultimately compromise VMware.
Also, every ESXi server has a root account.
And most of the time, every ESXi server
has the same root password,
which may be the organization default password,
which could be guest or found in a password file.
We can compromise the vCenter system,
we can compromise the vSphere environment.
And the VMware VIX API has functionality
that allows for direct access to guest OSs,
which is used by solutions
such as VMware Site Recovery Manager.
This requires VMware tools to be installed
on the guest OS or VM.
The VIX API allows guest operations
such as starting, stopping the application,
file directory manipulation,
uploading, downloading files,
all within the guest OS
without requiring any network connectivity
to the virtual machine.
This functionality may also be used
by vSphere accounts with limited privileges
to access a guest OS without the need to authenticate.
There is a scenario where an account
with vCenter privileges can use the vSphere API
to interact with the local OS installation
of VMware tools to allow a pass-through session
for an unprivileged account
to connect to that virtual machine
and execute local commands.
Since this applies to virtual domain controllers,
Active Directory could be compromised in this manner
by allowing execution of commands locally
on the virtual DC with accounts
not normally allowed local logon access to the DCs.
For Hyper-V, the attacks are similar to VMware.
We compromise an account with rights
or find a way to compromise a server,
either through a server admin account,
discover the local admin password,
or a modified group policy to gain access to the server.
So I decided I would take a look
at the so-called big three cloud hosting providers,
Managed Active Directory Environments,
and then focus on what this means
to Pentesters and Red Teamers.
The Managed Active Directory Environment
does not provide AD admin rights to the AD environment,
so the customer will never have domain admin rights
or even domain controller access.
The cloud provider controls AD infrastructure management
and delegates rights to OUs and AD components to the customer.
Let's start with Amazon AWS Directory Services
from Microsoft Active Directory.
That's quite the mouthful.
The AWS Managed AD has an AWS Delegated Groups
OU that contains the delegation groups
which the customer is provided membership in and access to.
The AWS Reserved OU is controlled exclusively
by the AWS system.
There is a customer OU created in the Managed AD domain
that is typically named after the AD subdomain
or the first part of the fully qualified domain name
or the domain's NetBIOS name.
This customer OU contains OUs for computers and users.
AWS Managed AD spins up two domain controllers
which are in different network segments
but in the same Active Directory site.
The default domain admin account
retains the default name administrator,
though this is moved to the AWS Reserved OU.
When a customer creates an AWS Managed AD,
their first account in the AD environment is called admin
and is granted full delegated rights to the AD environment
for that customer, not to the domain controllers
or AD admin rights to the full AD.
The domain password is default, still seven characters,
but there are five AWS-created fine-grained password policies
which are delegated to the customer to modify.
Based on the group policy linked to the domain controller's OU,
I was able to identify a pretty decent audit policy,
though it was missing Kerberos auditing,
which means Kerberos activity would be missed.
The first customer account, admin, is created with the description
that notes the customer OU distinguished name,
so it's very nice that they provide this description
so that way you can quickly and easily identify where this OU is.
There are several delegation groups in the AWS Managed AD environment.
All these are named with the prefix AWS-delegated
and are contained in the AWS Delegated Groups OU.
The AWS Delegated Administrators Group
is the primary customer admin group.
There are also delegation groups for managed service accounts,
adding workstations to the main Kerberos delegation,
DNS, and server administration.
Let's take a look at Azure Active Directory Domain Services.
While Azure Active Directory is not exactly Active Directory,
since it's a cloud directory with no Kerberos
or NTLM authentication, group policy, or LDAP,
Azure AD Domain Services is Active Directory,
just like you would have on a corporate network,
and is Microsoft's managed Active Directory.
So this environment, Azure AD DS,
has several top-level OUs that are created.
These are prefixed with AADD,
and the AADDC computers and AADDC users OUs
are the primary customer OUs.
The Azure AD DS domain controllers are in the same network,
and at least in this instance that I created,
had sequential IPs, .4 and .5.
The default domain admin account is called DCAASAdmin,
and remains in the default user's OU.
There is one fine-grained password policy that's pre-created.
I'm not going to try to read the name.
As it is, it's a bit tough with all the AA and DDs,
but the customer has delegated modifier rights
to this fine-grained password policy object.
I can't see any auditing configuration set in GPO,
so I have to assume that they are set via local policy,
which I can't see without domain controller access.
Azure AD DS has a few delegation groups
which are prefixed with AADDC, and are in the AADDC users OU.
The primary customer group is AADDC administrators,
which is delegated full control to many OUs and AD components.
I'm pretty sure when Microsoft came up with the names
for these groups,
they were not thinking about having to say this out loud in a row.
So thank you, Microsoft, for that.
Moving on to the third and final managed AD environment
that I reviewed recently,
this is the Google Cloud Platform Managed Active Directory.
It's interesting to note that the GCP-managed AD
seems to cost about twice what the Amazon AWS
and Microsoft Azure-managed AD environments cost,
because they're about $100 each.
The GCP-managed AD environment has a single root OU
for the customer called Cloud,
with a single child OU called Computers.
The Cloud Service Objects OU contains delegation groups.
GCP-managed AD is different from the others
that I just talked about in that domain controllers
are running Windows Server 2019,
though the Active Directory forest functional level
is still set to 2012 R2.
Also, the Active Directory Recycle Bin,
which enables rapid undelete capability,
is not enabled, which is interesting.
The default domain administrator account
keeps the default name Administrator,
and it's disabled.
So, GCP spins up a second domain admin account
called CloudSVCAdmin, and this account is enabled.
The domain password policy is default,
and the customer can create fine-grained password policies.
And with this, the DC event auditing
is not configured through group policy,
so again, I couldn't identify how it may be configured.
GCP-managed AD has several delegation groups.
They start with Cloud Service,
and these are stored in the Cloud Service Objects OU.
The Cloud Service All Administrators
and the Cloud Service Administrators
are the primary customer admin groups.
GCP delegation groups include computer admins,
managed service account admins, and others.
This slide captures some of the common themes
of the three managed AD environments I reviewed.
I used a version of the Trimark Active Directory
assessment tool that I wrote to perform the review
of the different AD environments,
which enabled me to capture data for all three
and identify the key security features
and delegation in about a day to put this talk together,
or at least this part of it.
The bottom of the slide will provide a link
to an Active Directory security review PowerShell script
that I wrote and published in June.
The link points to the article
where I highlight key AD security items
to check and what to check for,
and the PowerShell script simplifies
the data collection part, so analysis is much simpler.
So it's good to check that out. It would be helpful
if you're looking at an environment such as these.
So since the customer does not actually have AD admin rights
to a managed AD environment,
there's not much point in attempting to escalate to DA.
Better to identify which managed AD environment it is
and enumerate the managed AD privilege groups
to determine which accounts are highly privileged.
Interesting note about Azure Active Directory domain services,
Microsoft's managed AD.
It can be managed by Azure AD accounts,
and these are synchronized into Azure AD DS
or even by on-prem AD accounts
that are synced in from the on-prem AD environment.
And these are synced using the on-prem Azure AD
Connect system through Azure AD to Azure AD domain services.
And if password hash sync is enabled, which it usually is,
then the on-prem AD account password hash is included.
Normally, password hash sync does a hash of a hash,
but when syncing through to Azure AD domain services,
that original AD password hash is retained and synchronized,
so that way the password can actually be used
in Azure Active Directory domain services.
So, there are several cloud components
that connect on-prem to cloud.
This connectivity has some very interesting implications.
Amazon's AD Connector is effectively
an on-prem AD authentication proxy.
AD Connector is designed to provide an easy way
to establish a trusted relationship
between your Active Directory and AWS.
So, when AD Connector is configured,
it enables the ability to sign into AWS applications
with on-prem AD credentials,
join AWS Windows instances to the on-prem AD domain,
and provides federated sign-in to the AWS Management Console
by mapping these on-prem identities to AWS IAM roles.
And AD Connector can't be used with custom applications
since it's just for AWS.
Basically, a user opens the secure custom sign-in page,
supplies their Active Directory username and password.
This authentication request is sent over SSL to the AD Connector,
and then the AD Connector performs LDAP authentication
to Active Directory.
The interesting part happens at this point.
Once the user has been authenticated,
the AD Connector calls the STS assume role method
to get temporary security credentials for that user.
Using those temporary security credentials,
AD Connector constructs a sign-in URL
that users use to access the console.
And if a user is mapped to multiple roles,
the user will be presented with a choice at the sign-in time
as to which role they want to assume.
User session is valid for an hour,
which is pretty standard for cloud apps.
Since the AD Connector environment is hosted in AWS,
this would need to be targeted in order to potentially impersonate the users
and be able to leverage that STS assume role method.
Microsoft's pass-through authentication is similar
because with Microsoft pass-through authentication, or PTA,
it provides users the ability to authenticate to Azure Active Directory,
which is passed through to the on-prem AD
and validated in Azure AD by getting the status back
from the PTA agent in the on-prem environment.
PTA is enabled via Azure AD Connect.
The users use the same passwords
to sign into both on-prem and cloud-based applications.
The communication between an agent and Azure AD
is secured using certificate-based authentication,
and these certificates are automatically renewed
every few months by Azure AD.
I couldn't figure out what happens if you get one of these certificates,
but that could be pretty interesting.
But ultimately, this supports user sign-in
into all web browser-based applications
and into Microsoft Office client applications
that use modern authentication.
Adam Chester, aka XPN, has done a great blog post
on how to leverage a compromised Azure AD Connect server
with PTA enabled in order to gather usernames
and cleartext passwords that were used to authenticate via PTA.
So since password authentication is managed by Azure AD Connect,
you can compromise that server.
And then during this authentication process,
Azure AD sends the cleartext password,
not hashed, to authenticate the user.
So it's sent to Azure AD Connect.
Azure AD Connect PTA component is going to check
and validate the username and password with Active Directory,
then send back the status to Azure AD for that.
So Adam identified that you could inject a DLL
to compromise the credentials that were used for PTA
and then have those credentials for anyone that was using PTA.
Azure Active Directory Seamless Single Sign-On,
or Azure AD Seamless SSO, automatically signs users
in when they are on their corporate devices
connected to the corporate network.
This graphic is interesting and nice,
but I prefer this one because it has a lot more
of the flow.
But basically what ends up happening
is that users don't need to type in their passwords
to sign into Azure AD or even really type in their usernames.
It provides the users easy access
to their cloud-based applications
without any additional on-prem components like federation.
This is done by converting the on-prem authentication
to cloud authentication through a computer account
in the Active Directory.
So Michael Grafner from DS Internals
posted this article a few years ago,
or a couple of years ago, where he was talking about
how the Azure AD has a publicly available endpoint
that accepts these Kerberos tickets
and translates them into SAML and JWT tokens
that can be leveraged for cloud apps.
Effectively, if you can compromise the Azure AD
Seamless Single Sign-On computer account,
the password has to be associated with it.
The computer account is called Azure AD SSO ACC
and can be identified as in the computer's container.
If you're able to compromise this and get this password hash,
you can generate a silver ticket for any user
you want to impersonate,
and the service that you use in the silver ticket
is going to be aadg.windows.net.nsatc.net.
You inject this ticket into the local Kerberos cache,
and Azure AD Seamless Single Sign-On,
the computer account password doesn't actually change.
Microsoft does recommend that it gets rolled
and changed at least every 30 days.
What's interesting is because it's a computer account,
there is no actual computer that is behind the scenes
that is using that computer account
other than the Azure AD Connect client.
So normally a computer in Active Directory
would update the computer account password
that's associated with that computer.
Since there isn't really a computer here,
that password isn't getting changed.
So maybe in the future Microsoft will code into Azure AD Connect
to automatically update the computer account password for this.
Three years ago at DEF CON 25 in Las Vegas,
I co-presented a talk called Hacking the Cloud.
I talked about how Azure AD Connect's password
hash sync permissions provided Mimikatz DC-sync rights,
and that the author of Mimikatz DC-sync, Vincent Leteau,
was inspired by password hash sync to create DC-sync
because PHS requests password hashes
for sync users on a regular basis.
We can scan Active Directory domain permissions
looking for potential Azure AD Connect service counts.
Or if the Azure AD Connect install
would use the express install option,
then there's a helpful description on the account
noting on which server Azure AD Connect is installed.
We can then check in Active Directory
to figure out where that computer is stored.
Very often this is in the server's OU or somewhere similar.
Once we know the server name, we can check for,
and its location,
the actual distinguished name of the OU that it's in.
We can check AD for any group policies
that add AD groups to the local administrators group.
Compromising any account
that is a member of the local administrators group
on the Azure AD Connect server
would result in compromise of the Azure AD Connect server
and ultimately Active Directory,
assuming that it has password hash synchronization
rights configured in the environment.
Also, if we can modify a group policy
that links to the Azure AD Connect server OU,
then we can compromise the server that way as well.
When talking about cloud environments,
it's important to mention how administration is performed.
This is commonly referred to as Identity Access Management,
or IAM.
So we've gone from groups in our on-prem environment
with Active Directory to roles,
which is effectively these rights
that are packaged together.
Azure and AWS and GCP have their own methods,
but a lot of the concepts are effectively the same.
Azure IAM uses the concept of an owner,
a contributor, and a reader.
The owner has full access, the contributor has the ability
to create and manage the resources
but can't grant access to others,
and then a reader can just view things.
And so Azure has the tenant,
which is the top-level container of the Azure environment,
and this contains subscriptions.
The tenant admins are effectively the owner role on the tenant,
and subscription admin would be effectively the owner role
on the subscription.
What's interesting is that Microsoft notes
that Azure roles and Azure AD roles
are separate and different, mostly,
and I'll cover that in a little bit.
AWS has a root account, often called
or referred to as a payer account,
which is the primary account,
though it really shouldn't be used regularly.
Then there are account admins.
So in AWS, the account is similar to the Azure tenant.
It's the top-level container for the environment
for that customer.
The account admins have full control over the account.
If an account is a root account and an admin account,
then that account has full organizational control.
It's important to note here that any overscoped roles
provide escalation capability.
And Rhino Security Labs has an extensive article
covering 21 AWS IAM privilege escalation methods,
which allow an attacker to escalate
from a compromised low-privilege account
to full admin privileges.
So I've listed a bunch of them here,
I apologize for the small text,
but really what you want to do is you want to read
the blog article that they wrote,
which is linked at the bottom and in the slides.
They also released a scanning tool,
which is on GitHub, to identify these vulnerabilities
in an AWS user account.
And if you have an account with IAM read access for all users,
the script can be run against every user in the account
to detect these vulnerabilities account-wide.
So if you're asked to pen test AWS,
or you're providing security consulting services around AWS,
or even you work in an organization
and you're moving to AWS
and you have some of the IAM roles configured,
you definitely want to scan them
for potential escalation methods.
API keys, I'm just going to cover this really quickly.
They provide permanent access to an environment,
often with extensive privileges.
These keys can often bypass the typical username
and password used to authenticate as an account
if it's configured as such.
Well, I've covered a number of scenarios
on how to attack on-prem and hybrid cloud components.
Let's focus on on-prem Active Directory domain controllers
hosted in cloud infrastructure as a service.
So first up is Amazon AWS EC2.
Many organizations leverage Amazon AWS EC2
or IaaS as a cloud data center.
Some of these host on-prem AD domain controllers in AWS.
So we start with Acme Corporation,
which has the on-prem AD environment.
They have stood up an Amazon AWS account
to use EC2 as a cloud data center.
Then Acme configures an IAM role for the server admins
and they stand up a couple of on-prem Active Directory
domain controllers in AWS.
Acme then configures Federation
so that users in the on-prem AD
can authenticate to AWS using their AD credentials.
To simplify access for server admins,
the on-prem AD group AWS EC2 admins is created
and then that's associated with AWS IAM role.
The server team accounts are then added
to the on-prem AD group called AWS EC2 admins
to enable the team to easily administer the AWS environment.
But if an attacker is able to compromise
one of the user accounts
that's a member of the AD group AWS admins,
then they are able to connect to the AWS console
as an AWS admin.
And then now the attacker can configure
the virtual domain controller instance in AWS
and run commands on it.
Once the command runs to modify the domain controller's copy of AD,
the changes replicate back to the on-prem domain controllers
and then we end up with the on-prem Active Directory
being compromised.
So effectively in this scenario,
the attacker can compromise an account
a regular user account
that doesn't have any Active Directory rights.
It's just a member of a group in Active Directory.
This group is associated with the AWS IAM role
through federation.
And so once the attacker compromises this account,
which has these rights granted in the group in Active Directory,
but then are relayed and related to this AWS IAM role,
then they can leverage that in order to compromise
any of the systems that are in there
that they have rights to.
So I've got a summary slide here that covers this very briefly.
But keep in mind that Amazon SSM,
which is effectively the Amazon agent,
is installed by default on most Amazon-provided instances
or templates. You do need the role to execute,
but once you have the rights over that environment
as a server admin would, you can normally go ahead and get that.
Hopefully you're logging this and looking at the logs.
Amazon CloudTrail can pull these logs.
And the logs need to be configured to not be deleted.
So very often an attacker is just going to delete the logs
so you can't see what they did.
For most of 2019, I was digging in Office 365
and Azure Active Directory looking at features.
As I went through each of them, I found one
that was very interesting.
Remember earlier how I noted that Azure roles
and Azure AD roles are separate?
Well, not exactly. Here it describes a way
that global administrators could actually elevate their access
and have some control over Azure.
Surely the global administrator role description
notes this, right?
Well, no, it didn't, at least not as of May.
The global administrator role provides full admin rights
to Azure AD and ultimately all the Office 365 services.
The Microsoft Online document provides key information
about this, and there's nothing here showing Azure capability.
I took this screenshot in May.
After contacting Microsoft and providing information
about this escalation path, the document's been updated
in the past month or so, stating that, yes,
there is a way for a global administrator
to have Azure rights,
which is a bit different than what it said before.
Once we have access to the Azure AD portal,
which is typically all Azure AD users by default,
we can view several different configuration settings
for Azure Active Directory,
which controls many aspects of Office 365.
This page shows the directory properties,
which now includes the new managed security defaults.
Towards the bottom is access management
for Azure resources toggle.
So we can see what that is.
According to Microsoft documentation,
toggling this option from no to yes
adds the global administrator account
to the user access administrator role
in Azure RBAC at the root scope.
This has control over all subscriptions in the tenant.
And this option is only available to accounts
that are members of the global administrator role.
While this option is configured
in the directory properties section,
this is actually a per account configuration option.
So this magic button provides ability to manage Azure roles,
but no direct Azure rights as into VMs.
Further in the documents documentation,
there's additional context on what elevate access is
and how it works.
When developing Azure Active Directory in Azure,
Microsoft had to decide if the global admin
should have Azure rights initially,
and opted for no, they shouldn't.
But they provided this capability to gain Azure access.
This is that backdoor of sorts,
which has been somewhat highlighted now
that the managed security default option
is right below it.
This graphic shows what happens in the connection point
between Azure AD and Azure.
My biggest concern here is that for many organizations,
the group that manages Azure Active Directory
and Office 365 are often a different group
from those that manage Azure.
This means that someone could elevate access,
think a rogue admin, and no one probably would notice.
This is potentially a serious threat
from an insider threat perspective,
especially around the detection part of this issue,
which I'll cover in a bit.
This note in the Microsoft document
highlights an interesting phenomena with this option.
Only while the account is a member
of the global administrator role
can the account toggle this option to yes or no.
This means that if an account is no longer
in the global administrator role,
and the option has been set to yes,
they cannot change it back to no,
and no one else can change it either.
I also found an API that seems related,
which means that an attacker would not need to visit
the Azure AD portal to perform this action.
They just have to run this API call.
And Ryan added this REST API elevation method to PowerAzure,
which you should check out.
I have a link at the bottom of the slide.
Now, what I'm going to do is walk through a scenario
where we have Acme, and Acme has stood up.
They have a new Office 365 environment.
They've had Azure for a very long time.
They're using Azure as their cloud data center.
They've got domain controllers there.
They've got a bunch of different servers in Azure.
But they're just now spinning up Office 365.
They're going to use it for email.
They're going to use it for a number of different things.
So they have members of the Active Directory team,
Exchange, SharePoint, a bunch of different people,
probably 30 to 50 people
are members of the global administrator role.
Not everyone has had their account configured for MFA,
which is realistic since many admins,
this is from last year's Black Hat talk
that I gave with Mark Morrow,
and it was only about 8%.
Hopefully, it's much higher than that,
but it's still probably less than 15% if I had to guess.
And so this means that most global admin accounts
don't have MFA configured.
And so if an attacker is able to password spray
and guess the account password,
then they're able to take over this account.
So by default, this option is set to no.
So if an attacker compromises a global admin account,
they can go in and actually toggle this from no to yes,
which gives them Azure User Access Administrator role
rights at the root of Azure,
that Azure tenant that's configured on the Azure side.
So once that toggle option is set to yes,
we can see a new account appears in the User Access
Administrator Azure RBAC role.
I call this attack account hacker,
so I don't forget which one I'm using.
But you'll notice that I also added a couple others
that look like they belong.
There's really nothing that's supposed to be in this role.
Monitoring for this can be a challenge
since the primary method is running an Azure
CLI command at the root scope.
I can't find root level Azure RBAC roles listed in the portal,
so I can't go there to see what the membership is.
So something that could be monitored through an API
or if you're pulling logs into your sim,
you could alert on RBAC role or role membership modification.
But when I run that command, I get this output,
which means I would have to parse it.
And so this is the message that appears
when attempting to remove root level role
assignments from a subscription.
Removing an account from a root level Azure RBAC role
requires running an Azure PowerShell command.
When an account toggles elevate access from yes to no,
it is automatically removed from the User Access Administrator.
Once the attacker has User Access Administrator
rights at the root level,
this account can modify roles for any subscription in Azure.
Owner's obvious and probably monitored, maybe.
What about something more innocuous?
So I found Virtual Machine Contributor,
and this is pretty interesting because the documentation states
it lets you manage virtual machines,
but not access to them and not the virtual network
or storage account that they're connected to.
And it includes this ability,
which is Microsoft.compute slash virtual machine
slash run command.
This provides the ability to run commands on the VM
as system such as PowerShell.
And so we can click on run PowerShell script
and we can do that.
The other thing that's interesting here
is that it provides the ability
to re-enable the administrator account,
which on a domain controller in Azure
would be the RID 500 account for the domain.
So we have seen some environments
when we look at them from an Active Directory
assessment perspective,
they have had the default domain administrator account
disabled, but very often it has an old password,
say 10 years old or more.
And it may actually have a Kerberos service
principal name or spin associated with it,
which means that we can Kerberos it.
So if an attacker could just go ahead and Kerberos it,
then re-enable it through here,
they could potentially leverage that default admin account,
which was originally disabled.
So we're going to add the attacker controlled account
to the virtual machine contributor.
And now that our attacker has the ability to run commands,
we can update the administrators group.
This will happen on the Azure based domain controller
then replicate to the domain controllers that are on-prem.
Note that being able to run commands on an Azure VM
is not specific to customer on-prem
DCs hosted on Azure,
but also other systems hosted there as well.
So we take a look to see what would happen.
We can go switch to a computer on-prem
or run another command on the DC to see what the result is.
And we've now compromised the on-prem active directory
from Azure.
If PowerShell logging is configured appropriately
and sent to a sim and monitored,
this event should identify unusual activity
showing up in the PowerShell log.
Another option an attacker has with these rights
is running invoke mimikatz from the internet
or even a compromise system on the corporate network
or within the Azure tenant.
So here I'm running mimikatz
just to get the curb CD password hash
because with that I can create golden tickets,
which means I can persist forever
in the on-prem active directory
even though I've never touched that
or had access to the corporate network.
So that means once I get a foothold,
I can have domain admin possibly forever.
But what if instead of running invoke mimikatz
directly from GitHub, we loaded it into Azure Blob Storage?
So effectively, we're going to use a PowerShell script.
We're going to upload.
I'm not going to call it invoke mimikatz.
I'll call it invmmk.txt.
And Azure Blob Storage is very nice
because it gives me a URL in which I can interact
and connect to this file.
So that would work as well.
Let's take a moment to recap.
From global admin to Azure user access administrator
to Azure admin or virtual machine contributor.
So if an attacker compromises
an organization's global administrator account,
either because they're just starting with Office 365
or didn't realize the risks around protecting global admins,
either way, the global admin accounts
weren't locked down with PIM, conditional access, or MFA.
Or a global admin session token was stolen
because this admin was using their regular web browser
on the regular user workstation, which was compromised.
This would mean that the attacker was about an hour
to perform these actions, compromise the account,
elevate access to Azure, get access to get Azure rights
through Azure role membership, remove elevate access rights,
perform malicious actions on any or all Azure VMs
and all subscriptions,
then remove role membership in Azure or not.
The total time required only a few minutes,
perhaps as much as 15 minutes total.
In many organizations, there are two different groups
that manage Azure AD, Office 365, and Azure.
As I mentioned earlier, they have no expectations
that the other team has the ability
to affect their environment.
So why is this issue important?
Well, customers don't usually expect
that the global administrator role
that says it has Office 365 admin rights
can also control Azure subscriptions.
We need to be very careful what accounts
are members of roles in Azure at the root level,
as well as on the subscriptions,
since this impacts the security posture.
It's not obvious that removing the account
from global administrator role
does not remove the account
from the Azure RBAC role, user access administrator,
when the access management
for Azure resources toggle option is still set to yes.
And the key points around detection
is there's no way to detect this configuration
in Azure Active Directory at all.
There's no property to query on accounts.
You have to run an Azure CLI command
to check the role group membership,
but that's on the Azure side.
When I walked through the Azure AD
to Azure access elevation,
I attempted to identify a clear event
that I could alert on and was unable to.
The core directory, directory management set company
information log notes that something has changed,
but not what.
The capability for an Azure AD global admin
is quote unquote by design.
However, not expected, not well documented by Microsoft
when reading about what Azure AD global admins can do.
Certainly now the documentation has been updated
to mention this, but I feel like there should be better
logging and alerting around this sort of configuration.
I'm concerned that this is a scenario
where the Microsoft cloud customer can't detect,
can't remediate and ultimately can't prevent
since there's no real gate to lock
if an Azure AD global admin account is exposed.
And then they can jump over on the Azure side.
So the only real good answer I can find for this
is to ensure that global administrator groups
are managed or their roles are managed with Azure AD PIM.
But even this doesn't stop a rogue admin
from doing whatever they want.
The best mitigation is to prevent compromise
of global admin accounts.
Using PIM reduces this risk tremendously.
Outside of that, monitor global admin role membership
in Azure AD and the user access administrator group
at the Azure root level.
Regarding sensitive systems,
severely restrict who has access to them.
Placing them in a separate subscription
will protect against common over-permissioning,
but not against a sort of attack.
So leveraging a separate Azure tenant is the best method
for isolating and securing sensitive systems,
including domain controllers.
I reported this issue to Microsoft security
in September of 2019,
continued corresponding with them,
providing additional details through the rest of the year.
Again, my biggest concern with this
is that on the Azure AD side,
there is no way to detect, remediate,
or prevent a global admin from flipping the switch
and gaining full Azure rights.
Instead of an external attacker,
what if there was a malicious insider,
someone who did not benefit
from the company's spared no expense mantra?
And we know what happened in that scenario.
But how bad could it really get?
Let's play what if.
What if an attacker takes control of Azure resources?
What if the attacker then removes
all the accounts from all the roles?
And what if they decide to ransom the Azure environment?
Azure ransomware? Azureware?
Yeah, that could be pretty bad.
So let's take the cloud architecture concepts
to the next level,
which is pretty much where organizations are headed.
Let's say you start
with the typical Microsoft server architecture
with on-prem Active Directory,
and then we're going to add Azure to it.
And then we're going to add AWS to the mix.
And then we're going to add Google Cloud Platform.
And so I'm just going to provide a simplified view of this
because really the architecture diagrams get pretty messy.
We have AWS, we have Azure,
and we have Google Cloud Platform.
And I simplify this, like I said,
showing the on-prem data center
and federation to the three different cloud environments.
But ultimately this federation just means
that we have this sort of configuration
where everything can talk to everything else
and that they're interconnected.
And this means a compromise in one
will likely lead to a compromise in all cloud environments.
Since the IAM story is quite different in each
and difficult to track roles and privileges across them,
it's likely the accounts are over-permissioned
in this scenario.
And the use of federation provides the ability
to jump across between these different environments.
So a compromise of all three cloud environments
will likely result in on-prem compromise
since there's likely on-prem AD domain controllers
in one or more of the cloud environments.
Federation or any type of authentication flow,
really, between these environments
enables a new type of lateral movement.
The attacker could leverage over-privileged Azure
and AWS roles and the rights provided with them
and showing which ones can provide system access to VMs,
despite how they're described,
is one way that we can identify
and lock down those controls or those roles.
But the attacker could jump across the federation
to gain privileged access to cloud-hosted services
and systems once they compromise a single system
or even an account.
Because ultimately, gaining privileged access
to the IaaS service provider, so Azure, Amazon, AWS, GCP,
means that the attacker could do anything
with the VMs hosted in the environment,
including accessing and modifying data on the systems.
So it's very important that administrative roles
need to be limited and gated.
I recently heard a customer tell me
that an executive was concerned
about putting all their eggs in one basket.
So they signed up for Azure and AWS and Google Cloud Platform,
so now their eggs are in all the baskets.
And the security capabilities
and controls across these environments is quite different.
And it's up to the operation and security teams
to figure that part out.
So the cloud provides many benefits and power,
though cloud integration often complicates security.
Organizations will have to do better to map security
between on-prem and cloud systems and authentication.
It's important as a security professional
or security hobbyist to learn about these things.
The cloud is the Wild West at this point.
It is the whole brave new world.
If you want to go far in your career,
focus on cloud and cloud security because it is wide open.
There's a lot of areas that haven't been explored,
and especially in those areas across on-prem
and the hybrid environment.
So there's a lot of work that needs to be done there,
and there's a lot that we need to do moving forward
to improve the security in this environment.
That's been my time.
Thank you very much for yours.