Hello everyone and welcome to my talk container self staying undetected using the windows containers isolation framework The use of containers became an integral part of any resource efficient and secure environment starting for middle server 2016 Microsoft released its version of this solution called Windows containers Today we will break down the containers file system isolation framework and see how it can be abused to bypass security products file system based malware protection file write restrictions and ETW based correlations Before we begin a bit about myself, my name is Daniel Avinom. I'm a security researcher at Deep Instinct for the past few years It's my work. I focus mainly on Windows internals and low-level programming And I'm also a bigger motorsport fan. I'll even admit to you that I have a driving simulator at home, which costed me way too much Okay today's agenda We will start by Overviewing the basics of the windows containers framework and get to know some of its main concepts We then continue to analyze the WCFS driver Which is the main driver the framework uses for file system isolation and see how our findings can then be utilized to bypass security products Finally we will summarize and provide ways to effectively mitigate the bypasses Okay, let's start with the basics of Windows containers job objects Job objects have been around since the days of Windows Server 2003 these objects are designed to group several processes and manage them as one unit This allows the system to control the attributes of all processes associated with a job like limiting the CPU usage IO bandwidth virtual memory usage and network activity and Multiprocessed applications often use these objects to manage your child processes more easily These are also known as nested job For example this Google Drive processes uses uses a nested job to manage its child processes Although they make a good start jobs themselves are not enough to provide the isolation needed for a container Which is why Microsoft created silos? Silos can be considered an extension of jobs kind of super jobs Similarly to traditional jobs. These objects are used for process grouping but with additional features Containers use a type of silo called server silo which besides providing the basic job capabilities It also provides Redirection of various system resources like the registry networking and the object manager The windows kernel detects processes assigned to silos using API like PS is coinciding to in server silo and PS is processing silo Looking at the io-unload driver snippet taken from NTS kernel The windows kernel checks whether the process that attempted to unload the driver is within a server silo and if it is It locks the operation and blocks it my returning status privilege not held So silos do provide a good foundation to a container, but file system separation is still needed Microsoft implemented this concept using repos points So what is a repos point a repos point is an MFT attribute and it can be given to files or directories It's data type. So it's a repos point data is it's typed meaning a registered registered tag identifier or the repos tag Determines the data structure of the repos point data itself It can be given to files or directories using the device IO control function Alongside the set repos point file system control code note that right primitive is needed for this to succeed Now when a file with a repos point is being opened it is handled by a file system mini filter driver according to its repos tag a Good example of these attributes can be seen in symbolic links Directories that functions as a symbolic link to another Contains behind the scene a repos point with the path to the code destination Using the junction 64 to from system tell us so that you can see that the see users all users for them Redirects using a symbolic link to see program data. This redirection again is implemented by a repos point So we mentioned that the previous slide mini filter divers, so let's give a bit of a background about them as well Mini filter drivers were designed to make the IO filtering process much easier for developers Since implementing a legacy federal dollar from scotch is difficult Microsoft provided a solution in the form of its mid filter manager The filter manager is a legacy filter that manages other mini filter drivers according So when takes care of all the heavy lifting for them for example their insertion to the device tech Ignoring any relevant requests and the support for multiple platforms among other It also exposes a mini filter dedicated API that implements the common operations used by these drivers This is also known as the FLT API We will soon see some functions starting with the FLT prefix Okay, each mini filter can be attached by the manager to one or more volumes getting what is called a mini filter instance and A volume is simply a logical storage unit that is implemented So that is presented to the file system as a disk for example the C Drive or when it's anti device path the hard disk volume 3 Similarly to legacy filters and Mini filter instances can intercept the pre and post operations of numerous IO functions like create read and write and Another important concept the filter manager implements is the mini filter altitude system Each mini filter should specify an altitude which is a numeric value upon its registration to the manager The filter manager invokes its mini filter operations callbacks according to their altitudes a higher altitude mini filter will handle the pre operation before the ones below it and the post operation after them So I know that this was a long slide To make things a bit clearer. Let's see a demonstration of how many filters and riposte points work together When a call to any create file is made with a file containing a point the IO manager will be in that will build an IRP and interrupt request packet for this operation This IRP will then be sent down the device stack of the storage device the C Drive in our case until it will reach the filter manager as stated the filter manager organizes its mini filters in a stack Based on their assigned altitudes it will then invoke the pre create operation callbacks of its mini filters according to their altitudes from top to bottom And after all the pre-operation callbacks were involved The IRP will then be returned to the filter manager and then be sent down the storage device stack until it will reach the actual storage device until it will reach the device, the file will be read and the IRP will then go back up the device stack this time at the opposite direction And when it will reach the NTFS driver driver, it will recognize a file containing a point was being opened, so it will change the status of this IRP to status repos. The filter manager then will receive the IRP again, and this time it will invoke the post-operation callbacks of its minifilters, again according to the altitudes, this time at the opposite direction from bottom to top. Now when a minifilter that should handle a repos point gets an IRP with the status repos status code, it first needs to get the actual content of the repos point from the MFT attribute itself, and it does so by calling FLTFS control file with the get repos point file system control code, again this will read the actual content of the MFT attribute itself, the repos point data. Now here the minifilter faces two options. The first option is if the repos tag that is located in the repos data header is not associated with it. In this case it will ignore the request and will leave it to the drivers above it. The second option is if the tag is associated with it. The driver then will know how to read the repos point buffer and extract the correct destination path from the repos point itself, and in this case the minifilter usually replaces the request file object using IO replace file object name with the FLT set callback data dirty. The combination of these two functions will cause the IO manager to then repulse the name in the file object. And then pass the request back down, this time with the correct path. So eventually the destination path will be the destination file, so it will be returned to the caller. Okay so if you go back to containers, in order to avoid an additional copy of OS files, each container is using a dynamically generated image which points to the original using repos points. The result is images that contain ghost files. Which basically store no actual data, but point to a different volume on the system. When looking at a mounted container volume, we see that the Windows folder contains files in the size of 1.2 gigabytes, but has the actual size on disk of only 57 megabytes. Again this is happening because files on the container's volume doesn't contain the actual, doesn't store the actual content. They're simply links to the original files on the OS, the OS file system, and as stated these links are implemented by repos points. Okay so it was at this point that the idea struck me. What if I could use this redirection mechanism to obfuscate my file system operations and confuse security products? The route that I chose was not to escape the container from within, but intentionally use this feature while executing on the host. Okay let's start talking about the WCIFS driver. The Windows Container Isolation File System, or WCIFS, is the minifilter driver which is responsible for the file system separation between Windows containers and their host. We'll soon see exactly how. During my research, I was surprised to find that this driver is loaded on every Windows OS, starting from Windows 10, including servers by default. This is true even when the containers option is turned off in the Windows features menu. Meaning any potential abuse of this driver would affect most of today's maintenance. It's a very modern system, without relying on any third-party files or packages installed. We saw that repos points are parsed and handled by minifilters based on their tag. WCIFS parses repos points as well, and this is how it performs the ghost files redirection process. The main repos tags associated with this driver are IOReposTagWCI1 and IOReposTagWCILink1, which according to the Windows documentation are used to repos. After some basic reverse engineering, I managed to map both of the point's internal buffers, which were the same by the way. The buffer itself is pretty straight forward. It contains a lot of information. It contains a GUID, which is a hard-coded value I found on any of the WCI tags, and the path to the destination file, so nothing special here. And at this point, all that there was left for me to do was to attach a WCIFS driver to a volume, debug it using a debugger, and place a breakpoint on its post-operation callback to see how the repos points are being handled. And how can I potentially abuse it? So I've placed a breakpoint, and I've waited, and waited, and waited, and nothing happened. So it turns out that for the post-operation callback to invoke, the pre-operation callback must return either FLTPreop success with callback or FLTPreop synchronize, which didn't happen. So looking at the pre-create function of the WC driver, I saw that there was a function blocking me. This function called WC unions exists for instance. Stepping into this function, I saw that there were two conditions that had to be met. The first condition is a check of whether the process that originated the create call is not within the host silo. Host silo is the equivalent to the host OS, or in other words, if the originated process was not inside the silo at all. Simply being inside the silo is not enough, because the second condition is that the silo that originated the request is not within the host silo. The second condition is that the process will have a registered context within the driver's internal collections. Context management is another feature provided by the filter manager. A mini-filter can create custom-defined data, also known as union context, and link it to objects like files, instances, and silos using the FLT API. So to pass the pre-create checks of the WC driver, I had to do the following. First create a silo and insert my process into it. Then somehow inform the driver my silo is representing a container. So it will create a union context for it and handle it accordingly. The first requirement is not that difficult to fulfill, because in order to create a silo, we first need to create a job using create job object A and give it a name. Then convert it to a silo using set information job object with the job object create silo flag. This flag is undocumented, by the way. And finally assign our process into this new silo. So overall, three API functions, pretty easy. So let's get started. The second requirement, however, is a bit trickier. To communicate with the mini-filter driver, the filter manager offers the FLT send message function. This function receives the buffer without a specific structure and simply passes it to a handler function within the driver for further processing. In order to build a valid structure the driver will accept, I had to reverse engineer the get message handler function in the driver's binary. So I've examined the function. And with this, I managed to map both the function and the driver. So I managed to map the structure that will register a silo as a container. It turns out that there are several functionalities the driver offers to its user mode clients with the FLT send message function. The one that interests us is the code one message, or the set union message. Its data will contain the following fields. The notable ones are the number of unions where each union is representing a volume the container will work with, the name of the driver's instance. Both of these are the number of units where each union is representing a volume the container will work with. The first is the repos tag and repos tag links and a handle to our silo. At the end of the struct there are two extra structures. Again, one for each volume the container will work with. The first one is the volume union structure. It has a GUID, stores a GUID, which is the same GUID found on the repos points buffer. A Boolean that will represent if this volume is the source volume or destination volume. And an offset to the second structure, which is a container root ID which will store the the volume. Container root ID is just a different name I found in the driver symbols to a simple Unicode string, so it's just a simple string. To simplify this, let's see an example of a valid structure the driver will accept and will register our silo as a container. Again, we have the set union message, a code one message. The internal buffer will have two volumes, the source and destination volume, and both of the WCI repos tags. A handle to our silo and both the source and target volume unions and source and target container root IDs. Something interesting to note here, I haven't seen any validation of whether the source and target volumes are the same. This means that, in theory, it is possible to get a container that will redirect to the same volume. Now, this is interesting and remember this because we will use this later. Okay. So, now that we have fulfilled both requirements of the pre-create function, our post-create function will be the same. So, now that we have fulfilled both requirements of the pre-create function, our post-create function will invoke, allowing us to analyze how the driver handles his repos points. The first tag we will look at is the link one tag and, as you might guess, this tag acts as a regular link between two files. It is usually placed on files that should be open with read primitives only, for example, system files. The driver will read the destination file from the repos point buffer and will redirect the call to the volume the container directs to using IO replace file object name. So, if I go here, this is the file that will be read, so it Let's see an example. Let's say we have an open process that is inside a container. This container redirects IO calls from the container's volume, hard disk volume five in our case, to the host file system, the host volume. The open process opens a file with three primitives and the WC driver will see the status on the IRP and will see the link one which it's responsible for. So it will read the content of the point from the MFT attribute and will extract the relative part of the destination file. Then it will open a handle to the destination file on the destination volume. Again, the host volume. And then it will return this handle to the notepad process. Note that in this case the notepad process got a handle to a file that is outside the container dynamic images inside on the host volume. Now this is safe because the handle that is given to the process is with the primitives only so we cannot change it and cannot affect the files outside of its container. The second tab we'll look at is far more interesting. According to the driver symbols the WCI tag is responsible for file and directory expansion. Expansion is this driver's definition to copy an open protection. This tag is placed on files that can be opened with write primitives and can be edited. When encountering the WCA1 tag, the driver saves the report data in the file's object context and launches a work item that further handles the request. This work item copies the file into the source volume, the container's ghost image, so it would edit a copy of the file instead of the original using the FLT read file and write file functions. So going back to the same example as before, this time the NOPA process opens a file with write primitives, meaning it wants to edit its content. The driver again will see the status reparse and WCA1 tag, get the relative path of the destination file from the reparse point itself. This time it will read the content of the destination file from the host volume, note that the file must exist, and then it will write this content to the container's volume, the original file that had the reparse point, and return this handle to the NOPA process. Now, this time the NOPA process got a handle to the file it was originally requested, but the file is not a ghost file anymore. It has the actual content of the file, the destination file, the file that it was directing to. So the NOPA process can edit it without affecting the original file on the host volume. Now I've stated that the destination file must exist. In the case it doesn't exist, the original file that contained the reparse point will be deleted. Let's see this in this example. It's in the driver's decompiled code. It tries to open the reparse point, and if it gets back status object name not found, meaning the destination file doesn't exist, it will then allocate a new IRP using FLT allocate callback data, and will set the major function code of this IRP to set information alongside the file disposition delete flag. This will cause the original file to be deleted, and it will invoke this new IRP using FLT perform synchronous IO. Again same example. Another process opens the file, we cite primitives. Wcfs gets the relative path of the destination file. And this time, when it leads to the content of the destination file, it gets back such the subject name, not found because the file doesn't exist. So it will delete the original file on the container's volume and will return to the NOPA process because the file was requested doesn't exist anymore, was deleted. Okay. Okay. by this driver is the ability to copy and paste files without having to bother with entering a container or dealing with repost points. This is used when files are needed to be transferred between the container and host volume. So let's say we have a file on the container's dynamic image and we want to save it on our host volume. You can use this functionality by the WCAFS driver to do so. The copy and paste operations are done using, again, the FLT read file and write file functions, the same function as the WCA1 repost tag. To do this, we again send a message to the driver using FLT send message, this time with message code 4. The content of the message data will contain the source and target file paths and source and target volumes. Pretty simple, nothing interesting here. Okay, let's summarize what we've learned from reverse engineer, the WCAFS driver. We managed to create a silo, insert a process into it, and register it as a fake container by communicating with the WCAFS driver. This allowed us to resolve how the driver handles its repost points. Using the link one tag, we were able to open one file and receive a handle of another. Using the WCA1 tag, we were able to override a file with the content of another or delete a file. And by directly communicating with the driver, we were also able to copy and paste files. So, let's see how what we've seen can be utilized. File system protection is an essential feature any EDR must provide, and security vendors tend to use their own minifilter drivers to monitor the system's IO activity. We'll soon see exactly how. In the previous section, we saw that the WCA driver uses FLT read file, write file, and performs the Conus IO to perform IO operations from within the kernel. Now there is something special about this function. Looking at the MS-DN documentation of all these three functions, we will see the following remark. The function will cause the request to be sent to the minifilter driver instances attached below the initiating instance and to the file system. The specified instance and And each group is associated with a certain type of minifilters. For example, 320,000 to 330,000 is the range for security vendors' drivers. And going back to where our WC driver is loaded, it is lower at 190,000. And can be even lower if we attach it manually. Which will cause EDR minifilters to simply not be notified about any of our I.O. operations done using these three functions from within the kernel. Let's visualize this by using the same example as the beginning of the talk. This time, the file that is being opened has the WCI1 repos point. So again, the I.O. manager will create an I.O.P for this operation, which will go down the storage device stack until it will reach the filter manager. The filter manager will then invoke the pre-create operation callbacks of all of its minifilter instances. Note that this includes any EDR minifilter. So it will receive a notification. This notification of our process attempting to open a file with a repos point, which we don't really care about. We'll soon see why. The I.O.P again will then be returned to the filter manager. The file will be read. And the EFS driver will change the status of this I.O.P to status repos. And the request will eventually reach the filter manager again, where it will invoke the post-create operation callbacks of the WC driver. In this function, the driver will notice the WCI1 tag, which it's responsible for. And as we've seen. It will either override the source file with the content of the target file, or it will delete the source file in the case the target file doesn't exist. These functions will cause a new I.O.P to be created and go down the same path, down the storage device stack until it will reach the filter manager again, the same as any I.O.P. This time, however, if we will go back to the remark from the previous slide, any instances attached above the originating instance, our WC driver, including any EDR minifilter instances, will not receive a notification about this operation. Now the I.O.P again will be sent back down, go back up, and when it will reach the filter manager again, the remark will apply to the post-create callbacks as well, causing the post-create of any instances attached above the WC driver to not be notified, including any EDR minifilters. Eventually the call will be returned to the WC driver. When it will return, FFT post-create. This will cause the entire operation to be completed, causing the file to be repulsed, and as we've seen, any internal operations done using the WC driver will not be notified, EDR minifilters will not be notified about it. Okay, so in order to know how this behavior can be abused, we first need to understand how security products implement their protection. To combat file system-based malware, these products leverage advanced algorithms. They analyze minifilter I.O.Logs, actively searching for specific patterns to detect and prevent any potential irreversible damage. Most EDRs rely on a standard set of principles to identify processes associated with a ransom error wipers. These principles include monitoring processes that open a significant number of file handles and exhibit behaviors, such as reading data from a file and overwriting it, making the original data inaccessible. Knowing this, let's look at some examples. Let's create an undetectable wiper and ransomware using the WC driver. We'll start with the wiper because it's a bit simpler. We first need to create an empty file that will be our target file, and then write a buffer of zeros or random data to it, it doesn't matter. Then we'll traverse each file on the system, and for each file, we'll set an I.O. WCR1 repos point that will point to the target file, the file that contains the zero dot buffer. Then we'll create a silo. Assign our current position. Press into it and register it as a fabricated container, where both the source and target volumes are the same one, the host volume. We will then traverse each file on the system again. This time we'll simply try to open the file using create file. Now as we've seen, this will cause the WC driver to read the content of the destination file and write it for us on the file that we want to wipe, without triggering any security minifilter driver's callback functions. Let's visualize this. Again if we try to directly wipe the file. EDR algorithms will eventually detect and prevent us. Instead, we'll set a WCR1 repos point that will point on the source file that we want to wipe, that will point to the target file that contains the zero dot buffer, and we will insert our wiper into a container. Then we'll try to open the file that we want to wipe. The WC driver will then read the content of the destination file and write it for us, thus wiping the file without the detection of EDRs. Okay. This is the end. The ransomware algorithm is pretty much the same. We first need to traverse each file on the system, and for each one we'll read its content and encrypt it in memory. We'll then create a target file and write the encrypted data to it. Note that security minifilters will be notified about the file write, but will ignore it, because the encrypted content is written to a new file on the system, and an existing file is not being overridden. Then we'll set again a WCR1 repos point. A WCR1 repos point that will point to the target file that contains the encrypted data. Create a silo, assign a process into it, and register it as a fake container. Traverse each file on the system again and try to open it using create file, causing the encrypted data to be written by the WC driver. Let's see this ransomware in real time. Here we have two folders containing the exact same files and has the same size on disk. We have the secrets folder and the secrets folder. We will first try to encrypt the first folder, the secrets folder, with a traditional ransomware that simply opens, encrypts, and writes the encrypted data to the same file. The ransomware starts to encrypt some of the files, but eventually getting detected and terminated by the EDR installed on the system. Looking at the secrets folder. The one that we wanted to encrypt. You can see that some of the files were encrypted, but not all of them. For example, this test file was not encrypted. So the ransomware was prevented on time. We'll then execute our ransomware that uses the WC driver, encrypting the second folder, the secrets copy folder. So it first sets a repos point on the files, then encrypting them using the WC driver, and as you can see, the process was not terminated, and looking at the secrets copy folder, you can see that all of the files are encrypted. Including our test file. Another notable feature of security vendors' products is the ability to block file system write operations. This can be utilized by organizations to either enforce a read-only policy for mobile devices, thus effectively mitigating the risk of data exfiltration, or to block file writes, to folders housing sensitive data, which will add an extra layer of protection against unauthorized modifications. This protection is implemented by, you guessed it, a minifilter driver, which we can bypass as well. Let's see another demo. This time we have a different EDR than before. A USB stick is connected to the system, and when trying to copy a file to the USB stick, the operation is being blocked by the EDR due to a read-only policy. But, when copying the file using the copy and paste feature of the WC driver, so we copy the file to the E volume, the USB stick, the operation is completed successfully, and the file can be extracted from the system, and this is how you can extract the most precious file of all, the calculator from the organization. Okay, ETW. Okay, ETW. ETW, or event tracing for Windows, is a powerful and efficient logging mechanism built into the Windows operating system. The Windows kernel serves as a crucial log provider that captures a wide range of system operations, including those related to the file system. Security vendors leverage these events to analyze and identify potential threats, often creating attack flows by cross-referencing. Now, if you recall the WCI1 tag override process, the read and write operations occur within a kernel work item. Executing from a work item, which is a kernel thread, will cause the ETW log to attribute these actions to the system process instead of the actual process responsible. This will lead to misinformation for any vendor consuming events number 15 and 16 of read and write from the kernel file provider, bypassing any threat-handing correlation based on these events. An example of a built-in Windows tool that is ETW-based is SACL. Windows often uses SACL as an example. It offers the capability to establish an auditing policy for file system objects known as System Access Controllers, or SACL. This allows for extensive logging of all IOP operations performed on the specified objects. ETW-based Windows tools are intentionally designed to disregard logs originating from the system. This approach guarantees that such logs, which are typically irrelevant to the user monitoring the system, are not included to avoid unnecessary overhead. The result of all of this is that the ETW-based Windows tools are designed to disregard logs The reason for this is that our IOP requests will be absent from the logs altogether. Move on to the demo. Next we have a secret file that has the read and execute auditing rule, which will log any access to the file, so we are looking at the auditing policy of this file. You can see that we have a read and execute rule on any user on the system. Opening this file with no-tips, for example, will cause a log to be generated by the auditing policy. Looking at the event trás and an index Ari flagship, you will see that the announcement editing logs, you can see that logs were created stating that the secret file were accessed by the no-pair process. However, if you will create an empty file that will store the content of the secret that we want to steal, then set the WCR1 tag that will point to the actual secret file and overwrite it using the WC driver, so we set the reverse point and then overwrite it. The kernel work item will read and write the file's content for us without any log being created by the auditing policy, so the content of the secret was stolen, but looking at the auditing policy, you can see that no logs were creating stating that the secret, the original secret was accessed. . Okay, let's summarize. We've learned that the Windows container frameworks, the Windows container framework provides a file system isolation solution that is implemented by repass points and mini-filter drivers. Through reverse engineering of the main driver of the framework, the WCFS driver, we were able to create a counterfeit container and successfully insert our process into it. This allowed us to leverage the framework's IO redirection mechanism. causing files to be overridden, deleted, or copied without the detection of security products. As a result, we have identified new bypasses of file system-based model protection, file write restrictions, and EDW-based correlations without relying on any other known bypass techniques. So how to mitigate the bypasses? There are numerous routes you can take. I've found that the easiest way is by detecting the suspicious activity in the user mode and not the kernel mode. For example, detecting the suspicious activity in the user mode and not the kernel mode, you are selecting calls to device IO control with the set service.5 system control code, along the side, the any of the , from what i've seen, this activity is an issue it should not happen under a normal container operation as it is pretty much an anomaly on which looks suspicious. You can also check whether the WC communication blog was open or the silo was bans on this the process, or to check if the WC driver is attached to volume or instant connection. The window is cleaned up, both of those are indigenous tools. We're now정이 of the Navy Dual JPEG during the environmental issues, systems, incidents and while the container's feature is turned off. Again, keep in mind, these are just a few ways to detect the bypasses. There are plenty of other methods you can take as well, for example, from the kernel itself. So these are just a few. Okay, further research. The WCFS driver is just one of many mini-filters out there. I'm sure that there are plenty of other mini-filters that can be abused in a similar way to perform system operations from the kernel itself. It is also possible to set repass points on directories. The driver symbols reference directory expansion and possible redirection handling, so this is an interesting route you can take. And most importantly, mini-filter and ETW-based protection is everywhere, and I'm sure that there are more bypasses that can be achieved using the methods shown today. You can contact me on... If you have any questions, you can contact me on Twitter or X or whatever it is now. Or you can scan the QR code to access the POC tool shown on the demos today. Thank you. Thank you. Thank you.