Welcome to the 12 o'clock talk on All Roads Lead to JKE. Without any ado, Billy, are you starting off? All right. Hi. Hi, I'm Billy, and thanks for coming. Here is our title. All routes lead to the GKE Google Google Nest Engine host. Hosts refer to the Google GKE node. Four ways to escape. We will use four different color vulnerability, including one day and new day we found to escape from the container to take over the node in GKE. Today's presentation will be presented by me, Billy, and my partner, Land Head. Before going to the topic, let's introduce ourselves. My name is Billy, a security researcher in the software industry. I'm a security researcher in the software industry. I'm a security researcher in the software industry. I'm a security researcher in the software industry. My partner, Land Head, also works in the Star Lab. We usually work together to do research on the nearest kernel and hypervisor, like VirtualBus, QEMU, Parallels for bounty and exploitation. We successfully found Ubuntu in the Pound to Own 2021 and 2022, and VirtualBus in the Pound to Own 2022. We also identified various vulnerability in land. We also identified various Wowr players atIOU-A 그� zich 的 The reason why we are now like CBE 2021 direction labeled 4491 is because it is the nearest kernel migration vulnerability IOU , but then in rotation from virtual what else is that equipment measurement to do the decision us of players whether we support them or not. We found that various performance measures are more cotton, and each outperforms the operation of 41211. Let's verify a couple minutes behavior can clean up here and then let me do a little project here. Good day. This CVE shows we have some goals in the news kernel and hypervisor. Let's start today's outline. We will give you some background information, introduce a bit about what is Google Container Optimized OS, and explain KCTF-VRB Challenger, a Google-bounded programming focused on container escape exploitation. And then we will start show our detail on our four submissions and the exploitation we used to transform a limited preview to escape from the container. And finally, do the conclusion. Google Container Optimized OS basically is a news OS but optimized for running Docker container, which is default node OS. It is based in a Google Kubernetes engine. No package manager. Unable to install third-party kernel module. We only root file system. Every time when it boot, it always verify file system trace on to make sure the file system is not malfunction or backdoor. Usually, the news kernel config are stored in the etc folder. In the Google Container Optimized OS, this folder is reliable. But stateless. Which means every time it boot, it will start from a clean slate. And config is set up as runtime. For attack, install a persistent backdoor is hard. If we want to escape from a container, there are some limits here. There are some obstacles when we explore attack surface. And use some common exploit technique in a container. It disallow unprojected user to use ePBF. And not implement the user-for-fd file system call. It disallow to access internal networking. So we are unable to access Kubernetes API server. Also limit on the device node access. Unable to spin TTY structure. And unable to use FUSE. OK, there are some attack surface we can use in a container. First, we can create a new user netmate to explore more attack surface. Like cgroup, a news kernel feature that limit account for and isolate the resource usage. Or the file system content functionality. Or package socket. Or traffic control configuration. In IOU link is a news kernel system call interface for synchronous IOU operation. And there are some cp in this subsystem before. And it is reachable in a container. We see it is a great attack surface. Let's see what KCTFVRP environment architecture look like. Basically it is a GKE. Clouder have multiple node. A node you can see it is a VN. And each node has multiple part. A part you can see is a container. In a part it will launch a NSGL challenger. KCTF have two challenger. They are in different part. Your goal is to gather two flag. One flag is called KCTF flag. It's placed on the same part. When you connecting to the... And this challenger will give you a interactive facial. And another flag is called 4-chain flag. It's placed on the other part. The node OS and the KCTF version are upgrade automatically. But there is a time gap between the vulnerability fix and upgrade stream. So it's alone to use one day vulnerability. And we introduced before. All nodes are running on the container optimized OS. Public network access only. So you can download the exploit from the internet and launch. There are two attack scenario. One is breaking out of the NSGL same part. And gather flag on the same part. Another attack scenario is to... Spread the isolation that Kubernetes provide. And access the flag from the other challenger. But with the kernel vulnerability, we can directly escape privilege and take over the whole node. So you can gather all flag from a single kernel vulnerability. And next, let's show our four submissions. And left hand to show the first two submissions. Hello, everyone. I'm Ramdan. I will talk about our first submission. Our first submission is using CVE-2020-154. This bug is we used to exploit KCTF container. This bug is used after free bug in C group one function in C group kernel subsystem. So basically, this bug is caused by the lack of check in this function. There are source key that always need value that has string type. But before this page file, there's no check if the value is string or not. So we can pass a value with another type. So using the fsconfig syscall, we pass a value with the file type. So we pass a file descriptor as a value. Then param string will contain file strap address. Then it will start in fc source. We can reach this bug by calling fsconfig syscall. But by set the key as source. And we set the value with the file descriptor using fsconfig setfd. So the put fs context will... If we close the file descriptor, the put fs context will call and it will free the fc source that contain the file strap. But the file descriptor that associated with the free file strap still exist in the process. So we have use after free and in file strap object. So using this bug, we can create a bunch fc context pointing to the same strap file. So there are multiple fc source pointing to the same strap file. So if we close one of them, we have use after free. And we can close one another to have double free. Okay. So... We can free another fc context that associated with it. Okay. We can perform double free because we have the fc... The fs context still pointed to the free. So we can free stock file. So... So... So how to exploit this bug? We need to build our exploit strategy. In use after free scenario, usually we override some object with another type of object. But we have another problem to exploit. To exploiting this bug. Because the file strap is... First problem is the file strap is allocated in our slap cache. So we cannot override with another type of object in a common way. And the second problem, what type of object that we can use to extend the limit primitive of this bug. So... I'll present you the solution of the first problem. To override some object with another type of object that reside in different slap cache, we use the technique that called cross-cache technique. This is... The known technique to exploit use after free in Linux kernel. So there's two type of cross-cache technique. The first is cross-cache use after free. And the second one is cross-cache help overflow. So... So in cross-cache use after free, first we make slap pack free to body allocator. So first we fill the whole slap pack with some object. So for example there's slap of object A. So we allocate all of object A to fill the whole slap pack. We free... Then we free them all. So... So the slap pack will go to the body or pack allocator. Then we spray another type of object and it will... It will coming from the pack allocator and now the all free slap cache of object A owned by object X slap cache. So... Using this technique, we can solve the first problem. But I will... I will present to you the... The another cross-cache that called the cross-cache help overflow. In cross-cache help overflow, we make target slap cache adjacent next to another slap cache, the object that we want. So... For example the... In the right side image, we make the... We can override the object A from the object X. But they are into different slap cache. But it requires the sprayable object for both the object X and object A. But... Don't expect it will be reliable most of the time. Because... We... We also need to... To trigger the... The... The hop... Overflow bug multiple times. And we can also make it more reliable if we have the capability to gain some information if we already override the object or not. So... We... I already present you the first problem. Is that we can... We can actually to override the... The file struck in their own slap cache with another of object using the cross cache. And... For the second problem, what type of object that we can use to extend limited primitive. So... For the second problem, I assume some of you already know the... This is the MSG, MSG object. And MSG, MSG Sage. This is the crown object for exploiting Linux kernel bugs. For the future, I will refer the MSG underscore... MSG as MSG object. And MSG underscore... MSG Sage as MSG Sage object. So... This is the... Properties of MSG object. So... This is the object that... Already known for exploiting Linux kernel bug because of... Especially the heap... The heap bugs. Because of... Its flexibility of its size. And this is the... The sprayable object. And there are the second object... Called the MSG Sage. It's used to store extra message. If the... If the total... If the total of the message... Followed by the header... With the size of hex 30. So if the total... Object of the message is... More than 4K... The MSG Sage object... Will... Will use... And it pointed by the next pointer of the MSG object. So... This... This is the... The MSG object is a powerful object that can be used for many things. For example... Arbitrary write... Arbitrary free... Arbitrary read... We will go... Go deeper with some of this. And... And it's accessible via... MSG gate... For... For allocating... MSG queue... And... The MSG send function... To insert the message to the queue... And MSG receive to... Receive the message from the queue. And... So... There are... To... To allocate... The MSG... We can using the MSG gate... This is the... In the right side image... There's a definition of... Some of function... Interface that... We can use... So... As you can see... The... There are the... MSGP... Parameter... In the... Function... Interface... But it's... Actually defined as... The... MSG... And... The... MSG type... Is... Used for... Selection process... When we... Receive... Message from the queue... And... The... MSG flag... There are some useful... The flag... That we can use... For example... The... MSG... Underscore... Copy... This flag is... We can use... If we want to receive... The message from the queue... With... Without... Deleting them... Uh... From the queue... So... Uh... There's... Uh... The image... That... Uh... That shows... Uh... How... The... Message... Linked... Uh... To the queue... For example... We allocate... The... The message queue... Uh... And then... We... We send... The... The message... With the... M type one... And... And... The... And... With the... The... 4K... Bytes... Of message... We send... To the queue... And... And... We have... The... First message... Insert to the MSG queue... And... There are the MSG sets... Because... Of... Uh... The total... Uh... The total of... A counter... The size... Of MSG object... With the header... Is more than 4K... And... If we try... To add... Another... MSG... Object... With the... M type... Two... And... With the C... Of... Hex 200... Uh... It will be linked... To the queue... And... If we want... To retrieve... The... The second MSG... Uh... We can use the... MSG exit... With the... M type one... Uh... It will search... The message... That... Doesn't have... M type one... So... The... The... The second... The... The second message... Will retrieve... And print it... To the... To the terminal... So... I'm gonna explain... The... The tricks... That... We can use... Using the... MSG object... So... Suppose... Uh... Using the... The... Double free bug... We have before... Uh... In the... In the... Our first... Submission... CVE... We can... Convert this... To... To have use... After free... On MSG... And... After we have use... After... After free... On MSG... We override them... Using the... MSG... Search object... So... The first trick... Is... Uh... We can use... The... MSG object... Is... Have... Uh... Out of bounds read... We can override the... M... MTS... Uh... MTS is... Magic text size... If we... Override this... To large value... We can... Get the... Address leak... From the... Out of... Of... Out of bounds buffer... Via... MSG Chief... For... The second trick... Is... We can... Do... Arbitrary read... Uh... Uh... We can do this by... Uh... By controlling the next pointer... To the... To the... Address... Of that... We wanna leak... Because... When we... When we... Receive the message... And... We set the message text size... Uh... To the 4K... It will... Search... To the... Next pointer... Because... The... The... The... The kernel assumes that... The... This message is... Have more than... 4K size... It will... Go... It will copy... From the next pointer... To the... To the user... When we do... MSG Chief... Uh... Call... And... So... The third trick... Is... We can... Do... By using this... MSG object... Is arbitrary free... We can... Do this... If we have... Some specific... Uh... As kernel configuration... To... To get... Arbitrary free... Uh... We can see... If we... Receive the message... It will call... Do... Uh... MSG... Receive function... And... In... In the end... We... We have call to... Free MSG... And... Then... It will reach the security... MSG... MSG free... Uh... As you can see... It will... Call the... K free... On MSG security... Field... We... We... We have... Uh... Arbitrary free... But... Uh... Before we reach the free MSG... Uh... We... Uh... Encounter... The listdel... Uh... Listdel function... And... Need to take care of... Next and previous value... Uh... We... Actually... Can control them... Using the... MSG... Search object... But... But... We... Can... But the... MSG... Search object... Can... Can't control the first 8 byte... The... The first 8 byte... Uh... So... So... We... We... If we... Reach the... Listdel... Uh... Using... Using... Uh... The... MSG... Search... Uh... The kernel will crash... Because we cannot control the next pointer... So... In the M list... But... We can bypass them... Uh... If the... If the kernel have... The... Config debug list... Enabled... So... The... If we... Uh... Have this... In a... This config enabled... It will call the... Listdel... Entry... Valid... Uh... We... Can... Easily... Make it to... Return... Files... And... Skip... The... Listdel... By... By... By... Set... The... Proof... The... Proof value with the... List poison 2... This is... Uh... The sum constant defined in the kernel source... And... Fortunately... KSETF... Kernel config... Uh... Have that field enabled... So... We can... Use... The... Security... And... The config debug list enabled... To have arbitrary free using the... MSG object... So... To... Re-wrap... The... Our... Three... Tricks... Uh... We can combine this to... The... Kernel... Re-control... So... For arbitrary free... Uh... We can free the... Pipe buffer object in... Camelot... 1K... Cage... Uh... This free buffer... We can allocate... Using the... Pipes... Uh... So... We... We use... Uh... Our first two tricks... Uh... To do... Out of bounds read... And arbitrary read... To leak... The... The address of pipe buffer in the heap... And... We read the... The... The... Ops pointer... Uh... In the pipe buffer to... Get the kernel... Text address... And... By... Closing pipes... Uh... And controlling the... Ops... Uh... Field... We can control the... Kernel... Kernel... RIP pointer... So... We go back to the... Our first submission... When... When we have... Uh... Use after free... And... In struct file... And... We develop our... Express strategy... So... With the... For... The... With the first problem... And... The solution is... We can... Overwrite the struct file... With another type of object... Uh... We overwrite them using the... MSG object... Using... Our sketch technique... And... We free... One... Of... FC context... That's still pointed to... The MSG object... So... We have use after free... On the MSG object... So... We can reallocate... Them using the... MSG... Search object... So... We can... Fake the... MSG object... Using the... MSG... Search object... Then... Using... Using this... Condition... We can perform... Uh... Uh... Like... Our... To... Our tricks... Before... Auto... Open... Street... And arbitrary read... So... First... Uh... We need to lake the... The... The address of... Pipe buffer... In... Camelot... 1k... So... We need... Uh... Another of... MSG object... That... That adjacent... Or near... Uh... Near... From the... MSG... Search object... We send... We allocate... Another of... MSG object... Uh... In Camelot... 1k... And... The adjacent... MSG... Uh... Will have... Next pointer... Pointed to... Camelot... 1k... And... We can do... Auto... Open... Street... And... Lake the... Address of... Camelot... 1k... For free... The... MSG... Uh... MSG... Uh... That we... Talk before... The address... The MSG that allocated... To the... Camelot... 1k... We... We... We allocate the... Spray the... Pipe buffer... Uh... The pipe... The... All the... MSG... In the... Camelot... 1k... Will... Will replace it by the pipe... Buffer... So... From the... Address... Like... Before... From the... Auto... Street... We... We know the location of the pipe... Buffer... Then... We... Perform the arbitrary read... On the pipe buffer... Uh... To know... The value of the ops pointer... To... Get the... Kernel... Text... Address... And... Then... We do... Arbitrary free... Using our... Arbitrary free... Tricks... To... We... We... Control the... Security... Pointed to the pipe buffer... And... We know... Now... We have used... After free... On the pipe buffer... So... We just... Reallocate them... Uh... With the MSG... Search object... We have fake pipe buffer... And control over the... Pipe... Buffer content... And... We close the... Pipe... Pipe buffer... It will... Reach the pipe... Buffer release function... With our... Fake pipe buffer... And... We have... Kernel... Control... Control... Kernel... Repointer... So... After we have the... Kernel... Repointer... We... We need to convert this to... Rope chain... We... Can easily find the... Stack... Pivot... Pivot... Gadget... To convert this to... Rope chain... Execution... Uh... Using the... Commit... Kernel... And... Prepare... Kernel... We can... Install the... To the current process... Before... Uh... Our... Rope chain is... Execute... Uh... We have prepared the child process... Uh... The child process... Uh... Will... There's... That... If we... Already go through... It will break... If we already go through... And then... After... We install the... Kernel... The... The root... Credentials... We copy the... Credentials... To the... Child... Process... And... We switch the... Namespace... Of... PID1... To the... Init... Process... Using... The... Switch... Text... Namespaces... Function... We call this... Using our... Rope chain... And... Uh... The... Child... Process... That we... Prepared before... Will break the while loop... And... It will... Uh... Set the namespace... To the... Namespace... In PID1... And... We cat... We cat the flag... And... Spawn the root shell... So... In this demo... We... Connect to... So... In this demo... We... Connect to... In this instance... We download... And... We download our exploit... To the internet... And run the exploit... And... KCTF flag... Will be printed... And... Spawn the root shell... So... Now... I will talk about... Our second submission... For... The second submission... We use... The... Uh... CVE... 2021... Uh... 2 to... 600... To exploit the KCTF... This is double free bug... In packet settering function... In the... IF packet kernel subsystem... So... This is the snippet code... To trigger... Double free... Uh... I will explain... This bug code... One by one... The... Root cause... Of this bug... Is... Uh... Happen... Happens... When we switch... Version... From the version 3... To 2... But... Uh... There's some field... Big... Because of the bug... There's some field... That didn't clear... If we... If we... If we change... The... Version 3... To 2... So... There's some old state... That didn't clear... So... First... We... Create... The... IF packet... Socket... Uh... We set the version... To the version 3... By calling... Setsockof... And... The... TP version... Variable... Uh... Will... Assign... To the... To the... TP version... And... We call... The... Setsockof... With... With... Packet... Set... Packet... Rxring... Option... It will... Allocate... A variable... Called... pgfake... Uh... As you can see... Uh... We can control... The size... Of allocation... By setting the... TP block... And... R... So... The... Allocated... The... Pg... Pgfake... Uh... Is stored in... Pq... Bdq... Uh... But... It overlap... With the... Rx owner map... Because... Of... The... Union... Data type... And then... From the third... Setsockof... We free the... Pgfake... But... It didn't... But... It didn't clear the... Pq... Pkbdq... Now... We change the... Version to the... Version 2... By performing... Setsockof... Setsockof... Again... And... When we call the... Packet setring... Uh... Rx owner map... Uh... Uh... That... Actually... Coming from the old set... Is freed... So... We freed the... Pgfake... Of... Pkbdq... Toys... So... The Rx owner map is... Overlapped... Because... It's overlapped with the... Pkbdq... And... It didn't clear... Uh... From the version 3... And... In the version 2... It will be freed... So... We have double free... So... We can control the... Use... In this bug... We can control the target chunk size... And... We can... Control when to double free... So... We choose the... Timer... Fd object... As... A sprayable object... Uh... To fill the whole... Slab pack... Uh... Using... Cross sketch technique... Uh... We free all... Timer... Fd... And... Pgfake... Uh... To make slab pack... Instead... To insert... To the pack allocator... Uh... And then... Uh... We spray... With the msg... Object... And... Using the double free... Uh... We free the msg object... And... Override them... Use... With the msg... Such object... So... We can control... Over the msg... Object... Uh... In this... Condition... We can control... Uh... We have... Uh... Same case... As the... Previous... Submission... Our previous exploit... That we have... Just after free... On msg... Uh... And... Reallocate them... Using the msg... Such... We can just do... The same technique... First... We do... Out of bounds read... To like... The address of... Pipe buffer... On... Camelot... One... K... And... We do arbitrary read... On the... Pipe buffer... We override the... And then... We override the... Pipe buffer... Using the msg... Such... Then... We can... We close the pipes... And... Control... Ternality pointer... And... We can... Escape the container... So... This is the demo... Of... Our second... Submission... We... We... Download... We connect the... Kctf... And... Run our exploit... The... To... Download... The... KTK... Our... Web... So... Let... You... Join... All... The... In... The... Game... So... Let... Let us... To... To... To... You... To... One is CBE-2022 and Nero-185, it's the first time we use Nero-Day, but unfortunately we didn't properly report to the news security team after the KCTV submission. There is a description of the CBE. We highlight the keyword we care about, heat-based buffer overflow. Vulnerability is placed at the file system content subsystem, and in case of a privileged user namespace, we can trigger the vulnerability. Briefly speaking, there is an integer underflow when boundary tracking. So if we let the size larger than the page size minus two, it will cause integer underflow, and in the following four places will cause heap overflow on the legacy. We carefully append data in the legacy data buffer and cause heap overflow with one little byte. We treat this vulnerability as off-by-one. From the core legacy data allocate, it is in the KMLK 4K. So in summary, this vulnerability is off-by-one on the KMLK 4K cache. We use core cache heap overflow in order to override the messenger structure. Spring legacy data buffer with messenger structure at the same time. Trigger off-by-one vulnerability on every legacy data, and it will have a lot of chance to override the messenger disk next. Let's buy with zero. If we are lucky to override messenger next, let's buy with zero. We are. We can make the two messengers point to the same next. Free is for the one messenger queue, and another messenger queue will still point to the free chunk. At this point, we have made the same situation as previous submission. Do all OOV leak pipe buffer address, and do arbitrary to leak kernel test address. Next is to do arbitrary free on pipe buffer and override with the messenger segment. Close pipe to control the kernel IOP for escape container. There is our demo. We connecting to the challenger and get the exploit from the internal and launch. We successfully get the two phase and take over the node. There is a second little toy we use in the kctf. By this time, we properly report to the next kernel security and get the credit. Here is a description of the CVE. We also highlight the keyword we care about. He is a key collaboration in the IOU link. He is reachable in the container. This vulnerability is in the IOU link when request the same messenger or receive messenger operation. He will grab every structure reference before do operation. But if IOU link content has IOU link CR IOP, he forgot to release every structure when finish the operation. So if we key increase the users, he will cause integer overflow, and then the users reach zero again. When every structure reach zero again, we use another IOU link without IOU link CR IOP. It will free the every structure after finish the request. So we can have a free every structure as current in use. The key point about code cache is allocate a lot and free all at the same time. So we allocate a lot. We allocate a lot of all the structure and free all. Spring KMLK 128 messenger segment to reallocate current FH as messenger segment. And then free the current FH again by IOU link and code cache again with KMLK 64 messenger segment. Sign-in is still messenger segment. So we use messenger receive system code to read the next messenger to get the kernel address in KMLK 1024. Once we have read the read heap address, we can modify contents by free and allocate again. We can use it to fork the FH structure to do arbitrary read in order to read kernel address. With our FH FH structure, we are able to do arbitrary read by get CWD. System call. After we get the kernel address, we can prepare ROP payload and fork FH structure again. By system call or fchange directly, we can control the kernel RIP from fchange directly to set FH PwD to pass pool to D pool and in the end to the return entry. In the return D entry, we can control kernel RIP when code function pointer does not work. There is a demo. Original demo video need about 15 minutes to finish, so I speed up for the presentation. Again, connecting to the challenger and get exploit from the internal and launch. With the server get a two-frame and take over the GKE node. All vulnerability we use have already fit in a GKE. You can take a reference to the Google security bulletin GCP 2022.0.0.2 and GCP 2022.0.1.6. We believe with QuarkX attack with messenger, we can transform a limited procedure to arbitrary free to escape container. But there is a limiting, we can only allocate site under 4096. In our last CPE, in our last submission, we are able to cover UF to the common situation and preview G3 CPEs. But with some change, we still can escape container by QuarkX and messenger. As a result, we are award more than 100,000 bug bounty from the Google KCT VRB program. Thanks to Google for holding such a great VIP program to secure the KCT and GKE infrastructure. Here is our refund. And thank you.