All right, we're back live. Thank you, Liron, for your support and supporting DEF CON and the Red Team Village. It's a pleasure to have you here. So without further ado, take it away. All right, thank you. So thank you all for coming to this talk. It's my first time presenting at DEF CON. I think last year I got caught up in the DEF CON Blue Team Village trying to win that CTF from OpenSOC. It was pretty dope. But I think this time today, especially, we're going to talk about something more Red Team-ish. Why? Because I'm on the Red Team, and it just seems appropriate. So this is the talk, y'all trying to bypass Python 3.8 autohooks or nah? So who am I? My name is Liron Gray. I go by DaddyCocoman in most areas. You can check out my website, DaddyCocoman.dev. I haven't updated it in a while, but there's some cool stuff there, I think. You can also check out my GitHub. I have a couple of projects I've been working on, and hopefully I might have some more interesting projects up there very shortly. I was in the Navy for about ten years. I used to do electronics. I used to work on radar and communication systems, and I moved over into the cyber field. And I'm on the Microsoft Azure Red Team now. So you may hear some Azure-related things if you follow me on Twitter or wherever you may follow me. Hopefully it's only on Twitter and not anywhere else like real life. I also love Python and Pythonic things. So we'll talk about some other Pythonic things later in this talk that I really got into at a certain point in time. Because it was... it's pretty cool. I like the way Python is written as a language, more so than statically compiled languages. And I'm also a Nerdcore rapper. I go by OhMy. It's an I, not a one. You can find my music at MCOhMy.com. I'm actually on the DEF CON 28 tape. I got that hot track. I got that heat. Xs and Ram. If you want to listen to that, please do. And hopefully you enjoy all the music. So let's talk about the agenda. So first let's talk about why we're using Python 3. Then we'll talk about the two peps that have influenced audit hooks for Python 3.8. We'll talk about how to implement them, how to bypass them for Windows, how to bypass them for Linux, and how to bypass Python. And bypass Python is very... it's in quotes for a reason. But we'll talk about some interesting concepts that came up over the last year. So quick question. Why are you still using Python 2? I want you to look every single one of these people in the face and convince them that they have the time to listen to you explain why you're still using Python 2. I'll wait. All right. So you might have a legitimate reason, right? Sometimes there are tools out there that haven't been updated in the last 10 years or whatever. And if you are using something that is written in Python 2, I just suggest making a pull request, helping them move up to Python 3, and try not to break that backward compatibility if you really need it. So here's some Python 3 changes that have happened in the last couple of versions. Python 3.5, we had the async, await keywords that came out. Python 3.6, we had format strings. So you could say things like, hey, my name is Daddy Coco Man. And then you could print my name is in Curly Brace's name, and you would get my name is Daddy Coco Man. We also had ordered dictionaries. So now Python recognizes the insertion order for dictionaries, and you can always keep track of it. Previously, dictionaries would just be a random result every time you would try to print them or call them. But now they're ordered, which is great. In Python 3.7, async and await actually became reserved keywords. We had the breakpoint functions, so we could enter debuggers in the middle of our scripts. And we had the data class decorators, so we could make these struct-like classes. So you can say, here are these fields that a class is supposed to have, and here are some functions that it may want to implement, just like classes, but a little bit better, in my opinion. They serve different purposes. So for Python 3.8, we have this controversial thing called the walrus operator, where you can assign values while evaluating an expression. So on something like this, we can say, it reads, if the length of a is greater than 10, then we can assign it to n, and then be moving to the loop logic for print, list is too long, blah, blah, blah. But if length of a was not equal to 10, then we wouldn't assign it to n, and we would just move on. So it's nice, it saves you a line of code. A lot of Python purists really don't like it, and that's okay. People generally don't like changes to languages that they've been using for so long, but oh, well. So next, we have the async IO enabled REPL, so you can do Python dash M, async IO, and you can start using those await and async keywords inside of your REPL. And what we're here to talk about primarily are these audit hooks. So audit hooks are runtime events, are hooking those runtime events to perform some action, some arbitrary action, whatever you want to do. So the first pep we'll talk about explains how the audit hooks are supposed to be implemented, or how they're supposed to work, why you should have them. So the proposal was that sysadmins could use audit hooks in order to add some feature to their Python executable. So some suggestions that came in were integrating AMSI, script block for Windows, and then for Linux, you could send events to syslog or audit D. And if you... I meant to link it in here. I guess I didn't. I must have removed it by accident. There is a GitHub, GitHub.com slash ZOOBA, Z-O-O-B-A, from Steve Dower, who has a bunch of examples of what these audit hooks can look like in different implementations. So I would definitely check that out. Another thing you can do is restrict the importable modules. So you can say, well, maybe I need my people who are using Python in my environment to use the request module, but maybe for whatever reason, they don't need to use the mmap module. So you can say, well, I want them to use specific modules that are only for my environment, and I don't want them to use anything else. You can also do things like remove command line arguments. So before I mentioned with async IO or just a lot of other command line Python modules like HTTP server, you can actually remove that dash M or the dash C to prevent people from using those switches. But one thing to keep in mind is that the PEP actually reads that you should detect the code over preventing the code. And the reason for that is because sometimes you may use the Python dependency tree graph is out of control sometimes, right? You may try to use the NumPy module, and underneath it, it uses C types. So if you try to block C types completely because you didn't want someone importing it, you may actually mess up using NumPy. So it's important to just detect what's occurring rather than prevent what's occurring because you might mess up something that you had no idea was going to go wrong. Another thing that the PEP recommends you do is implement OS security level features to protect your modified Python. So that means device guard for Windows, maybe do some exe signing, use catalog files, that kind of stuff, and extended attributes for Unix. So maybe you want to add file attributes that have the hashes of the permitted Python files that are allowed to be used. PEP 578 says here's how we're going to implement or how you can implement these audit hooks. And they can be implemented inside the executable or inside of the script. So if you want to do it in C, you can actually recompile a new executable, a new Python.exe, and all the hooks that you apply to that code will be run by this new exe. Or if you want to just do it in Python inside of your script, you can, without compiling it, you can use the sys module, so import sys, and then sys.addAuditHook, and then the audit hook will be a function where you can handle all of your audit hook logic the same way that you can do it in C, except it will only apply to the script where it's implemented. So here's two examples. On the left side, we have the Python implementation of network prompt hook, right? We have, this is some bad Python, so ignore this, but we have an event that gets passed into the function, and then we're looking at this event and we're saying, does it equal socket.getAddressInfo? If so, I want to take some action. I want to warn the user that we're attempting to resolve some URL. If it's socket.connect, we're saying we're attempting to connect to this URL. Do you want to continue? On the right side, we have the C version, which you can use to compile a new Python executable. Notice it's considerably longer, and personally, I never really learned C and I don't really plan on writing C any time soon, so good luck to you if that's your thing. So here's a list of audit events that are currently implemented in Python 3.8. You can actually pull this from the documentation itself, but what I have highlighted are some of the things that you may really be interested in if you see these events occurring. Now, keep in mind that you have to modify this for your particular environment, but some things that you may find interesting are when if a script uses the compile function or the exec function or maybe some of these winreg functions on the right, things that you may not normally see in your environment, you have to be able to adjust your audit hooks appropriately. Pty.spawn, I don't know a single case of anyone using pty.spawn outside of trying to get a shell after they've already compromised your system. So things like that may be of interest to you, and you may want to capture those events as they occur. But again, the focus is on detection and not prevention. So we wanted to make sure that we are aware of these things that are happening, but we don't want to actually prevent them from happening unless you have a really good use case or use scenario to not let a particular event happen. I mean, you can totally do. You can block all events if you want to. It's not like a fake Python just sitting there. I don't know, honeypot Python or whatever. So bypassing audit hooks for Windows. So first we'll talk about how auditing works. So when an action is or when an audit hook is signaled, Python determines whether the action matches an audit rule. And that's kind of maybe not the best explanation. What happens is it will run through every single audit event or audit hook that you have, but you have to write your audit hooks to only care about the events that you want. So for example, in this network prompt hook, we only care about socket events. If it doesn't equal socket, then we're going to return because we don't want the rest of the hook to go through all that logic. So when you write your audit hooks, you have to make sure that you only process the events that you care about. Also, you probably shouldn't do anything like a while loop, some unbreakable condition, which is not a good idea. So the other thing to know is that all audit hooks are processed for all events. So even if I open a file, which is an audit hook event open, the open function, it's still going to get passed to this network prompt hook, but it won't do anything because I set a condition to only care about socket. So what's in the source code to figure out how these things work? So on the left side, we have the PySys add audit hook function. And what it does when it adds an audit hook, it literally just adds the hook to the end of a linked list. When the hook is called in the middle or the upper right, it will actually just go through each hook that has been defined and just goes through all those functions and determines whether or not it needs to do anything. Or based on... it will... you decide whether or not it needs to do anything, but it will just run all those functions at once. In order, I should say. On the right side, we also see that tracing is disallowed. Now, what I originally thought this meant was that I could not see into what an audit hook looks like. But it turns out that that tracing set to false is only for the Python debugger. So if you're using the Python debugger, for whatever reason, maybe in your IDE or maybe on the command line, you actually can't trace into the audit hooks. But if you're using an external debugger, so let's say maybe you grabbed a copy of a modified Python off of a system and you took it back to your station and you opened up a debugger, you can actually see what's going on. So here's an example, right? The same example where we had the socket.getaddress info. I can actually look into this audit hook to determine what it's doing. I'm not the greatest at assembly, and so I know what this says simply because I know what the code says. But you can still get a general idea. You can see the comparison strings, the compare, spython.string compare, and you can see that it's looking for certain audit hooks to occur before moving into some other logic. And you can also see the strings there attempt to resolve the URL that it tried to resolve. So here are some ideas for bypassing. By implementation, audit hooks can't be cleared once they're set. So there is a function, I think, to clear audit hooks, I think, but it doesn't actually work. You also can't overwrite. You can't, like, reload the sys module in order to clear all the audit hooks either. So one thing we could do is maybe overwrite the head of the linked list to point to null, but it might be possible. And attempting to locate the start of a linked list in memory might be a little bit difficult. But I'm actually really new to exploit development, so it might be possible. So if someone knows something I don't, please tell me, and maybe we'll figure that out. But in the meantime, maybe we can just overwrite the bytes in memory that calls the hook functions. And that smells like PowerShell, right? Everybody knows PowerShell and AMSI bypassing, and we can actually apply the same concept here. So there's plenty of resources to draw inspiration from. Adam Chester over at MDSec has plenty of articles about bypassing AMSI, so does RastaMouse, and there's a lot of inspiration just to draw just by reading their blogs. And it's also significantly easier to determine the address that you need to overwrite with these new bytes that you want to overwrite with. So let's go that way. So if I take the Python 3.8 DLL, and I open it up, or if I run Python, rather, this modified Python, and I go check out the Python 3.8 DLL, I can go find the address for PySysAudit, and we can see this is the 32-bit version. We can see the address and set a breakpoint on it. If I scroll further down, I'll get to a point where a function is called, and under call EAX, it has the name of the hook that is being called, right? So if I were to step into that call, I would actually see the logic that I showed you earlier. So the offending, I say offending, but the interesting call that we want to overwrite is where EIP is currently pointing, and that is call EAX for 32-bit, and call keyword pointer for 64-bit. So, again, it's very similar to PowerShell. We need to load the Python 3.8 DLL, we need to find the git address for PySysAudit, we need to change the memory permissions, and we need to just overwrite those bytes. And I learned this week, a couple of days ago, that I really need to go through all these versions of Python and grab their offsets. So I went through the 32 and 64-bit versions of Python, from 3.8 to 3.85, and even just for poops and laughter, I went to 3.9.0 Beta 5, just to see if it still worked, and it does. So we just have to set these offsets, and we can change the memory permissions and overwrite those offsets. Let's talk about Linux real quick. So I have never done any sort of, like, major Linux exploitation other than, like, a buffer overflow, right, that you find in, like, CTFs. I've never done any patching, so I didn't fully understand at the time how the Python packages worked on Linux. So there are no DLLs, right, because it's not Windows, and there was no sysmodule.so library that I thought was going to be there, and it just wasn't. So it's like the same, though, right? You can find the PySysAudit address by searching the Python executable itself. So I guess it's statically compiled into the binary, and you can just pull it out. So if you use the readelf, we can use readelf to figure out what the address of PySysAudit is, and we're going to cheat a bit, because I actually don't know how to parse ELF files. So in my code, I just subprocess readelf and grab the output, because I actually have no idea how to do it otherwise. And then we have to locate this address in memory. So once we grab that address here, it's 53007 foxtrot 0. We need to figure out where it's located and what region of memory that it's mapped to. And in this picture, we can see that it's under... it falls in the range of F230 and 6AE. So we need to modify those permissions. And we can do that, of course, with libc and mprotect. So we need the write execute permissions to change and to read write execute permissions. And sometimes, yes, sometimes it just be like that. I am watching the Discord, by the way. So what this looks like is we load the DLL for libc. Or we load libc, rather, I should say. And then we find the readelf. We find the address of pysis audit. We find the location of the memory mapped files. And then we determine in what region it falls under. And then we change the permissions. And then we just overwrite the the bytes in memory with these nops. So one thing I didn't go too far in depth in Linux is because every build for Python in Linux is different. So it's... I don't have the time of the day. Like, I'm stuck at home. I only go out to the supermarket if I need to. But I still don't have the time in the day to go through all these Linux versions and try to find these offsets. So later today I'll probably put this code up on GitHub. And if you have a version of Linux where you want to bypass audit hooks for 3.8, then you can send a pull request and I will happily accept once it's been tested. Let's talk about detection. While everything that I've mentioned works, everything up until how... up until you patch the binaries... the bytes, rather, can be detected. So importing C types can be detected, importing platform, importing sys. Every single function call that you make can be detected until you hit that men move and you overwrite the bytes. The way to defeat this is ironically better audit hook logging. Because if you can detect that these things are occurring and you send them off and you ship them to your seam or however you... your Windows event logs or however you collect your logs and you see that these are occurring, well, then you have a better chance of understanding what's going on with your Python executable on a random given station. So, yes, it's sort of like... it's like running AMSI bypass in PowerShell where you might get caught. It's the same thing. So that is something you should probably implement better audit hooks. And it might be harder for someone who doesn't program in C to implement, right? I don't think most sys admins have a lot of C knowledge. I could be wrong. I could be naive. I've never been a sys admin. I have no idea. But it's probably not something that you come across in the general resume where you need 10 years of Adobe XD that came out last year. So it would be great if somebody came up with a template of audit options and an action to help sys admins compile those custom Python executables. But as I mentioned, I don't have the time. So it's not me. So I want to give a demo real quick. Hopefully we can all still see the screen that I have up. I think it should be working. A Discord confirmation would be great if you can see this terminal. We can see it. Or a confirmation from Omar. Yeah, we can see it. Thank you. Great. So I have this executable here. This sPython that's been compiled. And I'm just going to enter it directly. And I'm going to try to run this code. And all I want to do is get the status code of the Red Team Village.io. I just want to make sure it's up. Because Red Team Village has been doing a great job. And I want to make sure that their website is up. But if I do that, I get this warning saying you're attempting to resolve Red Team Village.io. Do you want to continue? Yes, I do want to continue. Yes, I do want to handle the socket event. Yes, I do want to connect to this. And finally, I can get the HTTP status code. That's great. But I don't want to have to go through all that again. So maybe I'll just do this audit bypass. And we'll see. We're getting the handle. I'm printing out all this arbitrary stuff that doesn't really need to be printed out. We get the audit address. We find the protections. Using the offsets, we find the region of memory that we need to overwrite. And we successfully change the memory. And if you ever played NBA Jam, like the video game of your life, this is the point in time where I say, boom, shakalaka! And we bypass the audit hooks. So now when I run the same code, again, we just get status code 200. And yay! Right? We win. We don't have to worry about the audit hooks and going through all that stuff again. So that's great. Let's get back to these slides. Whoops. I lost my cursor. There we go. So let's talk about bypassing Python. Because as I mentioned, everything up until that point where you overwrite those bytes can be detected in memory. Or detected by an audit hook. And that could be bad. Right? So let's take this a step further. So since the bypass is easy to detect, are there any other methods that we can use to overwrite or to bypass Python in a sense? Right? So there are other ways to load and patch DLLs. I've had conversations with smart people about Python internals. And they've suggested other ways to do it. So, you know, just loading DLLs to kernel 32 and virtual protect and all that stuff is one way to do it. But there are other ways to do it. But everything that you do that way has to be run in the context of the Python interpreter. So no matter what I do, or no matter what you do, rather, you can be detected. No matter how you do it. So the question is, can we run Python without running the Python executable? And obviously, the answer is yes. Because I wouldn't be asking you this question and wasting your time with that bit of the slide. So there's a package called Python net, which is a legitimate package. It's great. And it uses it allows you to access the .NET CLR from normal Python. It's not Iron Python, right? It's not Iron Python at all. Iron Python is an implementation of Python that is, as far as I'm aware, not even MSIL compliant. But the syntax is still similar to Iron Python. And as of May 2020, it supports Python 3.8. That's great. And my camera just went out. So I'm going to turn my camera off, if that's okay. So we have the here's some syntax, right? We can import CLR from collections that we import counter. And then we can access the CLR the same way that we do it in sort of Iron Python-ish ways, right? Where from system.diagnostics, we can import process, and we can actually list all the processes in a CLR manner. CLR-like syntax. Or .NET-like syntax, I should say. You can actually also embed Python in your .NET executable. So Python.NET comes with a .NET assembly called Python.runtime.dll. And that can be used with .NET languages to run Python. The Python types for the basic primitive types are actually written in C sharp and compiled as part of this assembly. The only requirement is that your path variable must include a directory with the Python runtime installed. So in this case, we're looking for Python 3.8.dll. So here's an example of what it looks like in C sharp. We have this main function, and then we are using the Python gill, and we can import modules that Python knows about, such as numpy. And when we run this function, we get this output to the right, and it's pretty cool that we can actually run Python inside of .NET. And we can use C sharp to write this, but I like boolang. The reason I like boolang is because of Bytebleeder and the Silent Trinity project, that C2 framework that he has. It's pretty dope. I've contributed to it in the past, and I really like boolang because it's very Pythonic. If you look at the code that's on the screen right now, it almost seems like it's Python, but it's not. It's boolang. Also, support Bytebleeder in his open source sponsorship program that he has. It's definitely worth investing in people who have helped you in your corporate endeavors for pentesting and retting. So what we need to do, as I mentioned before, is point the path variable to a Python installation folder, which gives me an idea, right? Because the path variable can be set to any folder on the disk or on a network share. So you can already see where I'm going with this, hopefully. Hmm. I tried to find a black one, but there are no black emojis. So we can do Python across a network share by simply by pointing the Python path to it. So in this case, I had a share called winwork slash Python 3.7. And this is 3.7 because at the time I was testing it with 3.7 before 3.8 came out. It still works with 3.8, so ignore that. And that's what it looks like, right? So the path variable can be set to the server slash share, just as long as that share points to a Python 3x installation. And this is what it looks like. So you can see all this SMB traffic, and you can clearly see that I'm loading Python 3.7 across a share. Now, this isn't like an abuse of anything, right? You can actually install Python when you install Python normally on a network share and have everybody access it. It's slower because you have to load those resources remotely, but you can actually do that. But if we can use SMB, that probably means we can also use WebDAV. And sure enough, we can actually load Python across WebDAV by setting the server and the port. So whack whack server at port, and SSL is optional. There's a registry setting in Windows that determines how you can authenticate with WebDAV, and you may have to set that in some environments. I was having issues with this earlier today, or earlier, I guess it was today, or last night, whatever. And something broke, and I couldn't figure out what it was. So I can't show you what it looks like going across a WebDAV server. But it works. Here are the packets. It worked at some point in time. I just need to figure out what happened. So we can also host our own repo to remotely load packages, right? We don't need to pip install anything on the target box, assuming that we need extra modules that aren't already there on the Python 3.8 installation. Empire did this by implementing a custom import hook that allows you to modify the logic and how modules are loaded. So it allows you to load things remotely, and if you want more information on that, you can check out Chris Ross's blog on in-memory Python imports, or you can check out the Sulink's GitHub repo for remote importer, and you can check that out. It's really interesting stuff. There's also another module out there called HTTP importer, or something like that, that allows you to import modules over HTTP directly. And of course, Empire was written a long time ago. I know a new company is picking up Empire, I think it was BC Security, and I think they recently released a tool called Starkiller that seems really interesting. You should probably go check that out. If you want to load a package remotely, you need to just have it in the standard format. So here's the request module, and I want to just import this. This is what it needs to look like. It needs to be the zip file. Inside it, you need to have the name of the package, and then inside a folder with the name of the package, and inside of that, you need to have all the files that you need for the package. Let's do a demo of that real quick. All right. So in this folder, I have some code called scarysnake.boo. Everybody who writes C Sharp tools likes to call their tools Sharp something, but no, this is Boo. It's about ghosts, so we're doing scarysnake. And so we can actually compile this. I wrote a little thing using the compiler that comes with Boo, and we can just compile it real quick. We'll also use ILRepack to package the the additional assemblies that we need. That includes the Boo.lang assembly and some other additional things. Most importantly, the Python runtime.dll that you see here. That's the most important one, right? And now we have this tool called scarysnake, and it tells me that the Python path must be set. All right? So I'm going to set this Python path by just passing it as an argument. Sorry, I had to go find it. And then we can do something like maybe enter interactive mode. And this is the output that we get. It's a little, well, it's basically, I tried to imitate the same output that Python gives you when you start the interpreter. It shows us where the path is. And again, if you were doing this remotely, it would be over WebDAV. I think I had set up in the past, I had set up a server in Azure, and the load time was about 30 seconds just to run Python. But if it takes that much time to further escalate or do whatever you need to do with the Python and not get caught, I think it's worth the investment. So we can do things like, here we go again. And we're going to print out the status code, of course, from red team village.io. And without changing the normal Python executable, we basically brought our own interpreter to the system, and we were able to run Python, using the Python that's on the system, be able to run Python without worrying about all the hooks. Once again, boom, shakalaka, I'm on fire. I'm not literally on fire. Don't call the fire department. So some of the other options that are here include the local file. So you can run a local file. You can also run a remote file. I forgot to actually test this this morning, so I don't know if it works. So I'm not going to actually do it. Because that would be embarrassing. But it has worked in the past of trying to run a remote file. So you basically download the Python file, right, from GitHub or wherever you have it hosted, and you just sort of run it in memory. So let's try something real quick, right? I forget which system I'm on sometimes. There we go. Windows terminal. Split pane. Great. So I'm going to try doing this, right? I want to import the request module. But I don't have the request module. Just to show you that it works, that I'm actually using the actual Python that's hosted on the system, I'm going to Python-npip. Which, by the way, you should always do Python-npip because if you have several different Python versions, you don't want to confuse yourself. So you should always specify the version of Python that you're using when you run things like pip. It's just good practice. And we're going to just pip install request. And if I've installed request, and now I can import it with no problem. So, again, you can definitely use the Python libraries that are on the system to do it. Or you can host your own remotely and sort of just pull from a remote repo without actually touching anything on the disk. It's really up to you and what's in the best case scenario for your immediate situation. But I think it's great that we can actually just do this in the first place. So, so far, just to reiterate, we've talked about how to bypass autohooks for Windows, how to bypass autohooks for Linux, and how to not run Python on a system without running the Python executable whatsoever. For detections, it's a little bit difficult because everything is in .NET. So you have the same problem of .NET detections that you would find in any other .NET binary. If you're collecting ECW logs somehow and you have the space for that, wow, that's impressive. But if you have that space, you'll actually see that the Python runtime is loaded. So if you're in an environment where you don't normally see that, that could be something you detect. In this case, because I wrote it in Boo, it actually also detects the Boo.lang binary. But you could write this in C Sharp if you really wanted to. If you wanted to be a conformist and write all your tools in C Sharp, you could do that. And obviously, you wouldn't have the Boo.lang. So in conclusion, Python autohooks are great when they're implemented correctly. You should protect your custom Python executables with some OS-level protections. You need to do something, right? The same way you do any protection of any other binaries on your device. You need to make sure that your Python executables are protected. Because if I can pull your executable off of the system and you have autohooks enabled, I can debug it and get a pretty good idea of what autohooks you've implemented just by looking at the code or looking at the assembly. And of course, that ties in with don't rely solely on Python to protect your Python. So even if you had all these autohooks, I can pull your executable off and do some other nonsense with it and figure out how to bypass your autohooks. So thanks to everyone who has had some influence on this talk. Steve Dower, who works at Microsoft, he's a Python core dev, and he's actually the one who's done most of the implementation work for autohooks. Bytebleeder, who got me into Boo.lang and embedded scripting, Chris Ross over at SpecterOps, worked on Empire, and Adam Chester.