WEBVTT 00:13.740 --> 00:16.300 Hey, welcome back to AppSecVillage. 00:16.300 --> 00:18.860 Hope you're having a great time here at DEF CON 2020. 00:19.120 --> 00:21.260 Get ready for another wonderful talk. 00:21.260 --> 00:23.100 We've got Jared Overson. 00:23.560 --> 00:28.160 And he's going to talk about Hackium, a browser for web hackers. 00:28.160 --> 00:29.460 Okay, let's be honest. 00:29.480 --> 00:31.920 With a name like Hackium, you've got to kind of check it out. 00:31.920 --> 00:33.660 It's like the coolest name ever. 00:33.660 --> 00:35.500 I know I'm going to be watching this one. 00:35.620 --> 00:44.160 Jared is the Director of Engineering at Shape Security, now F5, where he designed and led the development of Shape's enterprise web security platform. 00:44.280 --> 00:51.640 Jared is a frequent speaker on modern web threats and has been quoted by Forbes, The Wall Street Journal, and CNET, along with many others. 00:52.060 --> 01:03.180 He co-authored O'Reilly's Developing Web Components book, delivered dozens of analysis and reverse engineering tools, and frequently writes about web development and fraud. 01:03.180 --> 01:07.740 Please welcome Jared to the AppSec security stage. 01:09.500 --> 01:10.700 Hey. 01:10.880 --> 01:11.740 Hi. 01:11.900 --> 01:12.980 Thank you. 01:13.440 --> 01:15.180 Thank you for being here. 01:15.600 --> 01:16.880 Thank you. 01:16.880 --> 01:17.880 This is awesome. 01:17.880 --> 01:19.960 This is so cool. 01:19.960 --> 01:22.320 This is strange. 01:22.320 --> 01:23.200 This is strange. 01:23.200 --> 01:24.560 I can't lie. 01:24.560 --> 01:25.780 This is strange. 01:25.960 --> 01:27.460 I can't see you. 01:27.460 --> 01:30.040 You're not actually here with me. 01:30.040 --> 01:36.860 And I was going to have slides and then it started feeling like a webinar with my little picture in the corner. 01:37.200 --> 01:42.080 And a webinar is the last thing I think of when I think of DEF CON. 01:42.080 --> 01:43.640 And I just I couldn't do it. 01:43.640 --> 01:46.820 I just physically could not do it. 01:46.960 --> 01:49.560 So we're just going to we're going to do this live. 01:49.780 --> 01:53.340 It's a prerecorded with a whole bunch of cuts and whatever. 01:54.800 --> 01:56.940 But it is it is what it is. 01:56.940 --> 01:58.880 This is this is a strange time. 01:58.880 --> 02:04.980 But anyway, I'm excited to talk to you about what I'm going to talk about. 02:04.980 --> 02:06.560 But first, who am I? 02:06.600 --> 02:08.460 I'm I'm Jared Oberson. 02:08.460 --> 02:11.820 I talk a lot about credential stuffing and automated attacks. 02:11.820 --> 02:14.120 And I'm just a web hacker. 02:14.120 --> 02:16.680 I've been hacking the web for a million years. 02:16.680 --> 02:26.240 I work at Shape Security, now part of F5, director of engineering, where I built Shape's enterprise web application security platform. 02:26.240 --> 02:35.900 And it was it was an awesome opportunity because it gave me the chance to basically make a product that would defeat me. 02:36.220 --> 02:38.400 And that was fun. 02:38.660 --> 02:43.440 I thought like, well, I could do this, but then I could beat me by doing this. 02:43.440 --> 02:45.640 And then if I did this, then I could do that. 02:45.640 --> 02:47.560 And if I did that, I could do this. 02:47.560 --> 02:52.120 And it was it was just a cool, fun game against myself. 02:52.120 --> 02:56.140 I ended up building something pretty cool, but I'm no longer building that anymore. 02:56.560 --> 03:02.140 And I'm building other things, which is what I'm going to talk to you today about. 03:02.140 --> 03:03.920 But enough about me. 03:04.080 --> 03:05.720 You, let's talk about you. 03:05.720 --> 03:07.920 I mean, everybody knows who you are. 03:07.920 --> 03:09.740 It's really awesome to have you. 03:09.740 --> 03:11.220 Thank you very much. 03:12.220 --> 03:14.360 So let's get started. 03:14.360 --> 03:28.840 So I am I am really excited to to introduce three major tools, but really a whole suite of tools that I guess I've been working on for my entire life. 03:28.940 --> 03:40.610 Every six to 12 months, I seem to rebuild part of these tools in order to do something with the Internet and the Web that I want to do, but can't given the way the Web works. 03:40.610 --> 03:50.230 So over the course of the past 20 plus years, I've been crafting and honing my strategy for manipulating Web stuff. 03:50.230 --> 03:59.530 And over the course of the past year, year and a half, I've started to to to to focus those into a few projects to see how good I could make it. 03:59.550 --> 04:02.610 And I'm really happy with what I've got. 04:02.610 --> 04:07.150 So today I'm talking about three major projects. 04:07.570 --> 04:13.510 Hackium is the one that's going to be most interesting to most people because it feels like a browser. 04:13.750 --> 04:26.470 But Shift Refactor and Shift Interpreter are much more technically complex and give you the ability to to to mold and twist JavaScript like you're a magician. 04:27.570 --> 04:36.730 JavaScript, I know, is not always the most well-respected language in security circles, but you can't get away from it on the Web. 04:36.730 --> 04:39.030 JavaScript is what you got and you have to deal with it. 04:39.030 --> 04:53.550 And when more and more business logic is showing up in websites, you have to understand more of it because what is transmitted from a Web page to a server is becoming less and less understandable on its own. 04:53.550 --> 05:07.130 So all these tools together give you the ability to manipulate Web stuff, understand Web stuff and transform Web stuff so that you can just control it better, which is awesome. 05:07.130 --> 05:12.070 So this session, I'm going to be going over three or four major topics. 05:12.070 --> 05:15.250 I'm going to be talking about what Hackium is. 05:15.290 --> 05:21.590 I'm going to be going over the REPL and the initialization command and configuration so you can get started. 05:21.590 --> 05:37.890 And then I'm going to be going over how to use Hackium with external services like 2CAPTCHA to solve CAPTCHAs and also how Hackium generates human behavior to make it bypass trivial defenses more easily. 05:37.890 --> 05:48.940 And finally, going to be going over how to use Hackium with Shift Refactor and Shift Interpreter to automatically programmatically deobfuscate JavaScript. 05:48.940 --> 05:51.500 Hackium is a command line tool. 05:51.500 --> 05:53.060 It can act as a browser. 05:53.060 --> 05:55.080 It's an automation framework. 05:55.120 --> 06:01.260 It is a foundation for building Web hacking stuff. 06:01.620 --> 06:03.540 So Hackium is a Node.js library. 06:03.540 --> 06:06.120 It is installable via NPM. 06:06.560 --> 06:15.640 You can install it via npm install-g hackium, which will install the Hackium library and the command line tool globally. 06:16.540 --> 06:18.460 Install Hackium. 06:18.840 --> 06:22.080 npm install-g hackium. 06:24.780 --> 06:28.760 NPM will do all of the stuff that it needs to do. 06:28.760 --> 06:32.920 It's downloading Chromium R782078. 06:33.080 --> 06:40.740 So Chromium is downloaded via Puppeteer right now, and it's a bundled Chromium that is guaranteed to work with that particular version of Puppeteer. 06:40.740 --> 06:50.980 Hackium started as a bunch of Puppeteer scripts that grew and were extended and copied over and over again over the course of a long period of time. 06:51.580 --> 06:59.480 And for the time being, Hackium extends a lot of Puppeteer's functionality and reuses the bundled version of Chromium. 06:59.480 --> 07:10.160 And it will continue to do this for as long as possible until the code bases of each diverge in a way where code reuse is no longer possible. 07:10.160 --> 07:14.300 All right, so now we have Hackium installed locally. 07:14.300 --> 07:16.900 Version 1.0.4. 07:16.960 --> 07:22.500 We start that up, we get what looks like a normal Chrome browser. 07:22.500 --> 07:27.000 Since we have Hackium running, we can use it just like we would any browser. 07:27.000 --> 07:34.740 Comes with one extension pre-installed as the Puppeteer extension bridge, one of the dependent projects for Hackium. 07:34.740 --> 07:41.900 This allows Hackium to communicate to and from the Chrome extension API. 07:41.900 --> 07:59.680 So Hackium also exposes all the Chrome extension API to Node.js scripts and automation scripts, so that you have access to a lot of additional functionality that is normally inaccessible from command line scripts. 07:59.680 --> 08:06.280 So Hackium also includes an in-page client for plugins and functionality to hook onto. 08:06.280 --> 08:16.680 So there's a consistent way to share and communicate between the extension and the automation script behind Hackium. 08:16.680 --> 08:21.780 So we can use the event bridge to communicate with the extension. 08:21.780 --> 08:26.440 We can post messages back to our automation script. 08:26.440 --> 08:31.780 And if we had anything else on here, we'd be able to access them. 08:31.780 --> 08:35.980 So Hackium also exposes a REPL to the command line. 08:35.980 --> 08:40.400 Let's see, we have access to page, which is our active page. 08:40.400 --> 08:49.800 page.gotoHPSexample.com Let's go back to Google. 08:53.930 --> 08:59.030 So you can use the REPL to do everything that you would normally do within a Puppeteer or Hackium script. 08:59.030 --> 09:07.010 Here we are typing into the input that has the attribute aria-label="search". 09:07.010 --> 09:18.190 We're typing Hackium with a new line at the end to basically just search for the term Hackium in Google's website. 09:18.230 --> 09:21.010 So this is a good way to experiment with automation. 09:21.010 --> 09:29.790 And you can also get out of here and use the REPL history to get a recording of everything that you've typed. 09:29.790 --> 09:37.010 So you can just copy and paste these directly into a Hackium script and share these with other people. 09:37.010 --> 09:49.750 The shareability was a very, very high priority for Hackium because historically it's been pretty difficult for people to share their work manipulating websites. 09:49.750 --> 10:04.590 If I were to make something neat that manipulated Google.com or change the functionality of Spotify or Twitter or whatever else, it would be fairly difficult for me to just quickly share that with somebody else. 10:04.590 --> 10:10.510 There are two main ways that people have historically manipulated websites. 10:10.510 --> 10:17.210 And that's either via the embedded browser dev tools or with a proxy or both. 10:17.210 --> 10:28.370 So a proxy would give you access to intercept and manipulate and change any transactions for the web, which is very, very powerful. 10:28.430 --> 10:43.310 But if you've worked with proxies extensively in the past, you know that they can be somewhat of a pain to install, especially if you're talking about different platforms, Windows, Mac, Linux, or whatever. 10:43.310 --> 10:48.930 And if you do go this route, then chances are you have a favorite proxy. 10:49.050 --> 10:59.830 And if you try to share it with somebody else, they've got a different favorite proxy or a different configuration for the same proxy or something that makes it difficult to just quickly share your work with somebody else. 11:00.450 --> 11:14.410 Now, if we do a lot of our work within the embedded browser developer tools, which are extremely powerful and fantastic, definitely, we have no way really whatsoever to share that with anybody else. 11:14.470 --> 11:16.370 A lot of the work is manual. 11:16.370 --> 11:19.990 It's difficult to automate if it's possible to automate at all. 11:19.990 --> 11:28.910 There are like content snippets and other little things that you can use to repeat repetitive work. 11:28.910 --> 11:30.470 But those aren't easily shareable. 11:30.470 --> 11:33.810 They're not easy to really configure in the first place for yourself. 11:33.810 --> 11:36.690 And all in all, it's kind of a pain. 11:36.690 --> 11:45.030 When you do this work, you get all set up yourself, and it becomes difficult for anyone to use your stuff. 11:45.170 --> 12:03.590 So with Hackium, it was a priority for me to get everything as self-contained and bundled into one project as possible, so that if you wanted to, you could share just a small configuration implementation to anybody, and they can see what you've done. 12:03.590 --> 12:19.090 One of the important aspects of shareability is making sure that the user this is destined for doesn't have to know anything secret or magic about your configuration to get up and running. 12:19.510 --> 12:31.670 So we take care of that with Hackium by making a lot of the base configuration stored in a configuration file that is just JSON or JavaScript and is portable so that you can deliver it to anyone else. 12:31.670 --> 12:35.930 You can initialize one easily just by using Hackium init. 12:36.370 --> 12:40.670 You can set up the URL you want to go to by default. 12:40.730 --> 12:47.690 You can configure whether or not you want DevTools to open automatically, which might be important for your script if you're communicating with DevTools. 12:47.730 --> 12:50.150 We are not going to do that here. 12:50.150 --> 12:53.470 We can create a blank JavaScript injection. 12:53.470 --> 13:18.230 One of the other top priorities for Hackium was to make sure that it was as trivial as possible to inject JavaScript before anything else is loaded, which is the only way that you can guarantee a pristine, stable environment for your code, and to also make it trivial to intercept and manipulate content coming in so that you didn't need a proxy at all. 13:18.230 --> 13:26.270 Hackium makes that possible with just generic injection files and interceptor modules. 13:26.270 --> 13:27.430 We're going to say no to that. 13:27.430 --> 13:33.590 We will create a boilerplate interceptor and, sure, a boilerplate Hackium script. 13:33.590 --> 13:44.590 We do not want to run headless, so now we have our hackium.config.js, which is a simple JavaScript file with the configurations that we set up. 13:44.590 --> 13:49.210 We can see that we have a boilerplate interceptor. 13:49.210 --> 13:54.870 An interceptor is just a standard Node.js module that exports two properties. 13:54.870 --> 14:06.410 One is the intercept property, which is an array of request patterns that this interceptor will be configured to intercept. 14:06.410 --> 14:18.310 So here, by default, you can see that we have just a wildcard asterisk for the URL pattern, and we are matching script resource types, and we are doing it at the response stage. 14:19.270 --> 14:31.970 And the second thing that an interceptor needs to export is the interceptor property, which takes a synchronous or asynchronous function that can just do something with a request. 14:31.970 --> 14:35.210 So to share this work, we can just archive this directory. 14:35.210 --> 14:36.590 We can upload it to GitHub. 14:36.590 --> 14:40.770 We can publish it to whatever publishee service you want. 14:40.770 --> 14:48.570 And anyone can download that and run Hackium with just the command hackium and see what you did. 14:48.570 --> 15:01.370 So now that we have gone over the REPL and the getting started, we can jump into what makes Hackium different than other browsers and how to use its APIs to connect to external services. 15:01.370 --> 15:10.290 So if you follow me at all, you know that I talk a lot about automated attacks, things like credential stuffing, scraping, and the trajectory of tools looking more and more human. 15:10.290 --> 15:33.030 Now, since this is a browser that is controlled partially by scripts or extensions or power tools that do stuff, it's important for the behavior that is being generated to look plausibly human so that it doesn't interrupt your browsing and your usage. 15:33.030 --> 15:40.910 So one of the extensions onto Puppeteer that Hackium provides is automatically generated simulated human behavior. 15:40.910 --> 15:51.150 Now Puppeteer and a lot of other automation tools have the ability to slow down behavior so that it doesn't come all puking out on the page all at once. 15:51.150 --> 15:55.930 You don't type large, large blocks of text within a fraction of a second all at once. 15:55.930 --> 15:57.250 You can slow it down. 15:57.250 --> 15:59.530 But still, that's not quite human enough. 15:59.530 --> 16:08.050 Humans don't have consistent durations between key presses or mouse movements that all take a certain number of steps. 16:08.050 --> 16:19.570 So one of the things I did with Hackium was to make sure that things like keyboard events happen at different intervals and durations all within a minimum and maximum expected human behavior. 16:19.590 --> 16:27.010 And things like mouse movements follow non-direct paths and sometimes make mistakes like this. 16:27.010 --> 16:29.110 So this is a slow, meandering move. 16:29.110 --> 16:33.510 You can see down over here, I kind of have to curve up to get to the point we're looking at. 16:33.830 --> 16:34.930 Refresh again. 16:34.930 --> 16:36.750 A random, larger curve. 16:37.050 --> 16:38.510 Much slower this time. 16:38.510 --> 16:40.830 We also overshoot and have to backtrack. 16:41.210 --> 16:42.290 Do this again. 16:42.290 --> 16:44.030 Overshoot and backtrack. 16:44.030 --> 16:50.510 Now things that, where if you're looking at it, it would not definitely be automated behavior. 16:50.750 --> 17:00.910 This is important because just naive automation detection techniques will check to see whether or not you're clicking consistently at like a 0, 0 position on every element. 17:00.910 --> 17:06.070 Or if you're zipping from one point to another with no mouse move events in between. 17:06.110 --> 17:09.650 Or if you're moving all on a straight line or at the same velocity. 17:09.650 --> 17:10.910 Things like that. 17:10.910 --> 17:19.910 So we have to account for that so that we can generate behavior that is not automatically blocked by everything that's out there. 17:19.910 --> 17:27.270 Now another thing that you'll probably know if you follow me is that I hate CAPTCHAs. 17:27.430 --> 17:30.110 CAPTCHAs are just such a pain in the ass. 17:30.110 --> 17:33.050 Most of them are really, really, really bad. 17:33.050 --> 17:40.850 And things like ReCAPTCHA, Google's ReCAPTCHA, seem like they've gotten much, much worse over time. 17:40.850 --> 17:47.990 Things like the little grid of pictures where you have to determine what a fire hydrant is or what a bus is, crosswalk, things like that. 17:48.410 --> 17:49.630 They're a pain. 17:50.010 --> 17:56.570 And CAPTCHAs are things that will pop up in the way of automation all the time. 17:57.070 --> 18:03.490 And it's an arms race back and forth trying to make an automated tool that bypasses CAPTCHAs. 18:03.490 --> 18:08.130 Then CAPTCHA makers get that tool, figure out how to block it, blah, blah, blah. 18:08.130 --> 18:23.810 Now rather than build in any CAPTCHA solving within Hackium, it's much easier to delegate that responsibility to services and companies whose core purpose is solving and bypassing CAPTCHAs. 18:23.810 --> 18:36.090 So you can wire up services like 2CAPTCHA very easily within Hackium scripts just by using standard node libraries and node modules to make requests and parse JSON responses. 18:36.090 --> 18:39.370 So this is a script that I put together. 18:39.370 --> 18:44.630 I am sucking in my API key here just so you don't see it. 18:45.230 --> 18:50.930 And next we're going to go to HTTPS old.reddit.com slash login. 18:50.930 --> 18:56.550 We are going to initiate a CAPTCHA request with our API key. 18:56.670 --> 18:58.450 We get a request ID. 18:58.450 --> 19:03.270 And then we type in our username into the user reg input. 19:03.270 --> 19:05.730 Samuel Clemens 90210. 19:06.110 --> 19:10.230 We type in our password twice on the password inputs. 19:10.430 --> 19:12.550 Ctechastronomy is our password. 19:12.690 --> 19:16.170 And then we pull for request results. 19:16.170 --> 19:19.430 So the way these CAPTCHA solvers work is that you send a request. 19:19.430 --> 19:21.050 It's like, hey, solve this CAPTCHA. 19:21.050 --> 19:25.110 And then you pull for a response for when that CAPTCHA is done. 19:25.110 --> 19:26.590 So you're just not sitting there waiting. 19:26.590 --> 19:33.170 You can do additional work while that CAPTCHA is being solved. 19:33.490 --> 19:43.610 So after we have solved that, we input the CAPTCHA response to an element on the page, g-recaptcha-response, and then we submit. 19:43.610 --> 19:51.610 Now we can run this script, hackium-e-index.js. 19:53.550 --> 19:55.250 It'll pop up in Hackium. 19:55.250 --> 19:58.230 Go to reddit.com's signup page. 19:59.110 --> 20:05.030 Notice how the keyboard typing is not instantaneous, but it's not consistently slow. 20:05.030 --> 20:06.790 It's kind of a blah blah blah. 20:07.170 --> 20:10.630 That's the technical term for that. 20:10.630 --> 20:12.870 So now that we're done, we're going to pop over to the console. 20:12.870 --> 20:19.770 See that we're waiting an initial timeout for two CAPTCHA. 20:19.770 --> 20:21.270 And then we are done waiting. 20:21.270 --> 20:24.530 So now we're pulling for our response. 20:25.310 --> 20:31.250 And now that we're done, pop over to the web page and bam, Samuel Clemens 90210. 20:31.250 --> 20:32.750 We are on Reddit. 20:32.750 --> 20:39.610 We are able to, I don't know, prop up whatever political candidate we are interested in this election cycle. 20:39.610 --> 20:45.630 And this is just one technique for getting by all the different defenses out there. 20:46.010 --> 20:48.010 And I want to make this clear. 20:48.010 --> 20:50.390 I do currently work at Shape and F5. 20:50.390 --> 20:54.490 Hackium is not a tool designed to bypass security defenses. 20:54.490 --> 20:59.670 Security defenses for things like credential stuffing or spamming or scraping. 20:59.670 --> 21:06.510 Those operate at high volumes. 21:06.890 --> 21:16.470 And the type of work that a legitimate and curious hacker using Hackium would do is much, much smaller volume. 21:16.470 --> 21:22.470 So that you can bypass things at small volume, but you'd be caught at high volume. 21:23.930 --> 21:26.390 Cover my ass. 21:26.390 --> 21:26.970 All right. 21:26.970 --> 21:33.730 So, so far I've talked a bit about just basic usage of Hackium with the REPL and other stuff. 21:33.730 --> 21:43.830 I talked about the extensions it has in order to make it look more human driven, as well as showed an example of how to tie it to other services. 21:43.830 --> 22:04.630 And I haven't even touched at all on the other parts of the suite of tools, Shift Refactor and Shift Interpreter, nor have I talked really much about the whole point I made at the start, which is about manipulating website content, which leads me into Hackium Interceptors. 22:04.630 --> 22:16.670 So Interceptors, like we've already gone over, are just basic node modules that intercept a particular URL or request pattern and just do something with it. 22:16.670 --> 22:23.370 You can use the Hackium init command to initialize some basic interceptors. 22:23.370 --> 22:26.030 I'll call ours interceptor.js. 22:26.030 --> 22:35.870 We can use a basic interceptor template, which just doesn't really do all that much, just gives you the boilerplate. 22:35.870 --> 22:40.550 A pretty printer, which uses Prettier to format JavaScript. 22:40.550 --> 22:45.090 So we're going to start a basic transformer using Shift Refactor. 22:45.310 --> 22:55.270 So Shift Refactor is a library I created that makes it easy to query and transform nodes of a JavaScript AST. 22:55.270 --> 23:02.870 An AST, an abstract syntax tree, is just a data structure that represents something that was parsed. 23:02.870 --> 23:10.510 So the AST I'm talking about is just a big old JavaScript object that represents JavaScript source code. 23:10.510 --> 23:18.410 You can use a tool like AST Explorer to just paste JavaScript in and see what AST it generates. 23:18.410 --> 23:25.090 So you can pop through each of the nodes and see what types they are, see how they're structured. 23:25.090 --> 23:32.230 And if you change any of those and regenerate source from that AST, then you change the JavaScript source. 23:32.230 --> 23:48.290 So Shift Refactor takes JavaScript source, parses it into an AST, and then analyzes the scope tree, creates parent mapping, basically does a lot of legwork that makes it easy to do queries and translation on it. 23:48.290 --> 23:56.350 It also leverages Shift Query, which allows you to query a JavaScript AST with CSS-like selectors. 23:56.350 --> 24:01.130 And I modeled Shift Refactor based off of the jQuery API. 24:01.130 --> 24:06.090 So it actually feels very, very similar to navigate and traverse and replace large trees. 24:06.930 --> 24:14.410 So we have a script object based around our parsed response body. 24:14.890 --> 24:21.350 This expression replaces all console log expressions with alert calls. 24:21.350 --> 24:28.110 So we make a query for call expressions that have an object name of console and a property of log. 24:28.110 --> 24:33.690 And we replace those with call expressions for alert. 24:33.690 --> 24:38.330 And we translate the arguments from the console to the arguments for the alert. 24:38.330 --> 24:50.350 As I'm breezing through this, I know that I am glossing over some fairly complex topics, if you haven't played with JavaScript ASTs before. 24:50.370 --> 24:53.450 But I can't stress this enough. 24:53.450 --> 25:07.070 Despite all the crazy terms and syntax like call expression and callee and identifier expressions and some more that you'll see, those are just nodes of a JavaScript tree. 25:07.070 --> 25:14.910 And all you're dealing with is a gigantic JavaScript tree in a similar way that you would navigate or traverse the DOM. 25:14.910 --> 25:24.230 So it might look foreign at the start, but don't let that dissuade you from jumping into this because it's not difficult to get a hold on it once you get the practice in. 25:24.230 --> 25:35.030 So here we've got a basic web page that exists only to simulate a real world scenario, because I'm going to be jumping into programmatically de-obfuscating JavaScript. 25:35.030 --> 25:39.010 And there are plenty of real world examples out there. 25:39.010 --> 25:44.910 And you probably have seen a few of those if you've been looking through a web page source. 25:45.130 --> 25:55.030 But we run into some sketchy legal ground if I'm de-obfuscating that live in front of people. 25:55.030 --> 26:11.970 So we've got a nice little sample website here that I've crafted to look like a payment form that has a, conveniently, an obfuscated bit of JavaScript just floating around on it. 26:11.970 --> 26:28.070 This is a sample of obfuscated JavaScript that was obfuscated via obfuscator.io, which is a pretty common JavaScript obfuscation website because it's the first URL that pops up when you search for JavaScript obfuscator. 26:28.070 --> 26:31.210 It actually does a pretty good job at obfuscating JavaScript. 26:31.210 --> 26:32.630 It can inject dead code. 26:32.630 --> 26:35.850 It can shuffle things. 26:36.210 --> 26:42.430 But it is extremely difficult to protect JavaScript. 26:42.950 --> 26:51.970 And obfuscation really doesn't actually put all that much of a barrier in front of an adversary trying to understand the JavaScript. 26:52.870 --> 27:01.550 So we're going to go through and show how this can be completely reversed and sent to the browser before the browser even gets it. 27:01.550 --> 27:06.610 So it looks as though you are browsing the web with de-obfuscated JavaScript. 27:06.610 --> 27:09.430 So this is using Firefox's DevTools. 27:09.430 --> 27:13.250 Even if we pretty print this, it's not all that much more readable. 27:13.490 --> 27:20.590 You see here that we start off with a list of encoded strings, clearly encoded strings. 27:20.690 --> 27:27.410 We are followed by a variable declaration that, I don't know, does stuff. 27:27.410 --> 27:30.910 It has an alphabet here. 27:30.910 --> 27:36.870 And given that these are encoded, this is probably the decoder. 27:36.910 --> 27:43.590 I would go down here and we see references to that function all over the place. 27:43.590 --> 27:50.930 So 0x227, 0x227a, 227a, 227a, 227a, 227a. 27:50.930 --> 28:03.850 This is a common method of making it more difficult for somebody to immediately understand what is going on with JavaScript. 28:03.850 --> 28:12.850 So I've created a complete interceptor here so that I don't have to code in front of all of you because that's not very exciting, is it? 28:13.370 --> 28:19.850 So we've set our URL pattern to the vendor file that I had on my local server. 28:19.850 --> 28:35.430 And we have fleshed out the interceptor so that we are running it through refactor and also loading up our interpreter via shift interpreter and massaging some JavaScript. 28:35.630 --> 28:37.930 So refactor, we already went over a little bit. 28:37.930 --> 28:42.890 We are sending it the response body and we're also adding a common methods plugin. 28:42.890 --> 28:54.570 Next, we are creating our interpreter instance and we're loading the scripts AST and we are passing it a context that we are requiring in from here. 28:54.570 --> 29:01.090 So the context is like the set of global variables that are accessible by JavaScript. 29:01.190 --> 29:07.530 And by default, this interpreter doesn't have anything in its global context. 29:07.530 --> 29:14.130 So if we want to expose things that the interpreter needs, then we're going to have to explicitly set them. 29:14.130 --> 29:15.090 And we can do it ourselves. 29:15.090 --> 29:17.290 It's just a plain old JavaScript object. 29:17.470 --> 29:29.530 I've put together a fairly basic and simple DOM lookalike context that uses JS DOM in order to simulate a browser DOM. 29:29.530 --> 29:44.410 So if you're interpreting browser side JavaScript, it'll have access to things like image elements, A to B, B to A, URL, URI decode something, whatever, all those things. 29:44.410 --> 29:52.770 One of the interesting things about this interpreter is that it was designed so that it can take statements or expressions piecemeal. 29:52.770 --> 30:02.990 So you don't have to execute a script statement by statement, expression by expression in the flow that a script would expect to be executed in. 30:02.990 --> 30:12.990 You can take any statement or expression out of the AST and pass it to the interpreter and interpret it as if that was the next expression or statement. 30:12.990 --> 30:22.990 This is very, very handy when you're dealing with JavaScript like the way I'm dealing with it, because sometimes we don't want to execute all the JavaScript. 30:22.990 --> 30:33.810 And we want to just execute only the pieces that we want to reuse so that we don't have to write our own decoder functions or copy and paste anything. 30:33.810 --> 30:41.430 We can just grab it, interpret bits and pieces of it, and then use that to deobfuscate the rest of the script. 30:41.430 --> 30:45.030 So it's like using JavaScript against itself. 30:45.450 --> 30:54.430 So here we're going to run the first statement in the script, which is this. 30:54.430 --> 31:03.660 So this basically primes this variable with these values so that anything else that we execute can access that variable. 31:04.460 --> 31:09.780 Next, we're going to be getting the decoder, which is the second statement there. 31:09.980 --> 31:14.220 Rather than run it directly, we're going to assign it to a variable so we can play with it later. 31:14.260 --> 31:21.240 And we're going to run that decoder statement, which is a variable declaration statement that assigns a function expression to a variable. 31:22.060 --> 31:33.800 Next up, we're going to query the innards of that decoder statement in order to get the first binding identifier. 31:33.920 --> 31:42.140 Now, a binding identifier is an identifier, like a variable name or something like that, that is basically being assigned to or having a value bound to it. 31:42.140 --> 31:48.740 And what that means here is basically we want to get that because we want to know how this function is referenced. 31:48.740 --> 31:51.600 So we're looking for this binding identifier. 31:51.700 --> 31:58.240 See, it's not that complicated once you start poking through and playing around with things. 31:58.240 --> 32:03.280 It sounds weird and scary to some people, but it's not that bad. 32:04.980 --> 32:09.040 Next up, we want to get all the references to that name. 32:09.040 --> 32:11.900 So that binding identifier is called throughout the script. 32:11.900 --> 32:15.780 We want to get all the references, the places where it's called, so that we can do stuff with it. 32:15.780 --> 32:22.000 And we're going to map those references and grab the node of that reference. 32:22.000 --> 32:36.680 So a reference is an object that has a node property in it, which is an AST node, an accessibility property, which tells you whether or not the reference is being read to, written to, or is read right. 32:36.840 --> 32:41.460 So then with those references, they're all going to be like an identifier expressions. 32:42.040 --> 32:44.060 So like, let's see down here. 32:44.060 --> 32:49.080 So an identifier expression is just this bit of code right here. 32:49.080 --> 33:00.920 So we want to get this bit of code, which is the reference where it's being used as an identifier expression as the child of a call expression. 33:00.920 --> 33:09.160 So here we're getting all of the parents of our references, and we're filtering out all of those, looking for call expressions. 33:09.160 --> 33:14.680 That just makes sure that we don't catch any references that aren't call expressions. 33:14.720 --> 33:20.000 I think almost all of them, if not all of them, are in the script, so it's not a big deal, but it's good to be safe. 33:20.000 --> 33:30.220 And we're going to replace all of those with the stringified result of the interpreter value executing those functions. 33:30.220 --> 33:32.260 So we need to JSON.stringify. 33:32.260 --> 33:38.080 So it's a string that contains a string, which is the return value of these functions. 33:39.520 --> 33:41.660 You still with me? 33:41.760 --> 33:46.260 Basically, what we're getting are the decoded values of these strings. 33:47.020 --> 33:50.340 So we've done a lot without actually seeing the effect yet. 33:50.340 --> 33:58.740 Let's comment out these statements so we can see our transformations take effect before we do too much magic. 33:58.740 --> 34:05.120 And then after that, we are printing out the refactored script to the response.body and returning the response. 34:05.120 --> 34:08.540 And bam, we are just about done. 34:08.540 --> 34:11.980 All right, we go automatically to our payment page. 34:11.980 --> 34:13.720 Let's check our resources. 34:13.960 --> 34:15.060 Here's our script. 34:15.060 --> 34:19.940 Still looks mostly the same, mostly because we didn't change those top two statements. 34:19.980 --> 34:26.660 But everything afterward, you'll notice that it no longer has those function calls anymore. 34:26.660 --> 34:31.940 Now you see those computed member access blocks right there? 34:33.020 --> 34:39.440 JavaScript still works, but they're not as easily parsable by at least my brain. 34:39.440 --> 34:46.360 And now that we've translated all the function calls to strings, we no longer need them to be computed anymore. 34:46.360 --> 34:49.460 We can translate them back to static properties. 34:49.460 --> 34:53.880 So instead of blah, square, string, we can do blah, dot, string. 34:53.880 --> 34:55.580 Well, not string, but the actual thing. 34:55.580 --> 34:57.280 You know what I'm talking about. 34:57.300 --> 35:01.520 And this is where those functions that we deleted come into play. 35:01.520 --> 35:06.760 Script.convertComputerToStatic is part of the Common Methods plugin. 35:06.820 --> 35:14.360 And the next two statements there, just delete the two statements above our main script because we don't need them anymore. 35:14.360 --> 35:19.240 The strings have been decoded, and since all the strings have been decoded, we no longer need the decoder function. 35:19.240 --> 35:23.500 So let's get rid of those and clean up our unobfuscated script. 35:23.580 --> 35:24.780 Deobfuscated script? 35:27.940 --> 35:28.820 Undeobfuscated? 35:29.040 --> 35:30.480 I don't know. 35:32.080 --> 35:36.380 So now let's load up Hackium again and check our script. 35:36.440 --> 35:38.880 And nothing's there. 35:38.880 --> 35:40.520 Oh, it is there. 35:40.580 --> 35:42.600 We actually had scrolled down. 35:42.600 --> 35:43.180 We were too low. 35:43.180 --> 35:44.620 But now this is our script. 35:44.640 --> 35:53.580 This is basically very close to the original script, minus some still the identifiers that are still wonky, but not much we can do about that. 35:53.860 --> 35:55.760 But you can see what it's doing. 35:55.760 --> 35:58.180 There's no sort of misdirection anymore. 35:58.460 --> 36:00.240 And it's real code. 36:01.200 --> 36:02.560 That's it. 36:02.560 --> 36:09.260 There's just a few lines of JavaScript that can just completely deobfuscate obfuscated JavaScript. 36:09.260 --> 36:18.720 Now, I'm talking a lot about deobfuscation, but there is a lot you can do with all the JavaScript across the web. 36:18.720 --> 36:38.720 Even a site like Twitter.com has a huge array of user-friendly, intuitive methods that could just provide you with an API to all of Twitter, but it's hidden away in the JavaScript so you can't touch it via the console or anything else. 36:38.720 --> 36:50.700 Now, all you have to do is find out where those beautiful little functions are and just expose them to the global namespace so that you can access them via the console or via hackymscripts or whatever. 36:50.880 --> 37:10.500 And if you're not a JavaScript expert, by exposing to the global namespace, all I'm saying is take a chunk of JavaScript and prepend to it like a window.myExposedVariable equals the stuff you want to expose. 37:10.700 --> 37:11.520 And that's it. 37:11.520 --> 37:26.060 You have an exposed API that you can access easily with hackymscripts, which then allows you to basically tweet via Node.js without using a developer API key or anything like that. 37:26.060 --> 37:28.480 Well, that's about all I can cover in the session. 37:28.480 --> 37:34.760 And we actually smashed a whole lot of content in a very, very short period of time. 37:35.040 --> 37:39.780 But because I was so excited and there's still so much more to talk about. 37:39.780 --> 37:41.580 This is just the tip of the iceberg. 37:41.580 --> 37:46.720 And I just barely dove into what shift refactor and shift interpreter can do. 37:46.720 --> 37:50.300 And I didn't even talk about any plugins. 37:50.300 --> 38:03.440 And I didn't talk about the Chrome extension API, talking with the Chrome DevTools protocol, and all sorts of stuff that I could just talk about for days or hours or so much time. 38:03.460 --> 38:12.240 I'm going to be putting more of this content on YouTube because that's, I guess, how people consume content nowadays. 38:12.680 --> 38:14.660 And I'll probably write some stuff up. 38:14.660 --> 38:18.580 But if you like anything I've talked about, you can follow me on Twitter. 38:18.580 --> 38:20.320 It's at JSOverson. 38:20.320 --> 38:22.800 There's probably a link somewhere nearby. 38:22.820 --> 38:26.560 Actually, I think I should be talking to somebody in Discord. 38:27.380 --> 38:28.720 Talk to me. 38:28.980 --> 38:30.640 But yeah, I like this stuff. 38:30.640 --> 38:36.320 I like making the web do things that I want it to do. 38:36.580 --> 38:39.120 And this is why I built this stuff. 38:39.120 --> 38:41.280 And I hope you enjoy it. 38:41.280 --> 38:42.500 And thank you. 38:42.500 --> 38:44.940 Thank you, AppSec Village. 38:44.940 --> 38:46.460 Thank you, DEF CON. 38:46.460 --> 38:49.500 Thank you, all of you, for watching this. 38:49.500 --> 38:50.860 This has been awesome. 38:50.860 --> 38:52.340 Thank you very much. 38:52.340 --> 38:53.220 Bye.