WEBVTT 00:01.280 --> 00:04.400 Welcome to this talk, When TLS Hacks You. 00:05.160 --> 00:07.040 I'm actually going to start with the demo. 00:07.100 --> 00:08.420 This is going to be weird. 00:08.420 --> 00:13.100 This isn't a concrete attack scenario, but hopefully should make everyone curious about what's going on. 00:13.100 --> 00:18.720 For those familiar with SSRF, the implications will be fairly clear, but I'll explain them right after the demo for everyone else. 00:19.080 --> 00:26.680 After that, the attack examples are going to gradually move towards real stuff, but I find it's easiest to look at the simplest cases and then start expanding to look at real-world stuff. 00:26.800 --> 00:31.120 For now, though, let's see the essence of the dangerous property of TLS that this talk is about. 00:33.460 --> 00:39.200 So, here's memcached, a popular cache that commonly runs on port 11211. 00:39.880 --> 00:44.880 I have it installed here, so you can first see for the key named Z, there currently is no entry. 00:45.060 --> 00:50.040 If you're not familiar with memcached, this is how stuff communicates with it, a newline-delimited TCP protocol. 00:50.260 --> 00:53.680 But now I'll do something that you might not expect to affect memcached. 00:53.680 --> 00:55.880 I'll make a request to an HTTPS URL. 00:55.880 --> 01:00.260 The "-l'' enables redirects, and the "-v'' lets you see all that's going on. 01:01.040 --> 01:04.780 So, you can see that the endpoint is redirecting to itself. 01:05.260 --> 01:09.340 I sped this up because this is a lightning talk, and the string of redirects takes a couple of minutes. 01:09.900 --> 01:11.580 But the redirects don't go on forever. 01:11.800 --> 01:14.880 After a bit, we get an error. 01:16.080 --> 01:17.380 And now for the magic. 01:17.760 --> 01:22.520 Going back to memcached, if we rerun the same git command, we see an entry appear. 01:23.040 --> 01:32.360 The odd thing is, that text never showed up in the curl command or in any of the redirects, so somehow the server we were talking to had to do something with TLS to trick curl into putting it in there. 01:32.900 --> 01:39.720 So even though I didn't just do anything other than hacking myself, what just happened was weird and potentially useful. 01:43.260 --> 01:45.020 But here's where I'm going with this. 01:45.060 --> 01:51.400 Because not everyone is into these sorts of attacks, I'm going to start off with exactly what makes this useful as an attacker. 01:51.400 --> 01:58.360 I'll then move on to some stuff that shows just how widely these attacks can be applied, and then get to some recommendations for what to do about them. 01:59.400 --> 02:03.280 So to start off with, there's an existing class of vulnerability called SSRF. 02:03.280 --> 02:05.460 It stands for server-side request forgery. 02:05.800 --> 02:09.600 It's where you effectively phish a server into doing something by sending it a URL. 02:11.320 --> 02:16.020 Last year at Black Hat, I demonstrated a bunch of these vulnerabilities in Apple Pay and in Webhook receivers. 02:16.240 --> 02:22.840 I'm not going to go into those as much here, but if you're curious, you can download these slides later and go to this YouTube link right here. 02:24.320 --> 02:28.900 SSRF has a ton of different approaches, but it was actually pretty easy in some cases. 02:28.960 --> 02:35.500 Webkit.org had a live sample implementation of Apple Pay, and it was even on AWS with the old Insys metadata service. 02:35.540 --> 02:40.380 So all I had to do was pass this IP, and I could poke around an internal API and get some tokens. 02:41.460 --> 02:43.260 But not everything was so simple. 02:43.260 --> 02:47.900 I had a lot of SSRF vulnerabilities that were almost there, but weren't vulnerable to this easier approach. 02:49.340 --> 02:54.840 Sometimes I'd have a Webhook where I could hand it a URL, and the server would even fetch the URL, but not give me a response. 02:55.000 --> 02:56.760 This is called blind SSRF. 02:57.760 --> 03:02.820 Other times the request in question would be a put request, which restricted what endpoints I could hit. 03:03.080 --> 03:08.920 Sometimes you can mess with the request method by setting up an endpoint that redirects the target server around, but that's hit or miss. 03:10.160 --> 03:16.400 Other times you might get some data back, but it's validated against a specific content type, or you only get back certain fields. 03:16.400 --> 03:19.680 Or even worse, it'll only accept HTTPS URLs. 03:21.320 --> 03:26.360 But what we really want in order to exploit this stuff is a generic blind SSRF approach. 03:26.360 --> 03:37.420 In other words, we'd like to see in the server a URL, have it fire a payload at something internal like SMTP or Redis, and get a security compromise just from that payload, without needing to see the response. 03:38.740 --> 03:40.880 Here are some of the common approaches. 03:41.000 --> 03:47.760 The issue is these are extremely platform-specific, to the point where I've never had any luck with these. 03:47.880 --> 03:51.700 But there are some useful ideas in here, particularly in SNI injection. 03:51.780 --> 04:01.540 You wouldn't think that TLS would be particularly useful, because its packets are mostly just encrypted data, binary flags and other stuff an attacker doesn't have much control over, but it's not all that limited. 04:01.540 --> 04:06.660 As OrangeSci noticed, the SNI, which is basically just the domain name, is part of the URL. 04:06.660 --> 04:16.000 So if you can trick the victim into accepting a weird domain name and parsing it incorrectly, you can include stuff like arbitrary bytes in there and smuggle stuff like SMTP commands. 04:16.400 --> 04:19.860 This approach is dependent on a platform-specific URL parsing book. 04:20.020 --> 04:23.400 But what if we had something that was inherent to the TLS protocol? 04:23.600 --> 04:25.180 This is where my work starts. 04:25.220 --> 04:28.460 Let's look at a TLS packet and see what's in there for us to use. 04:29.420 --> 04:36.420 As is often helpful when looking at protocols, we can open up Wireshark to see what happens in a TLS connection. 04:36.700 --> 04:41.800 Here's the SNI, or server name indication, that was previously explored in Orange's talk. 04:42.000 --> 04:46.540 His work involved putting new lines and SMTP commands into the section pointed to by the arrow here. 04:46.720 --> 04:51.980 If you poke around, you'll see stuff like random strings, key shares, and other stuff that isn't useful for SSRF. 04:52.240 --> 04:52.780 Why? 04:52.780 --> 05:01.120 Because the only part of this we can typically feed a server is a URL, and the only part of the URL that will make it into the client-hello packet is the domain name that we provide the victim machine. 05:03.120 --> 05:07.160 But there's a different way that we as an attacker can get payloads into client-hello messages. 05:07.180 --> 05:10.480 It involves a couple of steps and is the main focus of this talk. 05:10.740 --> 05:15.660 First, we send over a link to our own server, in this case jmatrix.com. 05:16.040 --> 05:21.100 Then jmatrix.com responds with a valid server hello, but it contains our payload. 05:22.620 --> 05:30.240 Now, it would be ideal if we as an attacker could get that payload that jmatrix.com sent the victim, and then get the victim to send it somewhere internal. 05:30.360 --> 05:31.820 But that raises a question. 05:31.860 --> 05:34.680 Is there such a field that we can use to deliver these payloads? 05:35.480 --> 05:41.380 It might seem too good to be true, but TLS actually provides exactly that in the form of a session ID. 05:41.600 --> 05:44.160 Most clients even persist this field for later use. 05:44.500 --> 05:51.600 Session IDs are limited to 32 bytes, as you can see here, but depending on the implementation, you might have session tickets. 05:51.780 --> 05:56.460 This one is about 200 bytes, but these can be up to around 65 kilobytes. 05:57.100 --> 06:03.560 Both session IDs and session tickets are mechanisms for a TLS client to go, hey, remember that cryptographic key exchange we did earlier? 06:03.560 --> 06:05.940 Let's just keep using that key in this new connection. 06:06.260 --> 06:13.680 TLS 1.3 includes a slightly more complicated but similar mechanism called a pre-shared key identity, which pretty much does the same thing for our purposes. 06:13.680 --> 06:17.700 All of these are about optimization, since key exchanges can be time-consuming. 06:18.000 --> 06:29.300 But they provide a way for a server to tell whatever is connecting to it to persist some data for later use, almost like a cookie that lives in plain text. 06:29.300 --> 06:31.820 So it's perfect for the SSRF attack we're trying to do. 06:33.060 --> 06:35.020 Now, you might anticipate a problem here. 06:35.500 --> 06:41.620 Obviously, if you go to google.com and then to apple.com, those TLS sessions are going to have distinct data from each other. 06:41.620 --> 06:44.580 The same applies between google.com and localhost. 06:44.640 --> 06:50.940 So we have a bit of a problem getting the target server to reuse the session when communicating with something internal. 06:52.140 --> 06:58.680 But when I looked at both the TLS spec and popular session cache implementations, I saw this wasn't quite the case. 06:58.680 --> 07:02.900 I first went to the curl source code, since libcurl is used by a lot of PHP stuff. 07:03.160 --> 07:09.260 Sessions are keyed by the four properties here, which belong to this struct. 07:09.260 --> 07:11.740 As you can see, none of these are the IP address. 07:12.140 --> 07:17.520 This means that no matter what IP address jmatrix.com resolves to, curl will attempt to reuse the session. 07:17.720 --> 07:21.040 This happens to be how just about everything else caches TLS sessions. 07:21.260 --> 07:29.740 So if you establish a TLS session with jmatrix.com, whether it resolves to something on the internet or your own machine, your HTTPS client will not know the difference. 07:29.880 --> 07:36.240 Since I, as an attacker, own jmatrix.com, I can combine all of this TLS stuff with a concept called DNS rebinding. 07:37.580 --> 07:43.440 So I resolve the first request to 35.x.x.x, some server I control. 07:44.840 --> 07:50.340 Then the target server picks up the payload from 35.x.x.x in the form of a session ticket. 07:52.080 --> 07:56.660 After some time, I send the exact same URL, except the DNS entry has expired. 07:56.740 --> 08:04.160 Again, since I control the DNS server, this time around I resolve jmatrix.com to something internal that I wouldn't normally have access to from the outside. 08:04.160 --> 08:06.620 To simplify things, I'll just use localhost. 08:07.800 --> 08:12.560 Once the DNS query finishes, the target server then makes an internal request with that payload. 08:14.540 --> 08:25.220 So, like what I did in the opening demo, you can manipulate whatever you're attacking to send a TLS session ID, ticket, or PSK identity like this to itself. 08:25.340 --> 08:28.400 Here I'm showing a PSK identity that does a memcache insert. 08:29.880 --> 08:33.140 And here's that same TCP stream, but just displayed raw. 08:33.140 --> 08:38.300 This view shows the really important property of my approach to combining TLS with DNS rebinding. 08:38.300 --> 08:42.200 You can get arbitrary characters to be sent, which includes new lines. 08:42.260 --> 08:48.980 This is what allows you to interact with a bunch of local services like memcached, just like people did in the past with Gopher URLs. 08:50.300 --> 08:54.300 And here's an exchange more typical of what you'd use to do real damage. 08:54.320 --> 09:00.380 This is a value that in most Python memcached libraries will execute a shell command immediately once it's retrieved from a cache. 09:00.380 --> 09:08.280 This is because it's quite common in Python and other languages to use built-in serialization methods like pickle to convert values to something that can be stored. 09:08.520 --> 09:12.600 Unfortunately, built-in serialization methods are also common RCE targets. 09:12.620 --> 09:14.340 We'll get to exploitation later. 09:15.220 --> 09:19.120 But since this is pretty common, I ended up testing a fair amount of slides. 09:19.120 --> 09:26.920 I'll get to some specific ones in a second, but first I'll go over some testing infrastructure I set up, along with some tips for actually performing these kinds of exploits. 09:27.980 --> 09:30.240 So here are all the VMs I have on the internet. 09:30.280 --> 09:35.640 Say for maybe some IP whitelisting, they need to be on the open internet so that the target boxes can reach them. 09:35.840 --> 09:45.180 I've got a nameserver record pointing ssltest.jmatrix.com queries to the DNS server on the top left, which is just a Python script running on port 53. 09:45.180 --> 09:50.080 It alternates between answering with the IP of the custom TLS server and localhost. 09:50.680 --> 09:55.980 Then over there on the right, I have my custom TLS server, which is there to deliver my payloads via session tickets. 09:55.980 --> 09:59.120 I configure those by just updating values in Redis. 09:59.380 --> 10:07.740 But when testing it, it proved to be difficult to just go straight to localhost, because depending on what library is making the requests, I didn't know if SSL sessions were being cached. 10:09.180 --> 10:14.760 To address this, I sometimes just ended up adding another box to see if the payload was being persisted. 10:14.820 --> 10:20.880 If you're not familiar, Netcat can be set up to just listen for TCP connections and log the results. 10:20.880 --> 10:24.840 If I saw my payload making it into the logs, I knew the general approach could work. 10:25.120 --> 10:30.580 And then go back to the setup from the previous slide to get the server to start delivering the payload to itself on different ports. 10:31.940 --> 10:34.020 And here are the links to all that I actually used. 10:34.220 --> 10:40.380 A big chunk of the work was saved by just messing around with a pure Rust implementation of TLS, which is linked to here. 10:40.560 --> 10:42.960 A big thanks goes out to everyone mentioned in this slide. 10:43.040 --> 10:47.120 I wouldn't have had the time to do any of this work if not for the code they had put out there. 10:48.980 --> 10:55.460 But I know with everything I just claimed, I need to back things up by showing what actual real-world attacks can happen with this. 10:55.880 --> 10:59.800 I'll start off conceptually and then get to some examples along with another demo. 11:01.560 --> 11:03.120 But here's the mullot I've got. 11:03.400 --> 11:06.500 If you're in the middle of this diagram, you're generally vulnerable. 11:07.520 --> 11:13.780 This top part refers to when you can send something a URL and it will fetch that URL, but you don't know of a way to exploit that behavior. 11:13.780 --> 11:19.840 It's interesting because you would hope by now people would try to eliminate anything even remotely looking like SSRF. 11:19.880 --> 11:23.040 But this pattern is in fact required by a few popular specs. 11:23.260 --> 11:29.060 This is even common enough that I've seen bug bounties mention SSRF with no proven security impact as an excluded category. 11:29.300 --> 11:30.960 But that's what this talk is about. 11:30.960 --> 11:35.840 Adding the security impact when you have just something that is just almost SSRF. 11:37.040 --> 11:39.520 This one on the left sounds weird, but it's getting bigger. 11:39.520 --> 11:46.140 It's just a matter of if whatever the SSRF is going through to make requests has implemented this feature according to the latest spec. 11:48.080 --> 11:55.340 Unfortunately, there are quite a few HTTPS client libraries out there that support full-blown TLS session caching. 11:55.520 --> 11:58.020 Here are a few that you may be wondering about. 11:58.060 --> 12:02.340 I reported this as a vulnerability to some of these, but not all of the entries with yes. 12:02.520 --> 12:04.120 And I haven't seen much traction. 12:04.480 --> 12:10.180 Understandably, since fixing this requires diverging from the TLS spec, not a whole lot of people are eager to do so. 12:12.060 --> 12:15.500 The thing you run into the most trouble with is this last circle. 12:15.500 --> 12:19.600 Because you have to first do some recon to see what ports might be internally exposed on the server. 12:19.900 --> 12:25.920 And then you need to see if what's on those ports is permissive enough to keep the connection open past the opening bytes of the TLS client hello. 12:27.420 --> 12:29.960 Here are some examples of what you might be looking for there. 12:29.960 --> 12:33.020 It's far from complete, but here are the most notable entries. 12:34.660 --> 12:41.240 So now we'll go through a couple of exploitable examples of these TLS session-based SSRF vulnerabilities that I reported and got fixed. 12:42.580 --> 12:44.120 Here's one from uTrack. 12:44.120 --> 12:51.060 It's rated low because all I did was hit local SMTP, and I didn't try poking around to see if I could exploit the SMTP server further. 12:51.300 --> 12:56.680 It originated from uTrack's import feature, which normally wouldn't be easy to exploit, being blind SSRF. 12:57.780 --> 13:02.940 But with my rebinding code, here's a section from the TLS packet I got uTrack to send itself. 13:03.420 --> 13:10.740 This works because SMTP is a newline-delimited protocol, so just like memcache, you can embed working commands within a TLS ticket. 13:12.920 --> 13:14.880 And here's what came of that packet. 13:14.940 --> 13:21.500 This alone wasn't super severe because it's not signed quite right, and I didn't dig too deeply because I wanted to just report it ASAP. 13:21.500 --> 13:28.320 But it did get initiated by a TCP stream I got uTrack to send itself, so it's still pretty bad. 13:29.700 --> 13:31.640 Here's the one I reported in Nextcloud. 13:32.080 --> 13:34.220 The idea behind Nextcloud is a cool one. 13:34.220 --> 13:39.880 You set up a server and host files, and can even share them with other people's servers using a share URL like this. 13:41.100 --> 13:44.220 But as it turns out, you can include port numbers in the share URL. 13:44.220 --> 13:52.380 Now, since Nextcloud automatically puts HTTPS in there, normally there wouldn't be much SSRF potential, and it would have to be blind SSRF. 13:53.460 --> 13:56.080 But luckily I was able to do just that. 13:56.080 --> 14:02.140 So as a non-admin user, I could use my TLS for binding attack infrastructure to perform arbitrary writes on memcached. 14:04.360 --> 14:07.920 After reporting this and working through the options, there weren't any great ones. 14:07.920 --> 14:11.780 Because not all request libraries allow you to disable TLS sessions. 14:11.780 --> 14:16.720 In this case, libcurl has the option, but it isn't exposed by PHP request libraries such as Guzzle. 14:17.180 --> 14:25.360 In any case, adding a request timeout made the attack pretty much infeasible, assuming that people don't turn the DNS time to live values super low on their servers. 14:25.740 --> 14:29.960 Nextcloud was still generous enough to give me a bounty, even though in my opinion this is libcurl's fault. 14:32.000 --> 14:38.060 Since the last couple were just text slides, let's see a real demo now, from the perspective of a Django web developer getting phished. 14:38.060 --> 14:45.280 The DNS for binding part has been done in the past, but what's new here is combining DNS for binding with TLS session persistence to make it more powerful. 14:45.500 --> 14:50.440 In the past, you had to get people to visit a fully malicious page to get anywhere with DNS for binding. 14:50.640 --> 14:53.260 But with my work, you can do the same with just an image tag. 14:53.380 --> 14:58.120 Because of that, you can phish people into RCE, even if they just view an email, but don't click any links. 14:58.140 --> 14:59.240 Let's see that now. 15:00.860 --> 15:02.820 And now for the actual demo. 15:04.920 --> 15:09.580 So as you can see, we're pretending I'm a Django application developer running a typical app. 15:09.760 --> 15:11.700 If we look at the app, it's pretty simple. 15:11.700 --> 15:16.500 We can view a sloth emoji, but if we refresh too fast, the sloth needs to go to sleep. 15:16.940 --> 15:19.040 And now for what happens when I get phished. 15:19.300 --> 15:25.160 As you can see, I have received some perfectly legitimate emails, and I'm viewing them. 15:25.160 --> 15:33.080 Normally, these image tags should be benign, but after viewing them, my sloth now opens a calculator via shell command. 15:33.620 --> 15:39.580 That's because those image tags pointed to the domain with my binding setup, and a simple Python pickle payload. 15:44.110 --> 15:47.870 So here's most of the source code of the example app I created. 15:47.950 --> 15:59.950 You can pretty easily find apps that call django.core.cache or on GitHub, but I didn't want to demo against a specific one, because that seems a little mean, and I wouldn't want people launching actual phishing campaigns against such projects. 16:00.050 --> 16:02.030 It's a little tough to assign blame here. 16:02.190 --> 16:08.470 Nothing is really breaking from established safe practices, but it's clear that there's a potential to compromise real-world developer laptops. 16:09.930 --> 16:13.790 On that note, there's a lot more to do that I couldn't possibly do on my own. 16:14.330 --> 16:19.710 These slides will be available online if you want to return to this, and if you do anything interesting, I'd love to hear about it. 16:19.970 --> 16:23.110 There are certainly a lot more potential vulnerabilities to explore. 16:23.910 --> 16:27.550 So all of that is bad, but what can be done about it? 16:27.570 --> 16:30.790 Let's start from the root of the problem, TLS clients. 16:32.550 --> 16:35.450 Here's something I recommended to the Chromium team a while back. 16:35.650 --> 16:42.110 We still aren't in agreement about whether or not this should be fixed, and my report was pretty hastily written, which probably didn't help things. 16:42.210 --> 16:46.350 Essentially, I'd like to remove the ability for sessions to be resumed between servers. 16:47.350 --> 16:54.570 But while the Chromium team has marked this as a won't-fix, they did suggest an alternative that maintains support for resuming sessions across servers. 16:54.690 --> 16:59.190 They probably care about that more than I do, but we can at least have the best of both worlds. 16:59.390 --> 17:05.210 This last option of just partitioning based upon whether the address is something local is a pretty good compromise. 17:05.430 --> 17:09.890 It wouldn't affect performance, but would protect against most of the attack scenarios I can think of. 17:10.230 --> 17:18.590 But based on my experiences so far, I'm not hopeful that there's going to be much proactive effort on the part of software packages that make HTTPS requests. 17:18.590 --> 17:24.630 For most of the audience, it's more useful to know what's susceptible, and what layers of defense can be applied. 17:25.270 --> 17:30.270 However, with all of this worth taking a step back to consider, is TLS session resumption even worth it? 17:31.710 --> 17:35.510 There are other issues with it, a couple of which are shown here. 17:35.690 --> 17:44.010 I'm not going to make the case that the industry as a whole should give up on this optimization, but we should take a step back and see if this is a worthwhile trade-off. 17:44.050 --> 17:47.670 We may, just like in the previous slide, find that the costs can be eliminated. 17:47.670 --> 17:51.850 But if we find we can't tackle these problems, we need to weigh them against the benefits. 17:52.930 --> 17:54.570 And here's the main benefit. 17:54.570 --> 17:59.850 You get a sizable performance improvement on the small subset of packets that are TLS handshakes. 17:59.870 --> 18:03.150 You probably care about this whenever you're in the position of serving up TLS. 18:03.430 --> 18:09.250 Whatever percentage of CPU usage is due to TLS handshakes, you probably want to keep that low to minimize costs. 18:11.050 --> 18:13.090 But if you're like me, you don't care. 18:13.290 --> 18:19.970 I'm taking on all the security and privacy risk of session resumption, but the amount that it saves on my power bill is pretty minimal. 18:20.150 --> 18:28.250 Additionally, if you're running a web application, you might care about session resumption on incoming connections, but when you're making external requests, it's probably not in your interests. 18:28.450 --> 18:33.370 You're placing a lot of trust on random websites not to redirect you and on your local services. 18:33.570 --> 18:35.470 So how might you turn it off? 18:36.890 --> 18:38.210 Here's what I found. 18:38.210 --> 18:46.470 It's kind of sad when stuff forces you into this, but at least as an internet user, you can pick Firefox and disable session identifiers if you're paranoid about this stuff. 18:46.470 --> 18:49.230 If you're running a web application, you're a bit stuck, though. 18:49.310 --> 18:50.350 So what should you do? 18:51.530 --> 18:58.790 First off, it's a good idea in general to run your outbound requests through a proxy, especially if you're supporting webhooks or some other SSRF-prone feature. 18:58.870 --> 19:01.930 But also reconsider your infrastructure decisions. 19:02.050 --> 19:09.890 You'd be surprised to know just how much stuff is still set up in a way that is just one locally initiated TCP stream away from compromise. 19:11.190 --> 19:12.870 But here are the general takeaways. 19:12.870 --> 19:19.690 The thing I'd like to highlight is if you're just starting out in the offensive side of security, I'd encourage you to look at the random standards and browser features that come out. 19:19.750 --> 19:25.190 Even if you don't fully understand them at first, find something that seems interesting and start hacking around in Wireshark. 19:25.190 --> 19:32.690 I know I didn't understand TLS at first, and I still don't really understand it very much after all of this. 19:32.690 --> 19:37.510 But as an attacker, you really only need to understand the worst parts, and that's what this is all about. 19:39.270 --> 19:40.230 And thank you. 19:40.230 --> 19:42.810 Here's my Twitter and a link to Latakora, where I work. 19:42.810 --> 19:47.170 They're a great place to work and have been amazingly supportive of random research directions I take. 19:47.330 --> 19:48.390 Definitely take a look.