1 00:00:01,820 --> 00:00:06,280 I'm Raven, and it's nice to see a bunch of familiar folks, and... 2 00:00:07,040 --> 00:00:08,340 Is that better? 3 00:00:08,840 --> 00:00:11,140 Alright, I will eat the microphone. 4 00:00:14,120 --> 00:00:16,140 Not like that. 5 00:00:17,400 --> 00:00:18,420 Alright. 6 00:00:21,500 --> 00:00:33,560 I'm here to talk about penetration testing for background networks, and I have these really interesting, great slides, and then, Wednesday, some guy came and revolutionized my field, so... 7 00:00:33,560 --> 00:00:38,060 I haven't gone to any parties this year, and I've done a lot of late-night rewriting. 8 00:00:38,280 --> 00:00:40,740 So, thank you, Michael Lin, and... 9 00:00:40,740 --> 00:00:42,620 The reason we were waiting... 10 00:00:43,420 --> 00:00:45,720 is that my lawyer isn't here yet. 11 00:00:46,820 --> 00:00:47,800 Oh, she's here. 12 00:00:47,800 --> 00:00:48,620 Fantastic. 13 00:00:49,740 --> 00:01:09,520 So, the first thing that you really need to address when you're looking at penetration testing for background networks is that there's a pretty severe culture gap between ISP folks and the hacker folks, and there's not really similar views on disclosure or vulnerabilities or anything like that, 14 00:01:09,520 --> 00:01:11,460 and that can be kind of problematic. 15 00:01:12,500 --> 00:01:16,300 So, how many of you do work or have worked for an ISP? 16 00:01:16,300 --> 00:01:17,320 Show of hands? 17 00:01:17,440 --> 00:01:20,300 Okay, you guys have probably run into this. 18 00:01:20,300 --> 00:01:24,480 How many of you were in the routing department rather than the security department? 19 00:01:24,480 --> 00:01:25,260 Show of hands? 20 00:01:26,000 --> 00:01:27,200 Okay. 21 00:01:28,900 --> 00:01:30,880 I used to work for ISPs. 22 00:01:30,880 --> 00:01:44,700 I spent a bunch of years doing senior ISP background engineering and things like that, and I eventually left for a more security-focused role because I got really frustrated with just the differences of opinion. 23 00:01:44,880 --> 00:01:53,820 A lot of ISP engineers, when they look at vulnerabilities, they tend to have a very, oh, well, no one's going to do that sort of view of things. 24 00:01:53,820 --> 00:02:02,280 And their views on actual attacks, if they don't see it in the wild, a lot of times they're very unwilling to actually treat something as a real problem. 25 00:02:02,560 --> 00:02:07,520 And so it takes something like, well, Wednesday to make people sit up and take notice. 26 00:02:08,360 --> 00:02:23,260 ISP engineers, in general, tend to go for a limited tiered disclosure method, so when something is found, you know, and people like that hear first, and then Cisco's next favorite children hear second, and so on and so on. 27 00:02:23,260 --> 00:02:29,100 And so if you have their gear, which are a small network near the edge, you will hear late, if at all. 28 00:02:29,380 --> 00:02:32,340 And most ISP engineers think that this is good. 29 00:02:32,360 --> 00:02:43,300 They believe that, you know, catching the core is the most important thing, and then moving outwards from there will release things in a more slow fashion and minimize the window of vulnerability. 30 00:02:43,300 --> 00:02:47,040 I have a feeling that most of you don't think this is a good idea. 31 00:02:48,900 --> 00:03:03,140 So when you're trying to do pen testing on a backbone, you're going to have to work with these ISP engineers, and you have to sort of be a little bit gentle with them, because otherwise they'll immediately say, evil hacksaw, and stop listening. 32 00:03:03,480 --> 00:03:20,220 So for maximum efficiency and security, generally, when you're looking at testing a backbone, I generally recommend doing it in a white box fashion rather than a black box, because you can do all sorts of recon and data gathering and such when you're actually doing it in a black box fashion, 33 00:03:20,220 --> 00:03:26,620 but so many of the vulnerabilities that there are in backbone gear are denial of service that people get a little pissed when you take down their network. 34 00:03:27,220 --> 00:03:39,300 And even if you've explained this to them, you've signed the contracts beforehand, you've made sure that you're legally covered, and say, no, look, I could knock over your router, they don't know what that means a lot of the time. 35 00:03:39,300 --> 00:03:46,520 And so then they get ticked off that their backup didn't go through, or suddenly they can't get to their router that you knocked over because it's offline. 36 00:03:46,760 --> 00:03:54,780 And so you want to be really careful when you're setting up your gig if you're going to pentest a backbone, because a lot of people don't really understand their ramifications. 37 00:03:55,160 --> 00:04:00,120 So when I do this, I generally go with a white box sort of testing methodology. 38 00:04:02,060 --> 00:04:05,120 So, when you're doing it, it's just like any other pentest. 39 00:04:05,120 --> 00:04:12,600 You do your initial block of recon, you look at the registrars for the address blocks and the autonomous systems that they have, and any other data that's associated. 40 00:04:12,600 --> 00:04:20,640 You can often get contact information for social engineering, you can pull all sorts of information about the network that you are intending to test. 41 00:04:21,040 --> 00:04:32,020 By doing a lot of these standard pentest techniques, you can identify hosts on the edge, but looking at the other IP addresses in their net blocks is more likely to give you addresses that they're using on their backbone. 42 00:04:32,020 --> 00:04:39,280 Now, a lot of people are, or should be, filtering information on the backbone from things past the edge of their network. 43 00:04:39,300 --> 00:04:47,480 In a perfect world, you shouldn't be able to hit the routers themselves from outside of the network, unless you are on a management station or something like that. 44 00:04:47,480 --> 00:04:50,620 It's just traffic should go through, and you shouldn't be able to target. 45 00:04:51,060 --> 00:04:54,260 This is really, really not a perfect world. 46 00:04:54,860 --> 00:05:02,800 A lot of ISPs fail at that level of filtering, and we'll look at some stuff from the routing protocol security group a bit later on that will show you some data on that. 47 00:05:03,580 --> 00:05:04,620 It is worthwhile. 48 00:05:04,620 --> 00:05:12,760 There are many public route servers, and it's worthwhile to look on those and internet maps to try and determine who is connected to whom and where the peering is. 49 00:05:12,760 --> 00:05:15,060 A lot of peering happens at public exchange points. 50 00:05:15,060 --> 00:05:16,420 A lot of it happens elsewhere. 51 00:05:16,420 --> 00:05:22,400 You have private interconnects between different ISPs, and those will be crucial points when you're actually mapping your network. 52 00:05:23,240 --> 00:05:31,800 You can often also, on route servers, get information about their BGP routing policy and try and figure out what they accept and what they send to where. 53 00:05:31,840 --> 00:05:34,100 Community strings will often be viewable. 54 00:05:34,780 --> 00:05:38,720 Important information about BGP and which routes are preferred. 55 00:05:38,740 --> 00:05:43,160 And there are many public route servers over there that you can look at things from their perspective. 56 00:05:43,280 --> 00:05:53,140 So if the network that you're analyzing is not one that is directly connected to a route server, looking through two or three will give you several points of view into how to look at it on the net. 57 00:05:53,720 --> 00:05:55,780 Whatever you get, dump it into Google. 58 00:05:55,780 --> 00:05:59,340 And if you get good results in that, dump them right back into Google. 59 00:05:59,340 --> 00:06:02,340 And that will really help you build a profile of the network. 60 00:06:02,620 --> 00:06:10,820 I believe it's really important not to just take the network maps that their admins give you in Visio or what have you, but to also try and build your own map non-destructively. 61 00:06:10,900 --> 00:06:12,680 Their engineers aren't aware of everything. 62 00:06:12,680 --> 00:06:14,520 Often they're not aware of much. 63 00:06:16,320 --> 00:06:20,900 And that can be really important when you're looking at things from a pentester's point of view. 64 00:06:20,980 --> 00:06:25,400 If they don't tell you about a link and you don't look and you don't find it, you're not going to test that. 65 00:06:26,020 --> 00:06:31,020 Also, a lot of ISP engineers aren't terribly security aware. 66 00:06:31,320 --> 00:06:36,320 And so they'll post a mailing list and say, here's my config, gee, this isn't working guys, what do you think? 67 00:06:36,320 --> 00:06:50,120 You can get all sorts of information about what they're running, how they're running it, what kind of gear they have, and all of that can be dumped into your favorite friendly neighborhood vulnerability database to build a likely profile of attacks and exploits that you could use against their network. 68 00:06:52,200 --> 00:06:56,940 Now, a lot of the backbone is built on gear from a very few vendors. 69 00:06:57,160 --> 00:07:01,080 And these vendors are generally not fans of full disclosure. 70 00:07:01,660 --> 00:07:07,920 Cisco and Juniper both have on their networks things that are behind the front end of their webpage. 71 00:07:07,920 --> 00:07:12,860 You have to have a login, in many cases you have to have a contract in order to see details. 72 00:07:12,860 --> 00:07:17,840 Cisco does release some vulnerability information publicly, but there's a lot that they don't. 73 00:07:17,840 --> 00:07:26,720 Juniper releases almost nothing publicly, I think I've seen like six advisories from them ever that didn't require some sort of back end acknowledgement. 74 00:07:27,000 --> 00:07:32,540 And so they believe that by doing this they are helping to keep it out of the hands of script kitties. 75 00:07:32,680 --> 00:07:43,000 If you have a contract with them, or if you have a login to their site or something like that, you will have a much greater database of information that's available to you when you're trying to determine what to test for. 76 00:07:43,700 --> 00:07:52,360 We're going to talk a lot about vendor negotiation and dealing with your vendors in order to secure the network as best as possible. 77 00:07:53,840 --> 00:08:02,060 If you are working for a company, they may allow you to use their CCO or Juniper logins to access the bug databases and look for vulnerability information. 78 00:08:02,640 --> 00:08:05,840 Also, don't forget that the backbone is more than just routers. 79 00:08:05,840 --> 00:08:10,760 There are switches, there's lots of stuff that gets done on Layer 2, and a lot of people just kind of ignore that. 80 00:08:10,760 --> 00:08:12,940 So test through your switch gear also. 81 00:08:17,380 --> 00:08:22,060 Cisco in particular has really complex code trains. 82 00:08:22,200 --> 00:08:33,320 iOS has a zillion different branches which support different features and for different hardware platforms and such, and you will need to be aware of what you are running in the network you're testing. 83 00:08:33,340 --> 00:08:38,240 If they've left CDP on or something stupid like that, then you can get them to tell you. 84 00:08:38,240 --> 00:08:41,600 If their engineers are friendly, you can get them to tell you. 85 00:08:41,600 --> 00:08:47,620 You can log into all the routers if they're giving you that level of access and do a chauffeur and pull details on that. 86 00:08:47,620 --> 00:08:52,560 But something that is vulnerable in one train may or may not be implemented in other trains. 87 00:08:53,100 --> 00:08:56,600 It's worth looking with a broad brush to start off with. 88 00:08:56,800 --> 00:09:05,600 Something might be vulnerable and then going, oh, okay, it's not a vulnerable version is much better than saying, ah, well, it's only this one version, let's not test that. 89 00:09:06,140 --> 00:09:10,600 It's also worthwhile in looking at code trains and where they're derived from. 90 00:09:10,600 --> 00:09:19,600 Code is forked all the time, and sometimes you'll come up with an error that was in one version and years and years down the road, oh, look, it's still there. 91 00:09:19,600 --> 00:09:29,800 There was a BSD vulnerability in Telnet a while back that whacked half the stuff on the Internet, almost regardless of operating system, because so much was derived from that old code base. 92 00:09:29,800 --> 00:09:34,060 And I have a link here to the vulnerability in Cisco's CatOS boxes. 93 00:09:34,060 --> 00:09:39,660 We use that same bit of code and we're vulnerable to that same bit of exploit. 94 00:09:40,180 --> 00:09:43,380 There are some scanners that will incorporate these checks for you. 95 00:09:43,380 --> 00:09:49,260 I know Nessus has some Cisco vulnerability information in it, but a lot of it isn't automated yet. 96 00:09:49,500 --> 00:10:01,040 And because of the great potential for denial of service, you can see why a lot of players got real thrilled to have that run against them and why a lot of people haven't been too eager to put it in the exploit tools. 97 00:10:01,040 --> 00:10:10,420 So this is something that, I think, pen testing like 5 or 10 years ago, you're going to have to do a lot of this by hand in order to make sure that you've got everything. 98 00:10:12,660 --> 00:10:17,280 As with any other pen test, failure paths and trust relationships are important. 99 00:10:17,340 --> 00:10:22,940 When you look at their architecture, look for redundancy, look for network robustness, and look for security. 100 00:10:23,620 --> 00:10:30,900 One thing that I think is really important when you're pen testing a backbone is to make sure that availability is a factor. 101 00:10:30,900 --> 00:10:38,200 I mean, denial of service is such a big deal for them, and yet so many people build networks with one path where a link goes down and you're screwed. 102 00:10:38,200 --> 00:10:43,440 Or they have two lines terminating into one card on one router, and a hardware failure will cause problems. 103 00:10:43,500 --> 00:10:49,700 I think, as security analysts, it's our job to point these things out and go, oh, you've got a single point of failure if someone knocks this out. 104 00:10:49,700 --> 00:10:53,200 If there's an exploit, if you have a hardware failure, you have an issue. 105 00:10:53,240 --> 00:10:57,300 I personally try and include things like that when I present a report to the client. 106 00:10:57,920 --> 00:11:02,780 Also, check for the authentication means and see whether those have redundancy. 107 00:11:02,780 --> 00:11:09,760 Just as with some of the SSL versions, you know, if you can force it back to the weakest point, then you can exploit that. 108 00:11:09,760 --> 00:11:18,840 There was a vulnerability on a RADIUS server in Cisco's implementation of RADIUS where you could give it several ways of authenticating and none was an option. 109 00:11:18,840 --> 00:11:21,540 So if you can just say, oh, well, I don't want to authenticate, no. 110 00:11:21,540 --> 00:11:23,620 You know, oh, okay then. 111 00:11:24,120 --> 00:11:26,040 Things like that are kind of bad. 112 00:11:29,500 --> 00:11:38,100 So with things like that, check for what the weakest way is because you know your attacker is going to take the weakest way and if Pentester is still awake. 113 00:11:38,980 --> 00:11:41,760 Once you're authenticated, is trust transitive? 114 00:11:41,760 --> 00:11:45,400 You know, if I get a login to that RADIUS server, what else does that give me? 115 00:11:45,480 --> 00:11:47,620 It's important to look at that. 116 00:11:47,620 --> 00:11:55,500 And so as soon as you get every new level of access, every new point, if you try and map the network again from that point of view, you may find extra stuff. 117 00:11:58,900 --> 00:12:06,060 One of the things you'll see a lot and one of the ways I've had most success in testing backbone networks is authentication servers. 118 00:12:06,100 --> 00:12:10,740 If you have a centralized authentication server, that's a really good point to start at. 119 00:12:10,740 --> 00:12:16,700 If you look for a login to that, you can sniff traffic and get someone's login if they are foolish enough not to encrypt it. 120 00:12:16,700 --> 00:12:24,380 If you can own one of the hosts that they use for management stations, you can then grab it before it's encrypted and sent on the wire with a keystroke logger or something like it. 121 00:12:24,460 --> 00:12:27,240 And that will often give you the keys to the kingdom. 122 00:12:27,240 --> 00:12:35,160 Very many places don't think to harden their authentication servers, and so that's a really common point of failure. 123 00:12:35,620 --> 00:12:50,480 Another thing that I have successfully done is if they have fallback mechanisms where they have a local login in case their RADIUS or their CACX or their whatever server is down, well, if you can knock that server offline and then brute force the local password, 124 00:12:50,480 --> 00:12:52,120 that will also get you in. 125 00:12:52,340 --> 00:13:01,680 And once you brute force the local password, depending on how they're storing it, like for a Cisco box, they use a 7 encoding, which it's not encryption, it's encoding. 126 00:13:01,680 --> 00:13:05,240 It's symmetric, it's easily reversible, alcrypto.co.uk. 127 00:13:05,240 --> 00:13:10,260 Cisco has an online decryptor, although you may wonder where that's going. 128 00:13:10,480 --> 00:13:14,620 So they also have programs you can download and run locally. 129 00:13:14,640 --> 00:13:22,920 But a lot of the passwords that are saved in the configs, you can just throw them in the internet, get the password out, and they may be using that in multiple places. 130 00:13:22,920 --> 00:13:29,860 But a lot of places do not do password management well, and so a local login at one place will get you a local login at another place. 131 00:13:29,860 --> 00:13:33,780 And as long as that central authentication server is down, there you go. 132 00:13:34,600 --> 00:13:39,980 Obviously, once you've logged into the boxes, if you have enabled, you can change that. 133 00:13:40,620 --> 00:13:42,500 But that will let them know you're there. 134 00:13:42,500 --> 00:13:44,220 You know, please be nice. 135 00:13:47,700 --> 00:13:56,400 Also, man-in-the-middling against the authentication server is usually a fairly successful technique, if you can get access to do that. 136 00:13:59,760 --> 00:14:03,600 It shouldn't need mentioning, but physical security is really important. 137 00:14:03,600 --> 00:14:05,580 A lot of places forget this. 138 00:14:05,620 --> 00:14:12,060 Everywhere you have a data center, and especially everywhere you're peering, please have good physical security. 139 00:14:12,280 --> 00:14:15,420 Please audit and make sure you have good physical security. 140 00:14:15,420 --> 00:14:20,380 Please don't let me walk into Macy's with a switch under my arm and a laptop under the other one and plug in. 141 00:14:20,380 --> 00:14:21,780 Yeah, bad. 142 00:14:23,180 --> 00:14:29,100 A lot of the time, if you have local access, local physical access, that will get you a lot. 143 00:14:29,340 --> 00:14:36,820 And many of the places, they have a very clear profile of what they're looking for. 144 00:14:36,980 --> 00:14:48,740 So, you know, if I was 40 and had a beard and came in, they might be a little more suspicious of me than if I'm 29 and got my eyelashes and go, gee, I don't know, I'm just supposed to go look at this router. 145 00:14:49,160 --> 00:14:58,600 They have poorly placed trust relationships in terms of local access, and if you want to take a social engineering approach to that, then that can work to your advantage. 146 00:14:59,240 --> 00:15:02,040 If you don't have physical security, you're screwed. 147 00:15:04,650 --> 00:15:14,330 One of the things that I've seen fairly often that's kind of appalling is how many people will leak data about the processes of the core of their network to the edge. 148 00:15:14,570 --> 00:15:22,410 So they'll have EIGRP as their internal routing protocol, and they won't turn it off for interfaces that face out towards the edge of their network. 149 00:15:22,410 --> 00:15:28,190 They just sort of send those multicast, hey, hey, who wants to be my neighbor, out to anyone. 150 00:15:28,210 --> 00:15:36,270 And, well, you shouldn't let people at the edge of your network establish a neighbor relationship and feed you a bunch of route data, which may be bad. 151 00:15:37,370 --> 00:15:39,210 Also, CDP. 152 00:15:39,510 --> 00:15:46,990 People will leave it on by default, they won't turn it off, and it will happily send you all kinds of information about the connected device. 153 00:15:46,990 --> 00:15:54,190 It will tell you what it is, what version of the operating system it's running, what sort of hardware it has, device ID, all kinds of stuff. 154 00:15:54,550 --> 00:15:59,530 And Cisco has designed this so that it's very useful when you're troubleshooting. 155 00:15:59,630 --> 00:16:09,930 I have had cases where I wanted to see what was on the other end of the link, and we weren't sure that the person at the peering site or the central office had us plugged into the right ports. 156 00:16:10,010 --> 00:16:16,290 So the other side turns on CDP, and you go, whoa, I'm not supposed to be connected to a 7206 VXR. 157 00:16:16,290 --> 00:16:17,670 Yeah, that's wrong. 158 00:16:17,670 --> 00:16:19,830 So when troubleshooting, it's a good tool. 159 00:16:19,830 --> 00:16:23,070 But when you're not troubleshooting, turn it off. 160 00:16:24,010 --> 00:16:24,870 Switching data. 161 00:16:24,870 --> 00:16:28,950 Also, switch management protocols, things like that, will be happily sent out. 162 00:16:28,950 --> 00:16:32,150 Hey, welcome to my spanning tree, who wants to be my root bridge? 163 00:16:32,150 --> 00:16:36,170 These things should not be seen by the end users, and they often are. 164 00:16:36,170 --> 00:16:38,470 So that's something for you pen testers to look for. 165 00:16:40,010 --> 00:16:43,150 If you can see the data, you can send things back. 166 00:16:43,150 --> 00:16:51,210 You can spoof packets, you can send real data, you can feed them information on who's supposed to be bridging to what, or who's supposed to be routing to whom. 167 00:16:51,210 --> 00:17:02,570 If you can get a more preferred route and inject it, then that will, depending on the implementation, of course, it may be able to take over the route that's actually on the legitimate internet. 168 00:17:03,690 --> 00:17:11,130 A lot of this is... when you're speaking internal routing protocols rather than BGP, it's a lot easier to deal with on the local net. 169 00:17:11,130 --> 00:17:17,730 BGP has its own separate set of rules because of how it's handled between providers and how you set up neighbor and peering relationships. 170 00:17:17,910 --> 00:17:21,570 But on the local network, it's often not that hard. 171 00:17:21,570 --> 00:17:30,030 If you want to do a really sort of not high-skilled but very effective DOS attack, you can tear down links with spoof packets. 172 00:17:30,030 --> 00:17:39,790 A lot of the time, they don't have authentication on there, and so many routing protocols support the capacity for even something as simple as an mb5 hash. 173 00:17:39,870 --> 00:17:44,450 It's often called BGP password, although many other protocols also support the same thing. 174 00:17:44,770 --> 00:17:50,410 I know you guys have heard about mb5 fairly recently, but, you know, it's still a whole lot better than nothing. 175 00:17:55,260 --> 00:18:07,240 Similar to the theory of rogue access points, if you have physical access to the site, you can just walk right in with a router and say, yeah, I'm supposed to install this, and some places aren't very good about saying, who the hell are you, why didn't I know you were coming, 176 00:18:07,240 --> 00:18:08,300 you know, go home. 177 00:18:08,620 --> 00:18:14,280 And you can plug in a router, you can put it into the peering switch, you can just, you know, start snipping traffic. 178 00:18:14,280 --> 00:18:21,100 If you can manage to get the switch ports set promiscuous or something like that, then you can see what they're speaking and start to speak it too. 179 00:18:21,440 --> 00:18:23,280 This also is obviously bad. 180 00:18:24,860 --> 00:18:37,660 Depending on what your contract is with the client, you can even go in there and say, oh, yeah, you know, I'm from this company, I'm supposed to work on their router, yank out theirs, put in yours with the configuration of whatever you want, and, hey, 181 00:18:37,660 --> 00:18:41,900 as long as you pass it through to them, they may not even notice except for the ten minutes of reboot time. 182 00:18:44,410 --> 00:18:48,330 A lot of places do not have good passwords. 183 00:18:48,370 --> 00:18:50,570 It's better than it used to be because of spammers. 184 00:18:50,570 --> 00:18:58,890 There have been cases of spammers taking over Cisco routers and then using the telnet function to send spam out of the router itself one character at a time. 185 00:18:59,030 --> 00:19:01,570 So, yeah, bad. 186 00:19:02,050 --> 00:19:08,310 But a lot of places have gotten better about not having, at least pathetically, default passwords. 187 00:19:08,690 --> 00:19:17,490 However, if I get called in and someone's locked out of their router, I always give Cisco Cisco a try, you know, maybe one time in four it works, and just, it shouldn't. 188 00:19:18,490 --> 00:19:21,710 You can redirect traffic through a GRE tunnel. 189 00:19:21,710 --> 00:19:29,190 I've seen that in the wild, and people were a little bit confused about what was going on there, and said, I don't remember setting that up. 190 00:19:29,190 --> 00:19:30,470 Bob, did you set that up? 191 00:19:30,470 --> 00:19:37,350 And meanwhile, it's terminating in, you know, Moldavia or somewhere, and there go all your packets. 192 00:19:38,990 --> 00:19:53,790 Another serious concern is that, especially when you're speaking BGP, you may be able to advertise additional net blocks out to the net, and if your route is preferred to other people's routes, you've just hijacked a net block. 193 00:19:54,170 --> 00:19:56,750 And this happened. 194 00:19:56,850 --> 00:20:03,790 There was a bunch of attackers that set up a site through an ISP that wasn't doing good security checks about who actually owned that IP space. 195 00:20:03,790 --> 00:20:17,250 They announced this block out to the world, and their route was preferred in most places because of the way that they did it and how their provider was set up, and they happily spent about a week sending out spam and sending out worms instead of a botnet, 196 00:20:17,250 --> 00:20:19,090 and were owning boxes left and right. 197 00:20:19,090 --> 00:20:23,510 Meanwhile, the legitimate users of that net block can't even get online because no one's routing to them. 198 00:20:23,510 --> 00:20:26,370 Everyone's routing to the own site. 199 00:20:26,450 --> 00:20:34,710 And so, when that was finally resolved at the ISP level, then the original owners are really left holding the bag there. 200 00:20:34,710 --> 00:20:42,530 Hi, you're on every blacklist on the planet, you have a whole bunch of angry people pounding down the door at your abuse desk, and you were offline for a week. 201 00:20:42,530 --> 00:20:52,110 And because some ISPs do not do good checking when they're hooking up a new client of do you really own this block, is this legitimate, that can happen, and it shouldn't. 202 00:20:52,110 --> 00:20:58,490 There are some really good people working ISP security that are cooperating between ISPs to try and prevent that. 203 00:20:58,610 --> 00:21:00,150 And, you know, more power to them. 204 00:21:00,150 --> 00:21:01,430 We need more of them. 205 00:21:04,290 --> 00:21:17,830 A great place to start when looking at the actual configurations of the backbone devices is to look at the secure BGP template, iOS template, Junos template, and Junos BGP template that the Cymru people do. 206 00:21:17,830 --> 00:21:20,690 These are very, very well done documents. 207 00:21:20,690 --> 00:21:27,490 They have bogon lists so that you cannot accept routes from IP addresses that haven't been allocated or don't exist. 208 00:21:27,490 --> 00:21:38,230 They do strong filtering so that you don't have, like, you know, RFC 1918 addresses speaking to you, you shouldn't allow people to advertise, you know, your own net blocks to you, for example, things like that. 209 00:21:38,230 --> 00:21:42,310 They have good checking for the strongest possible authentication methods. 210 00:21:42,310 --> 00:21:51,730 They try and turn off things like, oh, hey, let's speak finger, you know, ancient insecure daemons that you have no business really needing to run on your major backbone routers. 211 00:21:51,730 --> 00:21:53,050 I can't recommend these enough. 212 00:21:53,050 --> 00:22:01,330 Those guys do really good work, and it's also handy for a beginning place to start with an engagement if you're doing a white box design analysis. 213 00:22:03,010 --> 00:22:05,590 Peering security is particularly important. 214 00:22:05,910 --> 00:22:12,050 When you look at how someone is peering, look at the routes they're willing to accept and look at the routes that they're willing to send out. 215 00:22:12,410 --> 00:22:20,870 There was a case several years ago where someone actually advertised 10.0.0.0 to the whole internet and claimed that they owned it. 216 00:22:20,870 --> 00:22:23,910 And that got a lot of people a little bit irate. 217 00:22:24,130 --> 00:22:29,630 It broke a lot of people's private network implementations, and it broke a lot of VPN stuff. 218 00:22:30,530 --> 00:22:32,190 So that's worth checking. 219 00:22:32,310 --> 00:22:34,390 Again, route servers are your friend. 220 00:22:34,390 --> 00:22:38,250 And when you do peering changes, make sure you're doing good authentication. 221 00:22:38,250 --> 00:22:45,870 You don't want just anyone to call you up and say, Hi, I'm Raven from Uunet, and yeah, I want you to change this config. 222 00:22:45,870 --> 00:22:50,930 Can you take out this net block of this guy who, you know, stole my girlfriend last week? 223 00:22:51,430 --> 00:23:01,450 You want actual passwords, some sort of validation method to make sure the person who you're talking to to make these major affecting changes actually is the person that they claim to be. 224 00:23:02,030 --> 00:23:07,710 Also, make sure that the machines that you use for your network policy and your network admin are well secured. 225 00:23:07,730 --> 00:23:14,310 I have worked at a job where we had a couple of management stations, and they were Windows boxes, and wow, they got owned. 226 00:23:15,090 --> 00:23:16,750 That's really dumb. 227 00:23:16,750 --> 00:23:22,490 If you have only a few machines that are allowed to access these crucial backbone devices, make sure they're hardened. 228 00:23:22,490 --> 00:23:23,490 Make sure they're patched. 229 00:23:23,490 --> 00:23:26,450 Make sure that people can't get onto them and, you know... 230 00:23:28,370 --> 00:23:42,510 In that case, I think what happened was these machines that we did backbone stuff from were the same machines that we did abuse tests from, and so you're investigating an abuse case, and oh, our user's surfing porn, go to the porn site, porn site's full of malware, 231 00:23:42,510 --> 00:23:45,010 mandated corporate IE, boom. 232 00:23:45,390 --> 00:23:46,890 Don't let that happen. 233 00:23:49,030 --> 00:23:51,710 And filtering is also very important. 234 00:23:51,710 --> 00:23:57,390 Not just what you allow to your router for management traffic, but also what you allow as far as routes. 235 00:23:57,710 --> 00:24:03,890 You want to make sure that you do not allow internal routes that should stay internal to your network outside to the world. 236 00:24:03,890 --> 00:24:08,690 Yes, aggregation happens, but more precise routes are often preferred, and that can be a problem. 237 00:24:08,690 --> 00:24:16,510 It bloats the routing tables, it uses up memory, and it can be revealing more details about the interior of your network than you wanted. 238 00:24:17,090 --> 00:24:24,210 The RPSec Working Group did a small survey of people in the field, and they surveyed people who were peering at an exchange point. 239 00:24:24,210 --> 00:24:29,070 So these are some of the more aware, more security conscious people out there. 240 00:24:29,170 --> 00:24:32,090 And they still weren't using everything that was available. 241 00:24:32,710 --> 00:24:37,390 This may be because, for lack of a better term, ISP culture. 242 00:24:38,090 --> 00:24:47,950 They view change management in the time that it takes to do good password control and to change all these policies as more of a threat than the security implications. 243 00:24:48,110 --> 00:24:59,250 And so you've got to keep track of who's allowed to do what, and who's filtering where, and, you know, oh, we screwed up our peering list, and , you know, okay, now there's downtime because I wasn't allowing your route, or I had too many prefixes. 244 00:24:59,290 --> 00:25:05,070 And so, you know, they think that the hassle of management is often not worth the additional security. 245 00:25:05,390 --> 00:25:08,750 And from an ISP point of view, I can see why they take that approach. 246 00:25:08,750 --> 00:25:15,490 When you're driven by the bottom line, and you're a network engineer with too much work and too little time, it can be difficult to deal with these sorts of things. 247 00:25:15,630 --> 00:25:22,410 Nevertheless, I think this is going to become increasingly important as more attention focuses on the backbone as, you know, started this week. 248 00:25:24,050 --> 00:25:25,790 The RPSEC site security. 249 00:25:26,630 --> 00:25:32,110 Many people are actually doing an incoming prefix filter for each BGP customer, which is good. 250 00:25:32,110 --> 00:25:35,070 You should only allow people to advertise their own netblocks to you. 251 00:25:35,070 --> 00:25:40,470 You don't want them to start advertising your netblocks, or, you know, someone else's netblocks, or that guy who stole your girlfriend's netblocks. 252 00:25:40,470 --> 00:25:43,030 That can get you in legal hot water. 253 00:25:43,030 --> 00:25:47,810 A lot of them also are checking for incoming autonomous system paths. 254 00:25:47,890 --> 00:25:51,750 A lot of people use community strings in order to control this sort of thing. 255 00:25:51,830 --> 00:25:59,450 However, very few people are using an outgoing AS path filter, and in that case, it really depends on who you are. 256 00:25:59,470 --> 00:26:08,110 If you are a small ISP near the edge of the network, you have your AS, and you have a few others that are your customers, that may actually be a good idea. 257 00:26:08,130 --> 00:26:17,070 If you are a tier 1, and you have every AS in the world flowing through you, that can be a real change management challenge, and probably operationally unwise. 258 00:26:18,910 --> 00:26:20,850 Outgoing updates towards your peers. 259 00:26:20,890 --> 00:26:23,670 A lot of people use communities on that, but not everyone. 260 00:26:23,730 --> 00:26:30,050 And there is the URL here for the actual survey, if you want to look at the results from the RPSEC group. 261 00:26:30,050 --> 00:26:31,250 I didn't put them all up. 262 00:26:32,450 --> 00:26:37,430 So, when you're looking at actually doing a pentest, where do you get your backbone vulnerabilities? 263 00:26:37,890 --> 00:26:39,490 They're not all in the scanning tools. 264 00:26:39,490 --> 00:26:40,790 We've established that earlier. 265 00:26:40,790 --> 00:26:44,950 And Cisco and Juniper may have you log into their site to actually look at it. 266 00:26:45,070 --> 00:26:47,730 Mailing lists will have public vulnerability announcements. 267 00:26:47,790 --> 00:26:55,010 I've often found that it is a good idea to check the mailing list against the vendor site, because, hey, people change their webpages all the time. 268 00:26:55,030 --> 00:26:56,950 Mailing list archives are usually more static. 269 00:26:56,950 --> 00:26:59,750 You can at least see what the first release was of something. 270 00:26:59,810 --> 00:27:04,570 We saw this happen this week, when Cisco announced their IPv6 vulnerability. 271 00:27:04,790 --> 00:27:07,310 And if you look on their website now, it's 1.1. 272 00:27:07,310 --> 00:27:09,690 The original one that went full disclosure is 1.0. 273 00:27:09,690 --> 00:27:11,230 There have been some changes made. 274 00:27:11,830 --> 00:27:15,110 And it is valuable to Pentester to be able to see both versions. 275 00:27:15,370 --> 00:27:22,990 There's been, in the last five years, increasing interest in this field by hackers who don't come from an ISP background. 276 00:27:23,570 --> 00:27:31,350 FX and Paul Watson and Mike Lynn have all made valuable and important contributions that have really caused a bit of uproar in the ISP community. 277 00:27:31,570 --> 00:27:35,770 And after this week's uproar, I think there will be a whole lot more people looking for it. 278 00:27:39,140 --> 00:27:44,700 So, FX has done a lot of stuff in this field that is really useful. 279 00:27:45,040 --> 00:27:55,500 There is a EIGRP Spoof Packet Neighbor Saturation bug, where essentially, if you send it more than 256 neighbors, then it just falls over. 280 00:27:56,540 --> 00:27:59,140 EIGRP works by sending out multitasks. 281 00:27:59,140 --> 00:27:59,940 Hey, here I am! 282 00:27:59,940 --> 00:28:01,240 Who wants to be my neighbor? 283 00:28:01,360 --> 00:28:12,700 And so if you say, me, and me, and me, and me, and me, you can essentially overflow the amount of memory that it allocates for that, and, you know, it's at least a denial of service. 284 00:28:12,740 --> 00:28:14,000 It could be worse. 285 00:28:14,520 --> 00:28:26,980 Similarly, for the CBP neighbor announcement, if you send Cisco Discovery Protocol packets with device IDs that are really, really long, you can fill up the memory tables and possibly do evil after that. 286 00:28:27,100 --> 00:28:31,200 There's been GRE attacks, a TFTP server advisory, and a bunch of other stuff. 287 00:28:31,200 --> 00:28:35,660 If you look at FX's site on Phenolate, there's tons of stuff. 288 00:28:35,660 --> 00:28:40,540 There's actually a tool for pen testing that was released specifically to address these kinds of things. 289 00:28:40,540 --> 00:28:47,240 It only addresses the vulnerabilities that FX actually found, but it is well worth looking at. 290 00:28:47,860 --> 00:29:00,200 There's also been a really interesting paper about Cisco memory mapping, and I'm sure that any of you that were at Black Hat on Wednesday saw similar work in Mike Lynn's talk. 291 00:29:00,200 --> 00:29:08,820 The memory structures are still the same, the magic numbers are still the same, they're all online, and it is very useful if you want to dig into the assembler. 292 00:29:11,090 --> 00:29:18,570 Then about a year ago, any of you that were at CanSec probably heard the storm and fury surrounding Paul Watson and the TCP window. 293 00:29:18,850 --> 00:29:26,310 Essentially, TCP stacks on background equipment and many other things were not correctly looking at the window size and anything that hit spot on. 294 00:29:26,310 --> 00:29:28,890 Just anything in the general neighborhood was good enough. 295 00:29:28,890 --> 00:29:38,310 So if you send a reset that's in the general neighborhood, and you're able to correctly guess the IP addresses and port numbers on either side, you can knock down the session with a spoof packet. 296 00:29:38,830 --> 00:29:46,510 And this allowed unauthenticated BGP sessions, if you could guess it all correctly, to be knocked down. 297 00:29:46,910 --> 00:30:01,810 And this was really interesting when you look at disclosure and how the whole thing went out, because essentially what happened was when Cisco started notifying people of this, because Watson worked with Cisco and said, hey, look, we need to fix this. 298 00:30:02,110 --> 00:30:08,350 Many people on the backbone and in NANOG and similar ISP communities had known about this for years. 299 00:30:08,350 --> 00:30:10,050 They just didn't think it was very possible. 300 00:30:10,150 --> 00:30:14,570 And again, we see that attitude of, well, I'm really busy unless I see something in the wild. 301 00:30:14,570 --> 00:30:15,310 It's not a problem. 302 00:30:15,310 --> 00:30:16,490 It's not going to happen. 303 00:30:16,950 --> 00:30:23,890 And so Cisco said, okay, everyone, you need to put md5 authentication passwords on your BGP sessions now. 304 00:30:23,890 --> 00:30:26,210 Okay, you need to upgrade now. 305 00:30:26,210 --> 00:30:27,690 They didn't tell people why. 306 00:30:28,090 --> 00:30:33,410 And so they told their favorite people first, and then their second favorite people, and their third favorite people, and so on out. 307 00:30:33,450 --> 00:30:39,150 And so it took, I think, the better part of a month for it to actually get heard by most people. 308 00:30:39,150 --> 00:30:41,230 And then Watson gave his CanSec talk. 309 00:30:42,050 --> 00:30:49,650 Once the ISP operators found out what the actual flaw was, there was a very strong cultural backlash saying, oh, I knew all about that. 310 00:30:49,650 --> 00:30:52,430 You know, why are you making such a big deal? 311 00:30:52,430 --> 00:31:01,490 And I was really depressed by going to NANOG this year, where a lot of operators were like, oh, yeah, you know, we thought that was important, but wow, we really freaked out. 312 00:31:01,490 --> 00:31:02,190 And I've rolled back. 313 00:31:02,190 --> 00:31:04,650 I don't have any authentication on my sessions anymore. 314 00:31:04,650 --> 00:31:06,030 You know, stupid hackers. 315 00:31:06,630 --> 00:31:09,430 And that was upsetting. 316 00:31:09,490 --> 00:31:17,170 Because I really saw the community response to the Watson vulnerability as a strong positive step in securing the backbone. 317 00:31:17,430 --> 00:31:21,430 And then seeing everyone go, wow, we freaked out over nothing, did not help. 318 00:31:21,430 --> 00:31:31,850 I think Cisco's approach of we're not telling you why, just patch it because we say so, no really, just trust us, I think they hurt themselves because people trust them less than they used to. 319 00:31:34,550 --> 00:31:37,890 And then we come to Wednesday and what made me rewrite my slides. 320 00:31:39,070 --> 00:31:48,470 If you were at Black Hat and saw Michael in session on Wednesday morning, he essentially demonstrated what I view as a remote exploit in Cisco IOS. 321 00:31:48,470 --> 00:31:53,230 Cisco has announced in at least one of their vulnerability announcements that it was just a local. 322 00:31:54,010 --> 00:31:57,570 I think we have different meanings for the same word there. 323 00:32:00,510 --> 00:32:12,970 And again, the disclosure process here was important because when Mike gave his talk, he essentially talked about the internals of how it worked. 324 00:32:12,970 --> 00:32:26,890 That stack overflows are hard to find in Cisco IOS, that the code base looked strongly audited, and that you had to work with heap overflows, but that Cisco has a check-heaps process, which will go through and validate, make sure the previous and next values are the same, 325 00:32:26,890 --> 00:32:34,450 so that if you actually do corrupt the heap, then it will say, oh, this isn't okay, all right, I'm crashing the router because this is screwed up. 326 00:32:35,710 --> 00:32:39,830 He did not give details of what his entry vector was. 327 00:32:39,830 --> 00:32:46,030 He was very careful to basically try and not give enough information for people to recreate it. 328 00:32:46,330 --> 00:32:53,470 Now, even though the vulnerability has been patched and the old versions have been yanked off of Cisco's website, there are still vulnerable versions out there in the wild. 329 00:32:54,470 --> 00:33:08,790 Cisco, really shockingly to me, yesterday morning released an advisory to full disclosure that said, yeah, for the first time ever, we have a denial of service and possible remote code execution in our handling of IPv6 packets. 330 00:33:10,330 --> 00:33:12,190 Lynn didn't tell us that. 331 00:33:12,290 --> 00:33:13,750 Cisco told us that. 332 00:33:14,210 --> 00:33:22,950 In the latest version of their advisory, they say that it was disclosed at Black Hat, so that really does nail it down to, yes, this was the vulnerability. 333 00:33:23,970 --> 00:33:29,890 And this is really revolutionary for the field, because for the first time, it looks like you can remotely own a Cisco box. 334 00:33:29,890 --> 00:33:35,810 You can get it to shovel you back a shell that will allow you to have full enable access. 335 00:33:36,090 --> 00:33:38,810 And this is a scary thing if you're an ISP operator. 336 00:33:40,090 --> 00:33:51,950 I think that Lynn was pretty sensible in saying that people need to treat these like normal computers, they need to patch, they need to be aware of security, and they need to be diligent about making sure that they patch through the versions, because this is a real threat, 337 00:33:51,950 --> 00:33:52,270 people. 338 00:33:52,270 --> 00:33:53,930 We've now seen the remote route. 339 00:33:55,210 --> 00:34:15,410 Now, when you look at the Cisco advisory, what they appear to mean by local, to the best of my understanding, and, you know, disclaimer, I don't and never have worked for Cisco, I don't and never have worked for ISS, I don't speak for anyone but me, but Cisco claims that you have to be connected to the same local subnet in order for that to work, 340 00:34:15,410 --> 00:34:17,610 and that makes it a local exploit, right? 341 00:34:19,450 --> 00:34:20,330 Yeah. 342 00:34:21,670 --> 00:34:27,590 No, if you have to be on the same subnet from one remote box to another remote box, that's a remote route. 343 00:34:30,560 --> 00:34:32,380 And then came the cover-up. 344 00:34:33,580 --> 00:34:46,100 The talk's been ripped out of the books, the presentation's not on the CD, everyone got modified CDs, the slides aren't available, oh my god, there's suing, and there's suing, and there's the FBI, and there's restraining orders, and rumors, and news stories, 345 00:34:46,100 --> 00:34:50,140 and all sorts of stuff, and Cisco, you're really screwing up here. 346 00:34:50,840 --> 00:34:51,640 You know, 347 00:35:05,290 --> 00:35:12,650 to give you an idea of the attitude of the security community, there are people walking around wearing these. 348 00:35:19,820 --> 00:35:26,100 I think that gives you an idea of the mentality of what happens when you try and cover something up. 349 00:35:26,320 --> 00:35:28,500 Okay, so you're not really happy. 350 00:35:28,680 --> 00:35:29,640 Big pardon? 351 00:35:30,880 --> 00:35:32,740 It said, fuck Cisco. 352 00:35:34,420 --> 00:35:36,260 Let the record reflect. 353 00:35:41,360 --> 00:35:49,060 And I really think that Mike Lynn was right on when he talked about why he gave the speech, and why he released as much information as he did. 354 00:35:49,060 --> 00:35:57,940 I mean, okay, the reason he got onto it in the first place, again, from the best of my understanding, I don't know the guy, I don't work for anyone, yada yada. 355 00:35:58,380 --> 00:36:09,060 He said that he found some Mandarin Chinese documents, which when translated were talking about how to hook debugger hooks in iOS, and how to make those speak to your disassembler and your debuggers. 356 00:36:09,440 --> 00:36:11,300 This means people are working on it. 357 00:36:11,300 --> 00:36:16,860 Their source code has been stolen, and there's now a storm of interest. 358 00:36:16,860 --> 00:36:24,420 I know that there have been several people who have suddenly been diving into their assembler to attempt to figure out what's going on. 359 00:36:24,960 --> 00:36:27,640 Hiding your head in the sand is not going to help. 360 00:36:28,080 --> 00:36:35,760 If you have people who are attacking you that are actively looking for this, going, no, no, it doesn't happen, quick, shut everyone up, is not going to help. 361 00:36:35,860 --> 00:36:43,520 And even if you destroy all the slides, and you aren't going to let things get released, well, his slides are out there on the internet. 362 00:36:43,520 --> 00:36:46,960 Suing researchers is not going to achieve security. 363 00:36:46,960 --> 00:36:48,840 I mean, only speaking for myself. 364 00:36:49,300 --> 00:36:50,920 I was in his talk. 365 00:36:51,340 --> 00:36:52,680 I speak assembler. 366 00:36:52,680 --> 00:36:54,140 I know Cisco iOS. 367 00:36:54,920 --> 00:36:58,200 With enough work, I could probably recreate his exploit. 368 00:36:58,540 --> 00:37:00,840 I'm sure I'm not the only one. 369 00:37:01,160 --> 00:37:04,280 Trying to pretend it won't happen will not fix things. 370 00:37:04,380 --> 00:37:15,760 And alienating the security community, and making everyone think that you're going to be sued into oblivion if you ever speak up, that's not going to encourage people to come to you and say, hey, we've got a problem, let's work with you. 371 00:37:15,920 --> 00:37:23,040 When I was reading the reports of extant backbone vulnerabilities, it sounded like they treated FX a lot nicer than they treated Mike Lynn. 372 00:37:23,040 --> 00:37:29,660 FX actually thanks people on the Fino Elite Advisories, and says, hey, thanks for working with me, Cisco P-Cert, you guys were really great. 373 00:37:30,440 --> 00:37:32,400 They don't sound really great right now. 374 00:37:33,420 --> 00:37:37,660 They filed a temporary restraining order to prevent him from speaking about it. 375 00:37:37,660 --> 00:37:40,480 So, he can't talk about it, but I can. 376 00:37:53,960 --> 00:38:02,040 Well, his slides are easily findable on the internet, and not even the version that he presented at the talk. 377 00:38:02,040 --> 00:38:13,480 The original slides from before he got fired by his employer, I think like yesterday morning they were up on CryptoMay and several other sites, and this has just been a ridiculous media circus. 378 00:38:14,020 --> 00:38:23,020 They were put up on Infowarrior.org, and they got pulled down with a cease and desist by Cisco, and guys, just help us fix this. 379 00:38:23,560 --> 00:38:26,260 The CD for that was from ISS, apparently. 380 00:38:26,760 --> 00:38:33,320 So, he had to return his computer, he's not allowed to talk anymore, and this is really dumb. 381 00:38:34,540 --> 00:38:40,000 Dear Cisco and ISS, Responsible disclosure is good. 382 00:38:40,400 --> 00:38:43,440 Working with security researchers is good. 383 00:38:43,440 --> 00:38:52,600 Trying to cover up things that are known problems that have been released into the public domain that are on millions and millions of machines all over the internet now is good. 384 00:38:52,820 --> 00:38:57,940 If you try and shut people up, you're only going to alienate the very people that you need to help you. 385 00:38:59,780 --> 00:39:00,660 Duh. 386 00:39:05,140 --> 00:39:08,220 Alright, so, what Mike Lynn actually said. 387 00:39:09,180 --> 00:39:11,320 Heap overflows are way more common in iOS. 388 00:39:11,320 --> 00:39:20,600 He said a bunch of nice things about their code base, but he said that the watchdog process of check heaps will check for heap corruption, and if it finds it, it will crash it. 389 00:39:20,840 --> 00:39:31,600 And most of you that have worked with Cisco routers I'm sure will see that if you have a Cisco router and it has a fault, a lot of times, rather than actually trying to recover, it will just crash itself. 390 00:39:31,840 --> 00:39:38,620 And the really important thing that I think came out of Lynn's research was that the crash process itself is interruptible. 391 00:39:38,620 --> 00:39:42,740 So, you can interrupt it when it says, Hey, this bad thing has happened. 392 00:39:42,740 --> 00:39:44,360 My heap has gone badly. 393 00:39:44,360 --> 00:39:45,400 Am I crashing? 394 00:39:46,080 --> 00:39:49,200 And you can tell it, Why, yes I am. 395 00:39:49,200 --> 00:39:50,680 Okay, we're already crashing. 396 00:39:50,680 --> 00:39:51,880 I don't need to do anything then. 397 00:39:51,880 --> 00:39:55,100 Please continue executing whatever the heck it is you were executing there. 398 00:39:55,780 --> 00:39:57,800 So, you can buy yourself some time that way. 399 00:39:57,800 --> 00:40:12,400 He did mention that spreading memory corruption on the heap was going to be an issue, and that you might have two to five minutes before it exploded all over your memory, basically, unless you actually handle that in your shellcode, and anyone who's written shellcode knows that that can be tricky. 400 00:40:13,200 --> 00:40:18,540 But once you've actually gained execution, there are hardware interrupts that will try and interrupt you, but you can disable those. 401 00:40:18,540 --> 00:40:21,840 Once the CPU is yours, it's yours until the router crashes. 402 00:40:23,240 --> 00:40:30,340 And through methods like that, you can get the remote box to shovel an enabled shell back to your listener, which is what Lynn did in his demo. 403 00:40:31,280 --> 00:40:33,200 Enable is the Cisco equivalent of root. 404 00:40:33,200 --> 00:40:34,100 Game over. 405 00:40:35,260 --> 00:40:44,200 He indicated that though he didn't give us the specific vector, and Cisco did, that there were other vulnerabilities that you could do this for. 406 00:40:44,200 --> 00:40:47,720 And here's some of them that might be fruitful grounds for exploration. 407 00:40:49,420 --> 00:41:05,500 Anything that talks about memory corruption that leads to a denial of service is probably a likely factor for this sort of thing, simply because since the Cisco routers tend to crash, rather than actually trying to recover, if you screw up the memory enough to induce a crash, 408 00:41:05,500 --> 00:41:09,660 it may just be, okay, your offset was wrong, or something like that. 409 00:41:09,800 --> 00:41:22,660 I really strongly encourage Cisco to look at their code and look at the way that he was able to interrupt the CheckHeaps process and avert crashing, because that's the underlying problem. 410 00:41:22,660 --> 00:41:30,240 The front-end vector, whether it's IPv6 or CDP or OSPF or BGP, that's not really going to be your issue. 411 00:41:30,880 --> 00:41:34,860 The fact that you can defeat CheckHeaps, that's your problem. 412 00:41:36,620 --> 00:41:43,400 So, yeah, here's a link to the advisory about vulnerabilities in IPv6 processing that Cisco put up. 413 00:41:43,540 --> 00:41:53,880 And just in bold, because this is the first time I've seen it and I've been waiting for years to see when this was going to occur, because I believe it was possible, but Mike Lynn's shellcode is better than mine. 414 00:41:55,920 --> 00:42:02,800 Cisco Internetwork Operating System is vulnerable to a denial of service and potentially an arbitrary code execution attack. 415 00:42:02,800 --> 00:42:04,740 So, Cisco did admit it. 416 00:42:08,380 --> 00:42:18,520 Again, just to hammer home the point about local segment versus remote segment, local network segment does not mean that it is not a remote group. 417 00:42:18,940 --> 00:42:20,020 Thank you. 418 00:42:21,340 --> 00:42:23,280 So, look at this for yourself. 419 00:42:23,280 --> 00:42:24,300 Don't just take my word. 420 00:42:24,300 --> 00:42:28,980 Don't take FX's, don't take Mike Lynn's, don't take Paul Watson's. 421 00:42:28,980 --> 00:42:32,740 Go and look at the source of these sorts of things. 422 00:42:32,740 --> 00:42:40,980 Read the advisories, read the vulnerabilities, and check what Cisco says now versus what Cisco said on full disclosure versus what Cisco said before. 423 00:42:41,520 --> 00:42:45,260 Read FX's paper about how Cisco handles memory management. 424 00:42:46,120 --> 00:42:52,180 And there is a link to where these Lynn slides are, and I believe they were still up. 425 00:42:52,180 --> 00:42:55,980 They were still up when I talked about an hour or so before I got up here. 426 00:42:57,160 --> 00:42:57,980 They're up. 427 00:42:57,980 --> 00:42:58,720 All right. 428 00:43:00,840 --> 00:43:03,100 So, you network defender people. 429 00:43:04,560 --> 00:43:07,220 Start treating your backbone devices like computers. 430 00:43:07,220 --> 00:43:19,220 Realize that they will have vulnerabilities, that you need to allocate for security, that you need to work for this, that you can't just sort of set it up like an ancient BSD box and, oh yeah, no one's ever going to hack that. 431 00:43:19,220 --> 00:43:19,880 It's fine. 432 00:43:19,880 --> 00:43:20,620 Routing is hard. 433 00:43:20,620 --> 00:43:21,960 No one understands that. 434 00:43:22,000 --> 00:43:29,300 Well, I'd like to point out that of the people that I know that are working in this field, I think I'm the only one that actually has an ISP background. 435 00:43:29,660 --> 00:43:34,260 You can't just depend on it's hard and security through obscurity. 436 00:43:35,980 --> 00:43:43,900 So, yes, you need to treat them like computers, but when you're testing, because denial of service is so prevalent, treat them like really important computers. 437 00:43:43,900 --> 00:43:48,180 Don't knock them over without your client's explicit consent. 438 00:43:48,280 --> 00:43:49,520 Maybe packet log. 439 00:43:49,520 --> 00:43:55,980 When I'm pentesting, I run a packet log so that if they come and say, you know, you broke XYZ, I can go, really, where? 440 00:43:55,980 --> 00:43:58,940 Here's my timestamps, here's my packets, here's exactly what I did. 441 00:43:59,960 --> 00:44:03,720 So that may be a contract issue for those of you who are doing that. 442 00:44:05,020 --> 00:44:06,200 Attacks only get better. 443 00:44:06,200 --> 00:44:07,380 They don't get worse. 444 00:44:07,860 --> 00:44:12,020 In time, these test tools are going to become a regular part of pentesters' toolkits. 445 00:44:12,040 --> 00:44:21,040 I would not be shocked if next year I suddenly saw Cisco attacks in Metasploit or something like that, or more Cisco's vulnerabilities in Nessus. 446 00:44:21,040 --> 00:44:26,720 We're still kind of in the dark ages of pentesting through the backbone, but keep in mind that this is where things are going. 447 00:44:26,720 --> 00:44:29,580 Enough people are interested in working on it that it's going to happen. 448 00:44:30,600 --> 00:44:36,040 And please don't take me off the internet, or slap me with a gag order, or try to sue me, or anything like that. 449 00:44:36,040 --> 00:44:36,920 There are mirrors. 450 00:44:36,980 --> 00:44:40,120 Everyone, please go out and mirror my stuff, put it on BitTorrent. 451 00:44:40,120 --> 00:44:41,560 Don't let them shut me up. 452 00:44:58,020 --> 00:44:58,340 Just 453 00:45:01,460 --> 00:45:04,380 a few last comments, and then I believe we have a brief time for questions. 454 00:45:04,380 --> 00:45:10,980 You can reach me at raven at nmrc.org with feedback, additional ideas, comments, lawsuits. 455 00:45:11,700 --> 00:45:21,220 I would like to thank Jericho and the OSVDB people for promptly putting all these vulnerabilities up in a public space where everyone can see them and have equal access for research. 456 00:45:21,220 --> 00:45:26,060 I would like to thank people who are mirroring these for me, and also possibly staring down the eyes of lawsuits. 457 00:45:26,180 --> 00:45:28,620 And I'd like to thank the EFF for sitting in the front row. 458 00:45:28,620 --> 00:45:30,280 I love you, new best friends. 459 00:45:41,000 --> 00:45:42,160 Alright, we have three minutes. 460 00:45:42,160 --> 00:45:43,080 Any questions? 461 00:45:47,080 --> 00:45:48,240 Beg pardon? 462 00:45:56,120 --> 00:46:00,060 My updated PowerPoint will be up momentarily. 463 00:46:00,380 --> 00:46:03,400 I think people are going to run out after this and mirror it. 464 00:46:03,400 --> 00:46:11,340 I will put a site online where there can be a list of the extant mirrors, and if that gets knocked down, it gets knocked down. 465 00:46:11,820 --> 00:46:15,420 Also, there is a legal defense fund for Mike Lynn. 466 00:46:15,480 --> 00:46:21,780 If you PayPal to abaddon, A-B-A-D-D-O-N, at io.com, you may need it. 467 00:46:23,560 --> 00:46:25,980 Oh, apparently you will need it. 468 00:46:27,740 --> 00:46:28,680 Yes. 469 00:46:37,170 --> 00:46:46,150 The question was, given the negative reaction to the Paul Watson TCP issue, how confident am I that ISP engineers will actually pay attention to this? 470 00:46:46,410 --> 00:46:54,970 Well, despite working in this field, I'm perpetually optimistic, but I think the fact that we have enough people continually saying, Hey, look. 471 00:46:54,970 --> 00:46:55,550 Hey, look. 472 00:46:55,550 --> 00:46:56,470 Hey, look. 473 00:46:56,610 --> 00:47:13,250 I am cheered that even though I was not going to announce the details of what I was talking about beforehand, because I didn't want to be shut up, a lot of my ISP friends contacted me over the last two days and went, Oh, holy shit, what do we need to do? 474 00:47:13,590 --> 00:47:15,590 So, there are at least some people who care. 475 00:47:15,590 --> 00:47:18,270 There is an ISP community that's dedicated to security. 476 00:47:18,270 --> 00:47:21,650 There are both public and private mailing lists that are working on this. 477 00:47:21,650 --> 00:47:23,970 The problem is not that there aren't some people who care. 478 00:47:23,970 --> 00:47:26,290 The problem is that most of the people don't care. 479 00:47:26,290 --> 00:47:28,410 So , truthful people, be loud. 480 00:47:31,990 --> 00:47:33,430 Alright, one more question. 481 00:47:33,490 --> 00:47:34,870 Yes, in the back. 482 00:47:35,350 --> 00:47:36,190 That's you. 483 00:47:38,900 --> 00:47:40,400 I can't hear you. 484 00:47:54,400 --> 00:48:02,840 The question was, since I've talked about ISP infrastructure and such, can the ISP protect its hosts against zombies and things like that? 485 00:48:03,180 --> 00:48:05,200 Yes, the ISPs can to some degree. 486 00:48:05,200 --> 00:48:06,940 The question is whether they are willing to. 487 00:48:06,940 --> 00:48:10,500 A lot of times to do that level of filtering takes additional resources. 488 00:48:10,500 --> 00:48:11,880 You need stronger CPUs. 489 00:48:11,880 --> 00:48:14,580 You need more memory in the routers. 490 00:48:14,580 --> 00:48:17,880 And a lot of places are running pretty close to bare bones on that. 491 00:48:17,880 --> 00:48:21,200 So, they don't have a lot of resources to spare and they don't think it's a good allocation. 492 00:48:21,380 --> 00:48:26,520 What they need to be able to do that is to hear from people and say, No, really, I want you to do this. 493 00:48:26,520 --> 00:48:27,460 I want you to filter me. 494 00:48:27,460 --> 00:48:32,180 There is client demand for it because if they don't see operational demand, they won't do it. 495 00:48:47,510 --> 00:48:53,690 The comment was, don't you have to have an interest in keeping spam and zombies and such from happening in the first place? 496 00:48:53,690 --> 00:49:00,990 Alluding to the poor record of many ISPs who benefit economically from allowing such things to happen across their network. 497 00:49:01,290 --> 00:49:07,390 And, you know, I could go on for an hour about that, but suffice to say, market demand. 498 00:49:08,910 --> 00:49:10,630 They are following where the money is. 499 00:49:10,630 --> 00:49:15,570 If you inform them where you want the money to be, the more people that inform them, the more likely they will be to listen. 500 00:49:20,110 --> 00:49:22,230 Something that we can do to fix what quickly? 501 00:49:24,110 --> 00:49:25,310 It's already fixed. 502 00:49:25,310 --> 00:49:30,930 If you go and you update your IOS to the most recent version of firmware in your code train, you'll be safe. 503 00:49:32,150 --> 00:49:33,610 All right, I think we're out of time. 504 00:49:33,610 --> 00:49:34,670 Thank you very much.