1 00:00:00,365 --> 00:00:01,198 (audience clapping) 2 00:00:01,198 --> 00:00:04,020 - Well, Daniel Crowley is about to give an awesome talk 3 00:00:04,020 --> 00:00:06,540 on black box smart cards. 4 00:00:06,540 --> 00:00:07,593 So, take it away! 5 00:00:09,810 --> 00:00:10,643 - Hi, everybody. 6 00:00:11,580 --> 00:00:16,380 So, I am the head of the research program at X-Force Red. 7 00:00:16,380 --> 00:00:19,140 And that's about as much bio as I'll give. 8 00:00:19,140 --> 00:00:23,580 I think I have a fun bio in the DEFCON 9 00:00:23,580 --> 00:00:25,743 handout if you care to read it. 10 00:00:27,210 --> 00:00:29,970 But let's just get right into this topic. 11 00:00:29,970 --> 00:00:32,460 So, smart cards 12 00:00:32,460 --> 00:00:36,270 are a part of our lives in many ways, 13 00:00:36,270 --> 00:00:37,770 some of them that you wouldn't expect. 14 00:00:37,770 --> 00:00:39,630 So, credit cards and debit cards, 15 00:00:39,630 --> 00:00:42,873 you might recognize those immediately as smart cards. 16 00:00:44,100 --> 00:00:45,450 But then, SIM cards. 17 00:00:45,450 --> 00:00:49,290 SIM cards are one that is also in the realm of smart cards. 18 00:00:49,290 --> 00:00:53,520 And things like transit fare cards or electronic passports, 19 00:00:53,520 --> 00:00:56,847 are also examples of smart cards. 20 00:00:56,847 --> 00:01:00,420 And interestingly, all of the cards listed here 21 00:01:00,420 --> 00:01:01,770 and amiibos, by the way, 22 00:01:01,770 --> 00:01:05,190 those little statuettes that you can buy and put on 23 00:01:05,190 --> 00:01:07,080 your Nintendo Switch to unlock characters 24 00:01:07,080 --> 00:01:08,370 in different games. 25 00:01:08,370 --> 00:01:11,133 Also smart cards, interestingly enough. 26 00:01:12,570 --> 00:01:14,880 So, they are absolutely everywhere. 27 00:01:14,880 --> 00:01:16,950 And in many cases, unlike the amiibos, 28 00:01:16,950 --> 00:01:19,380 they handle very important things like money, 29 00:01:19,380 --> 00:01:23,160 or they control physical access to spaces. 30 00:01:23,160 --> 00:01:24,900 Or, maybe you store 31 00:01:24,900 --> 00:01:28,110 cryptographic keys on there, private keys, 32 00:01:28,110 --> 00:01:30,930 they're supposed to never come off, and all of this stuff. 33 00:01:30,930 --> 00:01:33,120 So, they handle some very important things in our lives, 34 00:01:33,120 --> 00:01:34,050 and they're everywhere. 35 00:01:34,050 --> 00:01:36,650 So, I thought that they were worth taking a look at. 36 00:01:38,400 --> 00:01:43,230 Now, all of the cards I'm talking about today are based on, 37 00:01:43,230 --> 00:01:44,430 they all kind of come back 38 00:01:44,430 --> 00:01:49,140 to the same standard of ISO 7816. 39 00:01:49,140 --> 00:01:52,530 And ISO 7816 defines a couple things. 40 00:01:52,530 --> 00:01:54,963 It's specifically for contact cards. 41 00:01:56,610 --> 00:02:00,600 And it defined a, in addition to, like the physical format 42 00:02:00,600 --> 00:02:02,760 of the connector and the pin out, and everything like that. 43 00:02:02,760 --> 00:02:05,430 So, you can actually make an electrical connection 44 00:02:05,430 --> 00:02:10,430 to the card, power it, and then communicate with the card. 45 00:02:10,707 --> 00:02:13,110 And it defines voltages, and things like that, 46 00:02:13,110 --> 00:02:15,300 that I haven't really touched. 47 00:02:15,300 --> 00:02:19,260 But it also defines a sort of command format, 48 00:02:19,260 --> 00:02:23,710 and a certain command set of standard commands 49 00:02:25,290 --> 00:02:27,450 referred to as APDU, is the format. 50 00:02:27,450 --> 00:02:30,090 It's Application Protocol Data Unit. 51 00:02:30,090 --> 00:02:32,290 We'll get a little bit more into that later. 52 00:02:33,210 --> 00:02:35,430 What it doesn't define interestingly, 53 00:02:35,430 --> 00:02:38,160 is which commands must be implemented. 54 00:02:38,160 --> 00:02:40,530 So, the standard defines something like 55 00:02:40,530 --> 00:02:42,453 20 or 30 different commands, 56 00:02:43,830 --> 00:02:46,803 but it says, you don't really have to implement these. 57 00:02:47,670 --> 00:02:49,950 If you do, here's what you should do. 58 00:02:49,950 --> 00:02:54,950 But if you don't, whatever, have a beer or something, 59 00:02:54,990 --> 00:02:55,823 I don't know. 60 00:02:55,823 --> 00:02:57,423 It's fine, I guess. 61 00:02:58,380 --> 00:03:01,680 And so like, sometimes, 62 00:03:01,680 --> 00:03:02,970 and the other thing is the particulars 63 00:03:02,970 --> 00:03:06,660 of the implementation aren't defined. 64 00:03:06,660 --> 00:03:09,600 So for instance, with a lot of cards, 65 00:03:09,600 --> 00:03:11,490 there is this challenge response mechanism, 66 00:03:11,490 --> 00:03:16,490 where you tell the card, I need some random data. 67 00:03:16,920 --> 00:03:19,410 The card gives you some random data. 68 00:03:19,410 --> 00:03:21,480 And you do something with it. 69 00:03:21,480 --> 00:03:24,030 Whether that's sign it with a cryptographic key, 70 00:03:24,030 --> 00:03:28,050 or encrypt it with some sort of secret key 71 00:03:28,050 --> 00:03:29,640 that you share with the card. 72 00:03:29,640 --> 00:03:31,620 And then you send the processed version 73 00:03:31,620 --> 00:03:33,510 of this challenge back. 74 00:03:33,510 --> 00:03:35,670 And the card can tell if you have 75 00:03:35,670 --> 00:03:39,303 the right key material cryptographically somehow. 76 00:03:40,320 --> 00:03:43,680 Some magic that is not defined in ISO 7816. 77 00:03:43,680 --> 00:03:46,803 So, there's potential for problems there. 78 00:03:48,630 --> 00:03:51,720 Then there's ISO 14443, 79 00:03:51,720 --> 00:03:54,390 which is a standard for contactless cards. 80 00:03:54,390 --> 00:03:57,900 Now, this builds on top of ISO 7816. 81 00:03:57,900 --> 00:04:01,050 So, ISO 14443 cards, 82 00:04:01,050 --> 00:04:06,050 NFC cards are an example of 14443. 83 00:04:06,330 --> 00:04:11,330 These built on 7816, and use the same APDU format, 84 00:04:11,880 --> 00:04:15,240 use some of the commands from this 85 00:04:15,240 --> 00:04:17,910 established command set from 7816. 86 00:04:17,910 --> 00:04:21,660 So, ISO 7816 standard is really important to 87 00:04:21,660 --> 00:04:23,973 how all these smart cards function. 88 00:04:25,380 --> 00:04:29,970 Now, how well do these cards actually follow the standards? 89 00:04:29,970 --> 00:04:33,360 That's, you can make a standard, but how well like, 90 00:04:33,360 --> 00:04:37,593 things actually follow it, is it depends, right? 91 00:04:38,640 --> 00:04:42,720 As far as like the physical connectors and stuff like that, 92 00:04:42,720 --> 00:04:44,640 as you would imagine, they follow the standard 93 00:04:44,640 --> 00:04:46,140 pretty well there, right? 94 00:04:46,140 --> 00:04:48,810 Because they have to where it's just not going 95 00:04:48,810 --> 00:04:49,713 to work at all. 96 00:04:51,090 --> 00:04:52,830 And then, there's certain commands such as 97 00:04:52,830 --> 00:04:55,170 select file and verify. 98 00:04:55,170 --> 00:04:57,360 Select file is like what you would expect. 99 00:04:57,360 --> 00:05:00,450 It's like, hey, there's file on the file system here. 100 00:05:00,450 --> 00:05:03,771 Although the file system is weird, we'll get to that. 101 00:05:03,771 --> 00:05:06,360 And then, there's verify, which is kind of like, 102 00:05:06,360 --> 00:05:08,730 you just give a password or a pin, 103 00:05:08,730 --> 00:05:10,740 or something like that to the card. 104 00:05:10,740 --> 00:05:15,060 And depending on whether it it's correct or not, it will, 105 00:05:15,060 --> 00:05:17,433 or won't unlock features of the card. 106 00:05:20,460 --> 00:05:25,460 The error codes in the 7816 documentation in the standard 107 00:05:26,880 --> 00:05:27,930 are pretty granular. 108 00:05:27,930 --> 00:05:30,150 They're saying like, this bit is wrong, 109 00:05:30,150 --> 00:05:31,920 that bit is wrong. 110 00:05:31,920 --> 00:05:35,040 Some of the cards just use really whatever error codes 111 00:05:35,040 --> 00:05:36,600 they feel like. 112 00:05:36,600 --> 00:05:38,250 They'll use different error codes 113 00:05:38,250 --> 00:05:41,130 for the same or similar situations. 114 00:05:41,130 --> 00:05:43,590 So, it can be very frustrating if you're expecting 115 00:05:43,590 --> 00:05:44,823 certain error codes. 116 00:05:46,680 --> 00:05:51,630 So, let's talk about APDU and what that looks like. 117 00:05:51,630 --> 00:05:56,130 So, the way that a command looks on the wire 118 00:05:56,130 --> 00:05:59,190 with smart cards is like this. 119 00:05:59,190 --> 00:06:03,090 So, this is the APDU for 120 00:06:03,090 --> 00:06:06,030 select file by dedicated file name, 121 00:06:06,030 --> 00:06:08,370 first occurrence, named ABC. 122 00:06:08,370 --> 00:06:11,550 So, let's break this down piece by piece. 123 00:06:11,550 --> 00:06:16,020 So, the first byte is the CLA or class byte. 124 00:06:16,020 --> 00:06:20,850 And the class byte defines what category of 125 00:06:20,850 --> 00:06:23,403 instruction to execute. 126 00:06:25,590 --> 00:06:29,250 There's a convention where 127 00:06:29,250 --> 00:06:32,190 if the CLA byte begins with 0, 128 00:06:32,190 --> 00:06:37,190 it follows the ISO 7816 standard, and is to be interpreted 129 00:06:37,290 --> 00:06:40,230 according to a series of byte flags. 130 00:06:40,230 --> 00:06:45,120 And if it begins with a one byte, so anything hex 80 131 00:06:45,120 --> 00:06:48,600 or above, that is going to be interpreted 132 00:06:48,600 --> 00:06:50,910 according to a proprietary standard, 133 00:06:50,910 --> 00:06:54,723 which is outside the scope of 7816. 134 00:06:56,610 --> 00:07:00,150 And then it also says, FF is invalid, can't use that. 135 00:07:00,150 --> 00:07:03,330 But there's another standard called PCSC, 136 00:07:03,330 --> 00:07:06,543 which is used by things like SIM cards, NFC tags, 137 00:07:08,430 --> 00:07:13,430 that you will actually see FF used in real smart cards. 138 00:07:14,010 --> 00:07:15,360 So, it's kind of like, well, 139 00:07:15,360 --> 00:07:17,130 we just ignore that part of the standard 140 00:07:17,130 --> 00:07:18,430 'cause everybody else has. 141 00:07:19,920 --> 00:07:21,840 The INS byte, which is the next one, 142 00:07:21,840 --> 00:07:25,623 is what instruction or command to execute. 143 00:07:26,608 --> 00:07:30,750 And so for this one, A4 is select file. 144 00:07:30,750 --> 00:07:33,540 This one is really well-supported across cards 145 00:07:33,540 --> 00:07:36,000 for reasons you might imagine. 146 00:07:36,000 --> 00:07:38,793 A lot of cards have reason to do things with files. 147 00:07:39,840 --> 00:07:43,290 We'll see why it's especially important for them to support 148 00:07:43,290 --> 00:07:46,380 a file selection, because there's this whole 149 00:07:46,380 --> 00:07:49,020 context thing, and file system is kinda weird. 150 00:07:49,020 --> 00:07:50,770 We'll talk more about that shortly. 151 00:07:51,900 --> 00:07:55,650 The next two bytes, P1 and P2, are parameter bytes. 152 00:07:55,650 --> 00:07:58,650 So, this says, we've got our application class. 153 00:07:58,650 --> 00:08:02,370 We've picked out a command from that application class. 154 00:08:02,370 --> 00:08:04,290 These are two byte parameters to it. 155 00:08:04,290 --> 00:08:07,980 So, this is mostly like, well, what variant of this command 156 00:08:07,980 --> 00:08:08,813 do you want to run? 157 00:08:08,813 --> 00:08:12,903 Or, what data do you want to feed to this? 158 00:08:13,800 --> 00:08:18,510 So in this case, 4 for P1, the first parameter means, 159 00:08:18,510 --> 00:08:19,710 gimme a name. 160 00:08:19,710 --> 00:08:22,233 There's a couple different ways to select files. 161 00:08:23,100 --> 00:08:27,180 But in this case, we're going first occurrence, 162 00:08:27,180 --> 00:08:29,910 which is P2 is 0. 163 00:08:29,910 --> 00:08:31,980 And don't worry if this doesn't make sense, 164 00:08:31,980 --> 00:08:33,720 this is just what they put in the standard. 165 00:08:33,720 --> 00:08:34,553 Right? 166 00:08:34,553 --> 00:08:37,263 I'm just trying to break down what's in what position. 167 00:08:39,090 --> 00:08:43,353 And then finally here, we have a single byte length, 168 00:08:44,220 --> 00:08:47,160 followed by data of that length, 169 00:08:47,160 --> 00:08:49,410 which is meant to be like an argument to it. 170 00:08:49,410 --> 00:08:50,460 So in this case, 171 00:08:50,460 --> 00:08:53,550 this is the file name that we're actually requesting 172 00:08:53,550 --> 00:08:56,343 on the card, starting with ABC. 173 00:08:58,410 --> 00:08:59,640 Now with this particular command, 174 00:08:59,640 --> 00:09:02,220 you can actually just request the first couple bytes, 175 00:09:02,220 --> 00:09:03,660 but again, I'm getting ahead of myself. 176 00:09:03,660 --> 00:09:08,660 So, there is also an optional field after this 177 00:09:08,700 --> 00:09:11,253 of the length expected, or LE. 178 00:09:12,270 --> 00:09:14,550 Now, we're not expecting any response here, 179 00:09:14,550 --> 00:09:15,383 and we can omit this. 180 00:09:15,383 --> 00:09:17,163 So, we just don't have that. 181 00:09:18,150 --> 00:09:19,203 So, 182 00:09:20,880 --> 00:09:23,430 the same goes for the data field. 183 00:09:23,430 --> 00:09:24,810 If you don't have any data to append, 184 00:09:24,810 --> 00:09:27,900 you can just stop after four bytes, and that's it. 185 00:09:27,900 --> 00:09:30,840 So, this is what APDU commands look like. 186 00:09:30,840 --> 00:09:33,079 So, let's look at the response. 187 00:09:33,079 --> 00:09:34,740 Responses are way simpler. 188 00:09:34,740 --> 00:09:37,230 They're just two status bytes. 189 00:09:37,230 --> 00:09:40,740 The first one defines sorta like the major response type. 190 00:09:40,740 --> 00:09:43,770 And the second one is the minor, like the subtype. 191 00:09:43,770 --> 00:09:47,760 So for instance, hex 90, that's success. 192 00:09:47,760 --> 00:09:49,890 Yay, you did it, it worked. 193 00:09:49,890 --> 00:09:53,790 And 0 is the only subtype of that, right? 194 00:09:53,790 --> 00:09:55,113 At least in the standard. 195 00:09:56,723 --> 00:09:58,950 And the response can also include data. 196 00:09:58,950 --> 00:10:03,950 Or, it can say I have data for you to be returned, 197 00:10:04,080 --> 00:10:06,360 but I'm not gonna give it to you right now. 198 00:10:06,360 --> 00:10:07,923 If you really want to see it, 199 00:10:08,970 --> 00:10:11,463 you can use the get response command. 200 00:10:13,590 --> 00:10:16,470 Now, here's an error response. 201 00:10:16,470 --> 00:10:20,193 6c means you didn't ask for the right length. 202 00:10:21,240 --> 00:10:24,270 You told me you wanted one length, or you didn't tell me 203 00:10:24,270 --> 00:10:27,363 what your expected response length is. 204 00:10:28,620 --> 00:10:31,110 Here is the expected response length. 205 00:10:31,110 --> 00:10:33,600 So, if this is one of those really, really helpful errors, 206 00:10:33,600 --> 00:10:36,060 that's like, if you don't know what you're doing, 207 00:10:36,060 --> 00:10:36,893 this is great. 208 00:10:36,893 --> 00:10:38,310 'Cause it's like, oh, thank you for giving me 209 00:10:38,310 --> 00:10:39,213 the right answer. 210 00:10:40,980 --> 00:10:43,563 But there's no data sent with this response, right? 211 00:10:44,430 --> 00:10:47,640 So, here's some more response code samples. 212 00:10:47,640 --> 00:10:49,920 Some of these get pretty specific. 213 00:10:49,920 --> 00:10:53,544 And you can kinda see how, if you are working 214 00:10:53,544 --> 00:10:54,690 with a smart card that 215 00:10:54,690 --> 00:10:57,420 you don't really understand, but like, 216 00:10:57,420 --> 00:10:59,700 it's not some specification that you're familiar with, 217 00:10:59,700 --> 00:11:02,640 or it's some proprietary specification. 218 00:11:02,640 --> 00:11:04,380 Or, something like that, 219 00:11:04,380 --> 00:11:06,960 or you don't want to follow a higher level specification, 220 00:11:06,960 --> 00:11:08,760 you want to map out the entirety of a card, 221 00:11:08,760 --> 00:11:12,093 not just what follows the expected specification. 222 00:11:13,230 --> 00:11:17,730 You get error codes that say wrong CLA, wrong INS, 223 00:11:17,730 --> 00:11:18,563 all this stuff. 224 00:11:18,563 --> 00:11:21,000 So, when you're trying to figure out what's a good CLA, 225 00:11:21,000 --> 00:11:24,000 what's a good INS, and so on and so forth, 226 00:11:24,000 --> 00:11:27,780 you have all these wonderful error codes that sometimes 227 00:11:27,780 --> 00:11:29,523 are not used correctly. 228 00:11:31,620 --> 00:11:34,561 That said, they are there, sometimes they work. 229 00:11:34,561 --> 00:11:39,360 So, let's talk about how easy or how feasible it is to take 230 00:11:39,360 --> 00:11:42,540 just any old smart card, whether that's debit card, 231 00:11:42,540 --> 00:11:46,920 SIM card, anything that follows this 7816 standard. 232 00:11:46,920 --> 00:11:50,190 And figure out what our valid CLA is, 233 00:11:50,190 --> 00:11:52,260 valid INS is, all that. 234 00:11:52,260 --> 00:11:57,260 So, with the CLA, if it's in following the 7816 standard, 235 00:11:58,500 --> 00:12:00,420 if it starts with a zero byte, 236 00:12:00,420 --> 00:12:03,570 then it follows a convention that we know about. 237 00:12:03,570 --> 00:12:07,470 So, we only really need to try zero, 238 00:12:07,470 --> 00:12:10,740 and then 80 through FF, right? 239 00:12:10,740 --> 00:12:11,573 So, 240 00:12:12,450 --> 00:12:16,833 that's 1 plus 128 for 129 possible values. 241 00:12:18,570 --> 00:12:22,500 So, we can try all of those and just get wrong CLA, 242 00:12:22,500 --> 00:12:26,130 and 129 values is very short amount to have to 243 00:12:26,130 --> 00:12:27,060 brute force through. 244 00:12:27,060 --> 00:12:28,320 So, that's very feasible. 245 00:12:28,320 --> 00:12:30,900 That's totally, we can totally do that. 246 00:12:30,900 --> 00:12:32,073 INS values. 247 00:12:33,390 --> 00:12:36,570 There's only 256 possible codes. 248 00:12:36,570 --> 00:12:37,863 It's only one byte long. 249 00:12:38,760 --> 00:12:41,340 Now, ISO 7816 forbids anything 250 00:12:41,340 --> 00:12:44,970 that starts with 6, like hex 6, whatever. 251 00:12:44,970 --> 00:12:47,160 Or 9, whatever. 252 00:12:47,160 --> 00:12:49,140 In reality, there's a ton of commands 253 00:12:49,140 --> 00:12:50,760 that actually views this. 254 00:12:50,760 --> 00:12:52,590 So, we just kind of, again, 255 00:12:52,590 --> 00:12:56,010 ignore this part of the standard 'cause everybody else does. 256 00:12:56,010 --> 00:12:59,670 But even still, we're talking about 257 00:12:59,670 --> 00:13:01,500 256 values. 258 00:13:01,500 --> 00:13:02,333 Right? 259 00:13:02,333 --> 00:13:04,560 And if we just look at a lot of different standards, 260 00:13:04,560 --> 00:13:06,180 like we look at EMV. 261 00:13:06,180 --> 00:13:07,620 We look at a global platform, 262 00:13:07,620 --> 00:13:11,430 we look at VSDC, and see all the commands 263 00:13:11,430 --> 00:13:12,310 that they have 264 00:13:14,640 --> 00:13:17,430 standardized on, we can try just those if we want 265 00:13:17,430 --> 00:13:18,263 to cut it down. 266 00:13:18,263 --> 00:13:21,120 But again, 256 values does not take a long time 267 00:13:21,120 --> 00:13:21,953 to brute force through. 268 00:13:21,953 --> 00:13:24,033 So INS, we literally do that. 269 00:13:25,110 --> 00:13:28,830 There are times when you get just generic errors, 270 00:13:28,830 --> 00:13:33,427 and you have to brute force through both CLA and INS for, 271 00:13:35,850 --> 00:13:37,590 can't do the math in my head right this second, 272 00:13:37,590 --> 00:13:40,710 but 256 times 129. 273 00:13:40,710 --> 00:13:44,550 Still, in the tens of thousands of requests, 274 00:13:44,550 --> 00:13:45,840 you're talking a couple minutes. 275 00:13:45,840 --> 00:13:47,673 So, that's again, that's feasible. 276 00:13:49,192 --> 00:13:50,760 P1 and P2, that's two bytes. 277 00:13:50,760 --> 00:13:52,653 Brute forcing them at the same time. 278 00:13:54,563 --> 00:13:58,230 65,536 values, totally feasible to brute force. 279 00:13:58,230 --> 00:14:02,040 So, so far, we can brute force through, 280 00:14:02,040 --> 00:14:05,850 and find like, whatever commands are available, 281 00:14:05,850 --> 00:14:09,810 and what parameter values are accepted. 282 00:14:09,810 --> 00:14:10,720 Now 283 00:14:11,580 --> 00:14:13,800 if we are 284 00:14:13,800 --> 00:14:18,120 trying every INS value, there are some potential pitfalls. 285 00:14:18,120 --> 00:14:23,120 Like, there is a terminate card usage slash blow fuse, 286 00:14:23,910 --> 00:14:28,910 which just means like fuckin' brick this card. (chuckles) 287 00:14:29,430 --> 00:14:30,730 You don't want to try that 288 00:14:31,770 --> 00:14:32,670 in most cases. 289 00:14:32,670 --> 00:14:34,980 Like if you're really desperate, I guess you could. 290 00:14:34,980 --> 00:14:38,580 But there's some pitfalls to be avoided. 291 00:14:38,580 --> 00:14:41,880 So, that is a consideration that you have to make 292 00:14:41,880 --> 00:14:43,680 when doing this. 293 00:14:43,680 --> 00:14:46,530 But also, the speed of a card 294 00:14:46,530 --> 00:14:48,870 might not matter if you have a couple of 295 00:14:48,870 --> 00:14:52,920 the same type of card, like you have several debit cards 296 00:14:52,920 --> 00:14:55,200 from the same issuer, same bank, 297 00:14:55,200 --> 00:15:00,200 whatever, you can parallelize this across several readers. 298 00:15:00,570 --> 00:15:03,300 So if you had a particularly slow card, 299 00:15:03,300 --> 00:15:05,850 but you had access to a bunch of them, 300 00:15:05,850 --> 00:15:07,743 you could parallelize this effort. 301 00:15:10,170 --> 00:15:15,120 Now, worst case scenario, you have generic error messages, 302 00:15:15,120 --> 00:15:18,030 whenever anything goes wrong, whatsoever. 303 00:15:18,030 --> 00:15:21,180 When the card doesn't like anything about what you're doing, 304 00:15:21,180 --> 00:15:23,013 it gives you the same error message. 305 00:15:23,850 --> 00:15:27,990 That means a lot of brute forcing. 306 00:15:27,990 --> 00:15:29,070 Right? 307 00:15:29,070 --> 00:15:31,500 If you have a slow card on top of that, 308 00:15:31,500 --> 00:15:34,140 that only supports 10 commands per second, 309 00:15:34,140 --> 00:15:35,490 which is pretty slow, 310 00:15:35,490 --> 00:15:38,610 but I can't imagine anything slower than that. 311 00:15:38,610 --> 00:15:42,270 We're talking about six years to try all the combinations. 312 00:15:42,270 --> 00:15:43,290 So, that's not great. 313 00:15:43,290 --> 00:15:47,070 But luckily, we're not in this situation often. 314 00:15:47,070 --> 00:15:49,470 I haven't seen a smart card that is this 315 00:15:49,470 --> 00:15:50,790 worst case scenario yet. 316 00:15:50,790 --> 00:15:55,290 And I have like, (shuffling) this like, 317 00:15:55,290 --> 00:15:57,000 stack that I've been playing with 318 00:15:57,000 --> 00:15:58,050 over the last couple of months. 319 00:15:58,050 --> 00:16:01,413 So, I've, (shuffling) it's rare. 320 00:16:02,910 --> 00:16:05,040 Best case scenario, we're done very, very quickly. 321 00:16:05,040 --> 00:16:07,770 And actually, I have a sort of best case scenario card 322 00:16:07,770 --> 00:16:09,570 that I'm gonna show you in a minute. 323 00:16:11,130 --> 00:16:15,030 So, when we're trying to identify files, 324 00:16:15,030 --> 00:16:16,260 which is another step, right? 325 00:16:16,260 --> 00:16:18,810 We have a file system, or account way of, 326 00:16:18,810 --> 00:16:20,250 sort of a file system. 327 00:16:20,250 --> 00:16:22,953 Yeah, I guess it's a file system on a smart card. 328 00:16:24,090 --> 00:16:27,843 We can brute force through file names. 329 00:16:33,229 --> 00:16:34,590 There's this concept, there's what's called 330 00:16:34,590 --> 00:16:37,380 right truncation, which is supported by a lot of cards, 331 00:16:37,380 --> 00:16:41,310 which means, if you have the file, ABCDEFGH, 332 00:16:41,310 --> 00:16:43,290 you can ask for ABC, 333 00:16:43,290 --> 00:16:46,260 and the card will often say, good enough. 334 00:16:46,260 --> 00:16:48,840 Yeah, here you go, I'll give you the file, 335 00:16:48,840 --> 00:16:50,490 that's enough with the file name. 336 00:16:51,330 --> 00:16:54,090 There's often a minimum amount of bytes 337 00:16:54,090 --> 00:16:56,190 that you must provide before it'll go, okay, 338 00:16:56,190 --> 00:16:58,110 yeah, that's good enough. 339 00:16:58,110 --> 00:17:00,510 Usually, that's five bytes. 340 00:17:00,510 --> 00:17:03,330 But then, there's these things, registered identifiers. 341 00:17:03,330 --> 00:17:06,840 A lot of file names will start with a byte sequence 342 00:17:06,840 --> 00:17:10,260 that corresponds to a card manufacturer. 343 00:17:10,260 --> 00:17:13,800 And these card manufacturers are supposed to get 344 00:17:13,800 --> 00:17:16,440 one of these five byte registered identifiers, 345 00:17:16,440 --> 00:17:17,790 and make it public. 346 00:17:17,790 --> 00:17:18,960 There are public lists of it, 347 00:17:18,960 --> 00:17:22,800 there's like 600 some, yeah, 652 348 00:17:22,800 --> 00:17:24,510 in a public list that I found. 349 00:17:24,510 --> 00:17:27,030 So, you can just brute force through all of those. 350 00:17:27,030 --> 00:17:28,380 That's pretty quick. 351 00:17:28,380 --> 00:17:32,550 That works extremely well, extremely well. 352 00:17:32,550 --> 00:17:35,130 However, there's also the possibility 353 00:17:35,130 --> 00:17:38,400 that you have a card that just accepts one byte. 354 00:17:38,400 --> 00:17:43,400 So, you can just brute force through 1, 2, 3, 355 00:17:43,910 --> 00:17:45,870 4, all the way up to FF. 356 00:17:45,870 --> 00:17:49,440 And then if you get a hit on A, and you get 357 00:17:49,440 --> 00:17:51,990 a hit on 13, 358 00:17:51,990 --> 00:17:52,930 you can try 359 00:17:54,512 --> 00:17:59,512 A1, A2, A3, and so on, and so forth until you've enumerated 360 00:17:59,880 --> 00:18:01,710 all files on the card. 361 00:18:01,710 --> 00:18:04,953 So, that's another option, right? 362 00:18:06,570 --> 00:18:10,290 But sometimes, cards also describe themselves to you. 363 00:18:10,290 --> 00:18:11,340 Right? 364 00:18:11,340 --> 00:18:14,190 They have the opportunity to do so. 365 00:18:14,190 --> 00:18:16,620 One through, what's called the answer to reset. 366 00:18:16,620 --> 00:18:19,470 So when you hook up a card, 367 00:18:19,470 --> 00:18:21,900 the first thing that you do is you reset the card. 368 00:18:21,900 --> 00:18:26,820 You say, let's reset and start a new session here. 369 00:18:26,820 --> 00:18:31,820 And that can come with a response that has some information 370 00:18:32,820 --> 00:18:34,830 about how the card works. 371 00:18:34,830 --> 00:18:38,730 It might tell you, hey, here's the first command 372 00:18:38,730 --> 00:18:42,003 that I expect from you, which is really nice. 373 00:18:43,170 --> 00:18:46,050 And this is part of what's called the historical bytes. 374 00:18:46,050 --> 00:18:48,750 So sometimes, the historical bytes are sent along 375 00:18:48,750 --> 00:18:49,583 with the ATR. 376 00:18:49,583 --> 00:18:51,540 And sometimes the ATR says, yeah, 377 00:18:51,540 --> 00:18:53,490 just go look in the usual place. 378 00:18:53,490 --> 00:18:55,500 And there's a standardized usual place, 379 00:18:55,500 --> 00:18:59,970 which is like a standard file identifier, right? 380 00:18:59,970 --> 00:19:04,120 And there, you can find what issuer 381 00:19:05,280 --> 00:19:06,450 gave out this card, 382 00:19:06,450 --> 00:19:08,010 which may give you more information 383 00:19:08,010 --> 00:19:10,590 for documentation to look up. 384 00:19:10,590 --> 00:19:12,540 It may give you an indication 385 00:19:12,540 --> 00:19:14,910 of what application is implicitly selected. 386 00:19:14,910 --> 00:19:19,170 So sometimes, when you reset a smart card upon reset, 387 00:19:19,170 --> 00:19:21,435 it selects a file for you. 388 00:19:21,435 --> 00:19:22,290 You don't have to select a file, 389 00:19:22,290 --> 00:19:24,603 it just says, I'm gonna do this, right? 390 00:19:26,820 --> 00:19:28,830 And that's important because files 391 00:19:28,830 --> 00:19:31,200 are also applications, sometimes. 392 00:19:31,200 --> 00:19:32,790 We'll talk more about the file structure, 393 00:19:32,790 --> 00:19:34,170 and I probably should have put that earlier 394 00:19:34,170 --> 00:19:36,220 in the presentation, but we'll get there. 395 00:19:38,100 --> 00:19:42,780 So, it will tell you this card has support for file 396 00:19:42,780 --> 00:19:46,320 or application selection by ID number, 397 00:19:46,320 --> 00:19:49,050 or by name, or by path. 398 00:19:49,050 --> 00:19:51,360 We'll talk about those in a second. 399 00:19:51,360 --> 00:19:53,190 But it can also tell you, hey, 400 00:19:53,190 --> 00:19:54,990 there's also a directory file. 401 00:19:54,990 --> 00:19:56,970 'Cause that's a thing as well. 402 00:19:56,970 --> 00:20:00,150 You can have a file that is a directory of all 403 00:20:00,150 --> 00:20:02,880 the files on the smart card, 404 00:20:02,880 --> 00:20:06,183 which means that brute forcing files is unnecessary. 405 00:20:07,470 --> 00:20:11,790 What that also unfortunately means is that most smart cards 406 00:20:11,790 --> 00:20:13,260 do not have such a directory. 407 00:20:13,260 --> 00:20:17,700 There is no way, without just knowing in advance what files 408 00:20:17,700 --> 00:20:21,810 are supposed to be there, to just map out the file system. 409 00:20:21,810 --> 00:20:24,000 You can't just say, hey, what's in route? 410 00:20:24,000 --> 00:20:24,840 What's below that? 411 00:20:24,840 --> 00:20:26,430 What's in this and that, right? 412 00:20:26,430 --> 00:20:28,110 You can't do that without brute force 413 00:20:28,110 --> 00:20:31,533 unless something like EF.DIR exists. 414 00:20:35,970 --> 00:20:40,530 So, I think, yeah, these two slides are redundant. 415 00:20:40,530 --> 00:20:41,970 Let's move on. 416 00:20:41,970 --> 00:20:45,543 So, EF.DIR just provides the directory via available files. 417 00:20:46,980 --> 00:20:49,470 Now, let's talk about the file system structure. 418 00:20:49,470 --> 00:20:50,303 Right? 419 00:20:50,303 --> 00:20:54,663 So, this diagram is maybe a little bit confusing. 420 00:20:55,650 --> 00:20:57,750 But if you wanna blame somebody for it, it's not me, 421 00:20:57,750 --> 00:20:59,853 this is from the 7816 standard. 422 00:21:01,140 --> 00:21:04,500 So, you have this thing called an MF or master file. 423 00:21:04,500 --> 00:21:07,530 This represents the root of the file system in the sort of 424 00:21:07,530 --> 00:21:10,920 file systems that you're used to from Linux, Windows, 425 00:21:10,920 --> 00:21:14,280 whatever, you have a master file. 426 00:21:14,280 --> 00:21:16,320 And under that 427 00:21:16,320 --> 00:21:19,200 is one or more 428 00:21:19,200 --> 00:21:21,030 EFs and DFs. 429 00:21:21,030 --> 00:21:25,170 Now, a DF is a dedicated file that can contain data, 430 00:21:25,170 --> 00:21:28,230 that can contain, that can be an application, 431 00:21:28,230 --> 00:21:29,820 that can be a directory. 432 00:21:29,820 --> 00:21:31,890 It can be all of those. 433 00:21:31,890 --> 00:21:33,033 So, that's fun. 434 00:21:34,530 --> 00:21:37,300 Not every DF is an application, but 435 00:21:39,420 --> 00:21:41,430 every application is, 436 00:21:41,430 --> 00:21:42,813 it has a DF, right? 437 00:21:44,010 --> 00:21:46,920 And, then EFs are just files, elementary files 438 00:21:46,920 --> 00:21:50,490 that just have some data attached to them. 439 00:21:50,490 --> 00:21:51,573 So, 440 00:21:54,570 --> 00:21:56,580 files are also contexts. 441 00:21:56,580 --> 00:22:00,870 So, one thing that's really important to know is that 442 00:22:00,870 --> 00:22:05,870 the commands that are supported by a smart card 443 00:22:06,270 --> 00:22:09,723 are dependent not only on the smart card, 444 00:22:11,190 --> 00:22:12,390 just like the smart card, 445 00:22:12,390 --> 00:22:14,370 but also the state of the smart card. 446 00:22:14,370 --> 00:22:15,690 Right? 447 00:22:15,690 --> 00:22:17,340 Like what file has been selected. 448 00:22:17,340 --> 00:22:20,640 So, you might have four different commands 449 00:22:20,640 --> 00:22:23,880 that are available with the file that's implicitly selected 450 00:22:23,880 --> 00:22:26,100 when you reset the smart card. 451 00:22:26,100 --> 00:22:30,000 And then, you may get access to 10 or more commands 452 00:22:30,000 --> 00:22:32,430 when you select some file on the smart card. 453 00:22:32,430 --> 00:22:35,400 Because you're changing the context, right? 454 00:22:35,400 --> 00:22:37,290 So, this is important too, 455 00:22:37,290 --> 00:22:42,000 because now, we have to select a file, and then brute force 456 00:22:42,000 --> 00:22:45,840 all the parameters of an APDU to find valid commands, 457 00:22:45,840 --> 00:22:46,673 and things like that. 458 00:22:46,673 --> 00:22:50,463 So, this is important for our enumeration efforts as well. 459 00:22:53,640 --> 00:22:57,090 When you, I already covered this. 460 00:22:57,090 --> 00:22:59,070 The one thing that I didn't cover from this slide 461 00:22:59,070 --> 00:23:01,410 is that there is a national ID. 462 00:23:01,410 --> 00:23:04,890 So, we talked about registered identifiers just briefly. 463 00:23:04,890 --> 00:23:07,740 The five byte identifiers. 464 00:23:07,740 --> 00:23:11,010 Every nation has a first byte. 465 00:23:11,010 --> 00:23:14,400 So if you know that a card comes from a certain country, 466 00:23:14,400 --> 00:23:18,630 you can try brute forcing just the second through 467 00:23:18,630 --> 00:23:22,710 the fifth byte of the file name, and just use, 468 00:23:22,710 --> 00:23:25,830 rely on that country byte being correct. 469 00:23:25,830 --> 00:23:28,773 So, but that's small change. 470 00:23:30,270 --> 00:23:34,380 Now, there are other ways to select files other than 471 00:23:34,380 --> 00:23:37,383 a name that I've alluded to so far. 472 00:23:39,540 --> 00:23:41,943 You have implicit file selection for instance, 473 00:23:50,100 --> 00:23:52,743 which is where the card select the file for you. 474 00:23:54,360 --> 00:23:56,700 But then, there's also a two byte identifier, 475 00:23:56,700 --> 00:23:59,220 which only two bytes, that's not so bad, 476 00:23:59,220 --> 00:24:01,653 we can brute force through that pretty quickly. 477 00:24:03,083 --> 00:24:06,030 And then, there's paths, which are, they're about 478 00:24:06,030 --> 00:24:09,993 the closest thing to a file path that you'll see, 479 00:24:10,830 --> 00:24:15,540 like you would see on a traditional file system. 480 00:24:15,540 --> 00:24:18,300 And you have these two byte identifiers that start 481 00:24:18,300 --> 00:24:19,200 with the root. 482 00:24:19,200 --> 00:24:20,820 And then, you have just two bytes, 483 00:24:20,820 --> 00:24:23,790 and every two bytes defines like another path down. 484 00:24:23,790 --> 00:24:26,850 So, when we go back to this, 485 00:24:26,850 --> 00:24:31,290 the MF is always 3F00 in this scheme. 486 00:24:31,290 --> 00:24:35,640 And so, you might have a DF underneath that that is 487 00:24:35,640 --> 00:24:37,620 0002. 488 00:24:37,620 --> 00:24:39,040 And a DF under that that's 489 00:24:41,010 --> 00:24:44,040 1415, or something like that. 490 00:24:44,040 --> 00:24:48,810 And so, the path to that would be 3F0000021415. 491 00:24:48,810 --> 00:24:49,643 Right? 492 00:24:49,643 --> 00:24:53,340 Just sort of, they're kinda like a file path, 493 00:24:53,340 --> 00:24:55,800 but without any slashes, there's no delimiter 494 00:24:55,800 --> 00:24:58,110 to separate the elements of the path, 495 00:24:58,110 --> 00:25:03,060 it's just based on the fields by a byte position. 496 00:25:03,060 --> 00:25:04,271 Right? 497 00:25:04,271 --> 00:25:05,104 They're fixed length. 498 00:25:06,870 --> 00:25:07,703 And then, 499 00:25:09,480 --> 00:25:12,210 with those DFs selected, 500 00:25:12,210 --> 00:25:15,030 there are elementary files underneath them that can be 501 00:25:15,030 --> 00:25:17,013 read out that have data in them, right? 502 00:25:18,150 --> 00:25:21,573 And that can be through a couple of different confusingly, 503 00:25:26,730 --> 00:25:30,510 a couple of different commands. 504 00:25:30,510 --> 00:25:31,343 Right? 505 00:25:31,343 --> 00:25:34,740 So, read record and read binary are both about reading out 506 00:25:34,740 --> 00:25:38,130 elementary files attached to data files, 507 00:25:38,130 --> 00:25:40,350 and then get data, read files, 508 00:25:40,350 --> 00:25:42,873 reads data attached to the DF directly. 509 00:25:45,330 --> 00:25:50,330 So, there is a potential for sensitive data exposure here on 510 00:25:50,850 --> 00:25:52,740 a card that you might be assessing. 511 00:25:52,740 --> 00:25:57,480 So, as you look through the card, as you pull the data out, 512 00:25:57,480 --> 00:26:00,570 look at that data and make sure that there's nothing that 513 00:26:00,570 --> 00:26:04,320 should be exposed before authentication, right? 514 00:26:04,320 --> 00:26:06,510 Or possibly after authentication. 515 00:26:06,510 --> 00:26:07,566 Right? 516 00:26:07,566 --> 00:26:10,440 So if you find the private key material 517 00:26:10,440 --> 00:26:12,783 on like an HSM SmartCard, 518 00:26:14,130 --> 00:26:18,900 through just browsing, through trying, get data commands, 519 00:26:18,900 --> 00:26:21,000 and stuff like that, that's not very good. 520 00:26:22,380 --> 00:26:24,570 So, authentication on a smart card. 521 00:26:24,570 --> 00:26:26,820 We alluded to that a little bit. 522 00:26:26,820 --> 00:26:27,653 The simplest one, 523 00:26:27,653 --> 00:26:30,570 the one that's supported frequently is verify. 524 00:26:30,570 --> 00:26:35,570 And this, you just provide in plain text, a pin or password, 525 00:26:36,420 --> 00:26:39,750 and you just hope that you're either using an encrypted 526 00:26:39,750 --> 00:26:42,360 communication channel with the card, which is supported, 527 00:26:42,360 --> 00:26:44,670 but not frequently. 528 00:26:44,670 --> 00:26:46,860 It is at least part of the standard. 529 00:26:46,860 --> 00:26:49,923 I don't see a lot of cards that support it, but that said, 530 00:26:52,110 --> 00:26:54,240 you provide a pin or password. 531 00:26:54,240 --> 00:26:57,390 It goes across in the clear, and it's compared directly 532 00:26:57,390 --> 00:26:59,850 to whatever secret is on the card. 533 00:26:59,850 --> 00:27:02,640 And this is usually like a pin to unlock 534 00:27:02,640 --> 00:27:04,323 a SIM card, for instance. 535 00:27:06,630 --> 00:27:09,540 If you do have somebody 536 00:27:09,540 --> 00:27:11,640 observing this transaction, 537 00:27:11,640 --> 00:27:14,070 they can see this authenticator. 538 00:27:14,070 --> 00:27:15,335 Right? 539 00:27:15,335 --> 00:27:18,750 So, that is just an inherent problem with verify. 540 00:27:18,750 --> 00:27:22,350 And if this smart card is supposed to be used 541 00:27:22,350 --> 00:27:23,850 for something sensitive, 542 00:27:23,850 --> 00:27:25,800 that probably shouldn't be what's used. 543 00:27:26,760 --> 00:27:28,770 There's also other problems. 544 00:27:28,770 --> 00:27:31,560 There's been some research in doing 545 00:27:31,560 --> 00:27:34,140 timing attacks, for instance. 546 00:27:34,140 --> 00:27:34,973 Because 547 00:27:38,103 --> 00:27:39,420 if the way that you validate 548 00:27:39,420 --> 00:27:43,293 the pin is to check each character, character by character, 549 00:27:44,130 --> 00:27:46,920 through the whole password or pin, and then bail out 550 00:27:46,920 --> 00:27:49,560 once you reach an incorrect character, 551 00:27:49,560 --> 00:27:54,560 then you can time how long it takes to check given pins. 552 00:27:54,870 --> 00:27:57,240 And when one takes longer than another, 553 00:27:57,240 --> 00:28:00,000 you know you've got more of the leading bytes correct. 554 00:28:00,000 --> 00:28:01,205 Right? 555 00:28:01,205 --> 00:28:02,520 So, this is something that's been done before, 556 00:28:02,520 --> 00:28:05,280 but it's definitely worth, 557 00:28:05,280 --> 00:28:07,560 it's definitely worth mentioning. 558 00:28:07,560 --> 00:28:11,280 Eavesdropping, that's, we mentioned that. 559 00:28:11,280 --> 00:28:13,290 There's also the fact that some 560 00:28:13,290 --> 00:28:18,270 smart cards will enforce an upper limit on how many times 561 00:28:18,270 --> 00:28:20,433 you can try and get it wrong. 562 00:28:22,140 --> 00:28:25,920 That said, there is also a command reset retry counter, 563 00:28:25,920 --> 00:28:28,110 which if that's not authenticated, 564 00:28:28,110 --> 00:28:31,470 like if you can hit that without authenticating, 565 00:28:31,470 --> 00:28:32,400 then that's a problem too, 566 00:28:32,400 --> 00:28:35,350 because that basically means that you have unlimited tries. 567 00:28:36,810 --> 00:28:40,440 And then, the alternative is a challenge response, 568 00:28:40,440 --> 00:28:42,990 which again, is not, the details of that 569 00:28:42,990 --> 00:28:44,610 are up to the implementer. 570 00:28:44,610 --> 00:28:46,650 You can kinda whatever you want. 571 00:28:46,650 --> 00:28:51,650 And that's interesting, and there's also no, 572 00:28:51,750 --> 00:28:54,090 there's nothing that mandates a particular 573 00:28:54,090 --> 00:28:56,400 type of random number generator. 574 00:28:56,400 --> 00:28:57,360 Right? 575 00:28:57,360 --> 00:29:00,300 In theory, you could use a really crappy pseudo 576 00:29:00,300 --> 00:29:05,300 random number generator, and seed it, or seed it poorly. 577 00:29:05,400 --> 00:29:08,160 One of the big problems with IoT devices 578 00:29:08,160 --> 00:29:10,800 that has been a constant problem, 579 00:29:10,800 --> 00:29:13,830 a very hard to solve problem for a long time, is how to 580 00:29:13,830 --> 00:29:16,710 generate high quality random numbers. 581 00:29:16,710 --> 00:29:19,230 Because even if you have like a good algorithm, 582 00:29:19,230 --> 00:29:22,187 if you're starting from the same place, right? 583 00:29:22,187 --> 00:29:23,793 If you start with the same seed. 584 00:29:25,590 --> 00:29:26,423 Or, 585 00:29:28,842 --> 00:29:32,850 if your device just hasn't been up long enough 586 00:29:32,850 --> 00:29:36,930 to collect enough entropy to make good random numbers, 587 00:29:36,930 --> 00:29:39,570 it might be possible to just, first thing you do, 588 00:29:39,570 --> 00:29:40,890 ask for a challenge 589 00:29:40,890 --> 00:29:44,460 as soon as you reset the card immediately. 590 00:29:44,460 --> 00:29:48,240 And then, there's a very low set of states 591 00:29:48,240 --> 00:29:49,710 that it could be in. 592 00:29:49,710 --> 00:29:53,760 And so, it may be possible to predict the random 593 00:29:53,760 --> 00:29:56,400 number generators that will come out of a card. 594 00:29:56,400 --> 00:29:59,310 Just because of a low entropy situation. 595 00:29:59,310 --> 00:30:02,943 So, the cryptography might be entirely broken. 596 00:30:03,810 --> 00:30:07,230 I found one card that did not enforce 597 00:30:07,230 --> 00:30:08,610 a minimum challenge length. 598 00:30:08,610 --> 00:30:11,013 So, you could ask for a one byte challenge, 599 00:30:11,970 --> 00:30:13,683 which is not great. 600 00:30:14,730 --> 00:30:18,930 And especially if you can sit in between a card 601 00:30:18,930 --> 00:30:23,550 and a terminal, and get the one byte challenge to be issued, 602 00:30:23,550 --> 00:30:27,990 then you can try and get that same challenge again, 603 00:30:27,990 --> 00:30:31,350 and then authenticate to the card later, right? 604 00:30:31,350 --> 00:30:34,380 So, and just replay the response for that one byte. 605 00:30:34,380 --> 00:30:37,530 'Cause you have a 1 and 256 chance at random, 606 00:30:37,530 --> 00:30:40,410 if it is truly random, of getting that same challenge. 607 00:30:40,410 --> 00:30:42,330 And being able to reuse that response. 608 00:30:42,330 --> 00:30:43,773 So, fun stuff there. 609 00:30:45,720 --> 00:30:48,240 Something that exists that I have not had a chance 610 00:30:48,240 --> 00:30:52,410 to look at on a real smart card is 611 00:30:52,410 --> 00:30:53,730 SCQL. 612 00:30:53,730 --> 00:30:58,730 Smart cards actually have their own sort of SCQL variant. 613 00:30:59,670 --> 00:31:04,353 And it's, in practice, it doesn't look a lot like SQL. 614 00:31:05,310 --> 00:31:08,040 There is an analog to all the different like, 615 00:31:08,040 --> 00:31:10,203 insert, delete, update, whatever. 616 00:31:12,570 --> 00:31:14,310 This was the slide 617 00:31:14,310 --> 00:31:17,670 that I had planned to use for this. 618 00:31:17,670 --> 00:31:18,750 But as of yesterday, 619 00:31:18,750 --> 00:31:22,260 I actually ran into somebody and got talking with them, 620 00:31:22,260 --> 00:31:24,930 and ran my tool that I'm going to release on 621 00:31:24,930 --> 00:31:28,885 one of their smart cards, and it happened to release, 622 00:31:28,885 --> 00:31:31,080 it happened to support SCQL. 623 00:31:31,080 --> 00:31:32,070 But that was yesterday. 624 00:31:32,070 --> 00:31:37,070 So, I don't have time to really figure out how to use SCQL, 625 00:31:38,550 --> 00:31:42,840 but I will be publishing information on that soon, 626 00:31:42,840 --> 00:31:45,150 because finally, I have a card that does that. 627 00:31:45,150 --> 00:31:47,163 So, awesome. 628 00:31:49,140 --> 00:31:52,443 So, let's talk about some things to look for. 629 00:31:54,150 --> 00:31:56,883 We talked about unauthenticated sensitive commands. 630 00:31:58,380 --> 00:32:00,720 Reset, retry, counter, we mentioned that one, 631 00:32:00,720 --> 00:32:04,170 if you can reset the retry counter, you get infinite tries. 632 00:32:04,170 --> 00:32:05,580 Oop. 633 00:32:05,580 --> 00:32:08,880 One interesting thing is the Java card platform. 634 00:32:08,880 --> 00:32:11,220 The Java card platform is exactly what it sounds like. 635 00:32:11,220 --> 00:32:14,520 It's a Java virtual machine that runs on a smart card. 636 00:32:14,520 --> 00:32:16,620 And there is a load applet command. 637 00:32:16,620 --> 00:32:20,910 Now, ideally, this command would be disabled 638 00:32:20,910 --> 00:32:22,890 once the card goes to production. 639 00:32:22,890 --> 00:32:25,980 And as we all know, all application developers 640 00:32:25,980 --> 00:32:28,230 disable anything that's for debugging 641 00:32:28,230 --> 00:32:30,930 once something goes to production, right? 642 00:32:30,930 --> 00:32:34,140 Everybody knows that, they don't do that, 643 00:32:34,140 --> 00:32:35,090 they don't do that. 644 00:32:36,360 --> 00:32:38,220 Disable verification requirement. 645 00:32:38,220 --> 00:32:42,300 That's, it does exactly what it says on the tin. 646 00:32:42,300 --> 00:32:44,673 If that's unauthenticated, that's a problem. 647 00:32:47,340 --> 00:32:51,210 Sometimes, you may have undocumented applications 648 00:32:51,210 --> 00:32:54,210 or instructions that are left on a card 649 00:32:54,210 --> 00:32:55,953 for debugging purposes. 650 00:32:58,534 --> 00:33:01,260 Maybe you didn't lock down load applet, 651 00:33:01,260 --> 00:33:02,793 or something like that. 652 00:33:04,020 --> 00:33:06,510 Sometimes, you have default keys or passwords 653 00:33:06,510 --> 00:33:07,890 that are on a card. 654 00:33:07,890 --> 00:33:10,320 And the verify 655 00:33:10,320 --> 00:33:11,950 is just 656 00:33:15,600 --> 00:33:18,480 something that's used to prove that you're the right kind 657 00:33:18,480 --> 00:33:19,650 of card reader. 658 00:33:19,650 --> 00:33:20,935 Right? 659 00:33:20,935 --> 00:33:24,840 So, a company that makes physical access cards for instance, 660 00:33:24,840 --> 00:33:29,010 might have a key that they keep secret. 661 00:33:29,010 --> 00:33:32,010 That if you disassemble the device and pull 662 00:33:32,010 --> 00:33:34,080 the firmware, and dig through it, 663 00:33:34,080 --> 00:33:37,470 you can find the key and be the right kind of card reader 664 00:33:37,470 --> 00:33:40,830 to access it, and do whatever you like with a card. 665 00:33:40,830 --> 00:33:43,263 Reprogram it, give it a new, whatever. 666 00:33:44,700 --> 00:33:48,780 There's also a classic problem with MIFARE cards. 667 00:33:48,780 --> 00:33:53,040 If anybody is staying at Harrah's or Flamingo, 668 00:33:53,040 --> 00:33:55,890 your cards are MIFARE 1K Classics, 669 00:33:55,890 --> 00:33:57,880 and they are actually using 670 00:34:00,527 --> 00:34:03,575 default keys for several of the sectors, 671 00:34:03,575 --> 00:34:04,408 which is interesting. 672 00:34:07,136 --> 00:34:09,210 There are a lot of cards that have a verify code 673 00:34:09,210 --> 00:34:10,950 of four zeros. 674 00:34:10,950 --> 00:34:11,783 Just 675 00:34:13,950 --> 00:34:15,510 in their normal state. 676 00:34:15,510 --> 00:34:17,073 So, that could be an issue. 677 00:34:18,540 --> 00:34:21,990 But then, you also have to think about like, smart cards 678 00:34:21,990 --> 00:34:23,910 are miniature computers. 679 00:34:23,910 --> 00:34:25,950 They are running code. 680 00:34:25,950 --> 00:34:30,150 APDUs are the way of accessing an API, right? 681 00:34:30,150 --> 00:34:32,313 We are issuing commands and giving data. 682 00:34:33,150 --> 00:34:36,880 One really interesting thing is that you have 683 00:34:39,776 --> 00:34:42,330 this TLV format, which is Tag Length Value. 684 00:34:42,330 --> 00:34:46,320 So, the way that a lot of data is expressed 685 00:34:46,320 --> 00:34:51,240 in smart card communications is, you have one byte. 686 00:34:51,240 --> 00:34:53,670 Sometimes, you can have multiple byte tags, 687 00:34:53,670 --> 00:34:54,750 but one byte that says, 688 00:34:54,750 --> 00:34:58,590 this is the type of data that I'm about to give to you. 689 00:34:58,590 --> 00:35:02,340 Then, you have a one byte, or it's a fixed field, 690 00:35:02,340 --> 00:35:04,540 sometimes it's one byte, sometimes it's two. 691 00:35:05,628 --> 00:35:06,690 In smart card world, it's, 692 00:35:06,690 --> 00:35:09,150 like, I've only ever seen one byte. 693 00:35:09,150 --> 00:35:11,580 So, you have a one byte value that says here's the length of 694 00:35:11,580 --> 00:35:13,350 the data that I'm about to give you. 695 00:35:13,350 --> 00:35:15,240 So, one byte to determine what kind of data, 696 00:35:15,240 --> 00:35:17,610 one byte to determine the length. 697 00:35:17,610 --> 00:35:19,080 And then, you just have the raw data 698 00:35:19,080 --> 00:35:21,090 that should match the length. 699 00:35:21,090 --> 00:35:25,710 Now, that data can itself be TLV values. 700 00:35:25,710 --> 00:35:28,680 So, you can have, that value has 701 00:35:28,680 --> 00:35:31,830 a tag length value structure as well. 702 00:35:31,830 --> 00:35:36,510 And when you're trying to parse these nested structures, 703 00:35:36,510 --> 00:35:40,440 there's a lot of opportunities for buffer mismanagement, 704 00:35:40,440 --> 00:35:43,470 especially when these length values 705 00:35:43,470 --> 00:35:45,963 or tag values are malformed. 706 00:35:47,700 --> 00:35:50,490 So, there's a lot of potential problems here 707 00:35:50,490 --> 00:35:52,320 with buffer overflows. 708 00:35:52,320 --> 00:35:53,751 Maybe not so much 709 00:35:53,751 --> 00:35:54,584 when you're using Java card, 710 00:35:54,584 --> 00:35:58,740 but there's plenty of parser issues that are 711 00:35:58,740 --> 00:36:00,540 potentially a problem. 712 00:36:00,540 --> 00:36:03,870 There's plenty of just generic application security bugs 713 00:36:03,870 --> 00:36:08,804 that could be present in the various commands. 714 00:36:08,804 --> 00:36:11,830 So, anything to do with just 715 00:36:13,020 --> 00:36:15,180 your generic application security bugs 716 00:36:15,180 --> 00:36:16,650 could be a problem here, 717 00:36:16,650 --> 00:36:19,650 depending on how the card is built, 718 00:36:19,650 --> 00:36:22,440 how the card is programmed, what libraries it uses 719 00:36:22,440 --> 00:36:25,773 to achieve its function, and all that. 720 00:36:27,540 --> 00:36:28,920 There may be cryptographic flaws. 721 00:36:28,920 --> 00:36:32,223 We talked about the one byte challenge. 722 00:36:33,390 --> 00:36:34,533 There is, 723 00:36:36,180 --> 00:36:40,590 the ISO 7816 standard is a little bit old. 724 00:36:40,590 --> 00:36:44,370 And some of the recommendations that it makes for things 725 00:36:44,370 --> 00:36:48,063 that cards might support are a little whack. 726 00:36:50,181 --> 00:36:51,991 Like, one of the things I've seen 727 00:36:51,991 --> 00:36:53,040 is a recommendation on RSA 728 00:36:53,040 --> 00:36:55,740 with no padding, which, if you're a cryptographer, 729 00:36:55,740 --> 00:36:56,673 you just winced. 730 00:36:58,020 --> 00:37:02,130 Because RSA with no padding is terrible. 731 00:37:02,130 --> 00:37:04,290 So, that's one of the things that comes up 732 00:37:04,290 --> 00:37:06,570 in the standard, is a possibility for what you could do 733 00:37:06,570 --> 00:37:07,403 with your card. 734 00:37:07,403 --> 00:37:10,233 So, somebody has probably implemented that. 735 00:37:11,580 --> 00:37:14,283 So, what did I find on real smart cards? 736 00:37:15,300 --> 00:37:17,103 Historical bytes are pretty common. 737 00:37:17,970 --> 00:37:21,420 They will often tell you at least one application ID 738 00:37:21,420 --> 00:37:23,190 that you can work off of. 739 00:37:23,190 --> 00:37:28,190 And it's usually fruitful to brute force 740 00:37:28,320 --> 00:37:30,840 based off of the couple bytes at the start of 741 00:37:30,840 --> 00:37:32,373 that application identifier. 742 00:37:33,960 --> 00:37:35,190 It often describes 743 00:37:35,190 --> 00:37:38,490 how you're supposed to select files from a card. 744 00:37:38,490 --> 00:37:41,430 I have never seen a card support EF.DIR. 745 00:37:41,430 --> 00:37:43,413 That I haven't seen yet. 746 00:37:45,240 --> 00:37:50,240 Most cards want you to select files by DF name. 747 00:37:50,430 --> 00:37:54,393 I've seen one card that supports selection by identifier. 748 00:37:57,060 --> 00:37:59,250 The whole DF name thing, like, 749 00:37:59,250 --> 00:38:02,430 there's cards that don't require 750 00:38:02,430 --> 00:38:03,900 a minimum number of bytes, 751 00:38:03,900 --> 00:38:06,633 but most of them require around five bytes. 752 00:38:08,460 --> 00:38:12,090 I found a card that supports the one byte challenge, 753 00:38:12,090 --> 00:38:13,830 as I mentioned earlier. 754 00:38:13,830 --> 00:38:15,810 I'm still talking with the vendor, 755 00:38:15,810 --> 00:38:20,193 so I can't tell you what type of card it was, unfortunately. 756 00:38:22,470 --> 00:38:25,443 But now, let me show you my tool that I built. 757 00:38:27,180 --> 00:38:29,013 So, it's called Wubblegum. 758 00:38:30,750 --> 00:38:33,750 It currently will 759 00:38:33,750 --> 00:38:38,640 enumerate through CLA values, files, INS 760 00:38:38,640 --> 00:38:42,660 commands, and then will dump any records or data attached 761 00:38:42,660 --> 00:38:44,880 to the files that it finds. 762 00:38:44,880 --> 00:38:48,390 And it will tell you, for any command that it discovers, 763 00:38:48,390 --> 00:38:52,023 if that byte corresponds to like a well-known standard, 764 00:38:54,970 --> 00:38:57,780 and what that command byte does. 765 00:38:57,780 --> 00:38:59,100 Right? 766 00:38:59,100 --> 00:39:00,633 So, let's take a look at that. 767 00:39:02,148 --> 00:39:05,853 I have an old expired debit card here. 768 00:39:07,350 --> 00:39:11,973 And we'll just see this in action. 769 00:39:19,380 --> 00:39:23,673 Now, let me just make sure that I've got the right reader. 770 00:39:25,772 --> 00:39:28,950 (keyboard keys clicking) 771 00:39:28,950 --> 00:39:31,950 And by the way, the YubiKey is kind of like 772 00:39:31,950 --> 00:39:35,910 a combination smart card and smart card reader, 773 00:39:35,910 --> 00:39:38,210 which is why it's showing up in the list here. 774 00:39:39,210 --> 00:39:41,200 So, I'm going to want to 775 00:39:43,380 --> 00:39:45,813 adjust which reader I'm using here. 776 00:39:47,370 --> 00:39:50,970 So, I'm using the options to enumerate 777 00:39:50,970 --> 00:39:54,900 class, files by name, INS, data. 778 00:39:54,900 --> 00:39:58,530 And I'm using the RID, the public RID list 779 00:39:58,530 --> 00:40:00,880 brute force technique that I discussed earlier. 780 00:40:02,910 --> 00:40:06,840 This card is pretty good about its error responses, 781 00:40:06,840 --> 00:40:09,240 which is why I wanted to show you this one. 782 00:40:09,240 --> 00:40:13,260 So, it gets through the CLA brute force really, really fast. 783 00:40:13,260 --> 00:40:16,500 Like, I didn't even get a chance to explain 784 00:40:16,500 --> 00:40:18,690 what it was going to do before it did it. 785 00:40:18,690 --> 00:40:22,080 Now, cards are a little bit weird about how they respond. 786 00:40:22,080 --> 00:40:26,490 So, I built in a thing where you can prune responses 787 00:40:26,490 --> 00:40:29,580 based on their response code. 788 00:40:29,580 --> 00:40:32,160 So, it gives like, it parses the response code, 789 00:40:32,160 --> 00:40:35,520 and tells you in plain English, like, what happened here? 790 00:40:35,520 --> 00:40:37,070 What is this complaining about? 791 00:40:38,730 --> 00:40:39,873 So, 792 00:40:43,290 --> 00:40:44,370 oh that's, 793 00:40:44,370 --> 00:40:46,200 I got the wrong card, my bad. 794 00:40:46,200 --> 00:40:47,310 That was the YubiKey. 795 00:40:47,310 --> 00:40:48,713 I didn't wanna (chuckles) do that. 796 00:40:51,360 --> 00:40:53,460 Let's switch to the right card. 797 00:40:53,460 --> 00:40:54,990 So, this takes slightly longer. 798 00:40:54,990 --> 00:40:57,060 This is a little slower, but here we go. 799 00:40:57,060 --> 00:41:01,560 So, this is saying, you see like, category three there, 800 00:41:01,560 --> 00:41:04,470 incorrect application, which is incorrect CLA. 801 00:41:04,470 --> 00:41:07,470 So, we're gonna say delete that. 802 00:41:07,470 --> 00:41:11,017 And then, we have 36 responses that say this, 803 00:41:11,017 --> 00:41:13,197 "Logical channel is not supported," 804 00:41:14,370 --> 00:41:17,490 which essentially means, like yeah, this is like, 805 00:41:17,490 --> 00:41:19,440 you've got the right prefix on this, 806 00:41:19,440 --> 00:41:22,200 but the bit flags aren't quite right. 807 00:41:22,200 --> 00:41:24,690 But logical channel support not supported means 808 00:41:24,690 --> 00:41:26,820 it's not gonna work no matter what we do. 809 00:41:26,820 --> 00:41:30,543 So, we're gonna wanna just say, yeah, no. 810 00:41:32,010 --> 00:41:34,710 So, we'll delete that as well. 811 00:41:34,710 --> 00:41:38,040 So, now we have incorrect parameters, which is like, great, 812 00:41:38,040 --> 00:41:41,100 we got a CLA and an INS correct at the same time. 813 00:41:41,100 --> 00:41:43,140 And then we got other responses that say, yeah, 814 00:41:43,140 --> 00:41:44,640 this INS is not supported on this. 815 00:41:44,640 --> 00:41:46,350 So, these are all good, we're gonna accept that 816 00:41:46,350 --> 00:41:47,520 and move on. 817 00:41:47,520 --> 00:41:49,350 Now, it's going to brute force through 818 00:41:49,350 --> 00:41:53,520 these five byte registered identifiers, 819 00:41:53,520 --> 00:41:56,160 and find whichever ones are available. 820 00:41:56,160 --> 00:42:00,600 So, since this is a credit card, 821 00:42:00,600 --> 00:42:02,940 debit card, like, it supports credit card and debit card. 822 00:42:02,940 --> 00:42:05,160 It's going to have applications for both 823 00:42:05,160 --> 00:42:07,230 of those usages on it. 824 00:42:07,230 --> 00:42:09,630 Corresponding to visa, which it is. 825 00:42:09,630 --> 00:42:13,233 So like, if you look up A0003, 826 00:42:14,790 --> 00:42:18,270 that's going to, you're gonna find visa, right? 827 00:42:18,270 --> 00:42:23,270 And so, it found a couple of different commands under 828 00:42:23,670 --> 00:42:26,070 this application identifier. 829 00:42:26,070 --> 00:42:27,750 And under this other one, 830 00:42:27,750 --> 00:42:30,720 we have a lot more complicated stuff of like, 831 00:42:30,720 --> 00:42:33,150 we have verify, and we have 832 00:42:33,150 --> 00:42:36,423 this get challenge, and authenticate. 833 00:42:37,703 --> 00:42:38,850 And we have Read Record(s). 834 00:42:38,850 --> 00:42:41,310 So, we have a bunch of records and data 835 00:42:41,310 --> 00:42:43,500 that are on this card. 836 00:42:43,500 --> 00:42:45,540 So, we see, 837 00:42:45,540 --> 00:42:49,620 we're dumping data from this application, 838 00:42:49,620 --> 00:42:51,390 and from that application. 839 00:42:51,390 --> 00:42:54,090 And some of this is actually readable. 840 00:42:54,090 --> 00:42:56,520 So, like, when we take this data, 841 00:42:56,520 --> 00:42:57,370 and when we 842 00:43:00,201 --> 00:43:03,701 (keyboard keys clicking) 843 00:43:04,560 --> 00:43:06,600 hex decode that, 844 00:43:06,600 --> 00:43:07,653 enhance. 845 00:43:10,290 --> 00:43:13,383 So, there's my name. 846 00:43:14,340 --> 00:43:16,533 So, we can pull some data from this. 847 00:43:18,210 --> 00:43:21,120 I don't believe that there is any sensitive data exposed 848 00:43:21,120 --> 00:43:23,130 by this card without authentication, 849 00:43:23,130 --> 00:43:26,520 which is why I'm happy to show you my expired credit card, 850 00:43:26,520 --> 00:43:27,450 also it's expired. 851 00:43:27,450 --> 00:43:29,460 So like, if you could manage to steal 852 00:43:29,460 --> 00:43:31,680 my credit card from this, then, like sorry, 853 00:43:31,680 --> 00:43:33,450 it's already expired. 854 00:43:33,450 --> 00:43:35,250 Unless you can manage to get around that, 855 00:43:35,250 --> 00:43:39,570 in which case, I don't know, I mean good for you, I guess. 856 00:43:39,570 --> 00:43:41,870 I guess I'll be impressed, and we should talk. 857 00:43:44,040 --> 00:43:45,294 But yeah. 858 00:43:45,294 --> 00:43:47,820 So, I already know that this card, it's going to brute force 859 00:43:47,820 --> 00:43:51,240 through a whole bunch of potential record IDs, 860 00:43:51,240 --> 00:43:52,710 and not find anything more here. 861 00:43:52,710 --> 00:43:54,900 So, I'm gonna cut this a little short. 862 00:43:54,900 --> 00:43:58,920 I've done, proven that it works. 863 00:43:58,920 --> 00:43:59,753 So, 864 00:44:02,543 --> 00:44:03,689 that's the tool. 865 00:44:03,689 --> 00:44:06,450 This will be made available, by the way, 866 00:44:06,450 --> 00:44:09,870 on the X-Force Red GitHub account, 867 00:44:09,870 --> 00:44:13,110 shortly after I get home from Vegas. 868 00:44:13,110 --> 00:44:15,723 So, expected in the next couple of days. 869 00:44:17,580 --> 00:44:20,100 And there will probably be some bug fixes that come along, 870 00:44:20,100 --> 00:44:21,870 some new features added over time. 871 00:44:21,870 --> 00:44:23,640 This is still very much a work in progress. 872 00:44:23,640 --> 00:44:26,790 There's still, there's plenty we talked about today 873 00:44:26,790 --> 00:44:28,830 that hasn't been implemented in the tool yet. 874 00:44:28,830 --> 00:44:30,370 So right now, it's just 875 00:44:32,310 --> 00:44:37,110 sort of like a primitive port scanner, but for smart cards. 876 00:44:37,110 --> 00:44:39,960 And I'll build some basic vulnerability checks, 877 00:44:39,960 --> 00:44:41,940 and things like that on top of it. 878 00:44:41,940 --> 00:44:43,740 But if you have a smart card reader, 879 00:44:44,670 --> 00:44:47,160 I can recommend the HID Omnikey. 880 00:44:47,160 --> 00:44:48,240 No, I don't work for them. 881 00:44:48,240 --> 00:44:50,640 No, I'm not being sponsored by them. 882 00:44:50,640 --> 00:44:53,790 It's not terribly expensive. 883 00:44:53,790 --> 00:44:56,580 If you want a smart card reader to start playing 884 00:44:56,580 --> 00:44:59,490 with your cards, and with this tool. 885 00:44:59,490 --> 00:45:01,940 There's some future work to be done, like I said. 886 00:45:02,790 --> 00:45:06,273 I would like to be able to enumerate things from, 887 00:45:10,110 --> 00:45:11,940 oh, I screwed up this slide. 888 00:45:11,940 --> 00:45:14,700 That was not supposed to say additional enumeration support. 889 00:45:14,700 --> 00:45:18,630 Yeah, so, I want to be able to attack these things 890 00:45:18,630 --> 00:45:19,860 using a card. 891 00:45:19,860 --> 00:45:20,693 Right? 892 00:45:20,693 --> 00:45:24,540 So, that's a big thing would be, card to terminal fuzzing 893 00:45:24,540 --> 00:45:25,800 would be very interesting. 894 00:45:25,800 --> 00:45:27,963 And like I said, vulnerability checks. 895 00:45:29,490 --> 00:45:30,780 But it would be very cool to, 896 00:45:30,780 --> 00:45:34,170 for instance, pop a card into an ATM 897 00:45:34,170 --> 00:45:37,290 and have a jackpot, because it exploits some sort of 898 00:45:37,290 --> 00:45:40,080 parsing error in the parser that takes in 899 00:45:40,080 --> 00:45:41,433 the response from the card. 900 00:45:42,450 --> 00:45:44,640 So with that, I've gotta wrap it up. 901 00:45:44,640 --> 00:45:46,860 I wanna thank Ludovic Russo for all of his work 902 00:45:46,860 --> 00:45:48,060 on smart cards in the past. 903 00:45:48,060 --> 00:45:50,250 I wanna thank Foundation for giving me a smattering 904 00:45:50,250 --> 00:45:52,050 of smart cards to play with. 905 00:45:52,050 --> 00:45:53,820 Jean Daniel (speaking in foreign language) 906 00:45:53,820 --> 00:45:57,690 wrote the Python library that my tool is based on. 907 00:45:57,690 --> 00:46:00,630 Christopher Boshman gave me, he was the guy I met yesterday, 908 00:46:00,630 --> 00:46:03,480 who, really nice dude. 909 00:46:03,480 --> 00:46:07,320 Chief has this great list of APDU commands from a bunch 910 00:46:07,320 --> 00:46:08,153 of different things. 911 00:46:08,153 --> 00:46:10,680 I don't know who the guy is, but he's a legend. 912 00:46:10,680 --> 00:46:13,200 And also, shout outs to Alaska and the Hack This Site crew, 913 00:46:13,200 --> 00:46:14,820 and X-Force. 914 00:46:14,820 --> 00:46:16,875 And that's it. 915 00:46:16,875 --> 00:46:17,708 I think I'm 916 00:46:17,708 --> 00:46:20,708 (audience clapping) 917 00:46:26,790 --> 00:46:27,623 flat out time. 918 00:46:27,623 --> 00:46:31,050 So, I will take questions elsewhere. 919 00:46:31,050 --> 00:46:33,844 Yeah, I'll take questions over there. 920 00:46:33,844 --> 00:46:34,677 There are ways. 921 00:46:34,677 --> 00:46:35,510 Thank you. 922 00:46:37,316 --> 00:46:40,316 (audience clapping)