1 00:00:00,000 --> 00:00:01,740 Thank you very much for the invite. 2 00:00:06,620 --> 00:00:09,120 I've never been to DEF CON or Black Hat before, 3 00:00:09,280 --> 00:00:12,780 so this is kind of an interesting phenomena for me. 4 00:00:15,980 --> 00:00:16,420 Thanks. 5 00:00:16,620 --> 00:00:18,560 I'm an engineer that got into security 6 00:00:18,560 --> 00:00:20,060 rather than the other way around. 7 00:00:20,460 --> 00:00:21,760 So what you're going to hear 8 00:00:21,760 --> 00:00:23,640 is the engineering view of the world 9 00:00:24,120 --> 00:00:26,760 and what's going on and what's needed and what's missing. 10 00:00:28,020 --> 00:00:32,800 I've deliberately put this slide up. 11 00:00:33,380 --> 00:00:35,560 Everything we're talking about is this guy right here. 12 00:00:36,700 --> 00:00:39,920 If you saw that in the village, that's a sensor. 13 00:00:40,480 --> 00:00:42,700 In this case, it's a 4 to 20 milliamp sensor. 14 00:00:43,840 --> 00:00:46,080 It's probably measuring flow. 15 00:00:46,340 --> 00:00:47,300 They put it in a line. 16 00:00:47,360 --> 00:00:50,360 It could be measuring pressure, flow, temperature, picket. 17 00:00:50,680 --> 00:00:52,440 What we're talking about are fingers. 18 00:00:54,040 --> 00:00:56,000 This is what measures things. 19 00:00:56,000 --> 00:00:58,600 And it's going to the brain, which would be the controllers. 20 00:00:59,840 --> 00:01:02,880 If you cut your fingers off, the brain can't do anything. 21 00:01:04,600 --> 00:01:06,300 And that's what all of this is about. 22 00:01:07,520 --> 00:01:09,640 I'm going to take it now back to the beginning. 23 00:01:13,340 --> 00:01:13,740 Okay. 24 00:01:17,240 --> 00:01:20,220 So, like I say, I wanted to thank Larry for inviting me 25 00:01:20,220 --> 00:01:23,340 because this is a really important area 26 00:01:23,340 --> 00:01:25,480 and one of the biggest things you're going to get out of this. 27 00:01:26,000 --> 00:01:29,240 Is we've got to, got to, got to have the engineers 28 00:01:29,240 --> 00:01:32,660 and the IT security people working together. 29 00:01:32,980 --> 00:01:35,680 And that is not happening in most cases. 30 00:01:36,320 --> 00:01:40,180 Because what we're talking about here is pure engineering. 31 00:01:40,420 --> 00:01:41,480 It is not IT. 32 00:01:45,680 --> 00:01:47,460 You've probably heard all of this. 33 00:01:48,220 --> 00:01:51,740 The big thing is that last bullet and what's in bold. 34 00:01:52,960 --> 00:01:55,980 Control systems monitor and control things. 35 00:01:56,000 --> 00:01:56,840 Physical processes. 36 00:01:57,680 --> 00:01:59,540 It doesn't matter what it is. 37 00:01:59,620 --> 00:02:04,720 It doesn't matter if it's a power plant, a pipe, a human body. 38 00:02:05,300 --> 00:02:08,680 If you've got physical processes, you know, 39 00:02:08,680 --> 00:02:10,440 and you've got to monitor, for example, 40 00:02:10,440 --> 00:02:13,160 temperature or pressure or flow or radiation, 41 00:02:13,520 --> 00:02:16,960 and then you want to control it, that's a control system. 42 00:02:17,620 --> 00:02:19,200 Industrial was a poor term. 43 00:02:19,220 --> 00:02:20,620 We didn't know what else to call it. 44 00:02:21,200 --> 00:02:25,900 Because all of the building controls here are essentially ICS. 45 00:02:25,900 --> 00:02:27,200 Because you're monitoring temperature. 46 00:02:27,960 --> 00:02:32,560 You're monitoring, you know, steam flow, water flow, humidity. 47 00:02:33,460 --> 00:02:35,040 All of the things you think about. 48 00:02:35,120 --> 00:02:35,940 There's nothing different. 49 00:02:36,820 --> 00:02:39,880 So everything we're talking about here crosses every border. 50 00:02:40,960 --> 00:02:45,180 But the big thing is, most of you, if you're coming from the IT world, 51 00:02:45,320 --> 00:02:46,560 you have the CIA paradigm. 52 00:02:46,880 --> 00:02:49,160 You know, confidentiality, integrity, availability. 53 00:02:50,080 --> 00:02:52,520 It isn't just that it's backwards for us. 54 00:02:53,000 --> 00:02:55,720 It's actually missing the most important letter. 55 00:02:55,900 --> 00:02:57,440 And that's the letter S. 56 00:02:58,440 --> 00:03:00,140 And the letter S is for safety. 57 00:03:01,320 --> 00:03:03,940 Because in our world, we can kill people. 58 00:03:05,400 --> 00:03:06,860 That's the most important. 59 00:03:07,120 --> 00:03:10,400 Everything we do is reliability and safety. 60 00:03:12,120 --> 00:03:14,920 Confidentiality, really one of the only places it's important, 61 00:03:15,760 --> 00:03:20,600 is if you have a meter, like a smart meter in a house or something like that. 62 00:03:21,380 --> 00:03:25,740 For almost everything else, confidentiality is very, very, 63 00:03:25,900 --> 00:03:26,320 very trivial. 64 00:03:27,200 --> 00:03:31,260 When you put it in a database and store it somewhere, 65 00:03:31,380 --> 00:03:34,100 then it becomes IT and confidentiality is a big thing. 66 00:03:34,740 --> 00:03:36,620 We're talking about data in motion. 67 00:03:38,000 --> 00:03:40,500 So one of the big things, and this is a problem in Europe. 68 00:03:40,600 --> 00:03:44,700 Europe is totally confidentiality based or privacy based. 69 00:03:45,080 --> 00:03:47,500 In the control system world, that's not us. 70 00:03:51,860 --> 00:03:54,940 This is just a Venn diagram I've had for years. 71 00:03:55,900 --> 00:03:59,980 Part of our problem is, you've got two worlds coming together. 72 00:04:00,440 --> 00:04:03,060 The IT world and the control system world. 73 00:04:03,480 --> 00:04:04,980 And you'll notice the number of arrows. 74 00:04:05,740 --> 00:04:09,020 Most of the people that are working this are coming from IT. 75 00:04:10,620 --> 00:04:12,560 Very few are coming from the domain. 76 00:04:13,860 --> 00:04:18,020 And part of it is, it's unfortunate, 77 00:04:18,200 --> 00:04:21,360 but control system people generally do not like security. 78 00:04:21,580 --> 00:04:22,980 It's an impediment to what they do. 79 00:04:24,360 --> 00:04:25,540 So you have... 80 00:04:25,900 --> 00:04:29,380 You don't have near enough people who actually understand the domain 81 00:04:29,380 --> 00:04:30,640 even wanting to be in this. 82 00:04:31,940 --> 00:04:33,660 And we need to figure out what to do. 83 00:04:34,300 --> 00:04:37,040 There is a really, really, really small dot 84 00:04:38,300 --> 00:04:42,880 where you have actual ICS security experts. 85 00:04:44,020 --> 00:04:45,960 And by the way, one of the things I always tell everybody, 86 00:04:46,600 --> 00:04:49,720 if somebody tells you they're a SCADA security expert, 87 00:04:49,860 --> 00:04:50,600 run the other way. 88 00:04:51,680 --> 00:04:55,640 It generally means they learned how to spell SCADA within a year or so. 89 00:04:56,980 --> 00:04:58,640 Because there's one other thing. 90 00:04:59,100 --> 00:05:03,140 How many people would go to Macworld and only want to talk about PCs? 91 00:05:03,900 --> 00:05:05,520 Sounds kind of silly, doesn't it? 92 00:05:06,540 --> 00:05:10,600 When you talk about SCADA, SCADA is not every control system. 93 00:05:11,400 --> 00:05:14,480 SCADA is only one type of control system. 94 00:05:14,940 --> 00:05:17,480 And it isn't used in a lot of places. 95 00:05:19,080 --> 00:05:21,260 So when you use the term SCADA, 96 00:05:21,860 --> 00:05:23,320 and you don't use it right, 97 00:05:23,440 --> 00:05:25,460 you will lose all credibility really fast. 98 00:05:30,660 --> 00:05:36,160 We are here in, you could either say the cyber-physical world, 99 00:05:36,280 --> 00:05:39,120 but really we're in the physical cyber world. 100 00:05:39,780 --> 00:05:42,280 Everything we're talking about is physical. 101 00:05:43,260 --> 00:05:45,620 Cyber happens to play into it. 102 00:05:46,740 --> 00:05:49,160 And you need to address it from that perspective. 103 00:05:52,080 --> 00:05:54,660 People hear about advanced persistent threats. 104 00:05:55,060 --> 00:05:55,880 When you hear about advanced persistent threats, 105 00:05:55,880 --> 00:05:57,500 when you look at things like Stuxnet, 106 00:05:59,700 --> 00:06:03,000 those were hard-coded default passwords 107 00:06:03,000 --> 00:06:07,260 that were part of the design of that controller. 108 00:06:07,680 --> 00:06:09,320 That wasn't an APT. 109 00:06:09,520 --> 00:06:12,640 In a sense, that was a persistent design vulnerability. 110 00:06:14,640 --> 00:06:16,220 You can't change it. 111 00:06:16,920 --> 00:06:18,600 And by the way, this isn't just Siemens. 112 00:06:18,760 --> 00:06:20,720 We have a lot of PLC manufacturers 113 00:06:21,580 --> 00:06:24,280 that have, if you will, 114 00:06:24,340 --> 00:06:25,860 used the password, 115 00:06:25,880 --> 00:06:32,940 as part of their actual software computing platform. 116 00:06:33,600 --> 00:06:36,340 So if you change the hard-coded default password, 117 00:06:36,480 --> 00:06:38,100 that PLC falls on the floor dead. 118 00:06:39,540 --> 00:06:42,120 This is not like what you would normally think. 119 00:06:42,220 --> 00:06:44,100 Well, geez, how stupid can you be? 120 00:06:44,500 --> 00:06:46,160 This was never, remember, 121 00:06:46,260 --> 00:06:48,700 we didn't start out life worrying about security. 122 00:06:49,680 --> 00:06:51,960 We worry about reliability and safety. 123 00:06:52,520 --> 00:06:55,720 And a lot of this came because we were trying to capture, 124 00:06:57,240 --> 00:06:59,420 meantime, between failure data, 125 00:06:59,600 --> 00:07:00,660 the only way to do that 126 00:07:00,660 --> 00:07:02,880 was to have a backdoor in 127 00:07:02,880 --> 00:07:04,460 to be able to continue to capture 128 00:07:04,460 --> 00:07:05,680 and get that data. 129 00:07:07,100 --> 00:07:08,420 Totally different reason. 130 00:07:09,080 --> 00:07:09,600 Okay? 131 00:07:11,740 --> 00:07:15,200 In IT, you normally want to stop data. 132 00:07:15,300 --> 00:07:16,520 You want to do a denial of service. 133 00:07:18,240 --> 00:07:20,280 When you're in the control system world, 134 00:07:20,400 --> 00:07:22,400 you don't, and you want to do something bad. 135 00:07:22,560 --> 00:07:24,160 It's not a denial of service. 136 00:07:24,940 --> 00:07:25,860 You want to call the police. 137 00:07:25,860 --> 00:07:26,660 You want to cause boom in the night. 138 00:07:28,360 --> 00:07:29,860 So what you want to do 139 00:07:29,860 --> 00:07:32,100 is you want to take control of the process. 140 00:07:34,180 --> 00:07:40,100 Or you want to take control of the operator displays 141 00:07:40,100 --> 00:07:42,760 and have the operator do the wrong thing. 142 00:07:43,520 --> 00:07:46,220 It is not a denial of service. 143 00:07:48,500 --> 00:07:51,480 What you're looking at, level 012, 144 00:07:53,080 --> 00:07:55,800 this is all from the Purdue reference model. 145 00:07:55,860 --> 00:07:59,480 This has nothing to do with the seven-layer OSI stack. 146 00:07:59,940 --> 00:08:00,660 Okay? 147 00:08:01,260 --> 00:08:03,200 So what you're talking about, 148 00:08:04,340 --> 00:08:06,020 this is the process. 149 00:08:06,140 --> 00:08:08,480 So this is like a boiler. 150 00:08:08,800 --> 00:08:12,820 Or it's like the electrical wires. 151 00:08:13,860 --> 00:08:14,580 Okay? 152 00:08:14,640 --> 00:08:17,500 Or this is like, you know, a, 153 00:08:18,820 --> 00:08:21,260 you know, the refinery process. 154 00:08:22,460 --> 00:08:24,220 Here, level 1, 155 00:08:24,260 --> 00:08:25,580 is basically, 156 00:08:25,860 --> 00:08:27,320 the real-time measurement. 157 00:08:28,660 --> 00:08:30,360 This is normally, 158 00:08:30,460 --> 00:08:31,540 now again, years ago, 159 00:08:31,600 --> 00:08:33,720 you could have subdivided this easily. 160 00:08:34,980 --> 00:08:37,220 What happened with Stuxnet, 161 00:08:37,240 --> 00:08:38,140 that was level 2. 162 00:08:38,680 --> 00:08:40,840 Now, many years ago, 163 00:08:40,860 --> 00:08:41,940 there was a distinct difference. 164 00:08:43,720 --> 00:08:49,860 Now, with networking and with ethernet, et cetera, 165 00:08:50,680 --> 00:08:51,880 it's blurring. 166 00:08:52,340 --> 00:08:54,660 Level 2, which used to be, 167 00:08:55,860 --> 00:08:57,940 totally isolated from the internet, 168 00:08:58,140 --> 00:09:00,000 is no longer isolated from the internet. 169 00:09:00,660 --> 00:09:03,300 So it starts blurring when you start talking about 170 00:09:03,300 --> 00:09:04,860 levels 2, 3, 4 on up. 171 00:09:05,580 --> 00:09:07,640 And it's even starting to blur 172 00:09:07,640 --> 00:09:09,100 even when you get down to here. 173 00:09:10,180 --> 00:09:12,540 Because you start talking about sensors on chips 174 00:09:12,540 --> 00:09:13,660 and things like that. 175 00:09:14,100 --> 00:09:14,340 Okay? 176 00:09:15,220 --> 00:09:17,400 But what we're really worried about 177 00:09:18,720 --> 00:09:22,240 is when you can compromise a system 178 00:09:24,080 --> 00:09:25,200 by physics, 179 00:09:26,560 --> 00:09:28,040 you can't stop that. 180 00:09:29,800 --> 00:09:32,420 One of the things Aurora does is use physics 181 00:09:33,600 --> 00:09:36,740 to cause essentially 182 00:09:39,340 --> 00:09:42,380 very, very, very large torques and current spikes 183 00:09:43,760 --> 00:09:44,640 in milliseconds. 184 00:09:45,680 --> 00:09:47,320 There's nothing you're gonna do about that. 185 00:09:48,240 --> 00:09:49,240 That's how, by the way, 186 00:09:49,260 --> 00:09:51,540 you bring the grid down for about 9 to 18 months. 187 00:09:53,560 --> 00:09:54,720 That's what we're talking about. 188 00:09:54,760 --> 00:09:55,420 This is not, 189 00:09:55,420 --> 00:09:57,700 this is not a day or two or anything else. 190 00:09:58,460 --> 00:10:01,580 This is how you break very large equipment 191 00:10:01,580 --> 00:10:06,200 like generators, motors, transformers. 192 00:10:07,580 --> 00:10:10,860 These are really long, you know, 193 00:10:11,860 --> 00:10:15,060 term equipment that, 194 00:10:15,180 --> 00:10:17,340 and you don't make very many of these, 195 00:10:17,420 --> 00:10:21,200 and these are almost always very specific 196 00:10:21,200 --> 00:10:22,860 to the specific process. 197 00:10:22,860 --> 00:10:25,480 And there aren't very many vendors 198 00:10:25,480 --> 00:10:26,560 who even make this anymore, 199 00:10:26,660 --> 00:10:27,540 and a lot of them are offshore. 200 00:10:28,620 --> 00:10:30,440 So when you start breaking things, 201 00:10:30,780 --> 00:10:32,800 there's a lot of questions that come in. 202 00:10:38,540 --> 00:10:42,120 Now, again, our communications are not native IP. 203 00:10:43,420 --> 00:10:45,980 You're talking about, say, Modbus over IP, 204 00:10:46,120 --> 00:10:47,480 or DNP over IP. 205 00:10:48,420 --> 00:10:50,560 Pick something, but it's not native IP. 206 00:10:50,760 --> 00:10:52,840 What we do is we use IP as a router. 207 00:10:52,840 --> 00:10:54,520 We use IP as a wrapper, okay? 208 00:10:55,840 --> 00:10:57,860 And there's work going on, 209 00:10:58,220 --> 00:11:02,400 but these protocols are not, like I say, native IP. 210 00:11:02,920 --> 00:11:05,640 And the real protocols we're worried about here 211 00:11:06,260 --> 00:11:08,520 aren't Modbus or DNP. 212 00:11:09,260 --> 00:11:12,300 They're things like HART or wireless HART 213 00:11:12,520 --> 00:11:15,020 or PROFIBUS, Fieldbus. 214 00:11:15,940 --> 00:11:21,560 These are your level zero, level one sensor protocols. 215 00:11:23,620 --> 00:11:25,800 There's not only no security in them, 216 00:11:25,840 --> 00:11:27,920 they were designed almost to be insecure. 217 00:11:29,140 --> 00:11:31,340 You'll find a lot of these have things like XML. 218 00:11:34,960 --> 00:11:37,140 So one of the other questions is, so what? 219 00:11:37,300 --> 00:11:38,320 Well, you can get to them. 220 00:11:41,560 --> 00:11:44,400 The I.O., the input, output cards, you'll see them. 221 00:11:44,540 --> 00:11:46,000 I mean, when you go to the board, 222 00:11:46,080 --> 00:11:48,160 you'll see things like Phoenix Contact or whatever. 223 00:11:50,140 --> 00:11:52,320 It allows the instruments to communicate, 224 00:11:52,320 --> 00:11:53,800 to communicate bi-directionally. 225 00:11:54,920 --> 00:11:56,720 You'd think about things like data diodes. 226 00:11:56,760 --> 00:11:57,860 Well, that doesn't do anything 227 00:11:57,860 --> 00:12:01,980 when you're talking about sensors, especially smart sensors. 228 00:12:02,020 --> 00:12:04,720 Smart sensors have to communicate bi-directionally. 229 00:12:05,580 --> 00:12:07,960 Smart meaning it's digital, 230 00:12:08,160 --> 00:12:15,620 and it's done all of the conversions at the sensor layer. 231 00:12:18,780 --> 00:12:20,160 And you'll notice, again, 232 00:12:20,320 --> 00:12:22,000 engineers can no longer simply measure 233 00:12:22,000 --> 00:12:23,060 the output analog signal. 234 00:12:23,080 --> 00:12:25,020 They need to communicate with the transmitter 235 00:12:26,140 --> 00:12:27,520 and read the digital signal. 236 00:12:27,980 --> 00:12:29,320 Well, there's no air gap. 237 00:12:29,580 --> 00:12:31,120 It's impossible to have an air gap. 238 00:12:31,860 --> 00:12:33,800 You know, everybody thinks, hey, I did this, 239 00:12:33,820 --> 00:12:34,540 therefore I'm fine. 240 00:12:34,660 --> 00:12:35,440 You can't do that. 241 00:12:37,300 --> 00:12:39,180 And the whole point about the presentation, 242 00:12:40,820 --> 00:12:44,840 there is almost no cyber security in them. 243 00:12:45,080 --> 00:12:46,140 There's no authentication. 244 00:12:47,380 --> 00:12:49,920 And from a cyber perspective, there's no forensics. 245 00:12:50,720 --> 00:12:52,780 One of the things that's really important 246 00:12:52,780 --> 00:12:54,880 to get across, as an engineer, 247 00:12:55,120 --> 00:12:57,380 I don't care if the process fails 248 00:12:57,380 --> 00:13:01,900 because it was a system problem, 249 00:13:02,120 --> 00:13:03,700 or because it was hacked. 250 00:13:04,260 --> 00:13:06,400 The process still failed. 251 00:13:08,060 --> 00:13:10,920 We need to understand when the process is starting to fail. 252 00:13:12,300 --> 00:13:14,000 And what you'll find in our world 253 00:13:14,000 --> 00:13:15,440 is often the only difference 254 00:13:15,440 --> 00:13:17,620 between malicious versus unintentional 255 00:13:17,620 --> 00:13:19,060 is the motivation 256 00:13:19,920 --> 00:13:20,280 of somebody. 257 00:13:20,580 --> 00:13:22,200 Why did they do what they did? 258 00:13:24,020 --> 00:13:24,920 It's different. 259 00:13:26,520 --> 00:13:28,180 And we need to think different. 260 00:13:28,300 --> 00:13:30,800 We need to think about how do you keep the boiler 261 00:13:31,800 --> 00:13:35,880 within a prescribed temperature and pressure range, 262 00:13:36,120 --> 00:13:37,980 regardless of why. 263 00:13:40,440 --> 00:13:42,800 And like I say, the protocols are vulnerable. 264 00:13:43,140 --> 00:13:43,960 I put this up. 265 00:13:44,100 --> 00:13:46,760 This was an ICS cert notification. 266 00:13:48,160 --> 00:13:49,200 Last revised. 267 00:13:49,200 --> 00:13:52,040 July 25th, 2017. 268 00:13:52,940 --> 00:13:54,400 We're talking Thursday. 269 00:13:56,240 --> 00:13:57,280 That's PROFINET. 270 00:13:59,320 --> 00:14:00,800 You know, when people think, 271 00:14:00,960 --> 00:14:02,740 hey, you know, this isn't vulnerable. 272 00:14:03,200 --> 00:14:04,060 Yes, it is. 273 00:14:04,780 --> 00:14:06,560 We've had any number of demonstrations 274 00:14:06,560 --> 00:14:08,660 of hacking, heart, wireless heart. 275 00:14:09,080 --> 00:14:11,120 A friend of mine just got his doctorate 276 00:14:11,880 --> 00:14:13,780 from the Air Force Institute of Technology 277 00:14:13,780 --> 00:14:15,300 at Wright-Patterson Air Force Base. 278 00:14:15,360 --> 00:14:16,120 He's now at Oak Ridge. 279 00:14:16,560 --> 00:14:19,180 His doctoral thesis was on, you know, 280 00:14:19,180 --> 00:14:22,660 hacking heart-enabled process sensors. 281 00:14:25,000 --> 00:14:26,840 What they're gonna be doing next year, 282 00:14:26,880 --> 00:14:28,220 even though he's already proven it, 283 00:14:28,900 --> 00:14:31,440 is the next one going through the PhD program 284 00:14:31,440 --> 00:14:34,640 is gonna go directly after wireless heart, 285 00:14:34,820 --> 00:14:36,500 even though we already know that's vulnerable. 286 00:14:37,840 --> 00:14:40,460 His thesis happened to have been on wired heart. 287 00:14:43,060 --> 00:14:44,580 So just so people understand, 288 00:14:44,700 --> 00:14:46,460 not only are we talking about a problem, 289 00:14:46,620 --> 00:14:48,440 it's a very exploitable problem. 290 00:14:51,260 --> 00:14:51,840 So, just want you to understand that. 291 00:14:53,780 --> 00:14:56,080 Control system cyber incidents are real. 292 00:14:57,080 --> 00:14:58,300 It was my database. 293 00:15:00,330 --> 00:15:03,400 Been well over 950 to date, okay. 294 00:15:03,540 --> 00:15:04,980 These are not vulnerabilities. 295 00:15:05,340 --> 00:15:06,700 These are real incidents. 296 00:15:09,220 --> 00:15:12,460 And you could see the impacts ranging all over the place. 297 00:15:12,600 --> 00:15:14,040 And you can see, you know, 298 00:15:14,040 --> 00:15:16,140 this is complete destruction of a turbine. 299 00:15:16,300 --> 00:15:17,640 That happens to be Aurora. 300 00:15:17,900 --> 00:15:18,920 That's DC Metro. 301 00:15:18,920 --> 00:15:22,680 metro train crash, San Bruno, Bellingham, that was Marucci. 302 00:15:23,160 --> 00:15:25,300 That was an offshore oil platform 303 00:15:25,300 --> 00:15:27,540 that they hacked the buoyancy 304 00:15:28,140 --> 00:15:30,980 and tilted an offshore oil rig. 305 00:15:32,040 --> 00:15:34,600 This is really different than things you're talking about. 306 00:15:34,700 --> 00:15:35,900 Gee, somebody stole my data. 307 00:15:38,320 --> 00:15:41,820 This is the dam in Russia. 308 00:15:42,040 --> 00:15:44,420 I don't even want to try to pronounce it. 309 00:15:45,340 --> 00:15:48,940 That lifted a 1,000-ton turbine off its pedestal. 310 00:15:49,720 --> 00:15:52,180 There were 75 people killed there. 311 00:15:52,400 --> 00:15:55,660 In fact, the Russians originally thought we did that. 312 00:15:58,920 --> 00:16:01,540 That happens to be a brand-new Navy ship 313 00:16:02,620 --> 00:16:05,920 that, because of control system cyber problems, 314 00:16:06,160 --> 00:16:07,500 on its maiden voyage, 315 00:16:07,600 --> 00:16:10,180 had to be towed in the last 40 miles into Norfolk. 316 00:16:12,360 --> 00:16:13,760 So when you start looking, 317 00:16:13,920 --> 00:16:14,400 you can only... 318 00:16:14,420 --> 00:16:16,680 You can also see there's a lot of different industries here. 319 00:16:21,140 --> 00:16:23,300 This is just kind of my summary. 320 00:16:25,780 --> 00:16:28,060 There have been over 1,000 deaths to date 321 00:16:28,820 --> 00:16:33,220 and well over $50 billion in direct damage. 322 00:16:35,080 --> 00:16:38,060 In fact, I'll give you an example of what we're talking about, 323 00:16:39,240 --> 00:16:42,040 and it also flows into what this presentation is about. 324 00:16:42,240 --> 00:16:44,320 I think most people know what Stuxnet is. 325 00:16:44,420 --> 00:16:47,280 It was changing the logic in a controller 326 00:16:47,280 --> 00:16:48,640 to change the process 327 00:16:48,640 --> 00:16:51,960 and then changing the logic back so nobody would know. 328 00:16:53,200 --> 00:16:54,640 You know what I just described? 329 00:16:56,120 --> 00:16:56,720 Volkswagen. 330 00:16:58,480 --> 00:17:03,600 And by the way, Fiat, Harley-Davidson, all of them, 331 00:17:03,640 --> 00:17:06,760 they all did exactly that 332 00:17:06,760 --> 00:17:09,300 because they couldn't meet the environmental controls. 333 00:17:11,080 --> 00:17:13,200 And when I had first... 334 00:17:14,420 --> 00:17:17,080 was looking at all of this about, quote-unquote, 335 00:17:17,160 --> 00:17:20,360 ICS sensors not being secure or authenticated, 336 00:17:21,040 --> 00:17:25,680 I called one of the professors at UC San Diego 337 00:17:25,680 --> 00:17:30,880 who was doing the analysis on Volkswagen and Fiat. 338 00:17:31,840 --> 00:17:32,920 And I asked him, 339 00:17:33,000 --> 00:17:35,260 are the sensors in cars, 340 00:17:35,960 --> 00:17:38,580 do they have authentication or security? 341 00:17:39,200 --> 00:17:40,640 You're right, no. 342 00:17:40,900 --> 00:17:41,300 No. 343 00:17:42,320 --> 00:17:44,300 There's nothing specific 344 00:17:44,420 --> 00:17:45,260 about this. 345 00:17:46,140 --> 00:17:47,540 But think about it. 346 00:17:47,580 --> 00:17:49,960 If the sensor data can't be trusted, 347 00:17:51,620 --> 00:17:54,180 what does it say about every single thing 348 00:17:54,180 --> 00:17:55,580 you're doing with network monitoring? 349 00:17:56,260 --> 00:17:58,940 If you're a doctor and you cannot trust 350 00:17:58,940 --> 00:18:01,200 your temperature or blood pressure readings, 351 00:18:01,460 --> 00:18:02,940 how can you make a diagnosis? 352 00:18:05,060 --> 00:18:06,360 That's where we are here. 353 00:18:07,740 --> 00:18:10,840 And you go think about Microsoft with the Azure Cloud 354 00:18:10,840 --> 00:18:12,360 or any of the others. 355 00:18:12,680 --> 00:18:13,540 What are they... 356 00:18:13,540 --> 00:18:14,100 What's their... 357 00:18:14,420 --> 00:18:15,720 What's their assumption on everything? 358 00:18:16,380 --> 00:18:19,100 The sensor data is authenticated and trusted. 359 00:18:20,180 --> 00:18:21,660 That's the entire... 360 00:18:21,660 --> 00:18:25,140 That is the assumption for the entire Azure Cloud. 361 00:18:26,160 --> 00:18:28,060 So think about what I just said 362 00:18:28,060 --> 00:18:31,140 with respect to the ICS world and cloud computing. 363 00:18:31,740 --> 00:18:33,060 It's really that simple. 364 00:18:37,680 --> 00:18:40,340 Here's the techie that explains most everything. 365 00:18:42,600 --> 00:18:44,400 What you have over here... 366 00:18:44,420 --> 00:18:45,760 What you have over here is what most people realize. 367 00:18:46,480 --> 00:18:47,560 Here's your... 368 00:18:47,560 --> 00:18:49,920 You know, your normal network... 369 00:18:49,920 --> 00:18:51,540 Your operator displays. 370 00:18:51,840 --> 00:18:52,680 This is all Windows. 371 00:18:52,840 --> 00:18:54,860 Everything here is generally IT. 372 00:18:55,700 --> 00:18:57,280 Prior to Stuxnet, 373 00:18:57,460 --> 00:18:58,580 don't even look there. 374 00:18:59,240 --> 00:19:02,580 Since Stuxnet, people are really worried about the PLC. 375 00:19:03,180 --> 00:19:03,480 Okay? 376 00:19:04,740 --> 00:19:05,340 Only... 377 00:19:05,340 --> 00:19:06,780 You know, this is a valve. 378 00:19:07,120 --> 00:19:08,120 Here's a motor. 379 00:19:08,240 --> 00:19:08,940 That's a sensor. 380 00:19:09,620 --> 00:19:10,740 This thing here, 381 00:19:11,460 --> 00:19:14,260 that's what makes that 1970s or 80s 382 00:19:14,260 --> 00:19:17,160 valve a 2015 valve. 383 00:19:17,700 --> 00:19:19,840 This is the smart electronics. 384 00:19:20,940 --> 00:19:23,060 That's the smart electronics for the motor. 385 00:19:24,180 --> 00:19:25,800 Same thing with the sensor. 386 00:19:26,000 --> 00:19:27,840 And that's all remotely accessible. 387 00:19:29,100 --> 00:19:29,420 Okay? 388 00:19:30,640 --> 00:19:32,140 So what you've got here, 389 00:19:32,280 --> 00:19:33,420 there's no security. 390 00:19:35,300 --> 00:19:35,940 Now, 391 00:19:36,640 --> 00:19:37,920 here's our process. 392 00:19:38,200 --> 00:19:40,620 That's the most important thing in the world to us. 393 00:19:41,840 --> 00:19:43,720 This stuff starts... 394 00:19:44,260 --> 00:19:44,860 as serial. 395 00:19:48,220 --> 00:19:50,940 It ends up going through the serial to Ethernet converter. 396 00:19:51,520 --> 00:19:53,580 And by the way, what did the Russians hack 397 00:19:54,000 --> 00:19:56,760 both in the... in the U.S. and in the Ukraine? 398 00:19:57,180 --> 00:19:59,240 It was the serial to Ethernet converters. 399 00:19:59,820 --> 00:20:02,620 Only they used it to go up into the network, 400 00:20:04,000 --> 00:20:05,120 not down... 401 00:20:06,200 --> 00:20:07,000 to the sensors. 402 00:20:10,760 --> 00:20:14,240 Here is where all of your network monitoring starts. 403 00:20:15,700 --> 00:20:20,200 They assume whatever happens here is correct. 404 00:20:22,480 --> 00:20:29,380 That's the flaw in every single network anomaly detection program today. 405 00:20:30,460 --> 00:20:36,300 They assume when you start here that that sensor value is correct. 406 00:20:37,580 --> 00:20:39,920 And they have no way of knowing that it isn't. 407 00:20:45,260 --> 00:20:47,040 Now, I showed you that picture there. 408 00:20:47,760 --> 00:20:49,820 Now, this is the Holy Grail. 409 00:20:50,820 --> 00:20:53,520 The Holy Grail is to know... 410 00:20:53,520 --> 00:20:55,160 In fact, I'll give it to you this way. 411 00:20:55,420 --> 00:20:57,600 About four or five weeks ago, 412 00:20:57,680 --> 00:21:00,820 Honda shut down an assembly plant in Japan. 413 00:21:01,100 --> 00:21:02,860 It cost them a thousand cars. 414 00:21:03,540 --> 00:21:04,100 Why? 415 00:21:04,540 --> 00:21:07,600 They found WannaCry on the networks. 416 00:21:08,440 --> 00:21:12,340 There wasn't one single alarm on the factory floor. 417 00:21:13,200 --> 00:21:15,660 They did a self-denial of service 418 00:21:16,620 --> 00:21:19,800 without having a clue whether it affected them or not. 419 00:21:21,640 --> 00:21:23,180 This is where we are today. 420 00:21:24,080 --> 00:21:25,960 You cannot, from here, 421 00:21:27,240 --> 00:21:29,260 make any real... 422 00:21:30,900 --> 00:21:33,380 view of what's going on in the real process. 423 00:21:33,940 --> 00:21:35,920 Because all you've got is 424 00:21:36,860 --> 00:21:39,920 the data that's going from that serial to Ethernet converter. 425 00:21:41,060 --> 00:21:45,460 The Holy Grail is to be able to look at the process 426 00:21:45,460 --> 00:21:49,540 and cross-correlate it to the network anomaly. 427 00:21:50,200 --> 00:21:51,920 To be able to say, 428 00:21:52,900 --> 00:21:54,320 hey, I found malware. 429 00:21:54,940 --> 00:21:57,920 Is my process changing? 430 00:22:00,920 --> 00:22:02,720 Assuming it shouldn't, 431 00:22:02,720 --> 00:22:05,580 and you're watching the sensors 432 00:22:05,580 --> 00:22:07,700 and you know precisely what's happening. 433 00:22:08,380 --> 00:22:09,620 Why do anything? 434 00:22:10,700 --> 00:22:14,040 This is probably the most important thing in the world 435 00:22:14,040 --> 00:22:16,180 when you talk about the industrial world 436 00:22:16,180 --> 00:22:17,560 or the commercial world. 437 00:22:17,800 --> 00:22:18,940 When do I care? 438 00:22:19,900 --> 00:22:22,810 Today, they cannot make that statement. 439 00:22:31,880 --> 00:22:33,460 This is just the sensor. 440 00:22:33,740 --> 00:22:35,420 And again, here's the point. 441 00:22:35,500 --> 00:22:36,140 What you're doing 442 00:22:36,140 --> 00:22:37,680 is you're coming back here. 443 00:22:37,680 --> 00:22:37,940 You're coming back in. 444 00:22:38,340 --> 00:22:40,860 And if you change things like zero and span, 445 00:22:41,460 --> 00:22:44,720 you can prevent a sensor from ever reaching a set point 446 00:22:44,720 --> 00:22:48,320 or to reach a set point long before it should. 447 00:22:49,560 --> 00:22:51,700 And you will never know that 448 00:22:51,700 --> 00:22:52,980 from network monitoring. 449 00:22:57,020 --> 00:22:58,740 This happens to be a smart sensor. 450 00:22:58,840 --> 00:22:59,540 The same thing. 451 00:23:00,360 --> 00:23:01,680 You get into here. 452 00:23:01,900 --> 00:23:03,080 You make your changes. 453 00:23:03,820 --> 00:23:05,300 Nobody will ever know. 454 00:23:07,680 --> 00:23:08,240 Now, 455 00:23:11,580 --> 00:23:12,680 what happens, 456 00:23:12,760 --> 00:23:14,940 this is what I was doing 20 some odd years ago 457 00:23:15,660 --> 00:23:18,320 when I was still at the Electric Power Research Institute 458 00:23:18,320 --> 00:23:20,020 before I ever heard of security. 459 00:23:20,460 --> 00:23:24,140 We were worried about doing root cause analysis on failures. 460 00:23:24,860 --> 00:23:28,020 But what we were doing was a snapshot in time 461 00:23:28,960 --> 00:23:30,080 doing postmortem. 462 00:23:30,080 --> 00:23:31,240 Well, there's technology now 463 00:23:31,240 --> 00:23:33,680 that allows you to do this in real time. 464 00:23:34,420 --> 00:23:36,040 This is where the Holy Grail is coming. 465 00:23:36,040 --> 00:23:39,500 But what's happening is 466 00:23:40,260 --> 00:23:42,680 this is what's filtered out 467 00:23:42,680 --> 00:23:45,880 as you go through the serial Ethernet converter. 468 00:23:46,120 --> 00:23:48,280 That's why nobody here on the network side 469 00:23:48,280 --> 00:23:51,520 is ever going to be able to really understand what's happening. 470 00:23:51,740 --> 00:23:53,460 What you're getting essentially 471 00:23:53,460 --> 00:23:55,660 is the absolute value of the signal. 472 00:23:57,600 --> 00:24:00,980 Not what's actually happening in the process. 473 00:24:03,200 --> 00:24:06,020 This is what is completely and totally different. 474 00:24:09,120 --> 00:24:11,040 Here are real things. 475 00:24:12,520 --> 00:24:15,360 And two weeks ago I was at a power plant. 476 00:24:15,720 --> 00:24:16,840 It had tripped. 477 00:24:16,860 --> 00:24:17,560 They had tripped it. 478 00:24:18,800 --> 00:24:20,460 In other words, shut it down. 479 00:24:20,520 --> 00:24:21,660 Very large plant. 480 00:24:22,940 --> 00:24:25,700 A pressure sensor reached a set point. 481 00:24:26,080 --> 00:24:26,680 Why? 482 00:24:26,840 --> 00:24:28,480 A sensing line clogged. 483 00:24:29,780 --> 00:24:32,660 It is impossible to see that from network monitoring. 484 00:24:34,020 --> 00:24:35,840 But if you're looking at the process, 485 00:24:36,040 --> 00:24:37,540 it becomes very obvious 486 00:24:37,540 --> 00:24:40,000 something is going on with that sensor. 487 00:24:41,260 --> 00:24:44,260 This is something I worked on a long time ago. 488 00:24:44,420 --> 00:24:45,780 This is a nuclear plant 489 00:24:45,780 --> 00:24:49,560 where a safety relief valve didn't lift. 490 00:24:50,000 --> 00:24:51,020 This is safety. 491 00:24:51,160 --> 00:24:52,740 This is as much safety as you can get. 492 00:24:52,980 --> 00:24:54,180 Why didn't it lift? 493 00:24:54,360 --> 00:24:57,200 Because the sensor could never reach its set point. 494 00:25:02,360 --> 00:25:03,520 What happened? 495 00:25:03,520 --> 00:25:06,020 That's Tom Sock, the dam in Missouri. 496 00:25:06,960 --> 00:25:09,060 Basically failed about five or six years ago. 497 00:25:09,080 --> 00:25:09,540 What happened? 498 00:25:09,560 --> 00:25:10,760 It had a couple of level sensors 499 00:25:11,260 --> 00:25:12,860 attached to the wall of the dam. 500 00:25:13,340 --> 00:25:14,620 The tie-downs broke, 501 00:25:14,700 --> 00:25:16,520 so the sensors came out from the wall. 502 00:25:17,460 --> 00:25:18,980 When they came out from the wall, 503 00:25:19,220 --> 00:25:21,420 the level readings changed. 504 00:25:22,340 --> 00:25:24,440 Because you measure level by elevation head. 505 00:25:24,560 --> 00:25:25,660 You measure level by height. 506 00:25:26,220 --> 00:25:27,140 Well, the height changed. 507 00:25:28,360 --> 00:25:30,500 So the SCADA system took the data, 508 00:25:30,560 --> 00:25:31,540 took it as real, 509 00:25:31,720 --> 00:25:33,120 had no reason not to. 510 00:25:33,520 --> 00:25:35,460 And when it said the level was low, 511 00:25:35,620 --> 00:25:38,460 it turned the pumps on until the reservoir overfilled 512 00:25:38,460 --> 00:25:40,200 and the earthen dam collapsed. 513 00:25:41,840 --> 00:25:43,700 There's no hypothetical here. 514 00:25:44,040 --> 00:25:46,540 This is the stuff that every bit of this 515 00:25:46,540 --> 00:25:48,300 has actually happened. 516 00:25:49,040 --> 00:25:51,040 You know, this was one of the biggest 517 00:25:53,980 --> 00:25:55,500 fires in the UK, 518 00:25:55,620 --> 00:25:57,740 which was a refinery fire 519 00:25:57,740 --> 00:26:00,220 because the sensors didn't work 520 00:26:00,220 --> 00:26:02,840 to tell them that the tank was being overfilled. 521 00:26:03,740 --> 00:26:06,560 These are huge catastrophic problems. 522 00:26:07,320 --> 00:26:09,660 The controller can't do anything, 523 00:26:09,880 --> 00:26:11,360 or it's different. 524 00:26:11,460 --> 00:26:13,220 The controller will do exactly 525 00:26:13,220 --> 00:26:15,140 what the sensor is telling them 526 00:26:15,140 --> 00:26:17,580 that the process seems to be. 527 00:26:21,140 --> 00:26:23,580 This is what is reality, 528 00:26:23,780 --> 00:26:25,840 and this is what network monitoring 529 00:26:25,840 --> 00:26:27,320 can't do today. 530 00:26:28,180 --> 00:26:30,980 Because it doesn't have a view into the process. 531 00:26:36,820 --> 00:26:39,320 Part of what's going on is this. 532 00:26:42,140 --> 00:26:44,580 Sensors, actuators, and drives to this day 533 00:26:44,580 --> 00:26:46,980 are still being made with no security 534 00:26:47,520 --> 00:26:48,520 and no authentication. 535 00:26:48,600 --> 00:26:49,040 Why? 536 00:26:49,200 --> 00:26:50,320 They're engineering systems. 537 00:26:51,780 --> 00:26:55,140 What our vendors do is they have the security done 538 00:26:55,140 --> 00:26:57,560 over here where you're worried about the network. 539 00:26:58,520 --> 00:27:00,500 Over here where they do the engineering 540 00:27:00,500 --> 00:27:01,520 for the engineering systems, 541 00:27:01,940 --> 00:27:03,400 there are no security requirements. 542 00:27:05,260 --> 00:27:06,280 That's to this day. 543 00:27:10,880 --> 00:27:12,500 This is a huge problem. 544 00:27:13,300 --> 00:27:15,960 In IT, it may be all about packets. 545 00:27:16,760 --> 00:27:18,180 For ICS, it is not. 546 00:27:21,040 --> 00:27:21,520 IoT. 547 00:27:22,340 --> 00:27:24,040 You know, IoT is a funny animal. 548 00:27:24,680 --> 00:27:25,900 You know, what is IoT? 549 00:27:26,060 --> 00:27:28,020 If it's Fitbits and refrigerators, who cares? 550 00:27:29,260 --> 00:27:31,380 But in a sense, we're part of IoT. 551 00:27:31,740 --> 00:27:33,380 We were IoT 20 years ago. 552 00:27:33,400 --> 00:27:34,460 20 years before there was IoT. 553 00:27:39,200 --> 00:27:42,400 And they're certainly not looking at anything like this. 554 00:27:42,940 --> 00:27:45,180 And then here's, even ICS-CERT, 555 00:27:45,260 --> 00:27:48,480 they put out their 2016, you know, statistics. 556 00:27:49,320 --> 00:27:51,040 There is no mention whatsoever 557 00:27:51,040 --> 00:27:52,940 of anything at level zero and one. 558 00:27:53,100 --> 00:27:55,380 The only thing they talk about are network issues. 559 00:27:56,420 --> 00:27:59,300 And it's the same thing from 2015 and 14, 560 00:27:59,420 --> 00:28:00,500 and you can keep on going. 561 00:28:00,840 --> 00:28:02,420 There's nothing that's ever changed. 562 00:28:07,220 --> 00:28:08,900 And what needs to be done, 563 00:28:08,980 --> 00:28:10,740 you've got to get the engineers involved. 564 00:28:12,700 --> 00:28:17,400 And reliability and safety have to be number one. 565 00:28:18,980 --> 00:28:21,780 Confidentiality is fine, maybe. 566 00:28:24,960 --> 00:28:28,780 But if you don't take reliability and safety there, 567 00:28:28,920 --> 00:28:29,640 why bother? 568 00:28:30,060 --> 00:28:31,800 And the other thing to realize, 569 00:28:33,400 --> 00:28:36,120 everything we're talking about in the sensor world 570 00:28:36,920 --> 00:28:39,200 is all reliability and safety. 571 00:28:39,200 --> 00:28:40,260 It's really not cyber. 572 00:28:40,640 --> 00:28:44,840 But it's the most important input to cyber. 573 00:28:46,240 --> 00:28:48,580 Because how can you monitor anything 574 00:28:48,580 --> 00:28:50,540 if you don't trust the input? 575 00:28:52,640 --> 00:28:54,400 And that's what this is all about. 576 00:28:55,000 --> 00:28:58,460 We have, you know, what do you do about supply chain 577 00:28:58,460 --> 00:29:00,240 when you talk about this layer? 578 00:29:00,240 --> 00:29:03,460 What do you do about risk assessment methodology? 579 00:29:04,980 --> 00:29:08,300 Risk assessment, certainly vulnerability assessment, 580 00:29:08,480 --> 00:29:11,320 assumes you're secure and it's a gap analysis 581 00:29:11,320 --> 00:29:13,720 away from what you think is secure. 582 00:29:14,600 --> 00:29:16,400 When these have zero security, 583 00:29:16,860 --> 00:29:19,120 what do you do about a vulnerability assessment 584 00:29:20,000 --> 00:29:20,980 with zero? 585 00:29:21,960 --> 00:29:23,420 It's effectively infinite. 586 00:29:30,240 --> 00:29:34,140 Like I say, the ultimate holy grail 587 00:29:34,140 --> 00:29:39,660 is to correlate process anomaly detection 588 00:29:40,240 --> 00:29:42,700 with network anomaly detection. 589 00:29:44,420 --> 00:29:47,700 Short of that, we're going to continue to have 590 00:29:47,700 --> 00:29:49,000 the Honda issue. 591 00:29:49,440 --> 00:29:53,220 And this occurred, by the way, 10, 15 years ago 592 00:29:53,220 --> 00:29:56,480 with Code Red and Slammer. 593 00:29:56,520 --> 00:29:58,260 We had utilities shut down. 594 00:29:58,380 --> 00:29:59,000 Not utilities. 595 00:29:59,000 --> 00:30:01,780 Paper and pulp plants, you name it. 596 00:30:01,820 --> 00:30:02,800 They shut them down 597 00:30:03,580 --> 00:30:06,800 because somebody said there's malware out there. 598 00:30:07,860 --> 00:30:09,760 Not because there was anything 599 00:30:09,760 --> 00:30:12,020 within the process that changed. 600 00:30:12,240 --> 00:30:12,920 Nobody knew. 601 00:30:15,540 --> 00:30:18,020 You want a funny form of ransomware, 602 00:30:18,080 --> 00:30:18,680 go do this. 603 00:30:18,700 --> 00:30:21,800 But malware, nobody has any idea what it means. 604 00:30:25,790 --> 00:30:27,080 So with that, 605 00:30:28,360 --> 00:30:29,220 any questions? 606 00:30:30,920 --> 00:30:32,580 Martina, I was going to put this back on. 607 00:30:32,640 --> 00:30:33,480 Here are some of those, 608 00:30:33,600 --> 00:30:34,440 because I know you were late. 609 00:30:35,420 --> 00:30:36,580 These are some of the real 610 00:30:37,940 --> 00:30:38,820 sensor-related 611 00:30:41,440 --> 00:30:43,080 incidents that have already occurred. 612 00:30:43,360 --> 00:30:45,340 I just went through my database quickly. 613 00:30:45,520 --> 00:30:48,180 This is not all of them by any stretch. 614 00:30:49,080 --> 00:30:51,540 But like I say, you're talking about 615 00:30:53,800 --> 00:30:55,380 hydro, water, waste water, 616 00:30:56,200 --> 00:30:56,640 fossil, 617 00:30:56,640 --> 00:30:57,260 nuclear. 618 00:30:59,390 --> 00:31:00,980 This was essentially mining, 619 00:31:04,140 --> 00:31:04,760 refineries. 620 00:31:05,240 --> 00:31:06,620 This is all over the world. 621 00:31:07,260 --> 00:31:08,580 This is not 622 00:31:08,580 --> 00:31:10,900 a utility problem. 623 00:31:11,180 --> 00:31:12,800 It's not a North American problem. 624 00:31:13,350 --> 00:31:15,020 It's not any one vendor's problem. 625 00:31:16,640 --> 00:31:18,040 Since we're talking about 626 00:31:18,640 --> 00:31:19,620 issues with sensors, 627 00:31:19,800 --> 00:31:22,040 what's the current state of practices, 628 00:31:22,260 --> 00:31:23,720 best practices or normal practices 629 00:31:23,720 --> 00:31:25,080 as it relates to, 630 00:31:25,720 --> 00:31:26,920 what's the mindset so far 631 00:31:26,920 --> 00:31:28,200 of redundancy in sensors? 632 00:31:28,880 --> 00:31:30,320 Multiple sensors, same type, 633 00:31:30,400 --> 00:31:31,420 multiple different types, 634 00:31:31,540 --> 00:31:32,300 out of hand. 635 00:31:32,800 --> 00:31:34,480 Well, one of the things, 636 00:31:34,640 --> 00:31:35,840 and then I'm going to have to get going 637 00:31:35,840 --> 00:31:37,120 because I've got to go to that thing too, 638 00:31:38,400 --> 00:31:39,460 where Chris is. 639 00:31:41,860 --> 00:31:44,580 Part of the reason I brought up risk assessments, 640 00:31:45,180 --> 00:31:47,080 we've not done very good risk assessments. 641 00:31:49,180 --> 00:31:50,400 One of the things, 642 00:31:50,500 --> 00:31:52,620 how could a single pressure sensor 643 00:31:53,240 --> 00:31:54,060 shut down 644 00:31:54,060 --> 00:31:55,420 a large power plant? 645 00:31:57,700 --> 00:31:58,300 That's, 646 00:31:58,720 --> 00:32:00,600 from an engineer's perspective, 647 00:32:00,720 --> 00:32:01,880 something is wrong here. 648 00:32:03,760 --> 00:32:04,360 Now, 649 00:32:05,300 --> 00:32:07,140 part of what it's saying is 650 00:32:07,140 --> 00:32:08,220 you better understand 651 00:32:08,220 --> 00:32:09,920 what is really truly critical. 652 00:32:11,180 --> 00:32:12,320 The other is, 653 00:32:12,500 --> 00:32:13,740 what happens if you put 654 00:32:13,740 --> 00:32:14,960 redundant sensors 655 00:32:14,960 --> 00:32:16,920 but they're on the same, 656 00:32:17,180 --> 00:32:18,080 if you will, 657 00:32:18,200 --> 00:32:18,900 Ethernet LAN? 658 00:32:20,280 --> 00:32:21,500 That's exactly right. 659 00:32:21,500 --> 00:32:25,280 I'm working with some technology 660 00:32:26,220 --> 00:32:28,160 to try to really understand 661 00:32:28,160 --> 00:32:30,040 what's going on at the sensor layer. 662 00:32:31,280 --> 00:32:31,880 But, 663 00:32:33,600 --> 00:32:35,000 first and foremost, 664 00:32:36,260 --> 00:32:38,420 you've got to have a good risk assessment. 665 00:32:38,500 --> 00:32:40,400 If you don't know what's really critical, 666 00:32:41,200 --> 00:32:42,440 you're dead on arrival. 667 00:32:44,320 --> 00:32:45,760 It's a really good question 668 00:32:45,760 --> 00:32:47,540 and this is such a great answer 669 00:32:47,540 --> 00:32:48,260 to your question. 670 00:32:48,700 --> 00:32:50,180 One pressure sensor 671 00:32:50,180 --> 00:32:52,420 isn't supposed to shut down 672 00:32:53,140 --> 00:32:55,260 a multi-unit power plant. 673 00:32:56,000 --> 00:32:56,800 And it did. 674 00:33:02,860 --> 00:33:03,300 Yeah? 675 00:33:03,380 --> 00:33:04,660 So if we're talking about security 676 00:33:04,660 --> 00:33:06,080 from the sensor perspective, 677 00:33:07,020 --> 00:33:09,280 what do you think is the solution to that? 678 00:33:09,360 --> 00:33:09,820 Because it's like 679 00:33:09,820 --> 00:33:12,040 you're trying to be aware of process issues 680 00:33:12,040 --> 00:33:13,160 and you have to use other sensors. 681 00:33:14,680 --> 00:33:16,140 So how do you use sensors 682 00:33:16,140 --> 00:33:17,040 to protect your sensors? 683 00:33:18,060 --> 00:33:19,700 This is the only way I know. 684 00:33:20,640 --> 00:33:22,660 I'm going back to what I did years ago. 685 00:33:23,100 --> 00:33:24,080 So, I mean, 686 00:33:24,080 --> 00:33:25,300 you need a sensor to detect 687 00:33:25,300 --> 00:33:26,320 that kind of stuff, right? 688 00:33:26,660 --> 00:33:28,720 Well, you need monitoring 689 00:33:28,720 --> 00:33:31,100 to detect what's going on. 690 00:33:32,260 --> 00:33:35,040 I'm working with one or two small companies 691 00:33:35,040 --> 00:33:36,900 that are actually looking at this 692 00:33:36,900 --> 00:33:38,260 and they've actually done 693 00:33:38,260 --> 00:33:39,880 some really neat, interesting work. 694 00:33:40,320 --> 00:33:42,700 And that's what's why I'm saying 695 00:33:42,700 --> 00:33:44,300 we have a shot at the Holy Grail 696 00:33:44,840 --> 00:33:46,340 of being able to correlate 697 00:33:47,080 --> 00:33:48,360 the actual process 698 00:33:50,180 --> 00:33:50,620 to the malware. 699 00:33:51,140 --> 00:33:53,120 And without that, it's not possible. 700 00:33:54,800 --> 00:33:56,380 We're mitigating some of that risk 701 00:33:56,380 --> 00:33:58,180 with things like Moxa firewalls 702 00:33:58,180 --> 00:33:59,040 through those devices 703 00:33:59,040 --> 00:34:02,500 and other devices like the Moxas. 704 00:34:02,920 --> 00:34:05,220 And the evolution, 705 00:34:05,460 --> 00:34:06,840 there's no IDS for this. 706 00:34:07,020 --> 00:34:07,900 They have Pi. 707 00:34:07,940 --> 00:34:09,880 The Pi is to collect the numbers from... 708 00:34:09,880 --> 00:34:10,440 Yeah, don't forget, 709 00:34:10,600 --> 00:34:11,920 where is Pi getting its data? 710 00:34:12,280 --> 00:34:13,020 From there. 711 00:34:13,820 --> 00:34:15,420 That's what I was trying to explain. 712 00:34:16,860 --> 00:34:18,860 This is the official Catch-22. 713 00:34:18,860 --> 00:34:21,340 Everything you're doing starts at the sensor. 714 00:34:21,580 --> 00:34:23,280 If you compromise the sensor, 715 00:34:23,480 --> 00:34:26,140 everything you look at downstream... 716 00:34:26,680 --> 00:34:27,200 Okay? 717 00:34:27,280 --> 00:34:29,480 I just had this discussion with somebody 718 00:34:29,480 --> 00:34:32,340 about, well, hey, I have redundancy 719 00:34:32,340 --> 00:34:33,180 because I have the historian. 720 00:34:33,900 --> 00:34:36,320 Yeah, but where did the historian data come from? 721 00:34:40,180 --> 00:34:42,680 That's what I heard 722 00:34:42,680 --> 00:34:44,380 because I got in yesterday 723 00:34:44,380 --> 00:34:45,320 and I didn't have a chance... 724 00:34:48,860 --> 00:34:49,080 Yes. 725 00:34:49,640 --> 00:34:50,140 Yes. 726 00:34:51,280 --> 00:34:51,780 Absolutely. 727 00:34:53,960 --> 00:34:55,860 And that's why I wanted to put this back up 728 00:34:55,860 --> 00:34:56,800 because you came in late. 729 00:34:57,100 --> 00:35:00,560 But this is where we are. 730 00:35:00,700 --> 00:35:01,320 And by the way, 731 00:35:01,380 --> 00:35:03,540 the serial to Ethernet converters 732 00:35:04,400 --> 00:35:06,800 were how the Russians put malware 733 00:35:06,800 --> 00:35:08,540 into our U.S. grids 734 00:35:08,540 --> 00:35:10,300 in October 2014 735 00:35:11,080 --> 00:35:12,180 and it's still there. 736 00:35:12,740 --> 00:35:15,820 It's how the Russians put the malware 737 00:35:15,820 --> 00:35:16,840 into the Ukraine. 738 00:35:20,240 --> 00:35:20,760 I will say... 739 00:35:20,760 --> 00:35:22,080 Let's go to the Russians. 740 00:35:22,340 --> 00:35:22,800 Let's go. 741 00:35:23,400 --> 00:35:23,920 Adversaries. 742 00:35:24,460 --> 00:35:25,580 I will say... 743 00:35:25,580 --> 00:35:27,920 I will say the attackers. 744 00:35:28,500 --> 00:35:29,380 There you go. 745 00:35:29,980 --> 00:35:31,180 Mark, you're right. 746 00:35:31,880 --> 00:35:33,420 How they get to the U.S. 747 00:35:33,420 --> 00:35:33,820 It's how they get to the U.S., right? 748 00:35:35,020 --> 00:35:38,740 It's how the attackers implement it. 749 00:35:38,820 --> 00:35:39,900 And I apologize... 750 00:35:39,900 --> 00:35:40,260 You're right. 751 00:35:41,360 --> 00:35:41,880 Okay. 752 00:35:42,280 --> 00:35:43,160 Last question 753 00:35:43,160 --> 00:35:44,380 because we have to wrap up. 754 00:35:44,400 --> 00:35:45,500 And I have to go, too. 755 00:35:45,940 --> 00:35:47,240 Just a quick comment. 756 00:35:47,460 --> 00:35:48,840 I grew up in Alaska, 757 00:35:48,840 --> 00:35:54,680 I worked on crude oil fields, and we were doing a retrofit of one of Arco's units, 758 00:35:54,960 --> 00:36:02,640 and one of my men who was out in the facility was climbing up, and he tripped, hit his foot, hit the sensor, 759 00:36:02,860 --> 00:36:05,320 and it sent the entire station into flare. 760 00:36:05,980 --> 00:36:12,020 All the processing went straight up, and the whole ground started shaking like there was an earthquake. 761 00:36:12,440 --> 00:36:16,360 And because we knew what had happened, we were able to tell the control center, 762 00:36:16,360 --> 00:36:20,200 and they were able to shut it down, that they would have had to troubleshoot that thing for hours 763 00:36:20,200 --> 00:36:21,900 to try to figure out what was going on. 764 00:36:22,020 --> 00:36:30,100 And that's, you know, for what it's worth, the city of San Francisco was brought to its knees April 21st 765 00:36:30,100 --> 00:36:32,540 by one relay and one substation. 766 00:36:33,880 --> 00:36:36,380 That should have never, ever, ever, ever happened. 767 00:36:38,440 --> 00:36:40,160 Joe, are you going to be back in the village later? 768 00:36:40,200 --> 00:36:42,180 Yeah, I just want to go to that console. 769 00:36:42,380 --> 00:36:43,600 Do you have any other questions? 770 00:36:43,700 --> 00:36:45,320 Are you going to be back in the ice in this place?