WEBVTT 00:18.590 --> 00:21.810 Will you guys let me know when to start? 00:27.480 --> 00:30.620 Good afternoon and good evening, everybody. 00:30.860 --> 00:32.960 Welcome to my talk. 00:33.060 --> 00:36.800 Thank you very much for being with me or watching this video. 00:36.800 --> 00:41.940 And thank you, ICS Village organization. 01:23.590 --> 01:25.470 Shall I continue? 01:26.130 --> 01:27.530 Go for it. 01:28.430 --> 01:30.090 Oh, okay. 02:10.630 --> 02:18.330 For those of you tuning into the ICS Village talk, this is Bryson Borth, the lead for the ICS Village. 02:18.330 --> 02:24.110 We're having some technical difficulties with Marina calling in from Europe. 02:24.250 --> 02:28.010 Her connection keeps dropping, so bear with us. 02:28.010 --> 02:29.710 We're trying to work through this. 02:29.710 --> 02:41.130 If we're not able to successfully pull this off during this hour, we're going to find another time to record a talk with Marina, and we will still share that with you after the conference. 04:10.670 --> 04:12.530 Hello, and I'm back. 04:12.630 --> 04:18.030 I don't know, I'm glad that I managed to get to internet connections for this talk. 04:18.030 --> 04:20.090 So now I'm on the backup connection. 04:20.090 --> 04:24.630 So I am a true good, I have a good cybersecurity program in here. 04:24.870 --> 04:28.710 So, well, and so I stopped that. 04:28.710 --> 04:32.110 I used to be this offensive vulnerability researcher. 04:32.110 --> 04:36.570 And in recent years, the word offensive is kind of have a very negative meaning. 04:36.570 --> 04:39.550 And that's why I call myself right now a vulnerability researcher. 04:39.550 --> 04:43.230 And I used to do very like crazy and exotic things. 04:43.230 --> 04:46.810 I have six accepted and five presented black hats. 04:46.810 --> 04:50.570 I have two different CCC troopers, SAS, hack in the box. 04:50.570 --> 04:53.270 I even keynoted at ICS JWG. 04:53.270 --> 04:55.090 So I've done offensive stuff. 04:55.090 --> 04:59.830 And I used to create this travels for asset owners and plant operators. 04:59.830 --> 05:03.970 And I believe that my threats are the most important and urgent to defend against. 05:03.970 --> 05:08.350 And I was very opinionated about that. 05:08.430 --> 05:11.570 But now I'm a manager of the cybersecurity program. 05:11.570 --> 05:16.770 And I'm responsible for keeping my customers plant safe and secure. 05:16.930 --> 05:20.870 Yes, I still think my past research was cool and very important. 05:21.930 --> 05:28.970 But no, I don't think that defending against my cool and very important threat is my priority right now. 05:28.970 --> 05:31.450 And this is what this talk is about. 05:33.630 --> 05:35.850 So this is what's my past life. 05:35.850 --> 05:37.910 And this is what you see here on the photo. 05:37.910 --> 05:45.490 It's how I was getting ready for my Black Hat USA 2017 talk. 05:45.490 --> 05:48.230 And this is somewhere 1am in the night. 05:48.230 --> 05:53.350 And I gladly spent time with all of my hardware and servers and troubleshooting. 05:53.350 --> 05:55.850 And it really was very satisfactory. 05:55.850 --> 05:59.370 But my life has changed entirely since then. 05:59.370 --> 06:04.010 And now I became a consumer of problems I used to create. 06:04.850 --> 06:08.870 So my current life is really extraordinarily boring. 06:08.870 --> 06:13.170 All of those things which all of us typically hate. 06:14.510 --> 06:17.710 Agenda is my second nature right now. 06:18.550 --> 06:23.710 I cooperate on terms like standards, compliance, KPI, policies, procedures. 06:23.710 --> 06:26.130 Everything has to be very cost-effective. 06:26.230 --> 06:28.270 And it's all about risks. 06:28.730 --> 06:33.470 So , you know, at the beginning I was kind of a little bit upset. 06:33.470 --> 06:37.090 Like, what did I agree to do for this job? 06:37.090 --> 06:43.870 And as my very good cybersecurity friend, also a researcher, told me, well, congratulations, dear, you are a manager now. 06:43.870 --> 06:45.850 So, yeah, my life has changed. 06:46.250 --> 06:56.670 And yes, some of you will probably argue, but Marina, you previously worked as a lead and senior cybersecurity consultant and an engineer and a principal threat analyst before. 06:56.670 --> 06:57.570 You've done this job. 06:57.570 --> 07:01.650 You've consulted customers about cybersecurity programs. 07:01.650 --> 07:04.350 You did threat assessments. 07:04.350 --> 07:05.570 You should know all of these things. 07:05.570 --> 07:07.730 This shouldn't be something new for you. 07:08.070 --> 07:09.470 And that is a difference. 07:09.470 --> 07:13.830 As a consultant, it's your job to actually point to the problems. 07:13.830 --> 07:15.130 You have to find them. 07:15.130 --> 07:18.070 And as an analyst, you also have to report about problems. 07:18.070 --> 07:23.430 And then you leave the customer with a recommendation, and you actually do not see the larger picture. 07:23.610 --> 07:29.690 So money is not involved because you're not actually remediating or building plans to remediate. 07:29.690 --> 07:38.770 You're not really going into the individual circumstances of individual plans, maybe internal politics or relationships and whatnot. 07:38.910 --> 07:42.610 So this part was not previously present. 07:42.610 --> 07:47.230 And now I'm responsible for the entire picture, because I'm going to the sales meetings. 07:47.230 --> 07:50.710 I then participate in bids and project proposals. 07:50.710 --> 07:58.310 I am then participating in the building and configuring systems, like entire solutions for the customers, networks. 07:58.410 --> 08:10.130 And then when the customer is operating the new environment, for example, or upgraded environment, or even if it is a brownfield, I also consult on the advanced service or provide advanced services. 08:10.630 --> 08:19.750 So basically, when I go already with the sales meeting into the customer, I already see in advance how we will be implementing it and building and what we will be consulting. 08:19.750 --> 08:26.410 And I need to consult customer in advance what they might need in the future, because I know what they will need. 08:26.410 --> 08:31.490 So basically, it's really a big difference between then and now. 08:31.490 --> 08:37.250 And maybe the best way to explain is that previously I've seen this local optimum. 08:37.250 --> 08:49.570 So I was working on one task, like, for example, hey, I just do assessment of one plant and then maybe I even propose mitigations and maybe even in a prioritized fashion, but it's a local optimum. 08:49.570 --> 08:55.630 Now I work, for example, for building cybersecurity programs for the entire company with multiple plants. 08:55.630 --> 08:58.330 So now I have to optimize the global optimum. 08:58.330 --> 09:11.130 And actually, for me personally, this is the best analogies to describe what I'm doing right now and overall what I need to achieve right now in my new role is the best example is linear programming. 09:11.130 --> 09:16.950 I don't know how many of you know about that, but as an engineer, this is what I used to do a lot for engineering problems. 09:17.390 --> 09:19.610 And I put the link into the Wikipedia. 09:19.610 --> 09:21.270 I highly recommend to check it out. 09:21.270 --> 09:42.570 And it's basically every problem you need to solve or optimize, because this problem, this linear program is frequently used for operating, for solving operational problems and finding there, optimizing, for example, profit or something else based on the constraints which need to be fulfilled. 09:42.570 --> 09:52.830 And you then formulate the problem which you need to optimize and you formulate your constraints and then specific applications will try to solve you that optimization problem. 09:52.830 --> 09:54.790 And sometimes it takes hours and whatnot. 09:54.790 --> 10:04.010 So this is how I actually think about pretty much every problem because we always try to find this optimum based on the constraints we have. 10:04.050 --> 10:11.050 And then my new role is just like the optimization equations got longer and the list of constraints got longer. 10:11.050 --> 10:15.550 So optimization is becoming more and more difficult and challenging and trickier. 10:15.750 --> 10:17.610 But that's the beauty about it. 10:17.970 --> 10:31.470 So, and maybe the situation in which I am right now, it reminds me a little bit of Carsten Noll's story, because he used to be a cybersecurity researcher, he is still, and he was a source of troubles. 10:31.470 --> 10:36.230 And at some point he became a CISO of the richest startup in the world. 10:36.230 --> 10:37.590 It was in India. 10:37.890 --> 10:41.070 And basically he became a consumer of own troubles. 10:41.610 --> 10:56.130 And he gives insightful talks on how it is impossible to build the perfect cybersecurity program or just security for any application or anything, even with all the money and knowledge in the world. 10:56.130 --> 10:59.830 And it's really fascinating talks which he gives. 10:59.830 --> 11:05.230 Unfortunately, there are no recordings, but I was lucky enough to see it in person a couple of times. 11:05.330 --> 11:14.650 But it's interesting to see that even if you have all the money and all the knowledge, you know, finding this, optimizing that equation with so many constraints is impossible. 11:14.650 --> 11:17.210 You just have to find like basically good enough. 11:18.050 --> 11:21.970 So before I go into jump into the talk, a couple of disclaimers. 11:21.970 --> 11:25.030 This is the least technical talk I've ever gave. 11:25.030 --> 11:26.250 Still too many slides. 11:26.250 --> 11:31.610 I'm still working on trying to have fewer slides for my talks. 11:31.750 --> 11:34.830 I think maybe some of you this talk may be boring. 11:34.830 --> 11:36.750 So I'm sorry for that. 11:38.150 --> 11:48.710 And I know that many of the professionals in this field, especially who has many years of experience after seeing this talk, they will say, well, I told you so. 11:48.710 --> 11:52.230 Yes, I know you did 10 and 7 or 5 years ago. 11:52.230 --> 11:54.250 But I was a different person at the time. 11:54.250 --> 11:55.850 I agree with you now. 11:56.850 --> 12:00.970 So this talk is based on my own thoughts and considerations. 12:00.970 --> 12:06.690 So whatever tips and suggestions I provide, no warranty they will work for you. 12:06.690 --> 12:13.530 And honestly, for my current life phase, I work on so many projects at the same time. 12:13.530 --> 12:14.770 I have to switch quickly. 12:14.770 --> 12:18.870 I realized I really became comfortable that good enough is actually perfect. 12:18.870 --> 12:21.130 I used to be such a perfectionist. 12:21.410 --> 12:27.010 So basically, like who I was in all good times, you know, that was my entire world. 12:27.010 --> 12:33.810 You know, like I've been looking for the most weird, difficult, exotic ways of exploiting cyber-physical systems. 12:33.810 --> 12:37.870 I was mostly working at the layout of controllers and physical processes. 12:38.030 --> 12:43.810 And the more difficult and challenging problem was, the more I was interested and determined to find the solutions. 12:43.870 --> 12:58.750 And , you know, like I used to have calls with my security researchers friends and bashing about like, oh my gosh, can you imagine they did not touch that vulnerability, how terrible it is. 12:58.750 --> 13:01.230 You know, I used to be all of that person. 13:01.230 --> 13:13.630 And now when somebody is telling me like my customers telling me, well, do you think we need to publish that remote code execution vulnerability in some device? 13:13.630 --> 13:19.790 And I say like, no, if you have some time and money to spend, please build your demilitarized zone. 13:20.550 --> 13:28.270 So and basically, like if I would try to explain like what suggestions I was previously given, this would be example. 13:28.930 --> 13:36.390 For example, this picture, this cluttered room is basically a good example of how traditional OT environments look like. 13:36.390 --> 13:41.610 Like many of them, you know, it's just like, like this cluttered room, you know, like there is everything everywhere. 13:41.610 --> 13:43.070 There is no really structure. 13:43.070 --> 13:45.930 Nobody knows what, where, somehow it works. 13:45.930 --> 13:48.550 And I guess many of my OT colleagues will understand. 13:48.550 --> 13:57.570 And I was saying that like if the context of this cluttered room, I was telling that if we would apply some very advanced cybersecurity solutions. 13:58.370 --> 14:07.890 You know, like for example, an analogy would be to put this beautiful chandelier in this room and it will suddenly this room will become beautiful and organized. 14:07.890 --> 14:19.870 And in reality, it won't, you know, like I was trying to tell like, like if you put this chandelier room or basically it will transform that room into the Omnia club in the Las Vegas. 14:19.870 --> 14:23.870 You know, I'm not nostalgic at this point, but of course it won't happen. 14:23.870 --> 14:31.670 So obviously when I was recommending like skip the basics, apply some advanced cybersecurity controls and you will be fine. 14:31.670 --> 14:37.410 Clearly that was a really wrong recommendation and I am admitting it now. 14:38.390 --> 14:52.970 So, and basically now that I was that person, and now like now when I basically my customer come to me and saying like hey this is my cybersecurity budget, and I need to understand how best I spend it. 14:53.390 --> 15:00.690 You know, like, I have to be pragmatic and cost effective, and I have to change the way how I think. 15:00.690 --> 15:05.030 And this is how I've learned actually to like secure architectures. 15:05.830 --> 15:17.890 So for example, we all know how the traditional ICS network would look like we have a corporate network and skater network and somewhere on the bottom of it we have a physical application which we are trying to protect. 15:20.370 --> 15:36.410 And basically, typically we build such networks in a layered fashion you know about that and it's all determined by the data flows at the bottom we have real time data exchange, and it's become less real time at the top. 15:36.410 --> 15:49.310 And in general, this could be basically divided into two parts like process control and monitoring at the bottom and process management in ideally you should build them in such a way that you could always decouple this upper part. 15:49.490 --> 16:00.290 So you may lose all of your corporate and layer three, like this process management, you should be able to operate, you might not be able to serve your customers like dispatch your chemicals or whatnot. 16:00.290 --> 16:09.990 But you still save and you still can produce and if you're, you know, like vessels are full you may just shut down plant, but in a safe manner. 16:10.370 --> 16:20.490 And like basically if you look at the entire architecture how it's like the very classic Purdue reference architecture look like and this is looks like this. 16:20.490 --> 16:32.830 And actually I previously, I mean of course it always made sense for me from the data processing standpoint but now I started to appreciate it from the organizing and building cybersecurity. 16:33.490 --> 16:35.450 And I will explain why. 16:35.450 --> 16:46.450 And this is basically it also helps me to explain customers, how they should prioritize their expenditures so that I could have a meaningful conversation which they also this argumentation which they can follow. 16:46.450 --> 17:01.350 And at the bottom, we have actually this is where our physical process and this is where the hazards leave, and we are trying to prevent hazardous situation and we have sensors and actuators, which are solid units, which are completely trusted. 17:01.350 --> 17:10.390 At the moment there is no security controls which we can apply there and it's considered to be trusted there is so basically there is nothing what we can do from the security standpoint there at the moment. 17:10.390 --> 17:13.870 But this is where we actually this is the layer which we are trying to protect. 17:13.870 --> 17:19.730 And then we have a very gigantic this layer one layer two ways you distributed control system. 17:19.730 --> 17:34.250 And here's actually one more disclaimer is that clearly I'm mostly specializing in chemical pharmaceutical oil and gas so basically large continuous processes so this is a background which I use in my presentation. 17:34.250 --> 17:38.490 So, this is why I'm talking in the context of distributed control system. 17:38.490 --> 17:45.950 And while this is already system with the computers and we could actually apply traditional security controls which we apply in the corporate network. 17:45.950 --> 17:50.190 Things are very difficult in here and this is why I put, it's complicated. 17:50.190 --> 18:04.830 At the moment, unfortunately, the situation is such that the market is full of really fantastic solutions which we could apply for security and a lot of IT companies also basically trying to push their solutions into this layer. 18:05.130 --> 18:15.470 But at the moment we still did not find a good way how to basically approve and manage and apply this security solutions in this layers. 18:15.470 --> 18:26.910 And the point is the truth is also that vendors like ABB and all others whatever you think of, like Siemens the most famous Yokogawa Honeywell and so on. 18:27.050 --> 18:30.790 Each vendor has to approve specific solutions to be applied. 18:30.790 --> 18:33.910 So the customer is pretty much limited to what they can use. 18:33.910 --> 18:52.270 And because at the moment there is such a competition and a variety of products they're still not even sure what is the best and what is most cost effective, that it's actually very difficult to apply like really implement proper cybersecurity technical cybersecurity controls in this layer. 18:52.590 --> 19:02.570 So where we actually left with is our layer three and layer and demilitarized zone, and this is where you actually can. 19:02.570 --> 19:15.510 This is already full fledged servers, you can patch them you can use a host agent for advanced threat detection you can you can do vulnerability scanning threat hunting whatnot. 19:15.570 --> 19:16.390 It's all yours. 19:16.390 --> 19:22.810 So this is where you can actually collect the logs from firewalls monitor data flows and whatnot. 19:22.910 --> 19:27.570 So this is actually you can implement the full fledged cybersecurity program in here. 19:29.950 --> 19:35.510 And basically apply patches and reboot service without interrupting productions. 19:35.930 --> 19:45.870 And you would probably argue, but Marina, we still want to have this defense in depth, you know, like we should still do something in layer two and level one. 19:45.870 --> 19:56.070 And here's where I actually find a good argument why we should still concentrate most of our efforts on demilitarized zone and level three. 19:56.070 --> 20:04.490 So when we talk about level one and level two, this is already they are actually part already of the safety protections and I will explain you why. 20:04.550 --> 20:12.570 So this slide you see actually the relationship between security and safety like if you put it like in a temporary temporary fashion. 20:12.750 --> 20:21.930 So we have layers of security protections which actually protects us from stress, and behind the every set of human is always behind me it's something intentional. 20:22.470 --> 20:47.690 And if the attacker was able to go through the all the layers of security protections we have already security incident, and then we say, well, this is where we rely on the layers of safety protections, so that the you can see it on the slide. 20:47.690 --> 20:51.630 They are actually already part of the security layers. 20:51.630 --> 20:53.150 These are layers of safety 20:57.850 --> 20:59.390 preventions measures. 20:59.510 --> 21:17.930 So we basically, it suddenly makes more sense to keep attacker as far away so we basically have to really build a very strong perimeter and make sure basically minimize chances that even the attacker will even approach our layers of safety protection because it's our last line of defense. 21:17.930 --> 21:39.450 But there is one more argument, and it goes into the finances, we all know that, you know, like operations, even a small hiccup on the network when maybe packets is not arrived, that already may cause process upset and majority of incidents or cyber related incident when which happens in the plans, 21:39.450 --> 21:47.770 they're not security they're cyber incident we are not enough memory not not enough storage network congestion, or whatnot. 21:47.770 --> 21:58.530 But all of those, they already introduced interruptions into the normal operations and actually reducing the efficiency of how we run the process. 21:59.230 --> 22:04.330 And this is a really big problem for the operators because they are operating on there. 22:04.330 --> 22:14.590 So, yes, the number one priority is safety but their second number two priority is to maintain actually their operations. 22:14.750 --> 22:23.030 And if you look at the, like on the left you have a slide, when we look, look at the about integrity protection layer three we have alarms in there. 22:23.050 --> 22:28.190 And there is even explicitly stated financial alarm, you know like we're losing money. 22:28.270 --> 22:38.210 And on the right you see another graph, where we still operating may still operate in the normal operational envelope is still producing our product, but we already not in profit. 22:38.210 --> 22:51.250 We already losing money because we are not producing effectively, the product is still up to specification but we are using too much energy, losing too much useful chemicals maybe in the purge or whatnot. 22:51.250 --> 22:53.010 So they're not. 22:53.170 --> 23:05.370 And this is why you know like we know that pretty much any hiccup on this on the DCS operations may cause this inefficiency in the productions. 23:05.370 --> 23:09.510 And this is one second reason why we really want to keep the attacker away. 23:09.510 --> 23:17.370 We want to minimize chances of attacker, even reaching that level we don't want any random person roaming in our industrial control network. 23:17.370 --> 23:22.170 And this is actually to give you an example. 23:22.810 --> 23:28.690 Like how much money we may lose if the operator will not, for example, react on time. 23:28.870 --> 23:38.810 So, on the left you see that, for example, when we talk about minor effects something what you know just minor it's not even medium it's not meant major it's minor. 23:38.810 --> 23:55.110 It's when you lose estimated like from 10k and 100k, you know, if you talk about the budget of 100k, I think people cannot even imagine such budgets you know like this will be fantastic pentest of the entire corporate network, and we fight for years to get that, 23:55.110 --> 24:17.930 you know, they basically have to react to emergency alarms first, and this is when they have very little if you can see we have very little reaction time five to 15 minutes. 24:18.410 --> 24:27.010 And if they will not react within that time, like for example within 15 minutes, the loss could be up to from one to 10 million or even more than 10 million. 24:27.010 --> 24:29.090 So imagine just those numbers. 24:29.090 --> 24:30.490 So imagine them. 24:30.770 --> 24:40.050 We do not really strong build strong perimeter, and we say well even if the attacker gets to the control layer. 24:40.390 --> 24:42.870 We will still try to detect him in there. 24:42.870 --> 24:49.370 But if the attacker will manage to suppress just one stream was one, this critical alarm. 24:49.390 --> 24:56.930 If the operator just basically lost his visibility for a few minutes, he fails to react on time, and the losses can go up to 10 million. 24:56.930 --> 25:01.670 So, as you can see because of this very short reaction time high losses. 25:01.850 --> 25:16.370 We really want to keep all the attackers away from our control environments, and this is why I still argue that we have to introduce as many technical controls in the layer three and demilitarized and try to detect. 25:16.370 --> 25:24.690 So make sure that the only conduit into your control network has so basically the attacker has to go through the demilitarized zone and through layer three. 25:24.690 --> 25:27.990 There is no any other way into the control network. 25:27.990 --> 25:35.650 So basically the attacker has to go there, and you try to maximize maximum to detect an intrusion in those layers. 25:35.650 --> 25:38.290 So this is why I say this is your top priority. 25:38.290 --> 25:43.470 And for example prioritize spending 80% of your expenditures in those layers. 25:43.470 --> 25:51.570 And that means also that you first have to actually build your proper reference architecture, you need to form a proper DMZ zone. 25:51.570 --> 26:06.830 You need to properly form your level three, and this is very expensive, it's very difficult, it's complex, but this is should be your number one priority and I can't believe I'm saying that because two years ago I would not believe that I would be saying something like that. 26:06.830 --> 26:21.570 And actually the layer one and two, this is where most your security controls at the moment can take form of policies, procedures, and common sense, prayers, you know, like check karma of your employees and whatnot. 26:22.690 --> 26:34.750 So, of course, you know, like, there is also slight deviation, you know, the standards will tell you how the absolutely ideas, ideal network architecture should look like. 26:34.750 --> 26:42.930 And then, you know, there is a reality because you know it's all about balancing security versus usability and versus cost. 26:42.930 --> 26:45.790 Again, we're talking about linear programming optimization. 26:45.790 --> 26:54.910 And for example for our ABB control system and I'm saying this is for ABB, it might not be applicable to every organization. 26:54.930 --> 27:15.090 We built for example, slightly, I would not want to say simplified because this is security level four architecture, but it's a little bit more manageable because, for example, for certain operations like for example for antivirus updates and backup, we do allow routing of the traffic between DMZ and level two, 27:15.090 --> 27:21.710 but that is because the way how we implement those services because we still do them in a very secure very conservative way. 27:21.710 --> 27:27.930 And we are very well aware that this is not a risky to do so but it is much more manageable architecture. 27:27.930 --> 27:34.310 And that is the point that for every organization that architecture security architecture will look. 27:34.650 --> 27:37.430 I mean it must be individual and might look differently. 27:37.430 --> 27:45.430 It's all about basically implementing all of your data flows and access control and identity management in a secure way. 27:45.430 --> 27:56.630 So for example, once you map all of your data flows and file exchange routes, and you make sure that all they are very secure, then you're good. 27:57.050 --> 28:00.270 So, but that is has to be properly analyzed. 28:00.330 --> 28:06.010 And again I'm saying that it might not be looking like a traditional Purdue architecture but it still has to be secure. 28:06.010 --> 28:14.610 But once you build a security architecture and unfortunately in this talk I do not even have time to go in all of the zones and conduits and why it is so amazing. 28:14.650 --> 28:30.470 But once you have this granular and properly build architecture which is capable of managing, then it is very easy to actually introduce any other service because, like I will jump back to the slide we have like layer three where you have operations management, 28:30.470 --> 28:32.250 this is basically your service layer. 28:32.510 --> 28:44.870 And you actually may share it so if you have a site with several plans, you may share that level three between multiple sites, you will have multiple level two but then one layer three which will be services for all sites. 28:44.870 --> 28:54.570 And again, once you build this properly format, and you're capable of managing your architecture, introducing new services become really plug and play. 28:54.570 --> 29:03.890 So this is an example I remove the name of the application but this is one of the ABP application advanced service, like advanced application for process control. 29:04.190 --> 29:17.390 And we already build them and implement them in such a way that it will you just plug and play to your existing infrastructure and integrating a new service and introducing it and configuring is extremely easy. 29:17.470 --> 29:22.650 And it's just example of ABB but it's could be any application and any vendor. 29:22.650 --> 29:29.450 So, when I work with a customer so this is exactly the processes and explanations which I'm going through. 29:30.390 --> 29:49.810 It's probably sounds for all of you so boring like Marina like ways vulnerability research where CV, but that is my new world right now and I have to break this complex problems into easy arguments so that I could actually implement the fundamentals right so that I can play to build the advanced security controls. 29:49.810 --> 29:51.990 And on. 29:52.190 --> 29:58.970 So , many customers rightfully argue that it's expensive to implement correctly the entire architecture. 29:58.970 --> 30:00.490 At the beginning. 30:00.510 --> 30:02.450 Yes, but in the long term. 30:02.450 --> 30:12.630 It's gigantic cost saving factor, and it's, it provides you really very manageable, very high level of security assurance. 30:12.630 --> 30:17.330 So, of course life would be boring if it would be easy. 30:17.410 --> 30:37.890 You know on one hand you know like it's so easy to say, well just put in firewall and then you're done you just like have your DMZ, the reality is not so easy because you know such projects so every customer is constrained by the available budgets and sometimes by schedule like hey I scheduled my downtime on this time and I need to be very quick. 30:37.890 --> 30:44.190 And you see like forming a DMZ, or even just adding a fire extra firewall. 30:44.770 --> 30:49.630 If they have already one firewall and we add in like North, they have already North and we add in South. 30:49.830 --> 30:54.510 It's not that easy it's not only hardware cost but it's also you know like a lot of. 30:54.570 --> 31:08.850 I put here on the slide, it's just a snippet an example from our real project where we are building a DMZ it's not complete I hide the details but just give you an idea how many activities goes into such projects, it all costs money. 31:08.850 --> 31:16.810 So eventually it's not just the buying a firewall but executing such project is expensive because a lot of activities goes in there just to configure everything. 31:17.310 --> 31:28.970 And, you know, eventually because also building firewall DMZ meaning you know like you have to proper build your services or servers and split services to make sure that everything is secure and everything. 31:28.970 --> 31:35.290 But then it's suddenly because you still have this amount of budget and it suddenly become this Tetris for adult game. 31:35.330 --> 31:40.630 You know you're trying to shuffle this budget like maybe we will remove a little bit in here and do there. 31:40.690 --> 31:48.510 We will remove switch but add a server in there and so on because you're trying to optimize customer value with available budget. 31:48.610 --> 31:57.330 So it's not that easy to build it and this is always you know like the most interesting things are like always in the details. 31:57.330 --> 32:15.310 But still, I would argue this point of my life and in the current role that for me the proper DMZ is a higher priority activity than for example, patching a code execution vulnerability in the control at some lower level, you know, like, and anyway, you know, 32:15.310 --> 32:25.630 like, at the moment patching controllers is, I mean depending of course on the CVE but you know like, since most of them do not even have authentication and you can directly talk to them. 32:25.630 --> 32:27.970 Why even bother patching. 32:28.130 --> 32:35.770 But again, probably many of my cyber security researchers don't like what I'm saying right now, and we can talk about that reach out on Twitter. 32:36.430 --> 32:53.290 Well, here's a pro tip, you know, as I said, building security architectures is expensive, and, you know, sometimes plant operator experience really difficulties when making those financial decisions whether or not to invest into extra firewall extra server and so on. 32:53.290 --> 33:13.870 So here's my pro tip, like, how to ease that decision making for the customer, conduct a high level simplified risk assessment of specific architecture in hands or data flow and document identify risk, explain risk and potential consequences to the operators. 33:15.110 --> 33:32.950 Remind them that ultimately they are responsible for all the risks, they're planned, and they shall eventually address them by accepting or mitigating and if mitigating they considered this point too expensive, they have to be aware that they will have to accept that risk and this is the potential consequences. 33:32.950 --> 33:38.630 And typically you know what the simple process makes the decision so much more easier. 33:38.630 --> 33:44.130 Okay, you know what, I don't want to be responsible for that, put the firewall, we have we will find budget. 33:44.390 --> 33:46.570 So, you're welcome. 33:46.570 --> 34:03.870 And, you know, sometimes, also like this is why I learned to like compliance because sometimes you would say well if they're still doubting shall I shall I not you will say hey the standards is telling you or the best practices are telling you and then it's actually makes the decision making also easier for the customer. 34:04.690 --> 34:11.030 So, the second part is how I learned to love standards, which I never done before. 34:11.190 --> 34:16.550 So, um, in June, last year I actually gave a talk. 34:17.470 --> 34:22.310 About 62 443 how it is applied about to this asset. 34:22.310 --> 34:34.350 And I actually was bashing about the standard, a little bit I, I was, of course, a factual but I kind of like express couple of disagreement. 34:34.730 --> 34:39.330 And because I have a lot of very good friends who actually build the standard. 34:39.330 --> 34:56.490 You know like I never published the slides because I was a little bit ashamed, like how could I say some negative things or basically point out to some maybe imperfections in this in the standard and gosh how I'm ashamed of myself because you know like right now. 34:56.650 --> 35:01.730 I absolutely love and adore 62 443 that's everything for me right now. 35:02.110 --> 35:03.770 And here's the point. 35:03.770 --> 35:06.270 So why I started liking it. 35:06.270 --> 35:12.230 So, you know, performing technical cybersecurity plans audits is not actually very hard. 35:12.230 --> 35:40.230 I have genetic like really extensive experience in auditing plans, and I've done it for multiple organizations for multiple industries globally so that's something where I'm like skilled and I have no issues with that, where it's becoming difficult and complex is when you actually have to build the entire cybersecurity program or build a program for mitigating and prioritize fashion with timeline cost, 35:40.230 --> 35:44.270 and suddenly it's becoming a very complex expensive and time consuming process. 35:44.270 --> 35:49.750 In addition to that, you know, because if it is a process and you invest money into it. 35:49.750 --> 36:01.990 You need actually to justify to find some measured value to justify the expenditures to the project sponsors sometimes it's government sees or whatnot. 36:01.990 --> 36:04.490 You actually need to measure your progress. 36:04.490 --> 36:19.650 And you actually need also to compare, you know, cybersecurity posture and pro mitigation progress of individual plans which are completely different, different maturity, but you need to compare it in some normalized metrics. 36:19.650 --> 36:25.370 And this is where this business risk KPIs all came back into my life, and I suddenly. 36:26.550 --> 36:41.650 It's reminded in all my knowledge from which I got so I have an MBA degree and suddenly all that knowledge came handy to me and I started like basically remembering all of what I've learned in my MBA, and I put it into practice. 36:42.410 --> 36:44.570 And here's another thing. 36:44.570 --> 36:48.530 So, the fourth principle of economics are 10 of 10 of them. 36:49.250 --> 37:02.130 The fourth one is my favorite people respond to incentives, it means that we do certain actions and make decisions based on their perceived value to us, it could be also negative and it could be positive value. 37:02.130 --> 37:11.990 And this is a fun story when I had a meeting with a season for very large industrial organization. 37:13.030 --> 37:36.890 And I will keep telling that him about our success stories where we've done fantastic projects for our customers and then I told him like, hey, we built for a large LNG plan to cyber security program where we matured customer from quadrant five to 1111 it's kind of recommended for oil and gas industry so basically security level three maturity level three. 37:37.650 --> 37:57.830 And, um, and his immediate questions was, and where we are now, you know like people are competitive, you know like they want to compare themselves, and I was just like, well, let's build your KPIs about those and let's explain you how you can get from some lower quadrants into the higher quadrant. 37:58.090 --> 38:10.510 And the, what I like about 62443 is that it's provides a lot of logical and meaningful ways to build your KPIs when building cyber security programs. 38:10.690 --> 38:18.350 So for example, even, I must admit I previously was skeptical about the security levels and how they are described. 38:18.350 --> 38:23.010 Right now I like them because they really helped me to. 38:23.010 --> 38:35.450 For example, when we talk about security architectures you know like we need to perform risk assessment of every system like fire and gas and emergency shutdown and process control and safety and whatnot. 38:35.590 --> 38:46.390 And then we need to assign security levels based on the criticality risk criticality and also sign those security levels to conduits and then we need to meet those requirements. 38:46.390 --> 38:51.990 And right now, I really like it because it gives me something tangible to work with. 38:52.110 --> 39:03.210 And we know that different plans are have different maturity so some of them terribly legacy some of them, a mixture of legacy new some very new. 39:03.210 --> 39:23.290 And the standards is also provide us with additional metrics so that you can for example measure capability security level like well like legacy system will not be very capable, but your target security level is, let's say your capability level is one and your target is three, 39:23.290 --> 39:25.190 then the standards will tell. 39:25.190 --> 39:33.730 Well, you can use compensating controls and see whether to achieve the target level and then you measure your progress. 39:34.570 --> 39:40.130 With this achieved security level you probably will go from level one to level two and level three. 39:40.210 --> 39:48.630 So all of the security levels in the standard they provide a really way to build very nice KPIs to measure the effectiveness of your program. 39:49.490 --> 39:52.790 And then of course you need to measure also how well you're doing. 39:53.310 --> 40:07.850 And that standard has a maturity levels, and this is virtual actually forms those quadrants, it's basically like which security level you were able to achieve and how mature you in maintaining that security. 40:08.010 --> 40:16.170 So security controls, and what I like about this maturity levels so for example, if you already have. 40:16.170 --> 40:31.750 So, level one is just like you have a security control and level two if you basically build your documented process about that you already updated in level two it's so easy to progress in those quadrants and level three is actually if you are practicing so for example, 40:31.750 --> 40:39.850 you have antivirus level two you have a documented process how to update an antivirus and level three you actually updating that antivirus and you already. 40:39.850 --> 41:04.110 So basically you can use this quadrants for pretty much every security controls and that will help you to actually monitor and measure the progress, your mitigations progress, which for any season governance will make very fantastic numbers and of course you can build much more smarter and more interesting KPI but this could be building you a very fantastic foundation. 41:04.110 --> 41:12.270 So, this is also what I can work with in terms of easy to explain to the customer, and it seems like an achievable process. 41:14.290 --> 41:22.290 So, and this is an interest not this interesting thing that I used to hate, even the word policies and procedure. 41:22.370 --> 41:34.470 I used to teach ISO 2701 at the university, and it was the least my favorite topics, I would have an allergy for the word policy and procedure. 41:34.750 --> 41:37.830 And now this is my number one to go. 41:37.830 --> 41:45.150 So when I start working as a customer with says like well we've never done anything with the security and now we understand it's an important part. 41:45.150 --> 41:47.170 Government wants it. 41:47.290 --> 41:52.270 We understand that the risks are higher the moment you want to build the cybersecurity program. 41:52.430 --> 41:59.110 And that is the number one what I need from them is like guys, we need to establish policies and procedures. 41:59.110 --> 42:02.970 Policies will tell you what I want in terms of security. 42:02.970 --> 42:18.770 Your all security goals need to be described in the policy it's very high level, very in short sentences and procedures will actually tell you how you're going to implement it, because even if you talk about the backup there is thousands of way to implement the backup. 42:18.870 --> 42:35.390 We need to describe procedure will tell you how what is acceptable level way of implementing backups in this specific organization which is based on their network architecture based on the risk assessment on perceived risk or acceptable risk personal preferences. 42:36.030 --> 42:43.970 And if this is not this documents do not exist, we will always be unsure, but what is a good way of implementing your backup. 42:44.090 --> 42:46.390 We need first to formulate the procedures. 42:46.390 --> 43:05.390 And then the beauty part of the procedures is that you number one you have roles and responsibilities, people will be assigned like a responsibility for specific tasks, and they will put their signature, and as soon as people put their signature they feel responsible things will start happening. 43:05.450 --> 43:20.810 If nothing is described nothing is defined, nobody's responsible there is no cyber security program and for me the number one working first step when working with the organization, developing policies and procedures, assigning roles and responsibilities, 43:20.810 --> 43:27.910 is to train those who are responsible and then on top we can suddenly start building a program, because somebody is responsible. 43:27.990 --> 43:37.090 And I hate, I remember how much I hate the word Rossi this responsibility assignment metrics, and now I love it. 43:37.690 --> 43:44.630 Well, and then, basically, the last tip is basically it's my. 43:44.670 --> 43:49.410 So how do you, because when I'm asked by the customer. 43:49.410 --> 43:58.130 Can you please build me the cyber security program that number one is that they know how it is expensive, and the stakeholders and the CEO they want to know. 43:58.410 --> 44:10.910 Are we doing well, like, was this even budget and just spending the budget was justified and we know that with a security it's difficult to measure because you know like if nothing is happening. 44:11.150 --> 44:15.910 People believe that why do we need that security we are not detecting even anything. 44:15.910 --> 44:20.550 And this is why I measure the effectiveness differently. 44:20.710 --> 44:39.490 So, what I do I take the corporate risk metrics, and I tell them that you know what, in order to make sure that you are socially responsible that you're minimizing your potential business risk, you want to basically in terms of cyber security risk you want to be in quadrant, 44:39.490 --> 44:42.830 like in those green area. 44:43.030 --> 44:52.150 And if we bring like if we implement security controls and we conduct all of this risk assessment and specific documented way on which we all agree. 44:52.150 --> 45:04.710 And we believe that with implement security controls we minimize risks to acceptable level, then the program was justified and then when the government will knock your door or who not. 45:04.710 --> 45:09.650 They will, you will show hey I'm managing my risk, I am responsible. 45:09.650 --> 45:18.930 And if you talk about the same things with a person, use the word license to operate, because they will understand what it means. 45:18.970 --> 45:31.450 For example, when we talk about the safety, you know, if you did not minimize your safety risks to gain this acceptable level so you never would be allowed to leave red. 45:31.450 --> 45:33.770 So it has to be at least amber and green. 45:33.910 --> 45:36.470 You will not get your license to operate. 45:36.470 --> 45:47.290 This is the same with security, tell them the word of license to operate that now your cyber security risks are minimized to the level which is acceptable and they will then understand, oh, then we did a good job. 45:47.450 --> 46:08.710 So you see, the work which I'm doing right now and my current responsibility sounds a little bit so boring high level, but in reality it's so complex and I need really like kind of have to also think hard how to break this complex program problems into manageable and achievable steps. 46:09.910 --> 46:31.350 So, and now that basically how I wrote in the, like in my abstract but yeah Marina but still people like you exist and there are advanced attacks like do you think your current approach, like basically concentrate a lot of your security controls in the upper layers is effective against advanced tax and I wanted to give you an example. 46:31.350 --> 46:36.630 So, for many years I was dreaming to do something like killing them. 46:36.630 --> 46:40.670 What a filter in the photo utility and last year. 46:42.130 --> 46:59.850 Yeah, the beginning of last year I was lucky enough that I work with a trust research center in Singapore and their team has nice kindly assisted me in the experiment we are like trying to kill this. 47:00.830 --> 47:03.530 Ultra filtration filter in the water utility. 47:03.530 --> 47:05.410 It looks like that. 47:05.630 --> 47:16.350 And of course when we try to cause a physical damage to a specific piece of equipment it's all have to start, how did that things can be damaged like what is harmful to it. 47:16.350 --> 47:23.590 So you have basically have to read the manual and see what are the harmful conditions, and on one hand you can kill it. 47:23.590 --> 47:37.590 If you have like too many impurities in water like oil or grease it will basically damage the membrane but I don't know how to implement it, how will they introduce oil into the closed system remotely. 47:37.590 --> 47:44.590 But the second condition is that is what will damage the membrane is high pressure. 47:44.590 --> 47:47.750 So for example in this specific model it's a two bar. 47:47.750 --> 47:58.750 So it's basically and based in overall a prolonged operation on the very peak load which is basically close to two bar will kill that filter. 47:58.750 --> 48:01.330 So okay we need to build the pressure in the pipe. 48:01.330 --> 48:05.950 So for that we need now to steal the process documentation. 48:06.710 --> 48:13.970 And this is the two things that are two ways how that I can find it and it's again it's all of course individual. 48:14.210 --> 48:30.270 So, if the entire process was built and programmed and designed in the company, you actually need to infiltrate the company and steal this documentation, but very frequently and it's like really very frequently. 48:30.470 --> 48:45.190 So, the designs and actually programming of the PLC is done by the subcontractor so you need to actually to find the subcontractor of the company which is of interest to you and basically in the typically smaller companies not so well protected. 48:45.570 --> 48:49.370 And then you have to steal basically the documentation is in there. 48:49.370 --> 49:04.190 And this is why in your cybersecurity policy, it has to be there that you need to require your third party providers to achieve certain security posture in their organization, and we know that for example that is a requirement in NERC SIP. 49:04.190 --> 49:18.990 So let's assume like I as an attacker I identify the suppliers, I got the documentation I now know that there are two data flows go into this filtering. 49:20.950 --> 49:28.450 Basically, I can do it via the HMI screens or PIND diagram. 49:28.450 --> 49:38.890 And because the second flow is go from backwash I probably need to take a look what's happening in the backwash and how things are working in there. 49:38.890 --> 49:49.890 And then I will understand, basically, which PLCs and which control logic I need to enter and which equipment I need to compromise to actually implement this attack. 49:49.890 --> 49:54.490 But then I still, you know, like I don't understand which conditions will trigger. 49:55.210 --> 50:01.770 Basically, because I want to flows to flow simultaneously and I don't know how to do that I need to find the conditions. 50:02.050 --> 50:10.430 And there's actually guided in the state machine in the PLC, and this is something that I already won't find on any diagram, I actually need the PLC code. 50:10.450 --> 50:13.590 So for example, the PLC was programmed in the organization. 50:13.590 --> 50:19.770 I need actually already to penetrate the organization and this is where you should really try to prevent me from doing that. 50:19.770 --> 50:24.770 Because I need to steal the PLC code and read it in order to identify those conditions. 50:26.890 --> 50:55.430 And basically, then I need to manipulate the PLC logic or packets on the network, depending really how things are implemented but let's assume that I found the way to actually pull off this attack and this is specific cases, like in the context of this process I needed basically to implement attacks on two PLCs I need to compromise two different PLC in two different parts of the process so it was stage three and stage four. 50:55.430 --> 50:59.010 So they are basically a lot of things for me to compromise. 50:59.250 --> 51:11.690 And then so basically we'll assume that from the cyber perspective, I was able to implement it, and now the water from two flows, which never should fall simultaneously it flows gushing through the filter. 51:11.690 --> 51:22.970 And I now want to see how much, because I still don't know whether I will be able to reach the pressure, which I need so I'm basically need to measure, am I able to achieve two bars. 51:23.470 --> 51:31.090 And we run those experiments that experiment, and it turned out that the maximum differential pressure we were able to achieve in the filter was just one bar. 51:31.170 --> 51:33.010 So it's not enough for breakage. 51:33.010 --> 51:36.050 So as you can see, the attack is not almighty. 51:36.990 --> 51:57.310 But the point the most important point in here that there is no way I can ever figure out this, what the pressure can achieve like for example one bar without actually interacting with the actual physical process, and I might have this fantastic idea this is what I'm saying that the attack is not almighty. 51:57.670 --> 52:09.830 And that's a successful implementation of a damage scenario and it's cyber execution will not necessarily result in successful attack because certain things, I will only be able to measure on the life process. 52:10.370 --> 52:14.410 And this is what the point which I'm saying that many targeted damage attacks. 52:14.410 --> 52:32.110 This is the scenarios where I mostly work in I enjoy working, and it's useful to conduct such research but the point is that such targeted damage attacks require prolonged access to the process equipment, and this is what I'm saying, limit or eliminate such option for the attacker. 52:32.110 --> 52:43.330 If the attacker don't have that option they will not be able to execute it or maybe they may try to guess like well I feel like if I will do that implement that attack and they will try to create an autonomous malware. 52:43.790 --> 52:49.030 Well, they don't know they will be successful if it's only one bar, you're not damaging anything. 52:49.870 --> 53:07.890 So, on the other hand, like myself and my co researchers we recently came up with a really worrisome and way of compiling completely automated and very targeted payloads for industrial processes. 53:07.890 --> 53:20.450 We actually were accepted with this talk for a black card but we had to withdraw because me and my co researchers we received new job role so we felt it was inappropriate if you will be talking about basically offensive research. 53:20.610 --> 53:22.130 So we have to withdraw. 53:22.810 --> 53:34.730 But it's possible but nevertheless, again, the number of scenarios you will be able to implement with such autonomous payloads is much more limited. 53:34.730 --> 53:55.990 In comparison, so that's why I'm saying, make sure that every conduit, which leads to the control equipment and physical process goes through the military zone and level three and monitor for the attacker that look for him in their limit eliminate his ways for each having persistent access. 53:56.390 --> 53:58.330 And this is where. 53:58.330 --> 54:01.510 So for example, yes, if we talk about defense in depth. 54:01.510 --> 54:19.790 We still cannot of course guarantee that the attacker will not get those persistent access or continuous access to the process so focus on protecting and monitoring high criticality events data flows, when, for example, something in the, especially the time of the situation which you have alert emergency alarms, 54:19.790 --> 54:29.910 and the reaction time five to 15 minutes, maybe have a redundancy so if the attacker managed to spoof or suppress alarm or drop the packet I don't know was this alert. 54:29.910 --> 54:39.090 Make a redundancy for every critical alarm that it goes maybe to communication path is so that the operator will not miss it. 54:39.570 --> 54:45.130 So, I guess, so that was pretty much it from me. 54:45.170 --> 54:57.230 So as you can see, basically the way how I'm building the cyber security program right now is kind of good to actually make life of the advanced attackers like myself. 54:57.230 --> 55:03.470 So it might mean we know that the security no guarantees but it's really make life so much more difficult. 55:04.170 --> 55:18.350 And I don't have really smart afterwards, as I say this was not a very technical talk it was just my confessions, how about how I think differently right now when I'm working in a different role. 55:18.350 --> 55:45.930 And, as I said, building enterprise plant, or plant by cyber security program is not simple but it can be broken into the simplest steps which makes you easier decisions, make your decision making process easier and maybe more transparent and then more prioritized fashion basically applying this how do you eat an elephant principle and in this talk I shared you how I broke down that for myself. 55:47.590 --> 55:59.870 Focus on establishing the foundation first, even if it is extremely boring, and, you know, like I can't believe like a couple years ago I wouldn't believe that I would say something like that but I'm saying that. 55:59.870 --> 56:15.490 And yes in in process control because you know like if you remember I told it, we don't want even that Tucker be in our control system not even messing up but even be there, it's such a sensitive system, it's just like thinking about like a belly of the cat. 56:15.490 --> 56:23.490 It's something soft, we need to build a very hard shell around it, we just want don't want anybody going be even being there. 56:23.490 --> 56:28.810 And the success of our security program is about minimizing chances of anybody reaching those layers. 56:29.570 --> 56:33.830 And yes, of course I'm missing my cyber security vulnerability research. 56:33.830 --> 56:39.690 So if I have a couple of minutes and I guess I have, I would take a couple of questions. 56:43.100 --> 56:44.680 Hi, can you hear me. 56:44.680 --> 56:45.480 Yes. 56:45.560 --> 56:54.860 Okay, so we don't have any questions yet but I do want to share some feedback that happened because there were multiple times in your, your talk that you were saying this. 56:55.440 --> 56:58.300 For the record, this is not boring at all. 56:58.300 --> 57:02.100 It's really good to see the higher level view, long term strategy, etc. 57:02.420 --> 57:04.480 Somebody else, I concur. 57:04.480 --> 57:07.880 This is a more realistic topic for asset owners and integrators. 57:08.160 --> 57:18.680 In my opinion, there is a large gap between academia, government research, security researchers, and the reality of real operational technology environments. 57:21.280 --> 57:24.780 Okay, so I'm very glad that people enjoy the talk. 57:24.780 --> 57:27.120 I'm really glad that somebody found it useful. 57:28.980 --> 57:35.300 I also informed everybody on the DEF CON discord channel that you would be trying to join there. 57:35.380 --> 57:37.300 If you were able to. 57:37.400 --> 57:39.320 So we are not live anymore. 57:39.380 --> 57:40.680 We are still live. 57:40.680 --> 57:41.420 Okay. 57:42.620 --> 57:49.660 All right, so I, I thank everybody for the attention and for being here. 57:49.820 --> 57:59.680 There was like also good feedback on the Twitter so thank you very much and I will try my best to join this, the channel on the DEF CON right now. 57:59.720 --> 58:02.080 Okay, well thank you very much for joining us Marina. 58:02.600 --> 58:03.940 Thank you for having me.