1 00:00:01,300 --> 00:00:04,040 Hi, thank you for attending our talk. 2 00:00:04,040 --> 00:00:09,800 I wish we were able to see you in person and hope that in the next year that things will get back to normal. 3 00:00:09,860 --> 00:00:20,500 As you can see from the title, the topic is very important because the Internet, the biggest storage of all your funny pictures and memes runs on electricity. 4 00:00:20,500 --> 00:00:33,760 And we are here to talk about internal self-power generation automations, about vulnerabilities that we were able to find, and about what you can do about that. 5 00:00:34,120 --> 00:00:40,700 We are security consultants who have been working with different industrial solutions for many years. 6 00:00:40,700 --> 00:00:51,180 Actually, we have been doing it for so long that we have a long list of contact information of different system integrators and industrial vendors. 7 00:00:51,380 --> 00:01:02,200 So when an asset owner, for instance, a power plant comes to us and asks for a service, we don't just give them information about vulnerabilities that we found. 8 00:01:02,200 --> 00:01:13,440 We closely work with all involved entities, including the vendor, to find together safe and reliable fixes and workarounds. 9 00:01:13,780 --> 00:01:25,720 We work at Kaspersky, and this research is a result of the teamwork, not just of the three of us, but also of Gleb Guritsay, Sergey Andreev, and Sergey Sidorov. 10 00:01:27,060 --> 00:01:36,260 Everything that we are going to discuss today was reported to the respective vendor, I'm talking about Siemens, a long time ago. 11 00:01:36,260 --> 00:01:40,120 But actually, this is not one vendor problem. 12 00:01:40,120 --> 00:01:45,050 You can find similar issues in systems of other vendors as well. 13 00:01:45,740 --> 00:01:54,860 We are going to talk about real vulnerabilities in real power plants out there, and at first glance, it may seem irresponsible. 14 00:01:54,860 --> 00:02:03,400 But if you think about it, for good guys to do such research, it's a challenge. 15 00:02:03,400 --> 00:02:08,280 You will need to have relevant experience. 16 00:02:08,280 --> 00:02:16,260 You will need to have a lot of time, and you, of course, you will need access to the industrial environment. 17 00:02:16,260 --> 00:02:21,480 There is no welcome sign on power plant doors, right? 18 00:02:21,980 --> 00:02:31,500 So for good guys, for penetration testers, for auditors, for power plant separators, it's challenging to get access to all of the things together. 19 00:02:31,540 --> 00:02:42,060 On the other hand, for bad guys, for those bad guys who would be willing and able to attack a power plant, it's their job to do such research. 20 00:02:42,060 --> 00:02:49,140 They have significant investments, they have a lot of time, but they keep their vulnerabilities to themselves for their malicious purposes. 21 00:02:49,420 --> 00:03:01,400 So we assume that those guys already have all this information, and from our side, we would like to share this information with good guys, with you, so you would be able to act upon this. 22 00:03:02,540 --> 00:03:08,620 Power plants is the main source of electricity on our planet. 23 00:03:08,620 --> 00:03:23,540 And thanks to carbon monitoring, anyone with internet access can get information about where different power plants are located on the world map, what fuel they use, and which is their capacity. 24 00:03:24,120 --> 00:03:27,600 The heart of every power plant is a turbine. 25 00:03:27,600 --> 00:03:34,260 We don't have a picture of a turbine here, but if you ever saw a modern airplane, I believe you also saw a turbine. 26 00:03:34,260 --> 00:03:37,700 This is a giant rotating thing generating electricity. 27 00:03:39,240 --> 00:03:43,440 And on power plants, they look and work quite similarly. 28 00:03:46,680 --> 00:03:53,380 Turbine manufacturers are generous enough to share information where the turbines are used. 29 00:03:53,380 --> 00:03:57,700 Here, for example, a screenshot from Siemens' website. 30 00:03:58,720 --> 00:04:07,340 Turbines are used not only on power plants, they are used in many different areas like chemical or oil and gas and many others. 31 00:04:07,340 --> 00:04:16,560 But if you correlate this picture with the previous one, you will be able to understand a solution from which vendor is used on a particular power plant. 32 00:04:17,400 --> 00:04:29,520 In addition to that, online you can find a lot of information about what solutions are used in certain power plants, in different press releases and marketing materials. 33 00:04:29,520 --> 00:04:36,720 You can find a lot of interesting information about software hardware version generations or different systems. 34 00:04:37,060 --> 00:04:39,740 Sometimes you can even find building plans. 35 00:04:39,740 --> 00:04:41,780 Here's just a small example. 36 00:04:41,780 --> 00:04:44,700 Okay, Google, show me some power plants of California. 37 00:04:45,820 --> 00:04:49,600 Here are just a few power plants of California. 38 00:04:49,600 --> 00:04:53,000 And they have one more thing in common. 39 00:04:53,120 --> 00:04:58,260 They use the same plant control system, SPPAT 3000 from Siemens. 40 00:04:58,260 --> 00:05:04,020 This is exactly the system that we are going to discuss today. 41 00:05:04,400 --> 00:05:16,500 But before we move on to power plant generation automation, let's talk a little bit about power generation process in general, what we are going to automate. 42 00:05:18,900 --> 00:05:32,300 Here and throughout the presentation, we will be intentionally oversimplifying a lot of things, partially to make them more suitable for the audience and partially because, to be honest, we don't fully understand them ourselves. 43 00:05:32,840 --> 00:05:35,700 So let's go from right to left. 44 00:05:37,280 --> 00:05:39,040 First, you will need some fuel. 45 00:05:39,040 --> 00:05:41,220 Here is coal, for example. 46 00:05:41,220 --> 00:05:49,280 You put fuel into a combustion chamber, put it on fire, and it generates pressure to rotate a turbine. 47 00:05:49,320 --> 00:05:58,080 The turbine is connected to an electricity generator through a shaft, so when it's rotating, the generator starts generating electricity. 48 00:05:58,280 --> 00:06:04,140 But it's important to mention that electricity doesn't go straight to your house or city. 49 00:06:04,440 --> 00:06:08,620 First, it goes to a special place called power grid. 50 00:06:08,620 --> 00:06:20,320 Power grid knows information about market demands, it receives energy from different power plants, and this is power grid who distributes electricity to consumers. 51 00:06:21,160 --> 00:06:31,400 During the burning process in the combustion chamber, you can have a lot of excessive heat, so-called cold waste heat, and there are different ways how to approach it. 52 00:06:31,400 --> 00:06:38,300 First, you can release it to the air through a condensing tower, or you can reuse it during recuperation. 53 00:06:38,800 --> 00:06:45,860 For example, warm water and send the steam to a turbine to generate more electricity. 54 00:06:47,480 --> 00:06:55,120 Systems that are used to automate this process are called distributed control systems, or DCS. 55 00:06:56,420 --> 00:07:02,140 They are designed to make life of power plant operators much, much easier. 56 00:07:02,240 --> 00:07:07,140 They allow you to conveniently start and stop the power generation process. 57 00:07:07,140 --> 00:07:15,660 They allow you to control the amount of electricity you want to generate, you just enter there the amount in megawatts that you want to have at the output. 58 00:07:15,660 --> 00:07:18,080 And they allow you to monitor everything. 59 00:07:18,080 --> 00:07:21,760 These systems are literally very powerful. 60 00:07:21,940 --> 00:07:29,620 They are connected through PLCs, they're connected to many parts of the plant, including the turbine. 61 00:07:29,620 --> 00:07:46,960 And they control such interesting things as the amount of fuel, the temperature in the combustion chamber, the rotation speed of the turbine, and even control the turbine not to go into different dangerous modes. 62 00:07:46,960 --> 00:07:53,280 Obviously, it's not a small piece of software that you just install on the server and it magically works. 63 00:07:53,280 --> 00:08:05,160 It's a very sophisticated system consisting of different hardware, software, PLCs, input-output modules, the turbine itself, and a lot, a lot of things. 64 00:08:05,480 --> 00:08:10,660 And often starting to create such a system starts even from building a structure. 65 00:08:10,660 --> 00:08:17,640 So someone comes to a vendor and says, we have an empty field, please build us a power plant. 66 00:08:17,820 --> 00:08:24,080 And there are many vendors who do this, but today we will be talking about Siemens. 67 00:08:26,020 --> 00:08:31,220 The DCS from Siemens that we analyzed is called SPPAT3000. 68 00:08:31,220 --> 00:08:45,220 Just like other DCSs, it consists of many different industrial components, PLCs, OPCs, different servers, HMI, a lot of things. 69 00:08:45,220 --> 00:08:55,680 And it may have very different architecture depending on the site, but there will be two components that are unique for SPPAT3000. 70 00:08:55,680 --> 00:08:58,800 They are called application server and automation server. 71 00:08:58,800 --> 00:09:01,140 And this is how we will structure our talk. 72 00:09:01,140 --> 00:09:08,820 First, we will talk about application server, then we will move on to automation server, and then we will move on to conclusions. 73 00:09:09,860 --> 00:09:18,760 In different manuals and documentation from Siemens, you will see how the system should be built in a perfect world. 74 00:09:18,760 --> 00:09:30,000 For example, they will say that the application network where application server is located should never be connected to any external networks, and how everything should be designed. 75 00:09:30,840 --> 00:09:37,920 However, also, we again here worry over simplifying everything. 76 00:09:39,040 --> 00:09:46,320 In reality, in real power plants, you will meet a lot of interconnections to other systems. 77 00:09:46,320 --> 00:09:52,580 For example, you will need a sensor network to monitor different vibrations inside the turbine. 78 00:09:52,980 --> 00:10:01,320 You will need a demilitarized zone because you will need to get some kind of remote support from the vendor. 79 00:10:01,320 --> 00:10:06,260 You will need to get updates for your operating systems, for antivirus, and so on. 80 00:10:06,260 --> 00:10:14,460 You will need to push out OPC traffic to your corporate network or to a regulator because this area is very strictly regulated. 81 00:10:14,460 --> 00:10:19,040 So, in practice, you will need a lot of interconnections. 82 00:10:20,280 --> 00:10:26,120 The life of our vulnerabilities started almost two years ago. 83 00:10:26,120 --> 00:10:45,860 Back in November 2018, we reported a bunch of vulnerabilities to Siemens, and about a year later, Siemens published an advisory containing information about how to approach these vulnerabilities, and also a set of other vulnerabilities that were reported by other vendors, 84 00:10:45,860 --> 00:10:52,140 other teams, and that's great that this area receives a lot of attention from security researchers. 85 00:10:52,820 --> 00:10:59,820 But also, about a year passed, it doesn't mean that during this year, Siemens didn't do anything. 86 00:11:00,600 --> 00:11:05,740 The thing is, SPPK3000 is an exclusively supported system. 87 00:11:05,740 --> 00:11:11,120 It means that a system integrator for this is Siemens themselves. 88 00:11:11,120 --> 00:11:22,900 So, soon after we reported these vulnerabilities, Siemens started to roll out patches and working directly with their clients to fix everything. 89 00:11:23,140 --> 00:11:26,180 So, December is just when this information became public. 90 00:11:28,160 --> 00:11:35,880 We don't have a lot of time to discuss in detail what kind of attacker, what can do with the system. 91 00:11:35,880 --> 00:11:48,140 I will just highlight that CVSS scores given here, obviously, do not represent how critical these vulnerabilities for real power generation process. 92 00:11:48,140 --> 00:11:53,840 They're just individual scores for vulnerabilities in the vacuum. 93 00:11:53,840 --> 00:12:04,140 To better understand what kind of attacker can do what, you can take a look at our threat model that is published in our white paper. 94 00:12:06,440 --> 00:12:10,120 Let's start with application server. 95 00:12:10,120 --> 00:12:14,960 Application server is a logical core of the entire system. 96 00:12:14,960 --> 00:12:21,100 Everything has connections to application servers in the logical sense. 97 00:12:21,240 --> 00:12:25,860 If anything needs to connect to the network, it will end up in application server. 98 00:12:25,860 --> 00:12:31,420 Other servers start their work from the loading software from application server and launching it. 99 00:12:31,420 --> 00:12:33,880 So, this is the heart of the system. 100 00:12:33,880 --> 00:12:38,960 And what can possibly go wrong if you open over 40 ports there? 101 00:12:39,540 --> 00:12:42,040 This is for an attacker. 102 00:12:42,040 --> 00:12:44,260 This is a huge attack surface. 103 00:12:44,980 --> 00:12:47,960 They are likely to get into their respective network. 104 00:12:47,960 --> 00:12:50,080 They can choose what they want to attack. 105 00:12:50,080 --> 00:12:55,040 Do they want to attack Windows operating system because this is simply a Windows server? 106 00:12:55,040 --> 00:12:57,200 Do they want to attack third-party components? 107 00:12:57,200 --> 00:13:02,800 Or they want to attack own SPP 83,000 services which are based on Java? 108 00:13:04,820 --> 00:13:08,460 Also, we hope that these vulnerabilities are already fixed. 109 00:13:09,180 --> 00:13:14,740 In industrial environment, usually, you don't update your systems very often. 110 00:13:14,740 --> 00:13:21,720 Usually, it's about half of a year or several months between major updates of operating system. 111 00:13:21,720 --> 00:13:33,560 So, it's always possible to find a time window with remotely exploitable vulnerabilities in Windows operating system. 112 00:13:33,560 --> 00:13:35,260 Configurations also could be better. 113 00:13:35,260 --> 00:13:39,480 You can find some statistics for security benchmarks on the right-hand side. 114 00:13:40,100 --> 00:13:44,020 But one of the biggest problems is actually passwords. 115 00:13:44,380 --> 00:13:49,620 The life of passwords has three stages. 116 00:13:49,620 --> 00:13:52,800 At first, passwords were the same for all power plants. 117 00:13:52,800 --> 00:13:54,860 They were default and the same everywhere. 118 00:13:54,860 --> 00:13:56,900 And you can find them online. 119 00:13:57,380 --> 00:14:01,720 Obviously, this screenshot were not from information published from Siemens. 120 00:14:01,720 --> 00:14:10,840 Some power plant operators decided that it would be a good idea to share passwords and a lot of other technical information with the Internet. 121 00:14:11,560 --> 00:14:19,360 But they are available online and we also have a work list in our white paper. 122 00:14:20,140 --> 00:14:27,280 Around 2014-2015, Siemens started to generate different passwords for different applications. 123 00:14:27,280 --> 00:14:30,740 However, the process of changing them was challenging. 124 00:14:30,740 --> 00:14:35,240 You would have to be familiar with the process to change the passwords by yourself. 125 00:14:35,280 --> 00:14:44,540 And only in the last year, in 2019, the process became easier and now you can do it yourself much easier. 126 00:14:44,560 --> 00:14:50,520 Now please welcome Radu who will tell you about vulnerabilities in Java. 127 00:14:52,040 --> 00:14:53,680 Hello everyone. 128 00:15:07,810 --> 00:15:12,730 Let's look how SPP software works on application server side. 129 00:15:12,830 --> 00:15:17,570 Operator can communicate with system through theme or fed client. 130 00:15:17,570 --> 00:15:29,090 In case of theme client, it uses Java plate of Internet Explorer and communicates with server over HTTPS. 131 00:15:29,090 --> 00:15:36,530 So it can be outside of application network and its communication can be constrained by firewall. 132 00:15:36,530 --> 00:15:42,770 In opposite, in case of fed client, operator should belong to application network. 133 00:15:42,770 --> 00:15:58,110 And fed client directly communicates with RMI registry to found RMI services and after that directly communicates with Sys services. 134 00:16:06,640 --> 00:16:12,080 Illustration of SPP architecture was kindly provided by system through a public URL. 135 00:16:12,080 --> 00:16:16,160 So not to be missed, let's divide it into spaces. 136 00:16:16,540 --> 00:16:22,100 In red zone there are items that process HTTPS requests. 137 00:16:22,100 --> 00:16:26,080 And in green zone there are RMI services. 138 00:16:26,080 --> 00:16:32,300 RMI service looks like network services which assert on dynamic TCP ports. 139 00:16:32,780 --> 00:16:36,360 SPP consists of containers. 140 00:16:36,860 --> 00:16:41,100 All types of containers are represented on these illustrations. 141 00:16:41,100 --> 00:16:47,060 And containers have self-explanatory names. 142 00:16:47,060 --> 00:16:55,540 So before we go deep inside internals of SPP, let me introduce some tools which used in this research. 143 00:16:55,540 --> 00:17:02,320 First of all, all jars files of SPP are obfuscated with commercial product. 144 00:17:02,320 --> 00:17:06,860 But this security measure can simply bypassed by public available tool. 145 00:17:06,860 --> 00:17:12,220 Secondly, sometimes it is useful to see how legit software communicates with server. 146 00:17:12,220 --> 00:17:18,040 It helps to understand architecture and to see client workflow. 147 00:17:18,520 --> 00:17:22,100 In case of SPP, RMI desector was written. 148 00:17:22,100 --> 00:17:27,000 It represents raw TCP streams and in human readable format. 149 00:17:27,000 --> 00:17:31,640 Inside it used read object method of SDK. 150 00:17:31,640 --> 00:17:38,900 It is known that this method suffers from insecure deserialization vulnerability. 151 00:17:39,180 --> 00:17:43,440 So be sure not to be exploited through remote pickup. 152 00:17:45,620 --> 00:17:49,560 The first pillar of SPP is Apache web server. 153 00:17:49,560 --> 00:17:56,360 According to its configuration, folder Orion Software Config can be accessed by unauthorized user. 154 00:17:56,360 --> 00:18:07,660 But in fact, this folder contains some critical information about application and about items of application and automation network. 155 00:18:07,660 --> 00:18:14,860 Also, configuration of Orion web application in Tomcat also can be accessed. 156 00:18:16,020 --> 00:18:21,400 Tomcat, there are three web applications registered in Tomcat. 157 00:18:28,010 --> 00:18:33,230 There are Remote Diagnostic Service, Manager and Orion. 158 00:18:33,290 --> 00:18:44,090 According to configurations of Apache and Tomcat, all these applications can be accessed by unauthorized user. 159 00:18:45,090 --> 00:18:50,230 Inside Orion web application there are seralettes which can be accessed. 160 00:18:50,230 --> 00:18:57,460 All of them are listed in configuration web.xml and the list is huge. 161 00:18:57,790 --> 00:19:04,150 Some of seralettes has attractive names for attacker. 162 00:19:05,150 --> 00:19:17,130 For example, BrowseSeralette allows unauthorized user perform directory listing on arbitrary folder. 163 00:19:17,130 --> 00:19:22,490 But in case of exploitation, another seralette is more useful. 164 00:19:24,650 --> 00:19:42,980 FileUploadSeralette allows unauthorized user perform file upload and folder is fully controlled by parameters based on target name of HTTP request. 165 00:19:43,150 --> 00:19:48,680 This vulnerability can simply be bypassed to remote code execution. 166 00:19:50,820 --> 00:20:08,120 For example, attacker can corrupt some startup scripts of SPPoA or simply inject JSP Shell in Tomcat web application and get remote code execution with system rights. 167 00:20:13,140 --> 00:20:17,280 Also there are seralettes which has service factory in its name. 168 00:20:17,280 --> 00:20:22,560 This seralette redirects HTTP request to RMI services. 169 00:20:22,800 --> 00:20:46,220 Inside it uses parameter service url of HTTP to identify RMI service which will be called and serialized object in data section of HTTP request contains information about method and arguments which will be called. 170 00:20:47,260 --> 00:20:58,300 So there are situations when themed client and fan client can communicate with RMI services. 171 00:20:58,300 --> 00:21:08,780 But in case of fed client, it can communicate with RMI registry. 172 00:21:08,780 --> 00:21:29,960 So if application server miss some important security updates of Java, then server contains insecure deserialization vulnerability and public available tool ySourceSerial can exploit it and perform remote code execution with system rights on server. 173 00:21:29,960 --> 00:21:43,800 So the next task will be to identify all input vectors for attacker and for this task we will try to list all RMI services in system. 174 00:21:43,800 --> 00:21:51,240 At first step we will use class LocateRegistryOffice.com and get big list of RMI services. 175 00:21:51,240 --> 00:22:02,840 All but one are Jamex RMI services and I assume that they used to control and manage containers of spp. 176 00:22:02,840 --> 00:22:08,860 For further investigation we will choose Lockup service. 177 00:22:08,860 --> 00:22:23,020 In fact, it looks like a collection of next level RMI services and using its public methods we can get the reference to this next level RMI services. 178 00:22:23,340 --> 00:22:29,920 Next level RMI services should implement InterfaceServiceFactory. 179 00:22:30,920 --> 00:22:51,720 So it also looks like some collections of another next level RMI services and using public methods getService and parameters such as clientId and name of the service. 180 00:22:51,720 --> 00:22:59,900 The instance of next level RMI service will be created and reference to it will be returned to client. 181 00:23:01,700 --> 00:23:11,880 And this next level RMI service is a server which performs real job of spp. 182 00:23:11,880 --> 00:23:22,380 But in fact it contains a lot of public methods which can be accessed by unauthorized user. 183 00:23:22,800 --> 00:23:28,340 So input vector of spp is very huge. 184 00:23:28,340 --> 00:23:35,760 The next question is to understand how authentication performs on a system. 185 00:23:35,760 --> 00:23:43,470 For this task let's look how client performs request to security service. 186 00:23:44,760 --> 00:23:56,060 To do this first of all client tries to get reference on security service with some clientId. 187 00:23:56,060 --> 00:24:02,420 It needs to PCServiceFactory use this clientId to get valid session in SessionManager. 188 00:24:02,420 --> 00:24:11,560 If SessionManager will fail then exception will thrown and client request will be rejected. 189 00:24:11,560 --> 00:24:18,900 But in case of success valid sessionId will be returned to PCServiceFactory. 190 00:24:18,900 --> 00:24:34,220 And in its turn PCServiceFactory create instance of security service where sessionId will start in loginId. 191 00:24:34,700 --> 00:24:39,240 And reference to this instance will be returned to client. 192 00:24:39,240 --> 00:24:45,420 Further client can call some public methods of this service. 193 00:24:46,900 --> 00:24:55,180 And in its turn these methods can perform some privileges checks of client. 194 00:24:55,240 --> 00:24:59,600 So we have two security measures of system. 195 00:25:00,180 --> 00:25:08,300 But there is a question how client can perform login operation on system if he doesn't have any valid clientId. 196 00:25:08,300 --> 00:25:18,340 For this task at startup of SVPR SessionManager create anonymous session with clientId 0. 197 00:25:18,460 --> 00:25:24,380 And client use this clientId and perform login operation. 198 00:25:24,380 --> 00:25:30,500 But attacker can also use this and simply bypass this security measure. 199 00:25:30,500 --> 00:25:44,780 To sum up, there are a lot of RMI services which has a lot of public methods. 200 00:25:45,240 --> 00:25:50,440 And permissions checks are fully delegated to these methods. 201 00:25:50,440 --> 00:25:56,840 So it's really difficult to perform security management of this system. 202 00:25:57,920 --> 00:26:03,360 So we understand all input vectors and security measures. 203 00:26:04,200 --> 00:26:08,190 So it's time to find vulnerabilities. 204 00:26:08,800 --> 00:26:13,800 In the list of RMI services there is admin service. 205 00:26:13,800 --> 00:26:16,680 It can be accessed by an authorized user. 206 00:26:16,680 --> 00:26:21,340 And its public method transcript doesn't have any privileges checks. 207 00:26:21,340 --> 00:26:30,680 In fact inside it first step create instance of class loader using bytes from arguments. 208 00:26:31,500 --> 00:26:36,540 In fact this step will load arbitrary Java class. 209 00:26:37,280 --> 00:26:43,160 This class should implement interface admin script and defined method execute. 210 00:26:43,160 --> 00:26:48,460 This method will be called by run script of RMI services. 211 00:26:48,460 --> 00:26:58,620 For this case we create Java class which simply run OS command. 212 00:26:58,920 --> 00:27:02,920 And this OS command will be run with system rights. 213 00:27:03,340 --> 00:27:13,840 In fact there is more powerful post exploitation because we inject arbitrary Java class around SPPS software. 214 00:27:13,840 --> 00:27:22,740 So we can use some Java reflection and patch some private variables of SPPS. 215 00:27:22,740 --> 00:27:32,320 And as a result we can corrupt some technological process of SPPS. 216 00:27:33,400 --> 00:27:42,340 To bypass privileges checks in methods we can use second vulnerability. 217 00:27:51,020 --> 00:27:58,800 It's using RMI session service and its public method get login sessions. 218 00:27:58,800 --> 00:28:05,640 Attacker can get information about all login user on the system. 219 00:28:05,640 --> 00:28:13,820 This information contains user names, IP and client ID of login users. 220 00:28:13,820 --> 00:28:42,660 Attacker can reuse this and if in this information there is a user with admin rights then attacker can simply reuse his client ID and get the reference to security service with more privileged session. 221 00:28:43,020 --> 00:28:57,580 And after that attacker can call public method get all users of security service and as a result get all private information about all users on system. 222 00:28:57,580 --> 00:29:02,740 Moreover password hashes also contains in this information. 223 00:29:04,900 --> 00:29:15,760 Both of these vulnerabilities can be accessed from either external or application network. 224 00:29:15,760 --> 00:29:25,700 All communications between RMI services are unencrypted so username and password hashes are transferred in plain text. 225 00:29:25,700 --> 00:29:31,820 Moreover system doesn't have any session protection mechanism. 226 00:29:32,060 --> 00:29:48,760 This fact is more critical in case of fed clients because of attacker can perform meat attack on the user of SPPS and get valid username and password hashes from the traffic of user. 227 00:29:48,760 --> 00:29:59,960 And after that simply reuse this username and password hashes and perform login operation on the system. 228 00:29:59,960 --> 00:30:07,940 Moreover attacker can also change the password of the user. 229 00:30:08,220 --> 00:30:15,740 I talked a lot about password hashes and users so it's time to understand how these items are organized on the system. 230 00:30:15,740 --> 00:30:17,220 Alex, you're welcome. 231 00:30:19,960 --> 00:30:21,580 Hello everyone. 232 00:30:21,580 --> 00:30:24,540 Let's continue our discussion about application server. 233 00:30:24,600 --> 00:30:28,420 On the previous slide you could see how remote authentication works. 234 00:30:28,420 --> 00:30:31,400 Now I'm going to tell you about how it's organized locally. 235 00:30:31,440 --> 00:30:42,180 After the system gets started it begins to read the content of files users1.xml and pdate1.xm to get user list and their password hashes. 236 00:30:42,180 --> 00:30:48,400 User1 is a simple XML while pdate1 has a slightly more complex structure. 237 00:30:48,400 --> 00:30:51,300 It's a gzip archive encoded in base64. 238 00:30:51,300 --> 00:30:58,200 There is a javascriptization object in the gzip archive containing a specific XML. 239 00:30:58,200 --> 00:31:03,020 The fields of this XML from pdate1 file are presented on the slide. 240 00:31:03,020 --> 00:31:09,490 They are used to calculate password hashes and check it during user authentication. 241 00:31:09,490 --> 00:31:16,930 At the bottom of the slide you can see a password check algorithm in a pseudocode. 242 00:31:16,930 --> 00:31:21,930 The cryptographic scheme is a typical crypt hashing scheme like in your Unix and Linux machines. 243 00:31:21,930 --> 00:31:29,210 It has solved different number of iterations and the only one thing which was added is hardcoded salt. 244 00:31:29,210 --> 00:31:34,530 The same for all users which is concatenated to the password. 245 00:31:35,670 --> 00:31:42,370 The tool to extract password hashes and their parameters from pdate1 file has been developed. 246 00:31:42,370 --> 00:31:45,370 Its output is presented on the slide. 247 00:31:45,750 --> 00:31:54,110 The tool can be used during password auditing to check weak or dictionary passwords and their hash calculation parameters. 248 00:31:54,110 --> 00:31:59,510 The tool is available in our GitHub repository. 249 00:32:00,650 --> 00:32:03,450 Draw the line under application server analysis. 250 00:32:03,450 --> 00:32:12,110 As you have seen, the attack surface is really huge and includes Java and MySQL systems, Tomcat applications and a lot of others. 251 00:32:12,270 --> 00:32:16,170 Secondly, it's about remote connections. 252 00:32:16,670 --> 00:32:24,250 Whether SPP has or has not remote connections according to vendor, system integrator or someone else, you should check it. 253 00:32:24,250 --> 00:32:28,630 And the good starting point for this is application server. 254 00:32:28,630 --> 00:32:37,530 Because OPC maintains remote operator, all of these things will end up on application server. 255 00:32:37,530 --> 00:32:50,170 Thirdly, despite the fact that there is automation network between application server and field devices, an attacker can affect generation process from application server. 256 00:32:50,170 --> 00:32:59,530 These actions include to start stop generation, change power output or just get information how power plant works. 257 00:33:01,370 --> 00:33:09,250 Of course, such manipulation will always be visible for operator, always monitoring technological process. 258 00:33:09,250 --> 00:33:17,690 So a real attacker will also be required to change or modify data on operator's HMIs. 259 00:33:17,690 --> 00:33:21,690 But it's also possible with real security issues. 260 00:33:22,470 --> 00:33:24,910 That's all about application server. 261 00:33:24,910 --> 00:33:31,930 Now let's move on to another main component of SPP infrastructure, automation server. 262 00:33:34,070 --> 00:33:42,870 The main goal of the automation server is to execute real-time automation functions and tasks for power plant control. 263 00:33:43,830 --> 00:33:55,050 Depending on power plant project architecture and features, the role of the automation server can be different. 264 00:33:55,050 --> 00:33:57,130 We have distinguished three roles. 265 00:33:57,130 --> 00:33:59,130 And the first one is automation. 266 00:33:59,510 --> 00:34:17,310 There may be slight confusion because the term automation is used both for server and its role, but analyzing automation server configuration and publicly available information about it, we have found that whatever the role is, almost the same hardware and part of software are used. 267 00:34:17,310 --> 00:34:23,170 So we have decided to use this kind of role classification, which seems less confusing to us. 268 00:34:23,170 --> 00:34:28,570 But at the same time, it's slightly different from the vendor's one. 269 00:34:28,570 --> 00:34:41,850 Anyway, having an automation role means that the server is responsible for interaction with input-output modules, which control and monitor power plant equipment such as turbines or electric generator. 270 00:34:42,130 --> 00:34:44,870 The second role is communication. 271 00:34:44,870 --> 00:34:49,810 This role is used to communicate with third-party systems. 272 00:34:49,810 --> 00:34:59,830 In another words, it's just a protocol converter, supporting such protocols as Modbus, IEC 101, 104, and some others. 273 00:35:00,090 --> 00:35:01,990 And the last role is migration. 274 00:35:02,730 --> 00:35:13,450 In this role, the server is used to connect previous versions of SPPA, such as SPPA-T2000 or Teleperm-ME. 275 00:35:13,450 --> 00:35:21,530 In the automation role, the automation server can be run both on schematic-assigned PLCs or industrial PCs. 276 00:35:21,530 --> 00:35:26,210 In the case of other roles, it can be run only on industrial PCs. 277 00:35:27,630 --> 00:35:36,390 Let's talk a little more about each role and let's start with the automation role based on PLCs. 278 00:35:36,430 --> 00:35:42,870 PLCs are what directly control field devices and access to them is game over for any security discussion. 279 00:35:42,870 --> 00:35:50,670 Any configuration changes and updates for PLC are required to stop technological process. 280 00:35:50,670 --> 00:36:00,070 Therefore, these devices usually have security misconfiguration and firmware without security updates. 281 00:36:00,090 --> 00:36:08,850 Another common security issue for PLCs is using unsecure industrial protocols. 282 00:36:08,850 --> 00:36:15,870 In the case of SPPA, there are schematic S7 protocol and PLC data protocol. 283 00:36:17,010 --> 00:36:23,010 There is quite a lot of information about S7, but not so much about PLC data protocol. 284 00:36:23,010 --> 00:36:26,510 So we had to deal with it and analyze it ourselves. 285 00:36:26,870 --> 00:36:29,330 It's not a special protocol for SPPA. 286 00:36:29,330 --> 00:36:37,730 When you program your PLCs and you need to exchange some data between them in real time, you use this protocol. 287 00:36:37,730 --> 00:36:45,210 It's pretty simple and maybe its description is available somewhere in the internet, but we couldn't find it. 288 00:36:45,210 --> 00:36:47,950 So just in case, I'll show its structure here. 289 00:36:48,510 --> 00:37:02,550 It doesn't have any security mechanism and the only obstacle while dooming in the middle attack to spoof data is a sequence number, which we can get from a packet and just a fuzz incrementation. 290 00:37:04,050 --> 00:37:10,790 For protocol analysis, Wireshark Dissector has been developed and is also available in our repository. 291 00:37:12,050 --> 00:37:23,790 During PLC security assessment, one of the main things which we check is unauthorized access to reading and writing the PLC memory. 292 00:37:23,790 --> 00:37:36,150 Availability of unauthorized access is determined by the position of mode selector of schematic PLCs and some other configuration parameters. 293 00:37:36,470 --> 00:37:45,230 The matrix on the slide shows unsecure states for schematic PLCs when unauthorized access is possible. 294 00:37:46,390 --> 00:37:58,310 The tool for gathering information from PLCs over the network and for its analysis has been developed by one of our colleagues and also available in our repository. 295 00:38:01,800 --> 00:38:06,440 Now let's talk about automation server based on industrial PC. 296 00:38:06,440 --> 00:38:11,520 The workflow of all roles in this case is quite similar to each other. 297 00:38:12,360 --> 00:38:17,880 When automation server is based on industrial PC, it's just a Linux box. 298 00:38:17,880 --> 00:38:23,740 During the start, it tries to download some additional files from application server. 299 00:38:23,740 --> 00:38:36,560 These files include JARs, which represent SPPA runtime containers, Barscript, some configuration files, and other. 300 00:38:36,560 --> 00:38:43,020 In order to execute downloaded JARs, PTC PERC virtual machine is used. 301 00:38:43,020 --> 00:38:51,920 It's a real-time Java virtual machine widely spread in industrial and military areas. 302 00:38:53,280 --> 00:39:01,600 After that, running JARs open RMI services or some of their extension. 303 00:39:01,600 --> 00:39:08,180 In case of migration server, Orion RPC services, which are extension of classic Java RMI services, are used. 304 00:39:08,180 --> 00:39:11,780 And you can see their listing on the slide. 305 00:39:13,260 --> 00:39:19,400 Automation server based on industrial PC has following security issues. 306 00:39:19,400 --> 00:39:27,600 Firstly, there is a possibility to spoof downloaded from application server files. 307 00:39:30,360 --> 00:39:39,080 These files are downloaded over HTTP and there are no security mechanism like authentication or integrity check during this process. 308 00:39:39,080 --> 00:39:42,120 Secondly, it's about using default credentials. 309 00:39:42,120 --> 00:39:44,600 Username admin with password cm. 310 00:39:44,640 --> 00:39:48,680 Thirdly, it's RMI services running on automation server. 311 00:39:48,680 --> 00:39:57,980 The research has found two vulnerabilities in Orion RPC services, which allow to perform sensitive data exposure and RCE. 312 00:39:57,980 --> 00:40:05,820 And the last group is vulnerabilities found in the software used to fulfill 313 00:40:09,300 --> 00:40:12,460 a migration role for connection previous version of SPPA. 314 00:40:13,540 --> 00:40:18,000 SPPA-T2000, also known as TXP. 315 00:40:21,020 --> 00:40:33,880 All these vulnerabilities found in migration server software are related to different kinds of overflow, stack, heap, integer, and other. 316 00:40:33,880 --> 00:40:47,300 Actually, there are so many overflows here that this talk would be overflown by that if we started to describe all these vulnerabilities in details. 317 00:40:48,580 --> 00:40:58,530 If you want to know about RPC vulnerabilities, these vulnerabilities have been found in runtime engineering service. 318 00:40:59,740 --> 00:41:11,440 This service has a method, requestRuntimeContainer, where the first argument defines an action to be executed. 319 00:41:11,440 --> 00:41:16,320 Using the action readFile, it's possible to read any file from the system. 320 00:41:16,320 --> 00:41:23,200 Using writeConfigFile action, it's possible to write any file to any folder in the system. 321 00:41:23,260 --> 00:41:32,380 For example, it can be jar file, which executes shell command from the command line. 322 00:41:32,380 --> 00:41:43,400 Then, using the same requestRuntimeContainer method, it's possible to execute this jar later. 323 00:41:44,640 --> 00:41:47,060 That's all about automation server. 324 00:41:47,060 --> 00:41:53,260 To sum up, automation server can be run on PLC or industrial PC. 325 00:41:53,260 --> 00:41:59,420 In case of PLC, it's a usual PLC with well-known security issues. 326 00:41:59,420 --> 00:42:11,500 In case of industrial PC, it's a Linux box, which tries to download jars from application server and then executes them with PHP or virtual machine. 327 00:42:11,500 --> 00:42:20,580 An attacker can spoof downloaded files or just exploit revealed vulnerabilities in network services. 328 00:42:22,140 --> 00:42:30,820 So far, I haven't mentioned any network equipment used in distributed control systems. 329 00:42:31,180 --> 00:42:37,280 During the research, we saw a wide variety of network devices and network infrastructure. 330 00:42:37,280 --> 00:42:46,760 They include different kinds of switches, routers, firewalls, and more rare devices, such as data diet, for example. 331 00:42:46,760 --> 00:42:55,740 We tried to summarize all this information and got common SPP network topology scheme presented on the slide. 332 00:42:57,520 --> 00:43:13,000 We have shown in purple usual places for network devices, but it also should be mentioned that the same devices in the same places can be found in other distributed control systems from other vendors. 333 00:43:14,720 --> 00:43:21,100 Network devices in industrial network usually have a lot of security issues. 334 00:43:21,100 --> 00:43:29,760 The reason is these devices don't require any configuration and can be run out of the box. 335 00:43:29,760 --> 00:43:45,180 Therefore, the things like weak community strings in SNMP, weak credentials in HTTP, FTP, Telnet, and other services, firmware with publicly available exploits, and just security misconfiguration. 336 00:43:45,480 --> 00:43:49,820 All these things are common and typical for industrial network devices. 337 00:43:49,820 --> 00:44:00,960 Additionally, some industrial protocols, for example, such as Profinet-DCP, can allow to get or set network configuration of these devices. 338 00:44:01,100 --> 00:44:13,260 In case of Profinet-DCP, it also can be very useful for device enumeration. 339 00:44:13,260 --> 00:44:32,300 And the last thing about industrial network devices is always remember that these devices with years of uptime can act unpredictably for completely legal and normal activities, such as browsing a webpage with statistics or logging into Telnet. 340 00:44:32,620 --> 00:44:37,460 Here, my part is over, and now Evgenia will sum up our speech. 341 00:44:37,460 --> 00:44:38,360 Thank you. 342 00:44:38,880 --> 00:44:40,680 Thank you, Alex. 343 00:44:40,680 --> 00:44:47,660 So, the topic of power plants is huge, and the topic of DTS is huge. 344 00:44:47,660 --> 00:45:09,400 We just tried to poke a couple of things here and there, and showed that to affect the power generation process, you don't have to go deep to the field level, to PLCs, but you can focus on higher logical levels, like application server and automation server. 345 00:45:09,400 --> 00:45:16,360 What we do not talk about today is havoc or damage that you can cause by such attacks. 346 00:45:16,360 --> 00:45:18,940 Actually, that's because it's not that bad. 347 00:45:18,940 --> 00:45:33,700 If you already imagine a hacker in a hoodie who writes a couple of lines, and after that, the entire city goes into the darkness, it's not like that, because it's power grid who is responsible for power distribution. 348 00:45:33,700 --> 00:45:36,500 This is not a power plant. 349 00:45:37,170 --> 00:45:46,500 So here we mostly talk about more local damage, like would it be possible to damage a turbine. 350 00:45:46,500 --> 00:46:11,460 Since a turbine is connected to a DCS through PLCs, and the turbine is a giant mechanical monster, and it self-degrades by working, probably by putting it into different uncomfortable modes or quickly starting and stopping it, it would be possible to make it degrade even faster or even break it. 351 00:46:12,240 --> 00:46:14,220 We were not able to check it. 352 00:46:14,220 --> 00:46:20,560 Unfortunately, or actually, fortunately, we were not able to find a spare turbine on eBay. 353 00:46:20,780 --> 00:46:29,120 But we are making an educated guess that the damage is out there based on the system architecture. 354 00:46:30,060 --> 00:46:46,460 We wanted to make life of power plants a little bit easier, and we prepared a small do-it-your-own assessment guide that you can use to check your SPPAT3000 system for vulnerabilities that we will discuss today. 355 00:46:47,860 --> 00:46:55,240 You just connect your laptop to a couple of places in the network, go through a simple checklist. 356 00:46:55,240 --> 00:46:59,600 You don't have to hire expensive security consultants for that. 357 00:46:59,600 --> 00:47:10,340 And after that, you can fix some parts yourself, or you can call your system integrators to discuss what to do further. 358 00:47:11,740 --> 00:47:15,660 Just any other industrial system. 359 00:47:16,120 --> 00:47:19,760 DCSs are not so good in terms of security. 360 00:47:19,760 --> 00:47:21,240 It could be better. 361 00:47:21,240 --> 00:47:50,260 And we recommend you to take a look at ISAIC 62443 set of standards to know what to do to improve security in terms of communicating with different entities, because in the industrial area, it's usually a challenge when you, for example, when a vendor is not interested in fixing something. 362 00:47:50,260 --> 00:48:02,440 So it describes in particular what kind of relationships you can build with regulators, with vendors, and other parties involved in the industrial area. 363 00:48:03,200 --> 00:48:11,980 Of course, update your systems, change your passwords, improve your configurations according to the vendor security guidelines. 364 00:48:11,980 --> 00:48:21,800 We also recommend you to set up monitoring since most servers are based on standard Windows and Linux boxes. 365 00:48:22,440 --> 00:48:30,800 You will not be able to detect different Java attacks, but at least you will be able to detect some parts of attacks. 366 00:48:31,060 --> 00:48:41,720 And again, this is more about DCS and industrial security in general, rather than by one system by a single vendor. 367 00:48:42,520 --> 00:48:45,940 We released a lot of information. 368 00:48:45,940 --> 00:48:50,080 We released a white paper, different tools that we mentioned today. 369 00:48:50,160 --> 00:48:55,580 I recommend you to take a look at our GitHub page to find all of that. 370 00:48:55,580 --> 00:49:03,020 And we would like to thank Siemens product search, who made all the communications very effective. 371 00:49:03,020 --> 00:49:04,560 They were very responsive. 372 00:49:04,560 --> 00:49:07,540 They allowed us to contact Siemens product team. 373 00:49:07,540 --> 00:49:12,700 And of course, they released the patch. 374 00:49:13,240 --> 00:49:24,900 Take a look at the advisory and Siemens always tries to raise awareness of their users, how to better build such systems. 375 00:49:24,900 --> 00:49:35,680 What are better configurations, but they also highlight that it's not the vendor who is solely responsible for security of an industrial environment. 376 00:49:35,680 --> 00:49:43,640 There are a lot of things that power plant separators can and must do themselves. 377 00:49:43,680 --> 00:49:47,480 So please follow the guidelines for that. 378 00:49:48,260 --> 00:49:51,180 That's all for our talk. 379 00:49:51,220 --> 00:49:55,680 Thank you very much for your attention and we will be glad to answer any questions.