Thanks, everybody. Our next speaker is Hari Hursti. He's a subject matter expert and co-founder of Nordic Innovation Labs and one of the foremost election security experts in the world. So let's give it up for Hari. All right. Thank you for having me. First, I want to tell the village is extremely busy. We are under one hour, 40 minutes from opening the doors, which in reality means less than one half hour starting people pouring in. We first had the first two hacks. We are right now seeing an extensive progress in three other things. So I don't think we get to early dinner time before those are complete. It's amazing progress. And one thing which is interesting is that people who haven't been part of the studies have never seen the results. So I don't think we get to the machines. They don't know what we did. And hence, they don't have the same tunnel vision as the people who have been 10 years looking into this. So they have a completely new approach. And if somebody would be educating them, they'd say, probably don't try it, but there's nobody telling that. So that's why they make a good success in completely new ways and completely new approaches. What was interesting was that the first hack was a professor from Denmark who was able to take control remotely. He was able to take control remotely over Wi-Fi, the win-vote voting machine, on operating system level, able to speak directly with the command line to the operating system, able to unrestrictedly go where the voter data is, et cetera, et cetera. So it was absolutely totally penetrated. Very interesting how quickly it happened from the point of you have never seen the machine to the place when you are, okay, proof of concept, I mean. Anyway, this talk was kind of having a different, the hacker perspective. So I think it was a very interesting talk. I think it was a very interesting talk. I don't know when it happened, but I'm going to give the speech which I wanted to give, which is to tell about what has happened since 2005 today. What are the lessons learned? And it also is a speech which I want to underline that don't make any assumptions, because most likely the assumptions you make are going to be wrong if you are a security researcher. And my statement from 2005 is still true. If I wouldn't have seen this, I wouldn't have been able to do this. So I think it was a very would only have said this with my own eyes, I don't think somebody would be able to convince me. These things are just a, this science which, there was never a prime time of this kind of design. So they should have never been in the market in the first place. So it's a, it's a sad state of affairs. So first of all, who I am and why I'm talking about this. I have been starting hacking election machines against actually my intention in 2005. And what happened to me was I was I was for over a year trying to be convinced that I should take a look into the election machines. I said no. I said no again until I decided that I want to get rid of the people who are asking me to vote the hacker machines by setting an impossible set of rules. And then my impossible set of rules were met by Ian Shanshaw from Tallahassee, Florida. He invited me over, invited me to take a look into the machines. And the rest is history. We discovered a vulnerability in design. It's not a bug. It's a vulnerability in design which allows you to pre-stack the electronic ballot box and gives you a capability of arbitrarily shift the outcome of the election. And the interesting feature of the voting machine is that the actual evidence, how much it was shifted, is self-destroying evidence. And the normal cause of the voting machine suffer. So you don't need to, as an attacker, worry about erasing the forensic evidence. The machine will do that for you. I participated in three government-sanctioned studies in the United States. Everest is, while it's old and should have been a long, long time ago replaced by a newer study, it's still the most comprehensive study. It took the California top-to-bottom review. Very fascinating. It verified all the findings of California top-to-bottom review and then added some new information. The redacted report is 316 pages. Wonderful bedtime reading. There is a – the report actually describes every vulnerability found. But the redacted version is the actual – some of the vulnerabilities are too dangerous. So the actual cookbook, a recipe, how it works, that is redacted. So you know there's a vulnerability. You just – The report doesn't tell in the most dangerous one how to access that vulnerability. And after that, I have been doing studies around the world. We are actually going to have a little bit of, in this presentation, experiences from Estonia, which Estonia has always referred. Why can we do internet voting? Because Estonia can. The truth is Estonia cannot either. But they just market it that way. So we will have a little bit of a look in that. And this talk is not about – the last presidential election, but in the last presidential election I participated in a number of lawsuits in the recount effort testifying about the vulnerabilities of the machines, et cetera. So, again, disclosure. I have only interest in security. I have a – I'm not an American citizen. I cannot vote. So I have no stick in this game. I have – I'm not talking about any specific election. These all vulnerabilities, all the things we are discussing has been used. They have been exploited. They are exploitable in number of elections. In case of Estonia, I think the number is nine elections. In the U.S., God knows how many, because these machines are used in a number of jurisdictions where, for example, in New Jersey, you can have 250 elections a year because of all the local elections. So we don't even know how many elections these have been potentially affecting the United States. So verifiability, transparency, integrity, security, those are my interests, not the politics or discussion about if an adversary, number one, or two, did anything. So, first of all, what we are not going to talk about. We are going to have a presentation at four by Matt Place. Matt is going to give you a demonstration with the real machine, the punch card machine, which started this mess in 2000 election in Florida. So he will show the actual machine. He will allow probably you to vote, and he will tell you the actual story, what is – what has been – missed in the weeds why this guy became the icon of year 2000 in count with trying to see the pregnant and hanging and pimpled sheds on the ballots. Again, how we got here, 2005 was the Diebold machines had. Ohio, those are the white independent studies. There's a lot of studies which are not independent, which are paid studies, or they're not independent. They are – the researchers are not independent. Those are different kinds of studies. The independent studies are when you have – when the study is made for – without any links or connection to the vendors or stakeholders in that, other than the government or Secretary of State or, in some cases, a lawsuit. And Estonian study is important for a perspective of internet voting. It's very – it's interesting to have the number. In 2016, 52 different models of voting machines were used in U.S. elections. The normal statement has been always to say that U.S. election machinery and infrastructure is protected by diversity. The argument is because there's so many different kind of machines, there's so many different deployments, that is protected. There's no single point of failure. First of all, from attacker's point of view, there's no single point of failure. Second point of view, it doesn't work that way. Attacker needs to know only for a few machines, and then you can choose where you attack, based on what you know. So the diversity actually is giving you a wider range of services, and it gives you an easier way to find where your skills meet your targets. So diversity, to a certain extent, is good, but it is not a – it's a false sense of security to say diversity itself is giving you a security. So diversity, to a certain extent, is good, but it is not a false sense of security, and it gives you a security. In that sense. The other thing is, the diversity itself is also undermined by a thing which is right now coming more to service, and it's the service companies. And we will have a slide later about service companies. In a number of states, a huge amount of jurisdiction is actually programmed and run by a very small number of service companies which are not certified, they are not catalogued, they have various practices in security level. We will have a little bit more discussion about that. Anyway, here we have... little bit of pictures of what has happened to the voting machines in the previous studies, replacing operating system, replacing the firmware, doing everything what attacker would do. So theory of election is simple. This is actually putting a little bit of a reference to what David Jefferson and Barbara Simons were talking. We all think this is what election should look like. We will take a vote and you add it to the candidate. That is what people think election machines are, and voting infrastructure. That is far from the truth. That is actually the real election systems. This is only the election part. The side here is a hard into civic infrastructure. The side on this side is ES&S. Those are two major manufacturers of their voting machine technology in the United States. This is what the actual architectural overview is. Humans are complex and these are complex. doesn't even touch the other areas, which is the election management part, the voter registration, e-poll books, yada, yada, yada. And it's very important to understand when we look to what the reports of the recent adversary attacks, they didn't go after the election machines. They went after the back end infrastructure. They went at the back end network, the back office. That was the target. Why attacking the voting machine when you can scratch your belly and send a couple of poison holes word documents and get your access that way to the back end network? So we have been looking at this area, and this is the voting technology everybody recognizes, but the actual infrastructure is way larger, and David Jefferson and Barbara Simons covered that at 10 a.m., so there will be a recording. You can see the recording and get yourself familiarized with that part of the structure. So when we now look at the voting machine and voting depilations system, what we have learned is that the voting machine and the voting depilation system, what we have learned is that we have a very complex system. First of all, what the systems look like. This is a one major vendor in the United States. This is the complexity of the code. The election management system, scripts, COBOL, visual basic. What could possibly go wrong with that and how modern it is? But if you look at the numbers, these are not numbers of code lines, comment lines. This is the actual active comment lines of the source code. These are humongously complex systems, and one would argue, and I think it's a right argument, unnecessarily complex. But elections in the U.S. are more complex than people think, and especially new legislation like rank voting is adding complexity, because the new laws allow new methods of voting, and you have to accommodate that. But just by looking at that and looking at how much assembler code we have here, how much we have a raw C, how much we have, I would say, obsolete things like COBOL, you immediately know this is a humongously expensive system to maintain, and when it's an expensive system to maintain with that amount of code, bad things will happen. Just to make sure that we understand that they are not alone, this is another manufacturer, yet another, it's more C++, maybe C sharp, assembler, no COBOL. I think that's about the same. But again, the management system, over 100,000 lines of code, the voting terminal, bootloader alone, 72,000 lines of code. And remember, these are not including your operating system, they are not including your driver code. There is millions and millions of lines of dependencies which come into place, and especially driver codes which are normally running in a kernel mode. If you don't know what they are, you don't know what your system is doing. So this is just to underline what is the complexity. Third vendor, more C++, but still, when you look at the numbers of code, humongous, and they don't run in the voting terminal standard operating system, they actually have a home-baked operating system. That number of lines is just the application. The operating system itself, which doesn't need to be certified under the theory that it's not vote-acting, humongous code, this is just to tell where, if you look at the code, it's not a vote-acting code. If you try to understand these two systems by a pure source code audit, you will discover something, but that is not enough. You are not going to be ever able to establish a sufficient amount of trust to the system integrity, sufficient amount of trust to the system performance and error-free producing by just looking at the source code. You have to go to fully integrated system, end-to-end, where you have the operating system, and you have the testing, where you have everything coming from drivers, everything, and then you test it as a whole. Otherwise, you are missing a huge part of the whole infrastructure. Now, if we compare that to what we have found here, and this is kind of a scary list of things. First of all, hard-wired PIN codes. We are talking about voting terminals where a supervisor PIN code, is either hard-wired as the only code, or there's a secondary code which is hard-wired, so even if the election supervisor changes their PIN code, there is another code which the election supervisor doesn't know, which can be used to log in. Hard-wired encryption keys. We have a lot of documentation that this has been across many of the vendors. Some of the vendors have gotten rid of that. Very bad practice. And, you know, we have a lot of documentation that the other practice is files starting with the negative encryption keys. I think that's very good for a key management perspective. If your threat model is that you lose your key, now it's very convenient. The key is always there, but it also renders the whole encryption meaningless. It's then purely advertising and decorated. We have a lot of variations of that, where the key itself is artificially mangled so that the entropy is lost. We talk about a key switch. We talk about a key switch that is where key strength is 16 or 24 bits, which means that your pocket calculator can track it in a matter of seconds. Even when you start with strong key, the normal way is you make the binary key to be converted to base 64, and then you truncate the base 64. Now you are done. Your entropy is gone. Again, marketing statement, everything is encrypted, technically means it's true, but when you look at the meaning of the key, it's a little bit different. It's a little bit meaningless. Is the encryption done in a meaningful way? Not at all. So don't assume when you are told that it's encrypted that it's encrypted in a meaningful way. Hard-wired secondary passwords. There is one vendor which their system has a secondary password called idiots. That's a wonderful idea. It's idiots with the ones and zeros, you know, you mangle. But in any username you can use with the real password of the user, or you can use it with the real password of the user, or you can just say idiots, and you are in. What it means is really scary, because now the system logs that that person came in, and if something is done wrong, there is now a log entry about wrong, as a real person would have been, the correct person would have been logged in. So it's bad practice, but also it is telling something about the attitude, if you choose hard-wired password, be idiots. Hard-wired Wi-Fi passwords. That's bad, but also it doesn't matter because they use the violent equal privacy, which is old standard and crackable in two and a half minutes anyway, so why not hard-wire it then? The real question would be, and discussion would be, why Wi-Fi? But that's another story. When we look at the newer generation of machines, an interesting choice of words has been started to be used, because modeming, where modems were called, they were called hard-wired passwords, and they were connected to Wi-Line, were in some parts of the US and earlier everywhere accepted as a way of transferring unofficial results. Now the wording has been changed to wireless modems, and the wireless modems happens to be connecting to Verizon or Sprint network. So it's not anymore that way of modem, as we think. But just to make it more interesting, there are a number of reasons, a couple of recent studies, public documents have been discovered when the vendors have been demonstrating the systems, and the demonstration has failed, and the vendor has been explaining why it failed by telling the voting machine had wrong IP address, and the firewall configuration script had a typo. It doesn't mean it's Internet, but it means that it uses IP addresses and goes wireless over telecoms network, so we don't know what it is. This is not the statement saying they are connected directly to Internet. It's just saying that we are discovering things which are raising interesting questions, and at one point of time, somebody independently should take a look and take a look into the system. Interesting fallback scenarios. Voter authorization smart card, which you are given when you go to the e-poll book, you get the card which allows you to vote once. You go with that to the voting terminal, and you get a card, and you jam it. When it tries to eject, you just keep your finger there, and there were a number of machines where previously when it panicked, it bumped you to vote election supervisor mode with no pin code or nothing. I mean, obviously, it's the supervisor who has to clear the jam, but it is a maybe not well thought through procedure. Another thing which is found in a number of manufacturers, none of these are actually, don't think about that I don't say a name of a vendor because there is only a vendor this applies to. It's more that these are seen out in the wild in more than one place. Secret functional is activated with hidden registry keys. In many cases, the registry is actually made to be remote registry, so inside of the election supervisor's closed network, you can remotely trigger the keys, and now the election supervisor has to make sure that the key is in the right place. So, the key is in the right place. software, whether it's the central tabulator, canvassing software, whatever part of the complex back end system, new features pop up. Again, slightly problematic from an auditor point of view, when you are auditing software and the features you are auditing are, you don't know all the features because they only become active, or in case of one vendor, this is one vendor, where it actually dynamically deloads and reloads a DLL. So, the actual library is not active, it's not even in the memory until the key is there. And a lot of weird misconceptions, like CRC considered as a cryptographically saved algorithm. It doesn't make any sense. But it is a common term, and especially 16, which is not really even giving you a sufficient protection against a non-malicious hardware errors, non-malicious storage errors. These are just examples, and the important thing here is these are not bugs. These are not buffer overflows, these are not mistakes made by the programmer, these are not mistakes, these are features which have been designed and programmed on purpose. We have a lot of bugs. These are different categories because they are not bugs. If we look the same on the side of our hardware and firmware, we see an old technology, which is a RAM, a removable RAM card, which actually plugs in the mapable, executable memory. How you can audit ever a system where removable media becomes part of your execution RAM? You don't. Anyway, that is a design still in use in the United States, and it makes it pretty unauditable. Custom operating system, I mentioned before that there are manufacturers who have in their terminal most of the machines are Windows and some of the new ones are Linux, but there are still manufacturers who have a completely custom made operating system. Or not an operating system at all. We had found a custom operating system where there is a remote control features, which means that inside of the election precinct network, you can remotely control the voting terminal as if there would be a voter. You can click up and down and you can click the candidate name and you say, I vote. And that all can be done remotely over the precinct network. Again, not a design feature which thankfully is being fixed by physical protections. Live open ports which are accessible to voter in booth. The interesting thing about secret ballot is you are in the booth alone and nobody is supposed to look over your shoulder. And if you then have a serial port which is connected to the computer inside of the voting terminal live, and you can plug something in the convenience of your booth and talk with the computer, that's generally speaking a bad idea. But it's all kinds of ports. We are talking serial ports, we are talking USB ports, we are talking about a potentially mountable device, because the smart card standard is another thing which people don't necessarily know. That the smart card standard which is commonly used in your prepaid telephone card, your credit card, that standard actually has an opportunity to have a file system. It's not a very big file system, it's a small file system. Now if the smart card standard reader, as a certain ATM manufacturer did, is a USB device, which is very common in this area, and in Windows you have auto mount and auto execution on. Now you plug your smart card and it takes their autorun.bat and runs it. In this example line, one thing which is recently has become a more known and understood problem in the voter registration system and e-poll books, people are using driving license and driving license barcodes as the way to register yourself in. And I know this is shocking, but I have heard a story that some underage kids might have a false ID. I know it's shocking, but that seems to happen. However, that means that you can have a rogue barcode. And what has been discovered is, first of all, a lot of these barcode readers, they read more standards than the standard you're done. Actually you cannot even limit the number of standards they read. Second thing is their keyboard emulators. Keyboard emulator means that you can emulate operating system commands, like command R to have your command line and type in the command line invisibly. Very convenient. It also works in point of sale systems in supermarkets. The other thing which it can do is if you don't have a sanitized input, which seems to be also thought that obviously we all know what the barcode is, so you don't have to sanitize it, a SQL inject directly from the barcode. It actually, a funny story, this is not an election story, but one guy in Poland figured out that SQL inject is possible in your license plate. So he made the license plate was as wide as a car, which was SQL inject, and deliberately throw through the red light cameras and speed cameras. And he just happened to know what is the name of the number of the table you have to drop. So if you ever read the XKCD, read Little Bobby tables, Little Bobby tables on license plate is a real thing. The same thing in voter registration system, e-poll books, it's something which is right now, thankfully, it's this year a lot of jurisdictions are looking to buy e-poll books. This is now flagged. You have to make certain that you are responsible in a way you treat unreliable input, which is the driving license, because they happen to have false ones. Now, internet voting side. The reason I'm talking about Estonia for a while here is because Estonia election, internet voting system is open source, so I can actually show a couple of code snippets. When we look, the Norwegian internet voting trial was stopped. This is a miniscule comparing the size of that code. We had the Washington D.C. mock election, which was hacked by Alex Halderman and his team. Way much bigger code. When we look, what is Estonia internet voting system, which is considered in a common knowledge as the gold standard, which it truthfully is not, what it looks like. It's 16,000 lines of code, and really we don't have to worry about counting the comment lines separately, because there's no comments. It's actually code which is uncommented completely. Well, not completely, but very little. So it's a 61% Python, 37% C++, and then shell scripts. And various shell scripts. It is really looking like a code trap. There is no docs, no harness, no test modules, no description of the protocol. Obviously you can learn the protocol by reading the source code. The documentation itself is interesting because almost all the comments which are here are actually from a borrowed code. So when they have cut and pasted a part of code from other source, accidentally the comments came in. And when you look at the lifted code, so the borrowed code, there is an undocumented code from GNU, which is an open source license requiring you to commit back the changes, in a sense which Joe would be telling, and Candice Hawke, it's a legal virus in a sense. It shouldn't be undocumented. That's a violation of terms. And the other code is that there's no attribution of source, no version, no authorship, no license, so there's a lot of code which is coming from somewhere, and it's not acknowledged where the code comes from. I would say this is a high school project with a teacher who is very relaxed. But it's not much better than that. It's a nasty thing to say, but it's true. So when they published the code, this is what I want to show. A security feature. And the actual code, when it was published in GitHub, says to do, implement security checks, verifying the correct size of the encrypted vote, returns okay. No test was ever programmed. And this, when this code was published, it had been used in seven real elections. Now, obviously it was slightly embarrassing when this was made publicly known, and so they fixed it. . I'm not making this up. This is actually the next commit to GitHub, they removed the command lines, because there was too much information anyway. So, again, if a voting system is not having the security check, obviously it will not tell you that there's a problem. So everything will go fine. I don't think this is a proper and prudent way of programming an election system, but people make different choices. Again, in the real world, you would never ever believe in any system, even including your point of sale system, a supermarket, or your calculator, that a key feature, year after year after year, is missing and just to do. There's a lot of other features there, and one thing that I want to underline, this is, the example is from Estonia, because I can show the code. This is not unique. When you go and look the other vendors, the vendors in America, vendors who are serving internationally, you see the same thing. You see key features, which somebody was too busy ten years ago to code, and nobody has had time ever since, because it's working, and obviously if you add that feature, you might start to have problems. . . Now, Estonia also is an interesting example, because they do publish videos. This is their government assurance, how the system works. They publish everything from the system being set up to election being run, the daily logins into the system while the election is live. Everything is published as a video by the government. There would be a lot of things I can tell about. We actually went, Alex Haldeman, Margaret McAlpine, the University of Michigan team, three people and two people from Google, we actually flew to Estonia to witness, because you are allowed to, in person, also to see the operations on a daily basis. First of all, because Estonia's system is widely open source, in Ann Arbor, in University of Michigan, Alex Haldeman and his team built a own Estonia. If you go to Estonia voting.org, you actually find a virtual images of the Estonia government systems, and you can be your own Estonia. Which means that you can take the binary code given to you as the voting software, and you can run it against a virtual Estonia, which is your Estonia, hence you can try your things safely, and you are not playing against the real system. The videos were actually very nice, because when Ann Arbor was building the copy of Estonia, they didn't know what everything means in Estonian, and as a result, every name of computer, every sticker was copied, and when Estonians were saying, well, how you knew this? Well, we don't know what that word means, but we put the sticker, because you have the sticker. And so that's how it was copied. Anyway, this evidence is interesting, because this is from the video, where they sign, cryptographically sign the client software the citizens will be using for casting their vote. So there's this video, the video goes on, and eventually there comes a signature and hash, and now as a voter, you can verify your binary to be the real binary, because you can compare it to the video, and they made a big thing that this computer, where they are doing the sign, this computer has never been connected to internet. That is the statement they make. So let's take a closer look. So we have a micro-torrent, we have the tournament, and poker's game. Online poker. So I think the statement, this has never been connected to internet. Maybe they are interested, they want to install all this software which is not going to use and they wonder why it doesn't use it. Or the statement might be false. I don't know which one. Anyway, operational security. I'm using Estonia as a, I could use many other jurisdictions in the US. I'm just showing that a lot of times these statements, we have as humans, we are trust-based animals. If somebody tells us sky is blue, we probably take it as a face value, we don't check. If somebody says in a video, in official evidence provided by government, this computer has never been connected to internet, very few people actually go and start checking. But the evidence is there that something is well, maybe that's offline poker stars online. Who knows? . I was talking before about, we've come back to US, about these small companies which are helping local governments to run elections. This is an actual website. This screenshot was taken, I think, three weeks ago. It's not from yesterday. I don't have any belief that election source has changed it. This is an actual company selling election services. First of all, this is a blow-up from here. Our election coding services saves you time and effort. If you would be running a national security level service, would you buy your coding services from a website looking like that? The answer is yes. They do buy it. That's why they advertise it. This is a not-embarrassing example, and I don't want to embarrass election source. They are not worse or particularly bad, but the whole concept is elections are, well, actually that's a different picture. There's also, say, Manning & President maintenance services. But these kind of companies are selling their services, and they are not certified. If this is a Joe's election services, who will ever check who Joe is, who owns Joe's services, how Joe is employing his people, how Joe is vetting his employees? What are the different stakeholders in Joe's company? What? Oh, microphone. Anyway, so this was just an example of what kind of companies and what kind of marketing methods actually appeal a real election official, local election officials, to use their services. On the same website, when you go to the production site, they have the new tag here, which is everything you need to secure your election, like seals. So they sell locks, they sell seals, all kind of things. And the one in the corner, this looks very interesting. It's red, and it's the seal of the state of Michigan. It is the great seal of the state of Michigan. This is actually the paper seal they use in elections. This is my hand. I used my own credit card, my personal credit card, and I ordered them. They were four and a half dollars each. So you don't need to forge a seal, you can just buy them. Election officials always make the correct statement. A physical protection is protecting the machine. When you talk about computer systems, the rule of the thumb is if you can physically access it, the game is over. So you have to have physical access. A lot of security studies have greatly improved the amount of physical security around the election machines. Great job done, more job needs to be done. However, that's not a problem if you can actually buy the original seals, and you don't need to even forge them. All right. Again, this is a machine which is in the hacking village. It's the Ivatronic. It has the security feature. You have a special cartridge. The machine only activates when the cartridge is present. The cartridge has a secret ingredient which no human can get access. It's called magnet. So when you have a magnet, the magnet actually activates the system, and the device is using infrared to communicate the ballot style, and also to do the tally, because it's also vote aggregation. So instead of using that device, you can use pound pilot. Again, when you look, it's actually very interesting. A lot of people who are not election security experts, they think when a specialized device they have never seen before, which has a weird shape slot which you put, they think it's secure because it's something they have never seen. It's not common. And because it's a magic, there's no connectors. It's smooth as ice. There's nothing metallic because people have a false sense of security it's safe. The reality is a magnet and mobile phone or pound pilot, that's how it works. That's how you talk with. Again, never make an assumption, when you see a machine, never make an assumption that you know how it works or the people who tell how it works would know how it works. And that's why we have the village. This is just a very quickly top line, one line per huge topic. Every single line would have been an hour presentation of its own. But to give you an idea why it's important that we take a look, fresh look, we analyze, we get the facts straight, and only people who are educated and know the facts can make an intellectual decisions and right choices. One more thing about seals. Yet another seal example. This is a D-Bold Aquavote precinct based scanner, tamper evidence seal, protecting the memory card so that nobody can tamper with that. But if you take a look inside, you see, hey, there's two screws. We have been recommending to put some seals to those screws because they didn't. And if you take them out, you take that out, now you can get to the memory card without breaking the seal. I have a number of pictures from Estonia where they have sealed the racks of the servers, and the seal didn't go from the door to the rim. So you can just open the door. The seal is not broken. Anyway, that's a brief thing about keeping your mind open when you analyze the systems. When something is presented to you and you want to make certain that you understand if it's secure or not, you don't need to have any special skills. You need to take a look and use common sense. If somebody says this is a tamper evidence seal, see where the seal is connected. And think like a criminal. Think about this is, I think, one of the most important things I have been saying in generally about hacking and security. Hacking is not intellectual challenge. Movies, and everybody tells hacker men are wise and cunning and smart. No. That's not the thing. Hackers want to steal your stuff. There's no style points. They look if there's a door. Oh, you have a door. That's great. Do you have a lock? Oh, you have a lock. Great. Let me go around the building to see if you have a back door. And do you have locks in the windows? And it's all about how you achieve your goal as a criminal. Not if you have style points. Because the best crime is a crime which was never discovered. So you don't even get to brag about it. Thank you. Questions? Are the, I'm talking loud so they can hear. Are the incidents of non-bugs in later generation machines becoming less prevalent? Most of the software hasn't been updated since 2007. So there is no new version. Okay. So the question is, is the newer version of the software, is the bug becoming less prevalent? And the answer was that when you go across the U.S. and you see the systems which are used, a lot of times there has been no software upgrade since this discovery. So it's still the same. About the newer systems, most of the newer systems have never been independently verified. So we don't have data one way or another. Is there anyone regulating these clouds? So the answer to that is a legal question. And that was Joe Hall and Candice who should be answering that. The, my answer is there's volunteer standards. And since they are volunteer, they are volunteer. I would also point out that the FEC, David when was FEC regulation? 2002. FEC. 2002 and then the EACVSS 2005, they both have a ban to use assembler language. And those machines had been certified and you see the assembler language there. So the EATs, the independent task authorities have not been, maybe they haven't considered that to be that important, but the systems which have been certified voluntarily but certified still don't necessarily meet the standard set. Anything else? Okay. I'm out of time. Oh. I think I actually went to YouTube live. I think I actually went to YouTube live. I thought one person there was already posting it as a live stream, so it should be there. Okay. Thank you.