00:00:00.042-->00:00:05.292 >>First off as you know I am a noob this is my first time speaking and also my first time 00:00:05.292-->00:00:10.292 to Def Con. [whoops from audience] So I wanna say thank you very much to this community 00:00:12.667-->00:00:16.958 for inviting me to share a little time with you and to explore what I think is a really 00:00:16.958-->00:00:22.750 fascinating and timely topic. As you can tell from the programs and I hope you know where you 00:00:22.750-->00:00:27.750 are, uh my name is Sean Kanuck for 5 years from 2011 to 2016 I was the United States First 00:00:31.667-->00:00:38.000 National Intelligence Officer for Cyber Issues. I was charged with leading strategic analysis 00:00:38.000-->00:00:42.833 including attribution assessments for a lot of the cyber breaches that happened 00:00:42.833-->00:00:48.667 during that period. As you can probably guess I'm quite happy to not be doing that anymore and 00:00:48.667-->00:00:53.667 I have great empathy and respect for my successor. [laughs from audience] [applause] But if 00:01:00.708-->00:01:07.000 you're familiar with the US government's attribution of the Sony Pictures Entertainment hack 00:01:07.000-->00:01:12.667 or Las Vegas Sands hack and a lot of those events that happened in years past, that 00:01:12.667-->00:01:17.458 would have been what my office was leading the intelligence community on. So I obviously 00:01:17.458-->00:01:21.875 come at this from a strategic perspective I imagine most of you or many of you in this 00:01:21.875-->00:01:27.875 audience are much more technically skilled than I am, much better at tactically 00:01:27.875-->00:01:33.250 hacking things. I will tell you my role as an international attorney and as a professional 00:01:33.250-->00:01:39.042 intelligence analyst was to look at where the greatest vulnerabilities are and where 00:01:39.042-->00:01:44.875 the greatest impact could be acheived through intelligence and military operations. 00:01:44.875-->00:01:50.375 Sometimes helping with strategic targeting in my nation's national security defense and 00:01:50.375-->00:01:57.000 often looking to prevent those kind of things happening to my nation, our allied nations and 00:01:57.000-->00:02:01.458 quite frankly, almost any nation in the world's critical infrastructure from being harmed 00:02:01.458-->00:02:07.333 and brought down. That's a little bit about me only to tell you where I'm coming from. When 00:02:07.333-->00:02:13.250 I think about systems I think a little the way you all do. I look at them as the mischief 00:02:13.250-->00:02:18.958 maker, of how could I harm this? Not as an engineer who builds them 'cause I haven't built 00:02:18.958-->00:02:24.083 systems and I've never administered a system. But what I have done is look at Al Qaeda 00:02:24.083-->00:02:30.250 financial networks, foreign information technology systems and say "Where is the weak 00:02:30.250-->00:02:35.958 point? How could this be exploited?" And then make a real effort to make sure that US 00:02:35.958-->00:02:42.250 systems and hardware and software that the whole world relies upon is protecting itself 00:02:42.250-->00:02:47.250 for our common interest. So let me start with 5 questions. This isn't the real Socratic part 00:02:49.458-->00:02:55.833 this is just a little in general. Raise your hand if you think there's anything such as a 00:02:55.833-->00:03:00.833 secure IT system. [laughter from crowd] Raise your hand if you think there's anything such as 00:03:03.417-->00:03:08.417 an air gapped network? Raise your hand if you believe James Clapper and Jeh Johnson that 00:03:12.958-->00:03:17.958 Russian government entities were behind the DNC hack in f- 2016? Raise your hand if you believe 00:03:22.833-->00:03:29.500 Vladimir Putin that the Russian government had nothing to do with that. [mumbling and 00:03:29.500-->00:03:33.417 laughter audience] Ok now I assume we have a very geographically disparate 00:03:33.417-->00:03:38.417 audience here so raise your hand if you believe whatever country you come from hasn't done 00:03:40.500-->00:03:45.500 similar things or wouldn't do similar things if they could. [audience laughs] Alright I like 00:03:47.750-->00:03:52.750 the laughter 'cause I think it means we're on the same page. [laughter] I try to think like 00:03:56.208-->00:04:02.042 an offender in order to improve defense. I'm gonna start here with a few minutes of where I 00:04:02.042-->00:04:07.792 come on a strategic trend analysis and then I have 16 questions I wanna do an exchange 00:04:07.792-->00:04:12.792 with you all on about hacking democracy. So just so you know where I'm coming from. I don't 00:04:15.417-->00:04:19.875 think there's any such thing as a secure system. Particularly when you allow for not only 00:04:19.875-->00:04:26.042 remote hacking but supply chain operations where either the vendor willingly sells you 00:04:26.042-->00:04:30.625 something that has added features, or where a 3rd party could have manipulated the 00:04:30.625-->00:04:35.625 manufacturing or transportation process to the end user. I think insider threats remain one of 00:04:38.083-->00:04:42.750 the greatest concerns for both government and, well government, military, and corporate 00:04:42.750-->00:04:47.917 environments. And that includes both stupid and malicious people, those are all insider 00:04:47.917-->00:04:52.917 threats. And I think when you have the ability to use dirty tricks the way intelligent 00:04:55.583-->00:05:00.583 services do, the way criminal syndicates do, and the way terrorists do, you really have 00:05:02.833-->00:05:09.375 to expand your thinking. Give you one quick example years ago I gave a NATO secret level 00:05:09.375-->00:05:15.917 speech in Europe and at the end of it a Hungarian secret service officer asked me "Mr. Kanuck 00:05:15.917-->00:05:19.500 what's the highest level of encryption that the national security agency can decrypt 00:05:19.500-->00:05:25.958 today?" [crowd laughs] So at that time I didn't actually know the answer so I told him "Sir I 00:05:25.958-->00:05:32.375 respect your question but I don't know the answer. Secondly, if I did I probably wouldn't be 00:05:32.375-->00:05:38.083 able to tell you at a NATO secret level. And third, how good is the lock on your 00:05:38.083-->00:05:44.958 upstairs back window?" Do you think we're gonna use a cray server farm in Maryland to get 00:05:44.958-->00:05:49.667 one person's password? If I'm a criminal I'm gonna beat you up in the alley and get it or 00:05:49.667-->00:05:55.958 kidnap a family member. If I'm an intel service, I'm gonna be providing you the cell tower 00:05:55.958-->00:06:02.792 you're using or the other links you're using, or I'm gonna be filming you doing it. Or finding 00:06:02.792-->00:06:07.083 out from one of your peers or from your assistant. So expand your thinking that's why I'm 00:06:07.083-->00:06:11.792 really excited to talk to this audience about democracy, because I've been doing a lot of 00:06:11.792-->00:06:17.167 research on this topic in the last year ever since CNN, ABC, NBC all started calling me in 00:06:17.167-->00:06:20.542 the fall. I was out of government so when the government says "We can't 00:06:20.542-->00:06:25.167 discuss that ongoing investigation" or "That's classified we can't discuss that 00:06:25.167-->00:06:29.083 with you now." The first thing the media does is call the people who used to be in that 00:06:29.083-->00:06:34.250 job, and hoping that they can now talk. But that got me thinking. I put on my hat and I 00:06:34.250-->00:06:39.250 said "If I were actually trying to undermine democracy how would I think about going about it?" 00:06:42.875-->00:06:49.583 Just like on September 13, 2001 a friend of mine at CIA and I wrote an unrequested paper 00:06:49.583-->00:06:55.458 called "What we would have done with 4 airplanes." All due respect to the 3-->000 lives we 00:06:55.458-->00:06:59.875 lost that day, and the property damage, but if you understand the infrastructures of the 00:06:59.875-->00:07:05.208 United States' financial system and government system 4 explosions of that size could 00:07:05.208-->00:07:10.208 have caused much more death, and much more destabilizing disruption. So what are the 00:07:12.292-->00:07:16.125 other assesment things I believe? I believe we have convergence and devices, 00:07:16.125-->00:07:23.000 protocols and infrastructure networks such that we're having many more single points of 00:07:23.000-->00:07:28.583 failure. I participated in a Lloyd's of London insurance report that was just aired about 00:07:28.583-->00:07:35.208 a week ago. And one of the scenarios was where a common hypervisor software upon which 00:07:35.208-->00:07:40.750 multiple cloud service providers rely was compromised. It was a hypothetical scenario but you 00:07:40.750-->00:07:47.000 think you have uh resiliency in your infrastructures or in your products and in many cases you 00:07:47.000-->00:07:51.375 don't because of the efficiencies of scale we're looking for. This is most easily 00:07:51.375-->00:07:56.000 demonstrated in the fact that the government and military have very few dedicated networks 00:07:56.000-->00:08:00.792 anymore. And the ones they do have have l- very limited bandwidth compared to the 00:08:00.792-->00:08:05.583 civilian fiber optic infrastructure and undersea cables. So I think about where 00:08:05.583-->00:08:11.208 the strategic nodes and the tipping points. I think the threat is accelerating, both in 00:08:11.208-->00:08:16.917 offense's capability over defense. I think defense is improving but I actually think 00:08:16.917-->00:08:22.417 the gap is still increasing because offense is increasing at even a faster rate. Part of that 00:08:22.417-->00:08:27.500 is due to the exponential increase in the threat surface area with the internet of things 00:08:27.500-->00:08:33.083 and a lot of other aspects. If you read the world wide threat assessments to congress each 00:08:33.083-->00:08:37.042 year that the director of national intelligence gives, which I highly recommend you do, 00:08:37.042-->00:08:43.625 they're awesome open source uh and every well cited occasionally resources. Uh I 00:08:43.625-->00:08:48.417 believe that integrity of information is going to supersede confidentiality and 00:08:48.417-->00:08:54.708 availability as the most significant vector and format of attack of concern. And most 00:08:54.708-->00:09:00.833 importantly and most relevant to this discussion is I think the fundamental distinction between 00:09:00.833-->00:09:06.417 how the United States, Western Europe and some other democracies think about cyber 00:09:06.417-->00:09:11.625 security is fundamentally different the way other nations do. We think about it as 00:09:11.625-->00:09:17.583 critical infrastructure protection protecting the systems and the pipes. Russia, 00:09:17.583-->00:09:22.375 China, some of the Gulf countries and others think about the content, the information uh- 00:09:22.375-->00:09:27.375 flow through it. We talk about cyber security, they talk about information warfare, influence 00:09:30.042-->00:09:35.375 operations, psychological operations and perception management. And that's what I 00:09:35.375-->00:09:40.375 wanna focus on here today. I've sworn an oath twice to protect the constitution of the United 00:09:42.958-->00:09:46.500 States and the form of government it represents from foreign and domestic adversaries 00:09:52.750-->00:09:56.917 [applause] OK that's actually something I wasn't expecting so I'm gonna say thank you very 00:09:56.917-->00:10:01.500 much because there are other people on stage at Def Con occasionally, there are people 00:10:01.500-->00:10:06.292 out in war zones, who dedicate and risk their lives. I mean I've done some fun stuff but I 00:10:06.292-->00:10:10.583 know the people the men and women who actually put their lives on the line to make sure 00:10:10.583-->00:10:14.833 we can have something like Def Con. Ok [audience applauds] I'm the guy who stands up and says 00:10:14.833-->00:10:19.833 "I love the fact that code pink was able to yell at a director of central intelligence nominee 00:10:23.333-->00:10:28.333 in the US capitol." I know of no other country where that hearing would have been occurred 00:10:31.250-->00:10:36.417 televised, open to the public and the people who swore at him to his face would have gone home 00:10:36.417-->00:10:41.375 to sleep in their own bed. So I've spent 16 years of my life to protect the right to do this 00:10:41.375-->00:10:45.833 and I thank you for recognizing that, for all the people who have also done that. So thank 00:10:45.833-->00:10:50.833 you and kudos to the rest of them. [applause] Alright after that heartfelt comment let's g- 00:10:56.792-->00:11:01.792 have some fun. When I think strategically about how to mess up a democracy I start with sort 00:11:05.958-->00:11:10.958 of a 4 grid. I have direct intervention and indirect- indirect intervention and overt 00:11:13.042-->00:11:18.042 and covert. What's an example of those? Direct intervention in an overt way: not letting people 00:11:23.917-->00:11:28.458 get to the polls. We saw this during the Civil Rights movement in the US where folks weren't 00:11:28.458-->00:11:32.500 actually allowed to vote even though they were entitled to. We've seen this in other 00:11:32.500-->00:11:37.500 countries like the Canadian 2011 election which ended up being a s- a very significant court case 00:11:40.042-->00:11:45.042 in their system where one party got the voter support list from the other party and very shortly 00:11:47.292-->00:11:52.500 before the election sent them all robocalls telling them the polling station had been moved. 00:11:52.500-->00:11:55.417 Kind of fun I tip my hat to that, that's good, but that's not what you want in a 00:11:55.417-->00:12:02.000 democracy. Direct intervention that's covert, easiest example of this is the ruling party 00:12:02.000-->00:12:06.833 stuffing ballot boxes or changing results. You can find great historical examples of 00:12:06.833-->00:12:11.833 that going all the way back to Napoleon Bonaparte. Indirect influence but overt. What does 00:12:15.333-->00:12:21.458 that look like? I think of President Obama's speech in London against Brexit. Public 00:12:21.458-->00:12:26.458 speech, invited, but he's not a voter in the UK. Not even a constituent or a candidate. Yet 00:12:28.500-->00:12:35.208 his foreign voice was invited to influence that election. And of course the obvious one on the 00:12:35.208-->00:12:40.750 indirect influence coovert side is something like what we saw in the US and French presidential 00:12:40.750-->00:12:45.750 elections recently. So how 'bout the systems themselves? When I think of voting systems and 00:12:48.375-->00:12:54.167 information resources to me they take on public and private aspects and in a s- open society 00:12:54.167-->00:12:59.333 like ours we entrust a lot of our democratic institutions, and that's small D, nothing I'm 00:12:59.333-->00:13:05.083 gonna say is gonna be party specific or lambasting either party here. Small d democratic 00:13:05.083-->00:13:11.958 institutions so the governments, the state governments, operate the polling stations. But who 00:13:11.958-->00:13:17.833 creates the voting machine software and hardware? Is that the federal government? Is that 00:13:17.833-->00:13:21.583 the state government? No. You have a wonderful voting village here where you're all getting a 00:13:21.583-->00:13:26.625 chance to hack those machines, right? I think that's awesome. [audience member whoops] On the 00:13:26.625-->00:13:32.208 information resources side we can see a lot of governments who have state news channels, TV and 00:13:32.208-->00:13:37.208 radio. You also have a lot of private resources. Historically that's been newspapers and radio 00:13:40.125-->00:13:45.708 stations, now you have social media and all kinds of other platforms. Gets very tricky when 00:13:45.708-->00:13:49.042 you start asking "How do you wanna regulate or protect things?" As we're gonna see in a 00:13:49.042-->00:13:54.042 few minutes. Lastly I think about intentional messaging and unwitting exploitation that's 00:13:57.208-->00:14:02.208 either meant to inform or deceive. And this is nothing new in the history of politics. You 00:14:04.583-->00:14:09.833 can go back to Thucydides's Peloponnesian war and can see the Athenians trying to convince 00:14:09.833-->00:14:14.833 the Melians to give up their city without a fight. That's on the direct lobbying. On the 00:14:18.125-->00:14:22.292 misleading and misperceiving if you understand the history behind the nomenclature of 00:14:22.292-->00:14:28.500 Iceland and Greenland or Bolshevik and Menshevik. The Bolsheviks were not the majority 00:14:28.500-->00:14:33.958 faction but they adopted that name. And if you think this only happens in foreign countries, go 00:14:33.958-->00:14:38.792 read the history of the words "federalist" and "antifederalist". Unwitting 00:14:38.792-->00:14:44.583 exploitation in more recent times. Think about Twitter that had been used as a platform for 00:14:44.583-->00:14:50.417 terrorist incitement speech. That company took active measures to start preventing 00:14:50.417-->00:14:56.667 some of those us incitements to violence on it's own platform. They weren't involved in it but 00:14:56.667-->00:15:02.708 it was uh- using their platform. And then of course we have issues about fake news and a 00:15:02.708-->00:15:07.708 good example in 2011, 13 was the Syrian Electronic Army's hack of the Associated Press uh Twitter 00:15:13.208-->00:15:18.208 account to put out a fake statement that there had been a bomb at the white house. And I 00:15:18.208-->00:15:22.708 hope some of you are familiar with this. For a lot of people the story ended with the fact 00:15:22.708-->00:15:28.333 that the I think it was about 140 point drop in the Dow Jones for 5 minutes. Most people were 00:15:28.333-->00:15:33.208 comforted to hear that that value came back into the financial marketplace. People 00:15:33.208-->00:15:37.125 like me who are worried about continuity of government and public confidence in critical 00:15:37.125-->00:15:42.458 infrastructures asked "Did the money go back into the same accounts?" And the answer is 00:15:42.458-->00:15:47.542 absolutely not. That was a multi-billion dollar redistributive economic event 00:15:47.542-->00:15:53.917 because someone put out a false tweet. Can any of those stock trades be reversed? Are you 00:15:53.917-->00:15:59.083 allowed to make a dumb stock trade based on inaccurate information? Are you allowed to 00:15:59.083-->00:16:04.292 pick your own information sources? Absolutely. The crime there was the crime of hacking 00:16:04.292-->00:16:09.292 the account. The stock trades were legal. So with that brief overview of how I view this 00:16:12.708-->00:16:19.708 space and with underscoring the fact that for me, our very competent competitors in 00:16:19.708-->00:16:23.375 cyberspace and I'm gonna speak as an American 'cause I am one. But I think a lot of the 00:16:23.375-->00:16:27.875 statements I'm gonna make are fairly universal. And yes I'm also not ignorant and naive to 00:16:27.875-->00:16:32.542 the fact that the majority of spam in the world comes through US gateways. And that there's 00:16:32.542-->00:16:38.125 plenty of criminal activities stemming from the United States. At the same time, my career has 00:16:38.125-->00:16:43.542 been focused on foreign and external threats so that's my point of relevance. Er. I think 00:16:43.542-->00:16:47.875 some of our very sophisticated adversaries along with some of our partners and allies are 00:16:47.875-->00:16:52.750 involved in a daily process of intelligence collection against each other. That's something 00:16:52.750-->00:16:57.208 that's not even prescribed under international law 'cause it's accepted to be the second oldest 00:16:57.208-->00:17:02.333 profession and something you'll never get rid of. The question becomes when is it too much and 00:17:02.333-->00:17:09.333 when does it cross lines? And that's what I wanna focus on here. How do we get that first 00:17:09.333-->00:17:14.333 slide up? What? >>[inaudible off mic] >>Oh play it out ok. So the first one is and I apologise my 00:17:24.125-->00:17:29.625 slide deck is literally just 16 questions in really bold print. So to start our Socratic 00:17:29.625-->00:17:36.583 dialogue. >>[inaudible off mic] >>Yeah. How do you know your vote was recorded correctly? And 00:17:36.583-->00:17:42.542 this is where we can start with. >>Yeah that's fine >>So. Someone from the audience, please offer 00:17:42.542-->00:17:46.292 a thought. How would you know if your vote was recorded correctly. I vote in Alexandria, 00:17:46.292-->00:17:51.625 Virginia. I filled in a paper ballot, I put it into a machine that electronically recorded it 00:17:51.625-->00:17:55.500 and it said "You vote has been recorded!" And I got a little green icon. It didn't even tell 00:17:55.500-->00:18:00.250 me who it was recorded for. Not that I would believe the name that came up on a monitor. How 00:18:00.250-->00:18:06.250 would I know my vote is recorded properly? >>Blockchain! >>Blockchain! OK. [sporadic 00:18:06.250-->00:18:10.458 applause] How many electoral systems are currently using blockchain for that? >>[unclear 00:18:10.458-->00:18:15.292 shouting from audience] None! >>You all know that Estonia has had 3 presidential elections 00:18:15.292-->00:18:20.292 completely online already right? How far before the US would have a complete online election for 00:18:22.667-->00:18:27.667 president. >>[crowd tittering] Never! >>[laughing] Second question. What's more important 00:18:34.042-->00:18:39.042 to you: secrecy or verification? >>[crowd murmuring] Verification >>So I'm clearly talking and I 00:18:43.375-->00:18:48.458 did hear a couple secrecies but most is verification. I could go to most people's social media 00:18:48.458-->00:18:55.000 websites and probably guess with 99 percent who they're gonna vote for. If you go back decades 00:18:55.000-->00:19:01.083 or centuries the idea of a secret ballot was a critical aspect of democracy in its 00:19:01.083-->00:19:07.542 evolution. Fear of the governmental authorities, fear of your neighbors, your spouses 00:19:07.542-->00:19:11.958 I mean this is why adults over 18 go into the voting booth alone in the United States. 00:19:11.958-->00:19:16.000 There are countries where they don't. I've gone in and watched someone vote in a foreign 00:19:16.000-->00:19:21.000 country in a way that I never could in the United States. And when I think about this in the 00:19:23.375-->00:19:28.208 sense of mutually exclusive options and blockchain may be a solution to give us our cake and 00:19:28.208-->00:19:34.500 eat it too, but if you wanted to p- be able to really know how your vote was recorded, should 00:19:34.500-->00:19:40.417 you be allowed to opt in to a system where you can check the voter database and how your vote 00:19:40.417-->00:19:47.042 was recorded? How many of you would prefer that opt-in option to verify your own vote even if 00:19:47.042-->00:19:52.292 it meant giving up a secret ballot? >>Hell no. >>I hear hell no. And I don't know if you can 00:19:52.292-->00:19:55.708 all see, I've got lights shining in my face, but I'd say it was about a third of the hands that 00:19:55.708-->00:20:00.833 went up. >>[Audience murmuring responses] I'd love to sell my vote so yeah, that's great. 00:20:00.833-->00:20:05.333 >>[laughs][audience laughs] That's good! What secondary market do we then t- create 00:20:05.333-->00:20:11.208 there? No those are really important things to consider. So what's the solution that gives 00:20:11.208-->00:20:17.042 us our cake and eat it too? Blackchain? >>[unclear audience response] >>Gimme another one. 00:20:17.042-->00:20:21.792 >>[unclear audience response] >>[laughs] The answer was there are more options and it would 00:20:21.792-->00:20:26.792 take too long to explain. That's fine. No I respect that, uh. My next question to you all: How 00:20:30.083-->00:20:35.083 much error is too much for legitimacy? I personally don't think the exact vote count I see 00:20:38.542-->00:20:43.542 rec- recorded and publicized that goes down to the singles digit in a multimillion dollar 00:20:43.542-->00:20:48.875 number is perfectly accurate. If you believe that vote counting is a science I'll simply remind 00:20:48.875-->00:20:55.042 you of the 2011 election and hanging chads. Where we had Cuban and Chinese governments 00:20:55.042-->00:21:00.458 offer to send election monitors to Florida to help us with our democracy. True story, true 00:21:00.458-->00:21:05.458 story! I I think that's great on their part. If you talk to an economist he or she would tell 00:21:09.500-->00:21:14.500 you you don't want to live in a society with zero crime. Actually getting that graph all 00:21:16.667-->00:21:21.833 the way to zero, to the limit, the marginal value of return and the expense it costs to get 00:21:21.833-->00:21:26.667 there is too much. All of us would need 2 policemen walking around us all the time. We've 00:21:26.667-->00:21:31.083 seen some models where it wasn't 2 to 1, it was probably 1 to 5, that was known as the Soviet 00:21:31.083-->00:21:36.083 Union and most of us don't think that was a great way to go through life. So what percentage 00:21:38.667-->00:21:43.667 of error, statistical error in a US presidential election, do you think is acceptable that you 00:21:47.417-->00:21:53.000 wouldn't question the legitimacy. Let's hear some numbers. >>One tenth of one 00:21:53.000-->00:21:59.833 percent! >>One tenth of one percent. >>Less than the margin. >>Less than the margin. 00:21:59.833-->00:22:04.375 >>[unclear audience response] >>Wh- [laughs] someone doesn't like first past the post voting. 00:22:04.375-->00:22:09.375 Okay. So now you're familiar with 2 standard deviations, the standard that's used in science 00:22:12.000-->00:22:18.292 for certainty levels and for the FDA to approve drugs? Well some of you probably take drugs with 00:22:18.292-->00:22:23.750 no FDA approval [laughs] [audience laughs] but those of you who stick to lawful 00:22:23.750-->00:22:29.792 prescriptions. Uh that's a 5 percent margin of error in those studies. You're putting toxins 00:22:29.792-->00:22:35.417 into your body with a 5 percent uncertainty and you expect one tenth of one percent, at least 00:22:35.417-->00:22:40.417 someone said that. We have a very low tolerance. Do we want to get to zero? And is it cost 00:22:42.958-->00:22:49.625 effective and feasible to get to zero? Who wants to get to completely verified zero 00:22:49.625-->00:22:54.625 statistical error in voting? >>If we don't know the cost, we can't answer that. >>We don't 00:22:56.875-->00:23:01.375 know the cost, so we can't answer the question. That's a great answer. You know if you 00:23:01.375-->00:23:05.875 spend time in government you realize a lot of good academic and business questions just 00:23:05.875-->00:23:11.250 become crazy complicated when you're in the fiduciary duty of spending taxpayer money on 00:23:11.250-->00:23:15.167 questions of uncertainty. So I really like that answer and unfortunately I don't have a 00:23:15.167-->00:23:18.792 great response to you on what this system would cost. But I hope it's something we're all 00:23:18.792-->00:23:22.458 thinking about I hope it's something the election commissions are thinking about 00:23:22.458-->00:23:26.250 and I hope it's things that groups like Verified Voting and others are thinking about to get 00:23:26.250-->00:23:31.250 us to that least level of error. So that's the election day and your actual vote tabulation. 00:23:33.250-->00:23:40.167 What does it take before you get to the voting booth? The next question I have for you is: Were 00:23:40.167-->00:23:45.167 you on the right list? Are you a registered voter? Who holds those databases? Can they be 00:23:47.833-->00:23:52.833 hacked and manipulated? What do you think? Voter registration lists, secure? >>No >>How much 00:23:59.458-->00:24:03.417 error do you want to accept in your voter registration lists? You all seem like it's they're 00:24:03.417-->00:24:09.083 compromised or they're compromisable, and then you laughed about it. What if I told 00:24:09.083-->00:24:13.792 you a specific ethnic group was the one getting excluded from that voter registration list by 00:24:13.792-->00:24:18.333 a hate group who was able to hack it? I'm not the least bit comfortable about that and I'm 00:24:18.333-->00:24:23.333 not gonna laugh about that. So who's helping secure those? Do we want the federal government 00:24:26.667-->00:24:31.833 involved? Or do we want that only at the state level? That public policy discussion is 00:24:31.833-->00:24:36.833 going on right now. >>[unclear audience response] >>More inclusive rather than less. I 00:24:39.042-->00:24:44.042 like that answer uh personally. >>[unclear audience response] >>Register everyone by default. 00:24:48.000-->00:24:52.542 Why bother with a list? >>[unclear audience response] >>What? [audience laughs] I 00:24:52.542-->00:24:58.292 missed that. >>[unclear audience response] >>Oh what do you mean by everyone? Well based on the 00:24:58.292-->00:25:02.833 constitution that would be people of all races and genders and race over the age of 18 00:25:02.833-->00:25:07.708 that's where we've currently set the bar. >>[unclear audience response] >>Oh there are- true 00:25:07.708-->00:25:12.833 there, so we heard "not felons" which means there are some people who temporarily or 00:25:12.833-->00:25:17.833 indefinitely lose their voting rights. >>[unclear audience response] >>Is it not my 00:25:19.917-->00:25:24.500 constitutional right to go play baseball on voting day? Isn't that what freedom's about? 00:25:27.292-->00:25:32.292 [hollers from crowd] 2 great successes of the Soviet Union: near perfect literacy much 00:25:35.000-->00:25:41.833 better than the current US rate, and almost perfect voter turnout. [crowd laughs] The 00:25:41.833-->00:25:46.583 third one is the beauty of their subway stations in Moscow, beautiful crystal chandeliers 00:25:46.583-->00:25:53.375 and mosaics on the walls. Ok. Were you on the list? How about the next question, were you in 00:25:53.375-->00:25:58.542 the right place on voting day? I already gave you the example of the deception in the Canadian 00:25:58.542-->00:26:04.292 election, what other things can happen? And when you think about being on the list and being in 00:26:04.292-->00:26:11.000 the right place to actually exercise your suffrage, do the policies in place 00:26:11.000-->00:26:17.375 disproportionately impact certain groups. >>[audience responding] Yes. >>I hear 00:26:17.375-->00:26:23.250 absentee voting and I hear a lot of yeses. And one of my great concerns is, so I'm an analyst I 00:26:23.250-->00:26:27.583 don't think anything is a hundred percent certain which means I believe there will be 00:26:27.583-->00:26:32.292 some statistical error in any system. I think if you get it to that one tenth of one percent or 00:26:32.292-->00:26:36.917 something incredibly low, that then we're doing well and you're not gonna have a crisis in 00:26:36.917-->00:26:41.917 legitimacy. At the same time I want that to be random error. I don't want that to be error that 00:26:45.500-->00:26:50.500 disproportionately affects people in a specific district, because they are in a low income 00:26:50.500-->00:26:55.500 area and their county can't buy the best machines. [audience members clap] I don't want it to 00:26:57.500-->00:27:02.625 disadvantage in a visual audit process, I don't want it to disadvantage visually impaired 00:27:02.625-->00:27:07.083 or people with other disabilities. So while I'm willing to stomach a little bit 00:27:07.083-->00:27:12.083 of statistical error I want it to be purely random and not harming any specific group in my 00:27:15.375-->00:27:22.208 electorate. To me that's a really important criteria. >>Do you know anything about uh, 00:27:22.208-->00:27:28.083 India's voting system? >>Do I know anything about India's voting system? I know India is 00:27:28.083-->00:27:32.833 the largest democracy and I know India has the world's largest biometric database, which so the 00:27:32.833-->00:27:37.083 government is using for things. But I get the sense you have a specific question or comment so 00:27:37.083-->00:27:42.542 please say it really loud and if you don't yell loud enough I'll repeat it. >>So uh >>Turn, turn 00:27:42.542-->00:27:48.958 around and yell. Come here. Come here. Come here. Get up here, expert. [audience laughs] Tell 00:27:48.958-->00:27:55.083 us all about it. >>Now, now I've gotta try to throw out my youtube knowledge. Uh so in 00:27:55.083-->00:28:01.542 India they have it setup where there has to be a voting poll I think it's like 2 miles within 00:28:01.542-->00:28:06.375 every person and uh there's actually a Buddhist temple that is out in the middle of nowhere 00:28:06.375-->00:28:11.750 where there's only one guy there and they have to go there every year and set up a booth just for 00:28:11.750-->00:28:16.542 him. >>Love it >>So... >>[from crowd] How much does that cost? >>How much- how much does it 00:28:16.542-->00:28:21.542 cost? But how much does freedom cost? Like how much, right? [crowd cheers] >>Alright so 00:28:27.583-->00:28:32.167 let's stay on India w- for one second because I actually do do a lot of work on cyber issues 00:28:32.167-->00:28:36.625 with India just not specifically their voting system in fact I'll be co chairing their 00:28:36.625-->00:28:42.333 international cyber conference with an Indian think tank in October. And to just put things 00:28:42.333-->00:28:47.917 in perspective about how you can manipulate populations. One of the problems they have in a very 00:28:47.917-->00:28:52.917 ethnically linguistic and religiously diverse society are some internism, competitions and 00:28:56.125-->00:29:00.708 unfortunately violence among different political and religious groups. A few years 00:29:00.708-->00:29:07.000 back there was actually fake SMS message- substance in SMS messages and social media that 00:29:07.000-->00:29:13.708 went viral about attacks on a certain religious and ethnic group in the northeast. Who a 00:29:13.708-->00:29:18.750 lot of those family members were working in the west and the south and sending money home to 00:29:18.750-->00:29:25.542 their families. This is very common in developing countries. Those threats of violence caused 00:29:25.542-->00:29:30.542 a mass migration of millions of Indian citizens to head back to the northeast to protect their 00:29:32.708-->00:29:37.708 families. You think about the impact of modern technologies of real and fake news and 00:29:40.542-->00:29:45.542 platforms, someone sitting in their basement could cause millions of people to migrate 00:29:48.625-->00:29:53.542 thinking their families were at risk of death. I'm not picking on India there, this is an 00:29:53.542-->00:29:57.708 example that actually happened there, I think this could happen in a lot of places and these are 00:29:57.708-->00:30:02.500 some of the issues that Modi's government and others in India are thinking about a lot and 00:30:02.500-->00:30:06.125 countries like Bangladesh and Pakistan and other neighboring countries are also worries 00:30:06.125-->00:30:11.167 about. So I welcome the comment about how India is dealing with some of that, and you know let's 00:30:11.167-->00:30:15.833 think about Estonia let's think about India, let's even think about our British friends when 00:30:15.833-->00:30:20.833 we think about elections and how the different systems play out. Um shifting gears a little. We 00:30:23.583-->00:30:26.750 talked about how you get on the lists and how you get to the voting booth, let's go further 00:30:26.750-->00:30:33.583 back. Let's pretend we're in campaign season. Who decides what becomes news? >>[unclear 00:30:33.583-->00:30:39.833 audience responses] >>Who? I heard me. [chuckles] who else? >>[unclear audience response] 00:30:39.833-->00:30:44.833 >>Tom Brokaw. Ok. [audience laughs] You're dating yourself. [more laughter] >>[unclear 00:30:47.042-->00:30:52.292 audience response] >>Large corporations. >>[unclear audience response] >>The 00:30:52.292-->00:30:57.417 government! [laughs] >> [audience laughter] [unclear audience response] >>Is "the 00:30:57.417-->00:31:02.792 government" the same people you congratulated for risking their lives to protect you 20 minutes 00:31:02.792-->00:31:07.792 ago? [chuckles] [audience laughs/groans] Uh lots of sources. Ok. We live in a multi 00:31:10.250-->00:31:15.792 uh various society where a lot of different entities can weigh in and do it but think about the 00:31:15.792-->00:31:21.667 process. Is it a blog post on reddit that ends up getting picked up and is it someone's 00:31:21.667-->00:31:27.292 tweet that goes viral? How- >>[unclear audience response] >>Robots on social media. We're 00:31:27.292-->00:31:32.292 gonna get to that. No we really will. Uh. Next question. And now my now my phraseology to the 00:31:36.750-->00:31:40.792 questions starts to get cheeky, so as I say 'em, try to figure out the real substance I'm 00:31:40.792-->00:31:45.792 getting at. Which runner is in the lead? Why would that matter to me for democracy? And of 00:31:48.500-->00:31:54.417 course runner is a euphemism. Which candidate is actually leading. There are actually 00:31:54.417-->00:31:58.708 people who vote based on what they see happening in the polls. Either because they wanna be on 00:31:58.708-->00:32:04.292 the winning team or because- you laugh at that, there are people who actually in many democracies 00:32:04.292-->00:32:09.500 who actually want to feel like they're with and supporting the group that's gonna be in power. 00:32:09.500-->00:32:14.042 But more importantly you have a lot of cases especially in a country where I'm not forced to 00:32:14.042-->00:32:19.167 vote if I want to play baseball, I may wait to look at the polls to decide if it's worth those 3 00:32:19.167-->00:32:25.667 hours of my time. If my candidate is so far behind in a district where I live, maybe I 00:32:25.667-->00:32:32.167 think it's useless to vote. So think about hacking polls if that can have an indirect 00:32:32.167-->00:32:37.250 effect. So name some polling organizations for me, nongovernmental, of course. 00:32:37.250-->00:32:41.208 >>[unclear audience response] >>Yell at each other, I'm not gonna repeat 'em all, but you've 00:32:41.208-->00:32:47.917 all heard of some of these, right? And what's their IT security like? Do they have any 00:32:47.917-->00:32:53.417 insight or threats? Or people with political agendas working in those companies? What IT 00:32:53.417-->00:32:59.417 hardware and software do they rely on? Remember I'm the annoying intel guy, if I can't 00:32:59.417-->00:33:06.000 get into your federal reserve bank, if I'm an Iranian person who is mad at the US, maybe I 00:33:06.000-->00:33:10.667 will DDoS all your commercial and investment banks instead 'cause it'll create economic 00:33:10.667-->00:33:15.792 harm. If I'm in China and I'm tryin' to steal intellectual property and I can't get into 00:33:15.792-->00:33:20.792 the pharmaceutical company or the defense contractor, or the Australian mining company, maybe 00:33:22.958-->00:33:27.167 I know that their auditors, their legal representatives and their financial consultants have 00:33:27.167-->00:33:33.458 that same information. Tons of law firms in the US and the UK have been cyber attacked to get 00:33:33.458-->00:33:40.292 their clients information. >>Panama papers! >>Panama papers. Thank you. So do you 00:33:40.292-->00:33:44.750 think people would be willing to try to hack polls if they thought it could have an effect? 00:33:44.750-->00:33:48.417 So just like I don't know if my vote is recorded properly, how much faith should I put in the 00:33:48.417-->00:33:54.833 fact that candidate A is 5 points ahead. I think that would be great to pretend that you had 00:33:54.833-->00:33:59.833 no reason to go vote in order to deter you voting in a very close race. >>[unclear audience 00:34:01.875-->00:34:08.583 response] >>Isn't this because of the spoiler effect because we use a first past the post voting 00:34:08.583-->00:34:14.458 system? We could spend all of your conference discussing the merits of first past the post 00:34:14.458-->00:34:18.333 constituencies, parliamentary systems and all the other variants that various countries 00:34:18.333-->00:34:22.792 use or that we've used in the past. I just want to acknowledge the importance of the question 00:34:22.792-->00:34:27.875 you asked and refer everyone to political science textbooks because how you do your 00:34:27.875-->00:34:33.583 elections, primaries to general and all kinds of things matter. If you want a really simple 00:34:33.583-->00:34:38.208 example compare the US system to the French system where they have multiple rounds of voting 00:34:38.208-->00:34:44.000 for president. OK. So it's a good point but I'm not going to go into it in full detail 'cause 00:34:44.000-->00:34:50.458 I probably wouldn't be able to fully answer that properly anyway. Next cheeky question. We 00:34:50.458-->00:34:55.458 asked which runner is leading on the field, is the field made of natural grass or astro turf? 00:34:58.792-->00:35:04.292 [audience laughs] That goes to our botnet question. OK. What is a grassroots movement today and 00:35:04.292-->00:35:09.292 how the hell do I know if it's real people or how many of them there are? >>You don't! >>You 00:35:13.708-->00:35:17.292 don't! [audience chuckles] And what's our problem there? Hopefully some of you are like 00:35:17.292-->00:35:23.083 "Well, that may not be bad if it's a little bit, but at what level does it become 00:35:23.083-->00:35:27.458 unacceptable?" And we're gonna come back to that question in a second. So we I think we all 00:35:27.458-->00:35:32.458 need to accept that what you see... >>Ten. >> Ok ten minutes left. What we see may or may not 00:35:36.208-->00:35:41.208 be real individual activity. >>Does it matter the scale? >>Does it matter the scale? 00:35:44.333-->00:35:49.875 >>Local grassroots level elections for example like for city council versus senator? 00:35:49.875-->00:35:53.500 >>My answer would be: how big is the data pool and what's the statistical level of error, 00:35:53.500-->00:36:00.042 right? What is the amount of error or risk you're willing to accept. Uh I'm gonna jump a 00:36:00.042-->00:36:03.708 little bit ahead here 'cause I think it's relevant to this discussion so. On the 00:36:03.708-->00:36:09.542 astroturfing question, for me it actually quickly degenerates into a competition between 2 00:36:09.542-->00:36:14.750 core values. Freedom of speech and equality. And we see this tension when we talk about 00:36:14.750-->00:36:20.417 campaign finance. We have put limits on what specific individuals and entities are 00:36:20.417-->00:36:24.917 allowed to donate or do as far as spending money. We want people to be able to have a 00:36:24.917-->00:36:30.375 disproportionate influence based on spending money in campaigns, but only up to a point. This 00:36:30.375-->00:36:35.875 speaks to your question ma'm. So when it comes to sharing information right, cause what's 00:36:35.875-->00:36:41.125 the money used for? The money is used for TV ads. But what if I can get the same publicity by 00:36:41.125-->00:36:46.125 doing social media. Should I as one person be able to have the influence of 5 million on social 00:36:48.708-->00:36:55.125 media, when I'm d- not allowed to have the right to do that in an economic sense in donating? I 00:36:55.125-->00:36:58.083 don't have the answer for this but I think we need to start thinking about it in those 00:36:58.083-->00:37:03.083 contexts. We want equality, one person one vote, we accept inequality in the campaign 00:37:05.208-->00:37:11.250 process but only up to a limit. And who should decide what that limit is? Should it be uniform 00:37:11.250-->00:37:15.083 over the United States of America or your respective countries? Or should it be 00:37:15.083-->00:37:20.083 county by county, polling station by polling station, or state by state? Do we want to 00:37:24.542-->00:37:29.542 invite foreign guests to our party? >>NO! >>[laughs] >>F**k no! >>We live we want to live in 00:37:34.375-->00:37:39.375 an open information space, we don't want foreign governments bankrolling specific campaigns. 00:37:44.333-->00:37:49.333 UK invited Obama to weigh in on Brexit. Should we be inviting President Xi to weigh in on the 00:37:52.083-->00:37:57.083 interim election? Or President Putin? >>[unclear audience response] >>Hu- hold on. The 00:37:59.625-->00:38:03.042 gentleman who was yelling one more time, there's no difference between public influence and... 00:38:03.042-->00:38:08.792 >>No there's a big difference. A public speech by Obama is totally different than a foreign 00:38:08.792-->00:38:13.250 power having thousands of Twitter bots. >>I agree with you. For me that's the 00:38:13.250-->00:38:17.167 distinction I gave in the beginning between the covert and the overt. Be- in my next 00:38:17.167-->00:38:20.958 question which I sh- remind me to give you something later as a plant, because that's the 00:38:20.958-->00:38:25.958 perfect segue. Are we inviting foreign guests to our party? And is it a masquerade ball? 00:38:29.625-->00:38:35.417 [audience laughs] Right? And then, is it just the foreigners who come wearing masks, or is it 00:38:35.417-->00:38:40.417 all kinds of interest groups? I mean one of the saddest examples that I remember for a while was 00:38:40.417-->00:38:45.083 in some of the unfortunate human rights turmoils in the middle east I think this was in Syria 00:38:45.083-->00:38:50.667 there was a large uh online following for a very sad story regarding a persecuted and 00:38:50.667-->00:38:55.667 oppressed Syrian lesbian. Turned out it was a 50 year old man in Iowa, I believe. Is that the 00:38:59.500-->00:39:05.500 kind of world we want to be living in? I don't know. But do I wanna let someone else be the 00:39:05.500-->00:39:10.208 sensor who decides what free speech is and is not allowed online? This get's us to the 00:39:10.208-->00:39:15.208 classic game theory question of who guards the guardians? Ok and that'll bring me into the next 00:39:18.750-->00:39:25.500 few questions I have. Which innocent victims deserve protection? Last time I checked 00:39:25.500-->00:39:32.000 the DNC is a private organization. Does it deserve taxpayer money protection to 00:39:32.000-->00:39:38.250 secure its networks? Yes or no? >>[audience answers mostly "no"] >>I was not expecting that 00:39:38.250-->00:39:44.000 uniform answer. I know that we regulate party activities your state legislators regulate the 00:39:44.000-->00:39:50.583 primary process to determine those private organizations' preferred candidate. So we do 00:39:50.583-->00:39:55.458 regulate it somewhat and we do spend some taxpayer money there. I did here a little yes over 00:39:55.458-->00:40:00.458 there so it wasn't unanimous but so I ask the question: which victims are entitled to state 00:40:05.750-->00:40:10.250 funded support and protecting their systems, but that comes with a very close second 00:40:10.250-->00:40:15.250 question: which entities are required to accept that help? Including if it means from the 00:40:18.167-->00:40:23.167 opposing political party that's currently in office. What do you think about that? >>Ask the 00:40:27.167-->00:40:29.625 Clinton body count. [audience whoops, laughs, and groans] >>[laughs] Ask the Clinton body 00:40:29.625-->00:40:35.000 count. So clearly we have recent reference points. As a strategic forecaster I'm always looking 00:40:35.000-->00:40:39.208 forward. I don't think we're gonna be able to fix our system before the midterms and maybe 00:40:39.208-->00:40:45.625 not even by 2020 but I sure hope we can get our act together a little better by 2024 and I hope 00:40:45.625-->00:40:51.500 we can start setting some good examples. And again I'm very nonpartisan, um by law I was not 00:40:51.500-->00:40:56.208 allowed to even participate in a lot of political activities beyond just merely voting while 00:40:56.208-->00:41:01.458 I was in the intelligence community. And part of that ensured the value and the um 00:41:01.458-->00:41:06.208 analytic authority of our team's work when we went to brief congress or the white house on 00:41:06.208-->00:41:11.542 things. So I respect that there are partisan motivations behind some of this but for me I use 00:41:11.542-->00:41:16.250 them as examples, because what happens to one group today or this year can happen to the 00:41:16.250-->00:41:20.833 other group or even a 3rd, 4th, and 5th party if we ever get a viable system with more than 2 00:41:20.833-->00:41:27.167 parties. >>[inaudible comment from audience member] >>Not so much the New York FBI? >>Right 00:41:27.167-->00:41:32.167 >>Tell me a little more >>Partisan. Conspiring with Rudy Giuliani. >>Ah! We have a 00:41:34.458-->00:41:39.458 comment that the New York FBI is conspiratorial with Giuliani. Just repeating it in case you 00:41:41.625-->00:41:46.625 didn't hear it. Ok. [audience chuckles] Ok last 2 questions. The problem with the elections 00:41:51.417-->00:41:57.708 and hacking democracy and undue influence from foreign entities. Is there actually anything new 00:41:57.708-->00:42:04.500 about this? >>[mixed audience response] >>Citizens United is mentioned. So if I'm sitting in 00:42:04.500-->00:42:11.417 the p- If I'm sitting in parliament in 1775 in London how do you think I characterize the 00:42:11.417-->00:42:16.583 pamphlets that Sam Adams and Thomas Paine are cranking out? Do you think that's terrorist 00:42:16.583-->00:42:21.417 activity inciting violence against the legitimate government authorities? >>Yes. 00:42:21.417-->00:42:27.458 Yes. >>How 'bout if I write a bunch of political papers, sign them in another name that 00:42:27.458-->00:42:32.458 suggests that I have much more support than I really do. Federalist Papers, Publius. 00:42:34.875-->00:42:39.875 Madison, Hamilton. The issues we're dealing with are not new. The question becomes: why does 00:42:42.208-->00:42:48.750 this matter and why is it so offensive today? Ok the scope and sophistication. And we heard 00:42:48.750-->00:42:53.000 that a little earlier a variant of that from a lady over here, and I agree with you. It's the 00:42:53.000-->00:42:59.208 question between qualitative and quantitative change. If you heard former NSA and CIA 00:42:59.208-->00:43:04.208 director Michael Hayden or former director of national intelligence Jim Clapper say 00:43:04.208-->00:43:07.583 that the Office of Personnel Management hack that stole a bunch of government employee 00:43:07.583-->00:43:12.833 data was a legitimate intelligence target but that they still want to prevent it, 00:43:12.833-->00:43:17.833 because it was so devastating. How 'bout the release of accurate documents, stolen 00:43:21.417-->00:43:26.417 documents, but accurate. Is it really wrong to share truth? So if we allow espionage and we 00:43:29.583-->00:43:35.583 believe that free information society and truth is desirable, those are 2 rights, how do 2 00:43:35.583-->00:43:41.750 rights make a wrong? And I think it goes to that scale and scope. It goes to the nearly costless 00:43:41.750-->00:43:48.042 ness level with which an entity even 1 or 2 people somewhere else in the world can have a 00:43:48.042-->00:43:54.333 hugely disproportionate impact. Without running the risk of actually running off the 00:43:54.333-->00:43:58.125 printing press and hand delivering the pieces, without entering the physical 00:43:58.125-->00:44:03.125 jurisdiction of those locations. >>Isn't this why we need intelligent voters? >>Isn't this 00:44:06.167-->00:44:10.208 why we need intelligent voters? [sporadic applause] I believe in voter education I unfortunately 00:44:10.208-->00:44:15.708 am also terribly aware that education is not perfectly consistent across our society 00:44:15.708-->00:44:20.417 and I don't want to exclude anyone who the constitution includes. So I don't wanna put 00:44:20.417-->00:44:25.167 education standards or informational standards on what constitutes an informed voter. 00:44:25.167-->00:44:30.875 >>[unclear response from audience member] >>All humans are subject to influence. Should 00:44:30.875-->00:44:36.083 I take that personally? [audience laughs] No, point well taken so we live in a world of 00:44:36.083-->00:44:40.917 humans with behavioral patterns that can be influenced. I'm well aware I've posed a bunch of 00:44:40.917-->00:44:45.292 questions to you that I don't have good answers for myself, but I respect that you've taken 00:44:45.292-->00:44:51.250 your time to join us and I hope I've at least been a little bit provocative and walked you 00:44:51.250-->00:44:57.208 through the various levels where someone like me who both tries to protect our continuity of 00:44:57.208-->00:45:03.042 government and continuity of our constitutional system thinks about these things because like 00:45:03.042-->00:45:08.125 yourselves, you think about breaking systems, not necessarily building them. And I 00:45:08.125-->00:45:14.458 think the more dialogue we have betweeen well minded offense with the defense who's either 00:45:14.458-->00:45:19.250 building or protecting these things, I think that's the path forward. So I think we're out of 00:45:19.250-->00:45:24.542 time. So with that I'm gonna say thank you very much for sharing your time thank you very much 00:45:24.542-->00:45:30.667 for your comments [audience applause] and please keep this dialogue going. [audience 00:45:30.667-->00:45:34.417 applauds and cheers]