Hi, welcome to Election Security Part Two, The Infrastructure Strikes Back. My name is Amelie Coran. I will be your panel moderator for this session here. Little did we suspect that this set of panelists would be back together six months later to discuss where we are versus where we were when it came to election security in the upcoming November general election. Since then, to say it lightly, things have gone off the rails, given that you now see us via video in our pandemic past. We've had a highly contentious Democratic primary season, some technical glitches supporting such primaries, court cases regarding in-person voting, and enough various disinformation campaigns to last another election. One thing that hasn't changed is the lineup of our esteemed panel from ShmoopCon. And tonight we have Kimber Dowsett, Casey John Ellis, Jack Cable, and Todd Beardsley. I will then let them introduce themselves with a short intro. I am Kimber. I am the director of security engineering at Trust. That's Trust.Works, a software infrastructure company based out of San Francisco that works with both the public and private sectors. Hi, my name is Casey Ellis. I'm the founder, chairman, and CTO of BugCrowd. We run crowdsource security as a service programs, including vuln disclosure, bug bounties, crowdsource pentest, and so on. And yeah, great to be... unusual to be talking about this with all the additional content, but very good to be talking about it again. Everyone, my name is Jack Cable. I am an election security technical advisor for the U.S. Cybersecurity and Infrastructure Security Agency, which is essentially the nation's risk advisors. We advise states and localities on the risks associated with different technologies, provide cybersecurity assessment services, so that they can make the best decisions to have a safe and secure election. Besides my work at CISA, I am a student at Stanford and a security researcher. Cool. And hi, I'm Todd Beardsley. I'm a director of research at Rapid7, a U.S.-based cybersecurity company. I personally care a lot about elections. I am usually an election judge in Texas, and I have a deep background in hacking, offensive security, research, vulnerability analysis, stuff like that. And congrats to Jack for the level up since our last meeting. He gets a bunch of power-up points on that one. That's awesome. So we're going to break this down into two sections, I believe, unlike last time, but mainly kind of a catch-up, a first section here about what has happened since February. So obviously a section B, we're coming up on about 90 days until the general election, and what we can do between now and then, since the timeline is definitely shorter, but also what kind of activities are going to be carried forward from them to the next election, next primaries, or just in general, lessons kind of learned. So with that, we have ourselves a first question here. So we find ourselves here again after six months, and there's been a lot going on that we didn't cover back in February. However, regarding one of the last takeaways from closing that panel, we noted it was important to engage your local board of elections. And with that, where do we stand? Wow. Well, let me start this off. I engage with my local board of elections by being an election judge. I ran a polling place not too long ago about, we're recording this on end of July. So for me in this time stream, this is about three weeks ago, there was a special election and a runoff election combined here in Texas. It was pretty fun. I never knew that, like, wiping down voting, you know, polling places would be so rewarding. I got to feel like I was battling COVID, like, every five minutes, you know, helping people out, and so like, and for that, at least for me, like, I felt like I was really, I was doing something. I did notice through training and then on election day, you know, the demographics have switched over quite a bit on who is working in the polls. Like, it is very common normally to see a lot of retirees and just, you know, older people who are there to help out and help out their communities in this way. I was not the youngest person at this polling place, which was first for me. So if you have the opportunity and the inclination, and don't mind doing a whole lot of cleaning all day long, you know, maybe volunteer to work in a polling place come November. Anybody else? Yeah, I mean, what has happened since then, I think, you know, in terms of rocking up and helping out, it's fair to say that any intention to do that would have been a little distracted by, you know, March and so on. But I think, you know, Todd's example of just doing what's needed, especially with the pandemic and, you know, the changes in operational considerations around actually running an election, still true. It's even more true now, than it was. As the token non-citizen on the talk, I mean, this is even more foreign interference now than it was when we gave the talk, because I'm actually in Sydney at the moment. But, you know, part of what we've been working on, what I've been working on, and a bunch of other people have been working on is, you know, standardization of, like, how do we make adoption of vulnerability disclosure programs and the implementation of policy, specifically for 2020, with all of the, you know, unique considerations this year has, how do we make that as easy as possible for the states and counties? So we updated a version of the language on disclose.io, which is an open source initiative to basically make it easy and make it as standardized as possible. That came out after the talk, and it's been good. I think, you know, at the very least, that's actually served to get a lot more people thinking about doing that, that maybe weren't before, because that kind of blocking function of how do I even engage with the hacker community in the first place was, I think, pretty difficult for a lot of people to even consider. Early to echo, yeah, Casey's point there, I think, yeah, something I've been involved with and pushing for is states and vendors to establish these vulnerability disclosure policies. On the CISA side, we are releasing guidance to election officials in order to establish vulnerability disclosure policies, essentially saying, if you want to do this, this is the best practices that you can follow. A lot of that is drawn from CISA's directive, the Binding Operational Directive 2001, which is a draft directive that will require all federal agencies to start a vulnerability disclosure policy. Yeah, that was a good deal, by the way. Yeah, I think, yeah, really looking forward to see that come out and see the positive security effects that can have all across the federal government. But of course, CISA doesn't have that same authority over states. So we're essentially putting out guidance, giving them the best practices and the resources they need to start this themselves if they want that. And then, of course, yeah, besides that, just phone work I've been doing, yeah, clearly I'm not at the local level, but the federal level. And I think that there's really a lot of ability there to kind of have an impact at scale of working with all 50 states, working with a significant portion of the localities of the counties that are out there. So I think that's a really great opportunity to be at CISA and have this kind of wide-ranging effort, wide-ranging effects that I'm not sure you can have anywhere else with election security. Kimber? Yeah, I'll jump in. It works out well since Jack touched on Casey's point. I'm going to touch on Todd's point. The prompt was, you know, what's happened since February? And the answer is a pandemic. So the reality of a lot of the election security things that we would normally talk about and that we will touch on today still rely on people being able to actually get to the polls to vote in states that aren't going to allow mail-in ballots. So I think it'll be a nice segue into a lot of the misinformation we're hearing about mail-in ballots. But to Todd's point, we can scream to the skies that mail-in ballots are perfectly safe and reasonable and actually help disenfranchise voters however voice. But there are going to be some places that insist that folks go to the polls and somebody's got to be there to man the polls or we're going to end up in a different type of disenfranchisement, right, where people are lined up for 20 hours because there's three poll workers, you know, for thousands of people who want to vote. So it's important to know what's going on in your voting district and if your voting district allows mail-in voting, great, cool. But if they don't, like, that's a perfect opportunity to get involved. And I understand that it's asking you to put yourself at risk, too, and that sucks, right? It is, yeah. That's where we're at. I am shocked. I had a COVID test about four days after election day and I am shocked I did not come up positive. But hey, you know, turns out masks and hand cleaning and surface cleaning works, though. Yeah, you know, kind of the follow-up on this, too, is, I mean, obviously, the curveball that we're thrown was the pandemic and, you know, as Todd mentioned that, you know, primarily a lot of the election workers, you know, that were counted on by various precincts and states in general were retirees and those who, you know, I hate to say it, have more time on their hands. You know, this is obviously going to be proving a challenge for staffing and it runs headlong into the issues, you know, obviously some of the disinformation that's been spread about mail-in voting. You know, are there any particular ways that we can kind of mitigate or address any of these issues that are novel? Obviously, we're running, you know, headlong against, you know, people pushing back on the mail-ins, but then we have the reality of, you know, folks potentially exposing them to a deadly virus. I hate to run the gambit of talking about, like, e-voting, but obviously there are other ways to, you know, look at, you know, potentially extending voting times, alternating places where people can vote to reduce exposure. Are there any other, you know, methods that, you know, potentially the EAC and others can address in this case? In before blockchain? Yeah, I wasn't there. You have to drink now. So, I mean, e-voting is a non-starter, right? Like, we're recording this and it is today, 97 days, by the time this airs, it'll be about 90 days before the election. And, you know, West Virginia is doing their thing and good for them, but no one else is. I don't see anybody having any plans for that right now. Maybe someday in the future, you know, e-voting will be a thing, but I don't, I don't, I think the easiest way to get people to the polls in states that don't have mail-in ballots is extending, you know, early voting. I mean, that's a thing. Texas, I'm in Texas right now, so great, Texas. We're bad at mail-in ballots, but we're apparently really good at early voting. My voting, my first day to vote in November will be October 13th, so that's a stupendous amount of time, way longer. And so that'll be, that will help at least give people an opportunity to get into a polling place where maybe it's not so crowded. Last day of early voting is super crowded and election day will be super crowded. So, you know, if you can vote in that early voting period, I strongly suggest you do. That, you know, it doesn't help any of the I.T. problems that we talk about and that nominally this panel is supposed to be about, but it does help the, like, not getting COVID. So, which, you know, might be a little more important. Amber? I want to, plus one to the adding more polling places because we know social distancing is huge to prevent the spread of COVID. When we have communities like mine where there's one polling place downtown and then one local school, then we have basically the town split in half to go to these two polling places and it gets kind of crazy. Holding people to districts, we see some gerrymandering, right? They'll draw a line right through the middle of the university so that half the university students think that they're supposed to vote at one place and it's really the other. So, you know, if they're going to say no mail-in ballots, then why not say, but all the schools in a single district can vote and if you're eligible to vote in one, you can vote in any of them so that folks can at least get to the closest, you know, place. And we do our best to, like, disperse the population, but a lot of towns have a couple polling places. They're almost always schools, which who knows if schools will even be open, but if they are, you sure don't want to have a hundred thousand people rolling through a school that children are going to be at the next day, right? Like, so there's physical considerations that certainly were not part of our equation in what, February 1st, when we jammed through all the things we think could go wrong. This was not on my bingo card. No, no, no. So the thing that's, to me, that's new is, and I've actually, I've heard Todd say this in a panel on this before, you know, democracy does rely on the peaceful concession of whoever loses. So the increased likelihood of a hanging count because of mail-in voting and the changes in the process and different things like that, I think there was a lot of conversation back in January and prior around the role of risk-limiting audits to basically, you know, say, no, this is not like any accusation of fraud can be basically confirmed or denied at that point. Projects, you know, to give a shout out to is Arlo. ARLO, which is essentially a framework for that, that I believe is funded by CISA and is open source. And something that I've been trying to encourage people in the security research community to do is to go bang on that, actually go look at it from a security standpoint, because ideally if there's any point in time over the next six months where Arlo itself gets called into question as a tool to rebuff, at least at that point we can say, no, we actually went through this and it seems legit. So that to me is new, like that was always going to be some, to some degree of risk, as it always is. But I think that's actually a far, that's going to play a far greater role actually post-election, on election day and post-election day in 2020. Yeah, I think they covered a little bit of that on the HBO special of Hari, as well as, you know, I think it was the second half of the documentary was regarding the risk-limiting audits. I don't know if they necessarily had a really good explanation of how it all works. That is a little extra math for most folks, but, you know, it's one of those good things that can be put in. I think, and looking at, looking at, so, you know, calling myself out as this was a theme, and the last time we got together as well, acronym, de-acronyming stuff. So risk-limiting audit is what RLA stands for. I think it's verified voting, who are running point on it, and they've done some pretty, I think, good work on explainer videos that take some, you know, fairly complicated math and kind of simplify the concept to the point where a non-technical potential voter can actually consume it and understand what's going on. It's essentially, you know, a cryptographically determined sample, random sample set that's then paired with verification of the outcome compared to what's recorded. And if there's any sort of deviational margin of error within that sample set, then it goes again and goes again and goes again until it can work out the scope of that. Or, you know, if everything checks out, then everything checks out and things are okay at that point. It's the randomization and the process around it that I think is, to your point, difficult to explain on a technical level to most people, but I think the concept itself is actually fairly easy to grow. Great. And just to kind of go back a little to our discussion on the different kinds of voting options that there are, it's clear that the election is going to be run a little differently this year. Just with the constraints we face, election officials have to provide an accessible and safe method of voting for their voters. And what this means is essentially from CISA's perspective, we want to limit the risk as much as possible with these options. So, for instance, talking about online voting, also called electronic ballot returns, CISA has assessed that that is high risk. Even with controls in place, the risk there still cannot be controlled. And it's not CISA's job to decide whether these are deployed, but it's our belief that the risk on these is much higher than, say, compared to in-person voting or mail-in ballots. So on that end, CISA has put out a series of documents essentially describing, from a procedural sense, what kinds of options election officials have, both to ensure safe in-person voting and then also to make sure that mail-in balloting process goes smoothly. And just to touch on some of the in-person voting options there, it is very true, like Todd was saying, that the truth is a lot of these poll workers are older and they face a higher risk of being impacted by the virus. So there's going to be very high poll worker shortages. And in a lot of cases, that means consolidation of polling places because they can't staff that many. And that, of course, can lead to problems because then you have more people in fewer places. With a pandemic, that's, of course, not ideal. But we have to make it work. So one option there is vote centers, for instance, where larger physical polling places that make it easier to maintain physical distance. There's, of course, still a polling shortage. I guess here I'll say to everyone who is young and healthy, the best thing you can do is serve as a poll worker and make sure that on a local level your elections run smoothly. But yes, it is going to be a challenge just because, yeah, of course, in-person voting carries some risks with it from a health perspective. So we encourage states to make the decisions that best fit them. But both mail invalidating and in-person voting we view as being low-risk options given that there's a paper trail and you can run, say, risk-only audits on those. Well, so I'm going to take kind of a little bit of a left turn. I know, you know, we just full transparency for folks who are watching this, like we have a list of questions that we've agreed on, but I'm going to kind of combine this because of the way the flow is. You know, one of the things that's amazing about, like, where we live, we're in the United States here, and Casey accepted, but yeah, we will adopt you on this one. Most of the time. You know, is the freedom of speech, you know, it's part of the our own constitution and whatnot. But, you know, as we mentioned in February, you know, one of the critical things about this election is how we talk about it, whether it be through discourse about outcomes, whether it be the primaries or the general election, the methodologies we use to do that. So we, you know, talk about the press about, like, how things have gone, the process of, you know, how we go about voting. But also, you know, it's another thing called disinformation or misinformation where, you know, what we talk about is willingly bad, essentially not right when fact-checked, or in some cases, is disinformation provided by an actual entity. I know, you know, with the 2016 and earlier, the recent, the midterms, we had influence from outside sources. And obviously, you know, Washington Post just recently kind of covered that we're potentially seeing some influence from China and Iran and some of our other, you know, would classically qualify them as adversaries, but yet we still find some ways to deal with them. You know, where do we kind of find ourselves in this case right now? Obviously, you know, six months later, you know, we had a little bit of a kind of, I wouldn't say necessarily contentious Democratic primary, but, you know, it was a lot more graceful when people, you know, basically, you know, said, yeah, I'm out, and let people carry forward. But also, you know, in recent news about how people are talking about, you know, the legitimacy of the methods that we're using, where do you kind of see ourselves now? And what can we do in the future here, as folks who are attendees to this video, but also, you know, as responsible citizens to kind of educate others, your parents, your friends, your peers, your neighbors, and so forth to be on the lookout for this? I'll rush into the fire. I think an interesting thing that I've seen is that, yes, when we did our panel back in early February, which seems like so long ago now, I think that we could pretty clearly say, like, Russia. We're seeing the Twitter bots, the farms, we're seeing, like, the disinformation campaigns on Facebook, Twitter, IG. Now, it's much more complicated. So, the interesting thing that we've seen now are, well, I feel like it's interesting, because I'm a, you know, social media nerd. The QAnon accounts that have popped up seem to span the gamut of countries. And you see a lot of activity from these QAnon accounts just coming from the U.S. And they're not, like, some, you know, complex, combative nation state, right? They're from just, like, diehard MAGA people who are like, I'm going to do my duty, and this is patriotic, and they are figuring out how to spin up bots. And so, that's pretty interesting. And then to see bots that will respond to Trump accounts, right? Or the interesting thing that I see a lot, too, are accounts that get a lot of followers because they'll post pornography, right? And then they get, like, loads of followers. And then they get verified in some cases. And then as soon as they get the checkmark, they switch to, like, QAnon accounts that have given themselves some name that you can recognize in the media. And all of a sudden, you think you're engaging with someone that you're not engaging with. But what that does is have this, like, celebrity or verified boost of this misinformation. So, for me, as a person who has a blue checkmark, I want to say, I don't know anything. I'm not an expert on any fucking thing. And I'm going to tell you flat out that, like, you'd be hard-pressed to find a blue checkmark that is an expert on everything. So, if someone gets their blue checkmark for being an actress, like, maybe don't just immediately trust that they're an expert on vaccination protocol, right? So, I think that it's really fascinating how the floods are coming and the stuff that Cambridge Analytica did, it's all still happening under a different name, a different company, but it's all still out there on Facebook and Twitter. It's just, like, now more people from different countries, including our own, are able to participate in the disinformation process. Yeah, I'll tag in on that. Just confirm what you're saying, like, the QAnon stuff and things of that nature, they're happening on the ground here in Australia. I think for, you know, for ostensibly different reasons from a partisan political standpoint, but it's kind of coming from the same mindset. And I think in part, like, we're all going a bit stir-crazy right now. It's good not to ignore the fact that society just in general is dealing with mental stress that we've not seen collectively for as long as Twitter's been around, definitely. So weird shit happens. But yeah, there's that piece of it. I think, Kimber, you touched on a really good point. I actually got invited to talk about disinformation on a friend who has a cooking channel. She's got, like, millions and millions of subscribers, but she saw basically bot, like, advertising-focused bot-generated content, ripping off her stuff, and then noticed that there was subversion starting to creep into that, and then the ability for that type of channel to be used. That's so crazy. It's nuts, man. And I'm like, what am I doing on a cooking channel? This is crazy. No, they have an entire channel on this, like, basically debunking some of these bots or the content farms. So it's a real thing, and I think the ability for that sort of thing to be deployed very rapidly, because these are businesses. It's businesses that exploit the things that are exploitable to build following on social media in some of the ways that Kimber just described, but then they sell that or rent that, or if they're owned, you know, potentially by, you know, an actor that can go hostile, it's redeployed into that. And that's happening across all sorts of different channels. The one you asked, Amelie, about, you know, things that we can do. I think something that we can all agree on. The great hack, for example, just as a, you know, a way to get people that aren't necessarily technical in a context that's apolitical. So you're not sort of going one way or the other too much. You're just explaining to them this general idea that, like, social media is a constructed reality that's been built just for you. And you actually need to be observing it like that. I think, you know, for the hackers that are kind of watching this, that's probably a thesis and something that's important that we could all agree on. And I found that to be fairly helpful. Great. And just to talk briefly on foreign disinformation, of course, that's a very large concern. We've seen in 2016 what happened, and in 2020, it seems to be shaping up again. We know, yes, our nation's adversaries, Russia, Trump, China, Iran, are all targeting, trying to interfere in our democratic processes. So from CIS's perspective, our number one priority is to ensure that Americans decide American elections. So that means ensuring that foreign adversaries are not able to interfere, whether that's by actually targeting election systems, whether that's disinformation campaigns, all that. It should be Americans who are deciding American elections. So that kind of leads us to the point, then, what steps can Americans take to mitigate the impact, say, of disinformation or just general confusion, say, on election night? I think the most important thing here is just to understand that elections are going to be different this year. Election night, November 3rd, is not going to be the same as election night in the past, because with many more mail-in ballots, they're going to take much longer to count, just due to state laws and processes around that, as well as just technical constraints, since some states are rapidly scaling out mail-in ballots at a scale that is maybe tenfold from what they previously had operated. So with that perspective, election night, it is entirely possible that it just isn't what the election results are. And it may take a week, it may take several weeks to actually learn what the final results are. So the best thing that Americans do is to just internalize this to understand that election results are not going to come out immediately. Media has an important role in this, that it can't just be election night, the final results, declare who won, because we have to acknowledge that might not be the case. So I think that if we all are on the same page, expecting this to be a slower process, and keeping in mind that a slower process means that there's more time to actually verify that results are correct, and to ensure that the final count is ultimately the right one. So I think just understanding that patience is needed here, and that election night, not going to know who won, may take some time, but we'll get there, and we can be confident then in the outcome of the election. That's the important thing. Yeah, and just to follow up on what Jack said, is like, election night is not the end of this, right? Like, for starters, like any kind of disinformation campaign that we've been talking about, that's going to happen way before election day. Like I mentioned, I get to vote on October 13th, so, you know, look for something exciting happening around, I don't know, first or second week of October. Almost, like, that's the time when your fear ganglia should flare up around what's going to be happening around disinformation. And just one other super quick point, Jack is also totally correct that I would be shocked if we had results election night. Now, it doesn't mean it's the end of democracy. Like, there will not be rioting in the streets over this. We've done this before. Like, the 2000... some people on this call are old enough to remember the 2000 election. And we remember that, like, that was weeks and weeks and weeks of will they, won't they, which ended up in the Supreme Court decision. So, like, that did not destroy America. And not having election results at, you know, 1am on November 4th is not going to, it's not going to kill everybody. Like, we'll be fine. We'll be fine. Yeah, that does bring up a good point, you know, or subsequent question here, you know, it's kind of talking some of the logistical errors. You know, just to put on an election is not as easy as everybody kind of thinks. Like, you just go in there and pull the handle if you're a manual or you... It is way more complicated. It is way more complicated. I mean, you know, just watching Matt Blaze's Twitter feed sometimes and just how simplistic some of the suggestions are. And then, of course, Matt being Matt kind of fires back in Matt's way and whatnot. And that's not a knock on him. It's just a try to educate people that, yeah, this this shit ain't simple. You know, as much as I railed on my trip to the DMV recently, you know, I sat in the car and kind of pondered everything required to kind of make my trip better. And I'm just like, oh, my God, that's that's a lot to move. That's Sisyphean in a way. But, you know, obviously, one of our one of our bigger challenges, obviously, the thing that made the biggest press right after our February conclave here was the Iowa caucus. And I wrote a long paper on this about the whole DevOps process in regards to how it was developed. But, you know, the Iowa caucus with the Georgia primaries, which, you know, some would say was kind of a predictable outcome of, you know, what kind of a clusterfuck it would be. But, you know, that's the other the other issues underscore the potential about how trust is eroded through procedural process error by no fault or intent of the creator of that error. It was more or less like we're forging new areas of election things we can do and mistakes will be made. You know, there was no necessarily evidence when one looked at that, you know, interference necessarily occurred. But, you know, when you know, basically, these things that we do in so nice a word, shit the bed, you know, what are the different ways that we can as professionals in the security and election security arena kind of capture the discussion and say, you know, this shouldn't erode trust. This is us trying something new. Mistakes will be made. Morale will be, you know, lowered. But, you know, what are some things at the technical level to kind of, as I mentioned, you know, you have a lot of technical people that will swoop in and say, oh, we can fix this with blockchain, for instance, as Casey so lightly joked about. But, you know, what are some practical techniques we can have to kind of educate some people on like, no, no, no, this is a big ship to steer, you know, this is what you can expect and, you know, don't lose trust in this. Great. So first, just to really underscore the point that running elections is incredibly hard. There's so much more than just kind of from a voter's perspective showing up to, say, a polling place, casting your ballot. There's so much more that goes into this process. So many months of preparation. That's a difficult task. And every single election official I've talked to is incredibly motivated and wants to make sure that elections run smoothly and that their people can, so just thinking about going back to February, say, was already shaping up to be perhaps one of the hardest elections that an election official has had to run just because we are in an incredibly polarized environment. We know that there is foreign interference that occurred in 2016 and we can expect again in 2020. So even from that perspective, this was a hard task. And then you add the pandemic and everything becomes so much more complicated because suddenly we can't vote entirely in the same way as we're used to voting and all of these processes have to change. In a lot of cases, like I said, election officials now have to scale out mail-in ballots at 10 times the capacity. And when your machines process those, we're only intended to do, say, a small percentage of voters in your jurisdiction. From a technical perspective, that can be very difficult and things can break because we're rapidly scaling out these technologies and things can and likely will go wrong. So from that perspective, what should voters expect? So I said before, be patient. Election results may not come in immediately and that's fine. I think the second point there is really to expect things to go wrong, but don't immediately believe that that is a result of interference of any kind because the most likely explanation is that's just some routine error that occurred and that will be worked through. There's processes in place in order to handle these types of failures. We have, for the most part, paper trails that allow verifying elections. So from that perspective, we have controls in place. And yes, technology can be brittle and stuff can break down, but a lot of times just look for the most likely explanation that, of course, interference is still possible and we should be very concerned if that does happen. But just looking from kind of what is most likely to happen, it's more likely that, I mean, we can almost assume that some technical failure in some capacity will occur, but that doesn't mean that's malicious and the people just have to view it as that way and understand that there are still controls in place. Yeah, Occam's razor is good. I can't remember if it's Hanlon's razor or Occam's razor, but it's one of the razors. I think it's Hanlon's. Yeah, anyway. Yeah, well, simple ends malice. Anyway, whatever, because we can probably look that one up after the fact and I'm outing myself for not knowing which razor is which right now. I'd add to that, no new stuff in 2020, like timeout, you know, like there's a whole bunch of innovation happening in the election space, which I think is fantastic. And I think it's important. It's going to be critical after this is done. But, you know, the addition of variables, you know, the idea that there's like software fails, which is the second point I'm going to make, but like it's the failure rate of software is directly proportional to how quickly it's been brought to market and oftentimes how mature it is. So this idea of like, cool, let's just blast 2020 with a whole bunch of brand new stuff that we haven't really tested. You know, ultimately, when you go back to Iowa and do a bit of a root cause analysis, that's sort of most of what happened there. It was less than six weeks of the analysis. Yeah. Yeah, it was. And it's logically what would happen again if we do it with other stuff. So no new stuff. But then this other idea of like software, you know, in terms of, again, coming back to how we can help, like humans aren't humans make mistakes, period. Like this is why, you know, we've got an industry is because, you know, while we come up with all these incredible ways to do stuff, including democracy itself, we do make the occasional spelling error. And then there's bad people that want to manipulate that to get what they want. So this idea of like to err is human, it's more about how you respond. Again, it's part of what I like so much about, you know, vulnerability disclosure as a process, but also as this like leading indicator of maturity when it comes to security of an organization that can translate to trust. I think that's a concept that isn't very well understood. And I think a lot of the time people, you know, on the operations side would prefer to do ostrich risk management and pretend it didn't exist. But I think it's going to become pretty important in context of all of the stuff that can and probably will go a bit funky this year. Casey, you have some of the best vuln disclosure jargon around. I've been practicing. Yeah, a little. Yeah, like I guess I would just say as technical people who are probably the only people watching this, you know, I think what you can you can do your part by not freaking the hell out when you see something that goes wrong. Like, you know, just to echo, you know, Jack and Casey, it's like it is it is a hand lens razor kind of thing. There will probably be mistakes. You know, I don't I don't think I would go so far as to say like it's super I've got the Metasploit shirt on. It's very hard for me to say like don't disclose vulnerabilities. But, you know, maybe not on Election Day and maybe not make a bunch of hay about like hackable voting machines. Like that is kind of the least of our worries. If all we had to worry about was a hackable voting machine like that physical device, boy, that would imply that we've fixed so many other problems in infrastructure, in disinformation, in everything up and down the line. So, you know, I would hope that, you know, the folks that work in the space who pay attention to things in Voting Village, you know, maybe not completely lose your cool over a, you know, a voting machine that can be hacked in person. So I'll do a quick response and I'm sure Amelie wants to move on. But this goes back to something that I said in February. And it's a recurring thing because I feel like I say it a lot. So if you've heard it before, suck it because I'm going to say it again. But the biggest disservice that we can do to the American people as security professionals is somehow convincing them that their votes don't count. And we do that by constantly preaching that the system's broken, the voting machines are hackable, the infrastructure's flawed, the voter registration system is, you know, something that can be tampered with. It's not to say these things aren't true. But also, like, you have to qualify those ramblings and announcements with how often that actually happens and what the likelihood of that happening actually is. And the idea that, you know, hacking 20 voting machines is going to sway an election without even, like, acknowledging what it would take to actually hack the voting machine, right? Or to tamper with the voter registration system without acknowledging that, like, you know, states do have some IDS systems in place. Like, sure, could it happen? Yeah, we can, like, what if all day long? But if we're, if we're putting information out there, that even makes one person think, well, my vote can just be changed anyway, so why would I bother voting? Like, then we've fucked up, like, bad, because we've kind of, we, like, shot ourselves in the foot with the thing that we were trying to make better. So we're now into Section B. So we've now pivoted from where we were six months ago to where we find ourselves in the last 90 days here. 90 days scares me, because, you know, coming from the federal government, it takes longer than that, mainly to fill out the paperwork for something. So 90 days for us in the real world, if in the commercial sector, will be totally interesting. But obviously, I'm going to highlight the fact that, you know, as Jack leveled up here, CISA has taken more of an active policy and assistance role for states. The Election Assistance Commission Committee has hired some really great new staff. In fact, some folks, I believe, Kimber and I were on the panel many years ago with, and, you know, the feeling that, while it's awesome, they hired these people. I don't know, I've tweeted out about it, it's a little too late in certain cases, but, you know, they hire great people. What is the feeling right now that these folks can actually make a difference between now and the election? Or, obviously, if we can't do it by then, what are the, what is the change that can be made for further elections, provided that the world isn't going to melt down? So yeah, I think, yeah, I can take this to start. Yeah, so it's true. Yes, CISA has brought on some more people to help out with election security. I'm part of a group of me and four other Stanford students who all came to CISA to work specifically on election security. And we've been having a lot of fun being able to work essentially on both the infrastructure component, building tools to allow organizations to better secure their systems, and allow CISA to say, aid and assessments to state and local election officials, as well as working on some that's a foreign disinformation component. So in terms of what both, say, CISA and EAC can do by November, I think there's a lot that can still be done. Of course, yes, we have, I believe, calculated this, I think it's 89 days from the time the talk this is airing until the election, and that's very little time, almost nothing, but we still can do a lot. We can help, say, states identify vulnerabilities in their systems. We continue to offer services that assess these systems and give guidance. I mentioned before, we have documents that we published along with the Election Assistance Commission, and we're working to support states in the capacity that we can. So I think that there's a lot that still, of course, needs to be improved, but we're getting there. And from my perspective, yes, the government plays a large role in this, the federal government. And I really do think that states, this is, I'd say, one of the major improvements since 2016. In 2016, the federal government's involvement with the states was not at all near where it was today. So much has improved since then that we are now working with each of the 50 states. We're working with a significant portion of the local election offices, and we're in a much better place, both to be protecting systems and then monitoring in case stuff does go wrong. So, you know, you speak to, obviously, kind of the involvement with CISA and, you know, kind of the states taking a more active role in their own survival in a way. You know, have any of the vendors, either of the e-poll books or the election systems, been more willing to kind of come forward and work proactively with the government or, you know, say any of the companies represented on here to kind of solve the problems? I know, you know, I've recently been involved with some workshopping with OECD on regards to vulnerability disclosure policies and digital product security. And one of those cases is finding a good mediator sometimes to kind of do that. Has anybody kind of moved that way, or are we still kind of like, you know, kind of finger pointing and moving forward there? So in terms of CISA's involvement with this, of course, CISA's preference is for vulnerabilities to be disclosed either directly to the state or to vendors when that is possible. And it is our view that, of course, vulnerability disclosure policies can be very helpful in this process. I'm not aware of any vendors that at the time of recording, or states for that matter, that have come up with vulnerability disclosure policies that could very well change between the two weeks when this airs. But what we do offer is resources for those that want to implement vulnerability disclosure policies to do so. So like I mentioned, we have our guide on vulnerability disclosure that will be live by the time this panel airs, as well as the fact that we do serve as a last resort for people who are unable to disclose vulnerabilities. For any reason, they can report it to US CERT, which is under CISA, and we will work to get that disclosed to the vendor in order to mitigate that vulnerability. So CISA does play an important role here. And yeah, it is our hope that of course, yes, that any vulnerabilities that either people come to vendors with, or come to us with, that they will be addressed. So I mean, and just to follow up on that, I mean, we're getting under the wire, at the wire, right, for the November 2020 election. If I were the king of vuln disclosure, I think I would direct people to disclose to you personally, and, you know, by extension to CISA, before vendors and states, like, I mean, I think that's kind of the way, like, let's say I'm sitting on a vulnerability, or I find a vulnerability in some election system or whatever, like, and I'm a, I'm a hacker guy, who wears bedspread to do this. Like, I have it, I don't want to not tell anyone about it, you know, there is this, like, it's okay to yell fire in a credit theater, if the theater is actually a fire business. Um, I think it's, it's probably not great to, like, drop that on Twitter and just full disclose and do that. I mean, that's not helpful, I don't think in, in, in the slightest. But I do think, like, you tell me, like, I, my, my instinct is, you know, tell CISA and hope for the best, and keep my mouth shut until November, I don't know, 10th, 15th or something. You know, so at least this is where they can do, I'm describing your job at you. Yeah, you can do instrumentation, right? Like, so even if there's no fixes, there's still ways to, to track the vulnerability. Yeah, yeah. And you're exactly right there that yes, the priority is for the vulnerability to get fixed as quickly as possible. And we want to support whatever will make that happen as efficiently and smoothly. So of course, it's ideal if it is possible to disclose directly to vendors, but that is hard when there are no policies today. Yes, exactly. So given that, yeah, the current landscape, yes, CISA does serve as a coordinating role there. For people who, yeah, can't really find the contact to disclose, they can come to CISA and CISA will work to make sure that the vendor or the state is made aware and that the vulnerability can be fixed. Yeah, I mean, hard, hard agree with, with Todd's suggestion of going to CISA, especially at this point. You know, keeping in mind as well that like with the 90 day kind of lead time that we've got, the vendors are very likely to be distracted and have lots of other things on their plate, just from a pure logistical standpoint, before you go layering on the pandemic and the fact that 2020 is generally a bit of a shit show. Yeah, the thing that I wanted to double click on is actually around basically non-disclosure of findings ahead of November at this point. And this is very much opposed to how I normally talk about non-disclosure. Yeah, it's really, it's a really, it's a really difficult thing to say. We actually talked about this in terms of the boilerplate election policy that we put up on Disclose.io. And we've got it in there. It's like, you know, basically the agreement is not to disclose until after the election is finished. Ordinarily, that timeline would serve as back pressure on the vendors to fix. And I think that's a really good and important thing for accountability and transparency. But the risk of frightening a non-technical voter into just giving up and not showing up to the poll booth as a product of trying to do something good, I think is extremely high on this particular topic at this particular point in time. So yeah, it's a hard pill to swallow. I think for security researchers in general, it was definitely, you know, from where I, from what we do and where I sit, it's a hard thing to say, but I actually do think it's the right thing for this year. Yeah. Yeah, I know. That's one of those things we've kind of, you know, as I mentioned with some of the policymaking, you know, obviously federal, international and whatnot. You know, we've set the, I wouldn't say necessarily artificial 90 day deadline, but obviously for inside the 90 days, it does kind of create an unworkable framework with both in the timing, I think the regulatory environment for whatever folks need to do for certifications, plus loading all the election information and the logistics of that. So, you know, it just creates this, this whirlwind of not a good situation for us to be in. So yeah. So yeah, we, we talked about this, you know, earlier in regards to kind of the effect that COVID-19 has had on how we staff the election, how we are attending the election and participating in it, obviously with retirees and whatnot. There is so much dumpster fires being poured into the alley right now. It's not even funny. And obviously with the disinformation and just all the stuff we talked about, if you were all betting people, and if we were in Las Vegas this year, instead of doing this virtual, what would you bet to be the first thing to crumble out of all of this? What do you think is the first thing that is just like, you know, the guy from Oz comes out behind the sheet and says, yep, everybody go home. We're fucked. If I was a gambling person, which I'm not, I think it would be if we're talking about the first thing to just go like shithouse on fire. What do we do? Is a bunch of maybe rebellious folks who would show up at polling places without masks and like fake cough and just make a big to do and just try to disrupt, you know, the peaceful line at the voting place, like, I think that, sure, I think we'll see people tweeting, oh, I got a mail-in ballot for my uncle who died last year, and then it gets 10,000 retweets and then somebody else tweets something similar. Like, I think we'll see that, but like just for shithouse election day, like, what are we going to see on the news? I think like polling places being disbanded for like civil unrest and not from the folks who are there just to vote peacefully. So this is starting to form like a John Carpenter movie in the worst way possible then. Well, I am a horror fan, so of course that's where I go. My hope is that people respect democracy, regardless of which side of the line you fall on, and just let people have their constitutional right to participate in the electoral process. However, I am currently disenchanted by the state of the country right now. I don't know. I think that your first sign of everything going to hell is going to be on like in the neighborhood of October 13th, October 14th. That's going to be where you have your last big push of whatever disinformation campaign is going on. I'm not a disinfo expert by any means, but if I were going to own stuff, I would definitely want to tell people. But like one of the tactics we've seen over and over again of people who are attacking election systems is that it's no good unless you tell people about it, unless you get noticed. And so you got to get noticed like early enough to sway elections, but not so early that, you know, there's enough from, you know, that Jack can fix it for us. So like October 15th, I think is the sweet spot for that. What is that? That's like Wednesday, I think, or Thursday. Non-Friday, non-Monday, early October, I would expect to see, I would expect to see big news to try to have that last push of, hey guys, don't bother voting. And you think that because that's when the mail-in voting window opens. That's when most states, in many states, absentee ballots are starting to get filled out then. Early voting starts then. And it's still early enough that you can make hay about it for the following two weeks. Like a losing side can call cyberfowl, pointing at that thing for, you know, and just eat up, eat up news for the rest of October. Just this is, you know, it's a bit personal, but it goes to one of the reasons that I'm not in the U.S. right now. I think, you know, we had the option to be near family and ride out the pandemic. You know, part of the concern that was in the back of my mind was how, you know, the potential for civil unrest and those sorts of things are amplified by the backdrop of the pandemic and economic depression, a lot of other stuff. So I think the number of things that are available for an actor to tweak on and the amount of leverage that's present, you know, as we do version two of this panel is radically different to what it was, you know, last time we got together and spoke. So, you know, from a mitigation standpoint, it really does come back, you know, for the typical audience of this panel in DEF CON is making sure that you're not adding to the problem. You know, the whole idea of, like, polarization, of just general distrust, like, nihilism, all of that sort of stuff. And I do believe, like, we are talking, you know, Armageddon-ish type stuff at the moment, but I do believe fundamentally in, like, working back from the worst case scenario and optimizing the critical path from there. So it's an important conversation to have. Yeah, that's a good segue into the last question we're going to do today. So, you know, obviously, we talked about mail-in voting as the next best mitigation for forcing people to kind of show up in person, and definitely a better alternative than, I'd say, any potential half-baked e-voting solution at the last minute that would come in and swoop. But obviously, with the rhetoric that's been, you know, spoken by various folks in the press from various levels of government and elsewhere about the validity and trustability of the Postal Service, as well as their own financial woes imposed upon them by Congress and pre-funding and so forth and so on, you know, it was just announced today that they had worked out a deal with a massive infusion slash loan from the U.S. Treasury. I think it was, like, 15 billion dollars, which is a huge chunk of change. It does keep people from necessarily having to rush out and, you know, running for stamps. But obviously, I have reports from some of the locals here in the D.C. area, specifically Baltimore, about potential fallout from the recent Postmaster General coming in and saying, please delay first-class mail. Obviously, that puts a downward pressure on delivery of mail-in voting, as well as, you know, returning that and making sure that everyone hits with the deadlines with the postmarks. So where we sit here is our last best effort to run a secure election as a Postal Service. It is in dire straits. It has potential leadership that is working antithetically against the essentially constitutionally or, you know, state that the, you know, the Postal Service exists in. You know, what are the last bits here that we can ensure that that is functioning for us to go forward? Are there ways that maybe we move up early voting even sooner so that we kind of play into the logistics of extended timelines? Is it write your senators and make sure that, you know, mail is delivered in a timely fashion? Or, you know, some other aspect of it, you know, with that, and especially, you know, as you mentioned earlier, too, I think it was Todd talking about the expected timelines being a lot longer for us to hear the outcomes. You know, if we have this extended timeline, what's our expectations to actually, you know, hear what the outcomes are going to be given this? It would be great if states would extend their deadlines. Like, I was shocked to see that Texas, for all the hand-wringing Texas has been doing about mail-in ballots and, like, trying to make that hard, the fact that Texas then turned around and extended early voting was was a sweet surprise. You know, I don't know. We do things randomly here in Texas. Some things are great. Some things are not so much. But I guess that's, I guess that's just here local in Texas. I don't know. Like, I feel like everyone should, you know, mentally hug a postal worker today. They do a lot of really hard work. A lot of people depend on them for a lot of things. You know, they are, in fact, constitutionally enshrined. It's an Article I power of Congress to establish the post office. And the fact that it became a target for disenfranchisement is just mind-boggling to me. But I think that we can all agree that the post office is kind of a wonderful Americanism, really. Like, it was a largely, this notion of a single stamp that carries something across the country. Like, that is, I'm not really sure. It might be an English thing. It might be a British thing. But one or the other, it's pretty great. So yeah, like, I mean, if you have the opportunity to vote absentee, absolutely do it. You know, absentee voting, you can get all nerdy about it and say, like, well, technically you're violating, like, the secrecy of the ballot by doing that because someone can watch you vote and direct you vote and see that you vote correctly and see that you put the thing in and mail it away. Like, but that's almost a, like, that is so low on my list of problems when it comes to democracy is vote selling. You know, if it turns out that's a big deal, great. Like, let's go tackle that. But it is, it is not, that has not been a problem since, like, the 19th century, so. Yeah, I want to say, too, we've seen the current administration, we've seen the current administration actively attack the Postal Service on social media. And so, you know, I would ask folks to understand that I don't think there was a long game there, but I don't think that it's going to be unreasonable to see more attacks on the Postal Service from the current administration. The unreliability, the conspiracies about deals with Amazon, and then how Jeff Bezos ties into Amazon, and then with the Clintons, like, there's a lot of, there's a lot of stuff to unpack there. But I would say, you know, at the end of the day, like, these are feds, these folks are feds, they took the same oath to the Constitution that other feds take. They're there to just do their jobs every day. And the idea that postal workers themselves would be tampering with mail-in ballots is just kind of ridiculous. And it's completely insane. It really is ridiculous. And, and if there were one, it's a blip on the radar if the numbers are out there of folks voting, right? So, so let's just keep it all in perspective. And to understand the importance of the Postal Service, when I was younger, the DMV test had a question that said, if you get to a four-way stop, and there's a fire truck, an ambulance, a police car, and a mail truck, who has the right of way? And everyone assumed, like, the ambulance or the fire truck, but it was, in fact, the mail truck, because they are protected under, you know, the, the guys of the federal government. Certainly mail truck wouldn't go first, but they could. And also, if you hit a mail truck, you get into a lot of trouble, too, because you've damaged federal property. So, you know, you probably don't want to go out and take your angst out on vandalizing mail trucks or bothering postal workers. So I just... That is a hot tip. I feel like... Robbing post offices used to be a hangin' crime. Yeah, well, they've also got that Crime a Day book that just came out from the Twitter feed. I would hope that there's a chapter in there about weird stuff like that. So, all right, then last thoughts on what we see as our future here. What you'll be doing, what you hope others will be doing, and then where do you hope we will be? I can go and take this first. So, yeah, nothing I'm gonna say here, really, is anything new, I would say, that I haven't said. But, yes, in terms of what I'm doing, what CIS is doing, we are going to be working through and after the election to support election offices at the state and local level to ensure that they have what they need from a security perspective. And we're committed to doing that. In terms of, I think, what's maybe more valuable is what people watching this, what steps they can take and what steps they can recommend others to take. And this goes back to the two points that you have to be patient and you have to expect that things may go wrong. But that doesn't necessarily mean there has been interference and that doesn't mean that the election is invalid. So, be patient. Do not assume that results will come out on election day. It may take some time, but have faith that election officials are doing their best to have an accurate result, that we have process in place, that if interference does occur, we can identify it. And just, yeah, ultimately the main thing is that the people of America need to have faith in their own elections and that can go away without any actual tampering occurring, without any interference. Just if the people do not believe that their result was valid, then the result is not valid. So, I think to everyone watching this, just to understand that what you believe happened matters and just to understand their process in place, their committed election officials, the federal government is here to support that process and, yeah, let's hope that we have a smooth, free, and fair election in November. Sure. So, I mean, I guess, like, to just kind of reiterate what, you know, everyone else says, like, you know, the best defense against any election shenanigans is voting and voting in numbers that are too hard to push one way or the other. If people go and they vote, especially people who have historically been disenfranchised or haven't felt the need to go vote, you know, it is here at every election, but this election is literally the most important election of your life so far. And so, go vote and hopefully if enough people do that, you know, any kind of shenanigans will be drowned out by the overwhelming signal that we have. Me personally, not only am I going to vote early, I'm very excited to do that, but I'll be working the polls. It's gonna be crazy. It'll be November and there will be no vaccine. And so, I will be doing a lot of cleaning and hoping that not too many polls close that day. You know, there were poll closures. In our last election here in Texas, there weren't any poll closures like on the day of. You know, people did show up. They got enough recruits to come and do the thing. I did have a couple poll workers not come to my polling place, but we had enough people to pull it off. So, hopefully that will remain the case, you know, but that's gonna be November. So, who knows like what the pandemic will bring us. If it becomes impossible to vote in person in any kind of crowded way, then, you know, we'll just have to deal with that as it comes. But if you have the bandwidth and the health to throw in for a, what is a super fun, sounds boring, but it's actually pretty fun, like 14-hour day, go with the polling place. I'll go next. You know, I will say that I'm very much looking forward to returning to the U.S. And this is honestly a part of that. So, again, on a, you know, speaking to the subject matter, but speaking to it from a very personal standpoint, it's like my adopted country is trying to figure all this stuff out. And I'm looking forward to being past it is something that, you know, it's heavy. It's everything. So, aside from that, practically, you know, for the hackers, find out where people are asking for help. Go help them. Like, look for the stuff that people have already volunteered. You know, the volunteering stuff that we talked about at the start, just to reiterate that. Some of that help might be IT. Go looking for it. See if you can find opportunities to provide your skills into those different areas. Help out on the open-source projects and some of the other things that are going on that have been volunteered. So, Arlo, we mentioned before, verified voting. It's up on GitHub. Go bang on the source code. And if it's legit, say so. If there's a problem, submit a PR. Help make it better. If these audits are a part of how we have a peaceful kind of acknowledgment of the count after the fact, then you'll have played a pretty big role in that, I think. And then, finally, you know, don't scare your grandma between now and November. If you're doing security research and you find something, you know, talk to Jack and the crew at CSIR and CERT. Try to talk to the vendor. Just be very mindful of the fact that dropping any kind of anything that looks like a vulnerability on the internet right now is highly, highly exploitable from actors, from a disinformation standpoint, and you don't want to be a part of the problem. Technically, over the last couple days, because July sucked when it came to VOMS, or at least people had to clean up after VOMS. So, let's make August better. It's DEF CON month, so the internet's on fire this month anyway, but maybe after that, I don't know. October, let October be quiet then, that's cool. Kimber, any last? It's unprecedented, so it's anyone's guess what actually happens on Election Day and the months following. Vote, just vote. Tell your friends, tell your family, vote. Vote safely. Vote mail-in if you can. If you pay attention, your state may have deadlines or your district on when you have to let them know that you're going to be voting by mail-in ballot. Some folks here didn't understand there was a two-part process to mail-in ballot. You had to request one to receive one. It didn't automatically get sent to you. So, just being aware of how your local districts work when it comes to mail-in ballots so that you can participate. And if you have to go to the polls, wear a mask, social distance if you can, and you don't have high-risk folks at home, volunteer at your local polling places and do what you can to make it safe for our most vulnerable populations to be able to get out and have their voices heard. Well, yeah, that's a great thing to end on. Obviously, I have a spouse who's immunocompromised, so I've preemptively requested a mail-in ballot. I know some states require an extenuating excuse in order to get an absentee ballot, so please check with your local officials on what you can do to do that. Obviously, you know, a safe and secure voting is important, but safe and secure also means individuals as well. With that, I'll close out our panel. I do appreciate Kimber, Jack, Casey, and Todd for joining us this evening for the recording. Again, this was the election security part two. The infrastructure strikes back. While I can't necessarily drop in some John Williams here, use your mind's eye as well as the Starfield background to kind of get it through. And hopefully, come November, we'll see what kind of shakes out. And geez, maybe next February we'll have a cleanup on this, maybe a part three. Hopefully, it doesn't end up with a bunch of Ewoks running around saying ugnug. But anyhow, once again, thank you very much and thank you for your time. Take care.