Hi, my name is John Odom, and I'm currently the city clerk in Montpelier, Vermont.
In that capacity, I'm also the election administrator.
And, boy, has this been a year for that.
We've got our own statewide primary coming up in about a week and a half.
So, even taking a little time to record this is kind of challenging.
I'm also a certified ethical hacker.
I've been in charge of a couple networks and been a database administrator for various non-profits.
So, I tend to have a little more hands-on knowledge than most of my colleagues in the industry,
but not as much as you might think, because my life's a little busy and I don't get to practice this stuff very much.
Anyways, I want to bring something to folks who participate in the Voting Village's attention.
And it's not something that has been ignored, but I think it's something that deserves a little more attention.
To make the point about how concerned I am, I'm going to tell you a little story.
And this is not exactly on topic, but I want to use it to make a point.
Several years back in Memphis, and some of you may have heard this story,
there was a local election, the mayor was being elected, various local officials,
and one clever person decided to compare the turnout from the tape,
and most tabulators generate some sort of tape, some sort of physical record of the time listing the votes,
and compare that number with what was reported through the GEMS system.
And I know also a lot of folks who've been participating in Voting Village share some concerns about that system.
Well, the numbers did not add up. According to the tape, 546 people in this particular precinct had voted,
according to the tape, and the system only showed 330.
They looked at this, they looked at other precincts, they found the same problem,
especially in districts with heavy minority populations.
Big problem, it's the problem we talk about at DEFCON, but I want to present another scenario to you.
Consider this happening on a statewide level, not district by district, not precinct by precinct,
not through any one individual network of voting machines, but an entire state.
There are statewide systems that manage databases. This is mandated by the Help America Vote Act,
which came in the wake of the 2000 presidential election voting debacle.
You know, the hanging chads, the numbers that didn't add up, and it all went to the courts and it went to the Supreme Court.
After that, there was a bipartisan group that got together, federally mandated, and came up with several recommendations.
Now, generally, they were very good recommendations. I don't want to disrespect HAVA at all.
One of the recommendations, understandably, was that states should be working on their voter rolls through one centralized system,
one centralized statewide database that would contain all the voter registration information.
But obviously, you can see where I'm going with this. There are concerns about, or we have to be concerned, about the security of these systems.
So these statewide systems don't all necessarily just hold the voter registration information.
A lot of them, including in my home state of Vermont, are actually election management systems.
Election administrators will report their information for election night reporting.
Sometimes we will work directly into these systems to manage it, to create our reports, to create our elections and local elections, and manage them directly out of that.
So there's a lot going on with these systems. They are very important, and local administrators have come to really depend on them.
Well, how are they doing? Could be better. I want to talk about the famous Mueller Report.
Now, tucked away in that is this little gem that I'm going to read here, part of it.
It says, in addition to targeting individuals involved in the Clinton campaign,
Mueller's operation also targeted individuals and entities involved in the administration of the elections.
Victims included U.S. state and local entities, such as state boards of elections, secretaries of state, and county governments, as well as individuals that work for the entities.
They also targeted private technology firms responsible for manufacturing and administering election-related software and hardware, such as voter registration software and electronic polling stations.
Now, here's the scary part. The report says that foreign actors targeted state and local databases of registered voters using a technique known as SQL injection,
by which malicious code was sent to the state or local website in order to run commands.
In one instance, in approximately June of 2016, the working group was able to compromise the computer network of the Illinois State Board of Elections by exploiting a vulnerability,
presumably a SQL injection-related vulnerability. This gave them access to a database containing information on millions of registered Illinois voters.
The group extracted data related to thousands of U.S. voters before the malicious activity was identified.
Alright, so beyond just the obvious scary here, you know, okay, a lot of you folks will know that code injection is a very big deal.
Code injection is, you know, it's where most of the hacks come from these days. But SQL? SQL injection is something we have been aware of for many years.
We know how to harden against it. So my question is, why wasn't it already hardened against it?
SQL injection is very easy. One line fed through to a database from a simple login screen can get you in. We know input validation is the solution. So where the hell was the input validation?
What that says to me is, and you, again, to refer to Voting Village in the last few years, what's gotten a lot of attention was the simulation of the statewide voter database.
It's got national coverage that kids could sit down at this simulation and they could hack right into our dummy statewide voter registration system.
Well, of course, the pushback and all the yelling and hollering from the secretaries of state were that this was a phony simulation.
That their systems are actually far more secure. Somehow this was set up to be hacked to make their point.
Well, if SQL injection is a way to get into this stuff, I would argue that those systems, those dummy systems we made up, maybe were not loose and accessible enough.
So, all right, so let's talk about HAVA, the Help America Vote Act.
And I'm going to read a little bit from it.
Part of the mandate is each state acting through the chief state election official shall implement in a uniform and nondiscriminatory manner a single uniform official centralized interactive computerized statewide voter registration list defined, maintained and administered at the state level.
So states have no choice but to do this. And again, populations, what they are, it's completely understandable.
So what do states do? Well, several do in-house development.
In Vermont, we used to, our first database, our first election management system out of the box, out of HAVA was an internet facing FoxPro application.
It was pretty crude.
But these days, and other states currently are still doing in-house solutions, states like Colorado, Illinois, Kentucky.
You could put together something pretty nice if you knew what you were doing.
Obviously, then the networks they sit on, the municipal networks, are potentially vulnerable too.
But if you can get in by SQL injection, why bother?
Okay, but more and more often, you see these states using vendors.
And these are vendors, you know, it's like any niche application, any niche market, you're going to get niche vendors who pop up specifically to serve that market.
So before this, we had somebody call around. I wanted to know what states used what.
How many of them used one vendor versus another? How many of them were designing in-house?
I don't know if I can really say this, but word got out that we were calling around and there were people who were less than thrilled.
And defensiveness in public systems should really make us all uncomfortable.
Now, I want to look at one particular vendor right now, because I have a little more firsthand familiarity with it, called PCC.
Now, here's a list from their website of their current clients.
Okay, now factoring out for the consulting only options here, that's 15 states doing database management in election application hosting for voter registration, election night reporting.
So who is PCC? Well, this is part of the problem.
Who knows? Go to PCC's website and no staff is listed. Only board officers, not even the whole board.
Now, I didn't do research on people who did show up. I probably should have, I could have, but I didn't. I don't want to knock them without any basis or unnecessarily then.
And, you know, they seem to be perfectly reputable people. I didn't see any, the one I did look at, I didn't see any obvious big political connections.
CEO Tom Ambergy is a big, was a big shot at Central Square Technology, for example, and they were recently at a major hack, a Magecart attack.
Now, you know, I don't want to beat them up too much for a Magecart attack. They're good.
But Magecart attacks generally use JavaScript injections. So it's something that could be hardened for.
And with election systems, there's just no margin for error. You've got to be ahead of these games.
So I wouldn't say that was a big concern of mine, but it does raise my eyebrow a little bit.
But let's look at some of these proposals that PCC or organizations like PCC have put forward to the states to try to get their business.
I want to show you the one from Delaware and just pieces of it. It's, of course, very, very long.
But I'm going to show you here a typical page from the publicly posted proposal from PCC to run their election systems.
OK, when you look at this, this is what you'll see. Not much.
I, you know, I didn't do the measurement and everything, but from my glance, I would say at the most about 20 percent of this entire proposal is visible.
So that's me being generous. Now, of course, I understand companies have proprietary information.
They have proprietary stuff that's standard for any RFP. You expect that.
But come on. Why even post it at all? I mean, this is something made available to the public, you know, public records.
But it's not. I mean, it's it's it's almost a joke. If I didn't know better, I'd say it was almost passive aggressive.
But what bothers me the most in terms of the redactions is all the staff is redacted. And this is typical.
So just like the website, we don't know who's working on this.
And that bothers me a lot because people have their own interests.
They come from backgrounds, partisan backgrounds, nonprofit backgrounds.
I think it's reasonable. And I don't think it necessarily, you know, reveals any particular corporate secrets that we could have some idea who the people are doing this stuff,
either in an individual state or even at the company proper.
Now, financials are also redacted. I know that's a thing that's very standard, but I would argue that it shouldn't be.
I think our right to know trumps any embarrassment or discomfort of big companies.
And publicly, we might not want one that's on the edge of bankruptcy.
And we might want to see that. So so here is the problem.
There are very few companies doing this. And they are opaque.
We don't know who they are. And that's basically it. We don't know who they are. And that's scary.
That's very, very scary, at least to me. And it does matter.
In Georgia, they had a recent problem, a debacle involving their voter rolls, involving the voter registration.
Now, this wasn't exactly what I'm talking about.
But it makes the point about how badly you can screw up an election simply by screwing up voter rolls.
You can disenfranchise people in a big election crush.
They're just not going to get to vote or they're going to have to fill out an enormous amount of provisional ballots,
which honestly might not necessarily get counted the way they should be.
Now, during a court case involving this whole debacle, a lot of insecurities,
a lot of vulnerabilities in the PCC system was was brought to the attention of the court.
After that, Georgia decided to pull back. Their contract ran out and they decided to pull it back in-house.
So that's a pretty unusual step to take. And it shows you just how concerning those vulnerabilities were.
Now, you know, we've got our public officials doing this.
Can we count on our public officials to be straight with us about this stuff if there's a problem?
Well, obviously not. And again, I don't mean to knock secretaries of states, but they have their own interest.
They have an interest in getting reelected, and that means they have interest in looking competent.
Now, some of them like to talk about internal security a lot. You know, we've upgraded this to make it better.
We've got better voting machines to make this better.
We're doing our due diligence within the sphere that we control in order to do a better job and get reelected.
They don't like to talk about the potential for things outside.
And that means they don't necessarily like to talk about the walls that they've built that they're responsible for
that are, you know, the dividing the firewall, pardon the expression, between the voters and the outside world where you could have malicious actors.
And again, with the Mueller report, we're talking about advanced persistent threats.
We're talking about state actors, but not necessarily.
So secretaries of state have a vested interest in saying everything's rosy and everything's wonderful.
So that, that is a problem.
So let's look at some of the other systems. I've been picking on PCC, but there are other systems out there.
And folks might recognize, Voting Village might recognize one of the other major, probably the other major election management system that's out there.
It's from ES&S.
ES&S are our old friends. For years, Voting Village has been hacking their machines.
And they, more than any of their companies, have been the most belligerent.
Last year, I believe it was, they actually had folks sort of roaming about trying to make people uncomfortable about hacking, suggesting they shouldn't.
These were the last folks to come around and say that Voting Village had a point.
And it was only after so much coverage. Again, I think it was last year.
These folks are not good partners. They are not reliable partners in our experience.
I don't want to get sued for slander here, but in my opinion, based on what we've seen, these are not good partners.
And we will remember how quickly the National Association of Secretaries of State last year was right there to defend them on their own terms, on ES&S's terms, and in some language that looked a little bit much like their own words sometimes.
So, scary. Scary stuff.
And are these things we can test in Voting Village?
The way we take apart the voting machines?
Of course not. Of course not.
We can make our own, you know, dummy systems like we have, and we should, and there's a lot to be made there.
But we can't go and test these systems. It's hacking. We can't hack, right?
So, I've thrown a bunch of terrible stuff at you here.
The question is, what do we need? Do I have any solutions here?
Well, first of all, transparency. Transparency, transparency, transparency.
We should know who these people are. We should know who runs them, what their interests are.
We should know what their background are.
We should know, we should need to know whether these folks even have the competence to do what they claim to do.
And I'm going to talk a little bit about the Iowa debacle,
where the, during the caucuses, or the primary caucuses this year,
the Democratic Party was using these little specially designed, custom designed apps, mobile apps,
to report the results of the caucuses to a central place.
You all probably heard about this. They were a disaster. They were a disaster.
It took a very long time to sort of rebuild the mess that they created,
and actually generate a final voting tally.
It was a big embarrassment to the Democratic Party.
They were going to be using the same systems in Nevada, and they pulled that out.
Now, what I would say, the biggest problem, conceptually, with that application,
was that it was made by, and you see this a lot,
made by folks in the industry who made a lot of personal connections.
You know, these were folks who had worked for the Democratic Party, done IT stuff,
and they decided to go out on their own, and they made a crappy product.
But the crappy product was bought up because, oh, we know these people.
These people are in our industry. They're in our world. We know them. We trust them.
So, I don't know if the same thing isn't going on with some of these voting applications.
PCC, who knows who ES&S hired? That is a real, real problem in this industry.
So, we need to know who's doing it, because we need to know if they're competent.
If they're reaching out for the best people, or if they're just reaching out for people who are connected with the company.
Which gets us back to another point. This needs to be opened up.
If we've got one, or two, or three companies doing this, that gives those one, or two, or three companies a lot of power.
We need to have more genuine RFPs get this out there.
I'm not saying there are angelic companies out there, but mixing it up a little would help.
Now, second, we've got to test this stuff.
I'm sure the Secretaries of State do pen tests on their systems, but it sure didn't help for 2016.
I don't know how a pen test misses a SQL injection attack.
There are pen tests, and there are pen tests.
And again, the Secretaries of State have a vested interest in not finding vulnerabilities.
So, and I've seen some of this, the pen tests tend to be minimal.
So, we talk about standards for voting machines.
We should have standards for these systems, and reports should be publicly available.
We need standards for penetration testing.
And that includes testing for social engineering all the way down to the user level.
All the way down to the level of the election administrator who, you know, has their own account and talks to this database system.
A thorough pen test is going to include tests for social engineering, and you don't generally see that.
Especially when you consider any kind of code injection.
Notwithstanding, the biggest problem will always be malware.
It will always be malware.
You can't really audit these systems.
So, we need those standards and, again, transparencies and transparency.
Yeah, you have to redact most of a pen test.
Sure, and that just makes sense.
But we could at least be able to see the executive summary of these pen tests as a public document.
So, I would argue standards. I would argue, again, transparency.
Those are the big two words. Standards, transparency.
That gets us a long way.
Now, I cannot stress how much of a problem this is.
You know, if we're talking about voting machines and systems, which we love to talk about and we need to talk about.
A presidential election on those terms would be hard to tank just by going after voting machines.
But not these systems.
There may be thousands.
I think 7,000 is what I read. Localized voting machine tabulation systems.
You know, that's a lot to get into and hack, although not for a local election.
And local elections are every bit as important as national elections. It's all democracy.
Although, again, not impossible.
But when you're talking about these online databases and these internet-facing user election management systems,
we go from 7,000 targets down to 50. 50 targets.
That's a lot more appealing. It's a lot more dangerous.
And these systems interact.
First of all, most states, you're going to see some kind of connection or some kind of ties to the DMVs, to the Department of Motor Vehicles.
You know, we always have to check off for, do I want to be registered to vote too?
There are states like Vermont, I'm proud to say, that have automatic voter registration.
So those systems have to talk to each other some ways.
There are safe ways to do it. There are unsafe ways to do it.
We don't know, again, how they're doing it.
Tax departments. Some states even connect to their tax departments.
So, obviously, any big extensive network-to-network, they're going to be as strong as their weakest links.
And those weakest links could be in the statewide networks talking to each other.
The weakest links, more often than not, are the users.
In New England, you can have election administrators running jurisdictions of as few as 70 people.
They don't have a lot of good equipment, and they don't necessarily have a lot of sophistication in how to do proper hygiene,
guard against social engineering, against spear phishing.
I mean, if I were somebody, I'd take aim at one of those folks, and I'd go right after them.
There's also ERIC, which is something I don't know about.
ERIC is a statewide system, which now covers about half of the states.
I think it's roughly 25, with more looking at it, whereby states interact their databases so they can track cross-state registrations.
As it is, it's been a challenge for those states to take someone off a roll in one state because they registered in another state.
That's a challenge. It's a weakness that a lot of folks have made a lot of fuss about, and honestly, they probably should.
That stuff has been a matter of sending a piece of paper or an email from one secretary of state's office to the other.
So, obviously, you're going to have systems like ERIC popping up.
Now, my understanding, which is limited, is that in ERIC, you're not having a situation where the statewide databases are talking directly to each other, which is great.
But, obviously, again, there is the malware issue. Malware can ride along with all kinds of things.
And at any given time, a third or more of the malware out there in the wild could be zero-day.
So, you know, it's the same problem we have with antivirus systems. They can only be so up-to-date. They can only be so current.
So, anyway, I'm not going to say, I hope I didn't scare you. I hope I did. That's why I'm here.
I think that's why a lot of us are here, are to scare people into action and to scare people into making things better.
Again, I'm not trying to knock anybody down. I'm just trying to draw a lot of attention to this problem.
And this is a problem we could go into a lot more technical detail on, and, you know, I could talk for, you know, an hour or two if I wanted to.
But I want to keep this accessible. I know for the election administrators who are watching this, and also I only have about 20 or 30 minutes to do it.
But it's a conversation I think we need to have a lot more of.
Thanks. Thanks very much for listening. Thanks for Voting Village for having me, and hopefully I'll see you next year, maybe even in person.