1
00:00:00,000 --> 00:00:02,640
You've been working so hard.

2
00:00:02,860 --> 00:00:04,140
Thank you, thank you.

3
00:00:04,380 --> 00:00:05,640
All right, folks.

4
00:00:05,640 --> 00:00:12,000
Looks like the stream is active, and I'd like to thank our speaker again for this presentation.

5
00:00:12,000 --> 00:00:13,540
Take it away, Reem.

6
00:00:14,140 --> 00:00:16,440
Okay, thank you.

7
00:00:16,640 --> 00:00:26,000
So, I'm Reem Buznek, as Steve already said, and I will be talking about the security of Pennsylvania election systems.

8
00:00:26,000 --> 00:00:32,800
This was a project that I conducted under the supervision of Professor Andrea Mitwishin.

9
00:00:33,080 --> 00:00:38,160
Can you go to the next slide, Steve?

10
00:00:38,600 --> 00:00:40,160
Yep, I got you.

11
00:00:40,280 --> 00:00:40,900
Thank you.

12
00:00:40,900 --> 00:00:42,780
This is Professor Andrea Mitwishin.

13
00:00:42,780 --> 00:00:47,640
I believe that everyone knows her, but it was good to put that picture there.

14
00:00:47,640 --> 00:00:51,780
And then, if you hit again, Steve, you will find my face.

15
00:00:53,700 --> 00:00:55,440
There you go.

16
00:00:55,440 --> 00:00:57,580
And who am I?

17
00:00:57,580 --> 00:01:00,340
I'm a law graduate.

18
00:01:00,340 --> 00:01:02,420
I had two master's degrees.

19
00:01:02,420 --> 00:01:11,900
I focused on, in the first one, I focused on internet regulations and freedom of expression in the MENA region, the Middle East and North Africa.

20
00:01:11,900 --> 00:01:18,440
And in this second master's, I focused on national security law and information security law.

21
00:01:18,440 --> 00:01:22,960
And I was working with Professor Mitwishin as a research assistant.

22
00:01:23,020 --> 00:01:28,700
And hopefully, and very soon, I will be joining the pilot lab as a postdoctoral fellow.

23
00:01:30,040 --> 00:01:36,640
So, I'm going to tell you now what we're going to be talking exactly about.

24
00:01:36,720 --> 00:01:43,960
First of all, I will start with a brief introduction to the history of Pennsylvania election system security.

25
00:01:43,960 --> 00:02:03,800
I will tell you a little bit about some fraud cases, some problems in the information in the voting systems, et cetera, and how the new administration improved the current election system.

26
00:02:03,840 --> 00:02:18,860
Then I will tell you more about the core of the project, which is more about sending write-to-no requests to every county in Pennsylvania, all of the 67.

27
00:02:18,940 --> 00:02:26,760
And then asking for copies of election system vendor agreements, related documents, technical specificities, et cetera.

28
00:02:26,760 --> 00:02:44,900
And then in the third part of my speech, I will tell you about my preliminary findings based on the review of the agreements and I will tell you all about how many documents we received, what types of documents, et cetera.

29
00:02:45,720 --> 00:02:47,680
You go, Steve.

30
00:02:48,560 --> 00:02:49,960
Thank you.

31
00:02:49,960 --> 00:02:54,740
So, as I said, I will start with the election history in PA.

32
00:02:54,740 --> 00:03:01,960
So, according to the Heritage Foundation, there have been only 23 fraud cases in the history of elections.

33
00:03:01,960 --> 00:03:08,440
But when we say only 23, it doesn't mean that they're not important.

34
00:03:09,360 --> 00:03:16,320
So, I gathered some examples of fraud cases in Pennsylvania.

35
00:03:16,320 --> 00:03:19,160
The first one is about Robin.

36
00:03:19,160 --> 00:03:30,060
Robin is a judge of elections and with the help of Laura, who is an election inspector, she signed under her son's name and cast a ballot in his place.

37
00:03:30,060 --> 00:03:35,120
So, it's an impersonation fraud, as opposed.

38
00:03:35,120 --> 00:03:37,960
The second case is about this guy, Mr.

39
00:03:37,960 --> 00:03:43,560
Victor, who voted in both Philadelphia and Lehigh County.

40
00:03:43,560 --> 00:03:46,000
So, he casted two ballots.

41
00:03:46,160 --> 00:03:50,520
The third case is Mr.

42
00:03:50,520 --> 00:03:51,180
Richard.

43
00:03:51,180 --> 00:03:57,940
Actually, he's a former police chief of Harmon Township in Allegheny County.

44
00:03:57,940 --> 00:04:05,720
And he applied for absentee ballots and made people not eligible for the absentee ballot to cast them.

45
00:04:05,720 --> 00:04:12,320
And in a while, I will tell you about the absentee ballot and what it exactly means.

46
00:04:12,800 --> 00:04:21,100
A fourth example is the former congressman, Austin Murphy.

47
00:04:21,100 --> 00:04:26,040
He actually also committed absentee ballot fraud.

48
00:04:26,040 --> 00:04:32,580
He made senior citizens in nursing homes to vote in a certain way, the way he wanted.

49
00:04:32,580 --> 00:04:36,120
But he said that he was just helping them.

50
00:04:36,280 --> 00:04:37,980
So, we believe him.

51
00:04:38,000 --> 00:04:40,980
And the last case is Harry.

52
00:04:40,980 --> 00:04:43,520
It's a new one in 2019.

53
00:04:43,520 --> 00:04:45,820
And he actually picked up girls.

54
00:04:45,820 --> 00:05:13,140
He made girls sign absentee ballots in the names of deceased people, which makes me a little bit sorry because with the advanced systems of ensuring that there are no duplicate registration, there are still, until 2019, it is still possible to impersonate a deceased person.

55
00:05:15,640 --> 00:05:18,040
Next slide, please.

56
00:05:18,780 --> 00:05:29,580
So, the problems in the election system in Pennsylvania isn't only about fraud cases, but also about election system malfunctions.

57
00:05:29,580 --> 00:05:34,860
So, these are some examples prior to 2005 that I gathered.

58
00:05:34,860 --> 00:05:40,660
And they're mostly about the DRE malfunctions.

59
00:05:40,660 --> 00:05:51,820
DRE, it means direct recording electronic voting machines, which means that these voting machines do not use paper ballot systems.

60
00:05:51,820 --> 00:06:01,400
So, the only record of the votes that have been cast is electronic.

61
00:06:01,400 --> 00:06:05,380
That's why there was this case in 2004.

62
00:06:05,380 --> 00:06:14,380
DRE records 83% blank presidential votes, which doesn't make sense.

63
00:06:14,380 --> 00:06:19,020
And that's why they knew that there were these malfunctions.

64
00:06:19,040 --> 00:06:24,340
And this is a funny story, another one in 2012.

65
00:06:24,380 --> 00:06:31,740
There is this person who actually took a video with his smartphone of the malfunction.

66
00:06:31,740 --> 00:06:45,000
He made a video of the voting system, how it changed his vote from Obama to another candidate.

67
00:06:45,240 --> 00:06:47,830
Next slide, please.

68
00:06:51,160 --> 00:07:00,800
So, before the enactment of HAVA, Pennsylvania's election law provided new statewide guidelines.

69
00:07:00,800 --> 00:07:11,920
Basically, all the 67 counties were autonomous, and they had different election systems using a variety of voting machines.

70
00:07:11,920 --> 00:07:14,980
The HAVA is the Help America Vote Act.

71
00:07:14,980 --> 00:07:28,300
It was enacted in 2002, and it created a new mandatory minimum standards for states to follow in several key areas of election administration.

72
00:07:28,300 --> 00:07:56,580
It also required states to establish a statewide voter registration database, and it made the state follow certain systems over others and made high standards for the voting machines that can be used for elections.

73
00:07:56,600 --> 00:08:13,800
About the statewide voter registration base, Pennsylvania has a top-down system, which means that it makes a central platform at the state level that connects with all the local jurisdictions.

74
00:08:13,800 --> 00:08:15,680
So, it goes top to down.

75
00:08:15,680 --> 00:08:21,200
They gather information on a state level.

76
00:08:21,200 --> 00:08:30,920
Also, before, there weren't any information security training programs before 2018.

77
00:08:30,920 --> 00:09:08,840
And now, since many requirements became, like many things became required under Act 77, I will tell you about it in a while, companies started having the responsibilities and the obligations to provide their customers with training regarding the usage of the voting systems or how to maintain their security.

78
00:09:09,440 --> 00:09:21,300
The other thing is that, so, as I said, the DRE voting machines do not use paper ballots.

79
00:09:21,300 --> 00:09:27,040
So, the only records that are kept are electronic.

80
00:09:27,040 --> 00:09:33,940
And now, Pennsylvania has changed all its voting systems, voting machines.

81
00:09:33,940 --> 00:09:39,200
Over 25,000 voting machines have been replaced.

82
00:09:39,200 --> 00:09:46,740
And the governor made them use all voter-verifiable paper records.

83
00:09:46,740 --> 00:10:03,440
And actually, it helped, because I will tell you in a little bit about some cases in 2019 that some malfunctions happened in the voting machines and they had to turn to those paper ballots.

84
00:10:04,620 --> 00:10:07,020
Can we go to the next slide?

85
00:10:07,020 --> 00:10:08,260
Thank you.

86
00:10:08,260 --> 00:10:13,360
So, the new administration brought improvements.

87
00:10:13,360 --> 00:10:22,160
First, by purchasing new, more accessible, auditable, and secure voting systems.

88
00:10:22,160 --> 00:10:40,990
The second thing is that it enacted Act 77, which I will talk a little bit more about in a minute.

89
00:10:40,990 --> 00:10:55,270
But thanks to this Act, all Pennsylvanians can now vote by mailing ballots without having to provide an excuse, which helps people to register more.

90
00:10:55,270 --> 00:11:06,050
And because the way it used to be, it was that you can only cast your ballot through the mail if you have an excuse.

91
00:11:06,050 --> 00:11:07,850
Now, you don't need an excuse.

92
00:11:07,850 --> 00:11:10,250
You just apply for it.

93
00:11:11,490 --> 00:11:20,430
Also, now counties make election officials do random tests of voting machines on election days.

94
00:11:20,430 --> 00:11:30,290
And they incorporated added checks and other improvements into the SURE system to reduce data errors and improve accuracy.

95
00:11:30,290 --> 00:11:38,570
About the SURE system, so it's an abbreviation of Statewide Uniform Registry of Electors.

96
00:11:38,570 --> 00:11:49,310
And it's a system that Pennsylvania has to assist the counties with training and user development, continuous improvement, etc.

97
00:11:49,310 --> 00:12:15,290
And there were, so as we talked a little bit earlier about impersonating deceased people, so developing the SURE system helps minimize the number of duplicates or the use of registered deceased people.

98
00:12:15,450 --> 00:12:17,870
Next, please.

99
00:12:18,330 --> 00:12:19,430
So here you go.

100
00:12:19,430 --> 00:12:22,630
Act 77, election reform bill.

101
00:12:22,630 --> 00:12:32,890
It was enacted in 2019, so mainly the biggest thing about this bill is that it allows mail-in ballots.

102
00:12:32,890 --> 00:12:38,750
As I said, anyone can apply for a mail-in ballot.

103
00:12:38,750 --> 00:12:48,350
So basically, if you are eligible to vote, you are eligible to apply for a mail-in ballot.

104
00:12:48,350 --> 00:12:55,990
And you can apply for it until the Tuesday before the election.

105
00:12:55,990 --> 00:12:59,850
The absentee ballot is the same as it was.

106
00:12:59,850 --> 00:13:20,430
If you have a disability or illness, if you are in the military, if you have a religious celebration that conflicts with the date of the elections, you can apply for an absentee ballot.

107
00:13:20,430 --> 00:13:23,910
So this is for people with an excuse.

108
00:13:27,840 --> 00:13:42,700
And under Act 77, the shortcut straight party button, which is also known as the straight ticket to vote for all candidates of one party, is no longer available.

109
00:13:42,700 --> 00:14:02,000
And there was a lot of tension about this because both sides say that it will make the voter reluctant as to vote for all candidates and look for who is Republican, who is Democrat.

110
00:14:02,160 --> 00:14:04,360
But it was taken off.

111
00:14:04,360 --> 00:14:12,220
And also, another thing that was prohibited is the use of stickers for writing candidates.

112
00:14:12,360 --> 00:14:21,800
Writing candidates are people who you may vote for despite not being in the ballot.

113
00:14:21,800 --> 00:14:34,520
And what I mean by sticker is that in their campaigns, these people can distribute some stickers to the people having their names on.

114
00:14:34,520 --> 00:14:46,620
And it used to be that the voters can just put the sticker on the paper ballot and it will be counted.

115
00:14:46,620 --> 00:14:57,820
Now, the stickers are prohibited to be used and the stamps are very discouraged to be used because of the wet ink.

116
00:14:57,820 --> 00:15:00,800
But it's not prohibited under Act 77.

117
00:15:02,160 --> 00:15:04,280
Next slide.

118
00:15:08,070 --> 00:15:17,170
So the use of mail-in ballots and this new Act encouraged more people to enroll in the elections.

119
00:15:17,170 --> 00:15:31,090
So this year, 1.8 million Pennsylvanians applied for mail-in ballots and it is 17 times more than the elections of 2016.

120
00:15:31,790 --> 00:15:41,170
But there is a problem with that, is that the current election code doesn't allow election officials to begin counting ballots until 7 a.m.

121
00:15:41,170 --> 00:15:42,750
on the election day.

122
00:15:42,750 --> 00:15:55,730
And election officers are worried that they wouldn't be able to meet the deadline and to announce the results the night of November 3.

123
00:15:55,730 --> 00:16:08,590
And even the former governor is calling the election officials to make it possible to start counting the ballots earlier.

124
00:16:08,590 --> 00:16:11,270
But we will see what happens.

125
00:16:11,350 --> 00:16:26,870
And even with all these improvements, some counties still experience election system malfunctions in this June elections.

126
00:16:26,870 --> 00:16:28,670
The next, please.

127
00:16:29,670 --> 00:16:35,730
So many problems, many malfunctions were reported.

128
00:16:35,810 --> 00:16:38,430
I will give you three examples.

129
00:16:38,430 --> 00:16:43,630
York County, Northampton County and Bucks County.

130
00:16:43,630 --> 00:16:59,970
In York County, the paper ballot system was defective because the ballots were printed on shorter paper and the scanners could not read those ballots.

131
00:16:59,970 --> 00:17:06,840
And for some papers, some other papers, it took longer than it should be to read those ballots.

132
00:17:06,840 --> 00:17:18,660
The case is worse in Northampton County, because in this case, there was a problem with the tabulation in some of the precincts.

133
00:17:18,660 --> 00:17:26,640
And the state ordered the county to count the paper ballots rather than rely on machine totals.

134
00:17:26,640 --> 00:17:47,360
And why it says human error late, it's because... so the manufacturer later explained and told everyone that 50% of the machines were actually improperly manufactured.

135
00:17:48,680 --> 00:17:51,680
So who to blame?

136
00:17:51,900 --> 00:17:55,240
And the last example is Bucks County.

137
00:17:55,240 --> 00:18:02,440
In Bucks County, there was also a problem with the paper ballots because of the irregularities.

138
00:18:02,440 --> 00:18:12,540
And the irregularities made the papers a fraction of an inch too wide to fit into the new built-in machine scanners.

139
00:18:13,540 --> 00:18:21,800
So before, the problem was we do not have paper ballot machines.

140
00:18:21,800 --> 00:18:31,260
Now we have them, but the papers they used, they didn't make sure that the paper ballots fit the scanners.

141
00:18:31,260 --> 00:18:36,720
Now whose problem is this?

142
00:18:36,720 --> 00:18:53,940
It should be the manufacturer, because they should provide the counties with all the systems and everything that goes with them to perform as expected.

143
00:18:57,550 --> 00:19:00,310
Can I have the next, please?

144
00:19:01,090 --> 00:19:14,630
So I told why I talked in the beginning about this in order to tell you a little bit why we made this project, why we started this project.

145
00:19:14,630 --> 00:19:19,250
Because there is a natural problem in the election system.

146
00:19:19,250 --> 00:19:26,150
There are malfunctions and they are affecting the polls and the counts.

147
00:19:26,150 --> 00:19:29,150
So we decided to do this project.

148
00:19:29,150 --> 00:19:30,850
Next slide, please.

149
00:19:30,850 --> 00:19:47,550
We decided to do this project and to have an idea about the contracts that have been signed between the counties and the voting system companies.

150
00:19:47,550 --> 00:19:58,690
We also wanted to have a sense of which systems are used in which counties, because there are a lot of different systems used.

151
00:19:58,690 --> 00:20:17,570
And we also wanted to have a closer read on the agreements in order to see what are the system vendors' duties towards the counties and how would they assure maintaining the information security of the systems.

152
00:20:17,890 --> 00:20:19,750
That's our background.

153
00:20:20,210 --> 00:20:21,770
Next, please.

154
00:20:22,330 --> 00:20:25,610
So how we started doing this?

155
00:20:26,290 --> 00:20:39,250
Pennsylvania has a right to know law, number 65, and it provides for access to public information obtained from local government or state agencies.

156
00:20:39,250 --> 00:20:44,990
Pursuant to this law, we sent 67 requests, one to each county.

157
00:20:44,990 --> 00:20:56,090
We started on the 15th of April, 2020, and most counties invoked the 30 days extension.

158
00:20:56,230 --> 00:21:06,450
And we are still waiting for many counties to respond and to provide us with the documents we requested.

159
00:21:06,450 --> 00:21:11,090
And in a little bit, I will give you the exact numbers.

160
00:21:11,620 --> 00:21:14,990
So here are the responses we got.

161
00:21:15,190 --> 00:21:19,950
The counties varied in dealing with our right to know request.

162
00:21:20,170 --> 00:21:34,130
44 out of 67 counties provided us with the documents, but they also varied in what documents they provided us with.

163
00:21:35,030 --> 00:21:47,810
And 14 counties invoked the 30 days extension, but they still didn't provide us with the documents.

164
00:21:47,810 --> 00:21:54,970
And it has been three months now, so maybe we should re-request.

165
00:21:57,130 --> 00:22:02,270
So seven counties didn't respond at all to our request.

166
00:22:02,270 --> 00:22:08,710
And let me, just to clarify, all the requests were sent by email, online.

167
00:22:08,710 --> 00:22:17,150
And we made sure, and I made sure, that we contacted the right people, the open records officers.

168
00:22:18,750 --> 00:22:24,470
And then two counties were hard to reach.

169
00:22:24,470 --> 00:22:34,290
One didn't have online applications, so we will have to send them a mail or a fax.

170
00:22:35,250 --> 00:22:37,650
Yeah, they still use that.

171
00:22:37,650 --> 00:22:43,130
And two counties, only two counties requested payments.

172
00:22:43,130 --> 00:22:51,250
So these counties requested payment for printing and for the flash drives, not for anything else.

173
00:22:51,250 --> 00:22:57,070
All the other counties were nice enough to send us everything without requiring any payment.

174
00:22:57,750 --> 00:22:59,550
Next please.

175
00:23:00,550 --> 00:23:06,590
So as I said, now I was talking about 67 counties.

176
00:23:06,590 --> 00:23:20,590
Now I'm only going to talk about 44 counties, the counties that provided us with the documents and the ones that responded to our requests.

177
00:23:20,590 --> 00:23:28,310
I noticed that there are four companies the counties have been dealing with.

178
00:23:28,310 --> 00:23:34,910
Dominion Voting Systems Incorporated, 10 counties signed with them.

179
00:23:34,910 --> 00:23:41,430
Election Systems and Software, ES&S, 25 counties signed with them.

180
00:23:41,430 --> 00:23:46,870
Clear Ballot Group, seven counties signed with them.

181
00:23:46,870 --> 00:23:52,570
And Election IQ, two counties signed with them.

182
00:23:53,450 --> 00:24:04,090
And let me say that the terms of the contracts that we receive vary from a company to another.

183
00:24:04,250 --> 00:24:07,430
They don't all have the same terms.

184
00:24:07,430 --> 00:24:11,330
And I will be talking about the terms in a little bit.

185
00:24:11,330 --> 00:24:14,070
Can we have the next slide, please?

186
00:24:14,070 --> 00:24:15,210
And here we go.

187
00:24:15,210 --> 00:24:17,770
The documents received in total.

188
00:24:17,770 --> 00:24:28,070
As I said, each county sent us the documents they assumed fit.

189
00:24:28,070 --> 00:24:37,870
So we received 40 sales agreements, 14 sales orders, 11 purchase orders, four invoices, 10 bids.

190
00:24:37,870 --> 00:24:44,830
About the bids, some counties sent us only the bid of the company they signed with.

191
00:24:44,830 --> 00:24:52,370
And some counties sent us all the bids they received from all companies and the agreement.

192
00:24:52,670 --> 00:24:56,210
Six counties sent us the manuals.

193
00:24:56,210 --> 00:25:09,030
By manuals, I mean the documents containing technical specifications about the system, the voting system used.

194
00:25:09,310 --> 00:25:14,350
14 counties sent us marketing materials.

195
00:25:14,350 --> 00:25:26,910
Marketing materials are mostly flyers and papers showing how exactly as a voter you use the election system.

196
00:25:26,910 --> 00:25:28,730
So nothing elaborate.

197
00:25:28,730 --> 00:25:32,210
Two reports, interior reports.

198
00:25:32,210 --> 00:25:39,630
So they made a diagnosis before the usage of the voting machines.

199
00:25:42,210 --> 00:25:44,790
So they sent us the reports.

200
00:25:45,090 --> 00:25:47,350
Two counties sent us their loan agreements.

201
00:25:47,350 --> 00:25:55,710
So they had loan agreements in order to be able to buy and purchase the new voting systems.

202
00:25:55,710 --> 00:26:01,130
Two counties sent us their codes and two counties sent us product schedule.

203
00:26:01,690 --> 00:26:08,330
So we were lucky enough to have at least 40 agreements at hand.

204
00:26:08,530 --> 00:26:17,830
Now, about the NDA, 18 counties invoked NDA.

205
00:26:17,830 --> 00:26:27,750
And it was mainly about the technical specifications of the voting machines.

206
00:26:27,750 --> 00:26:31,750
They mostly, they all provided us with the agreement.

207
00:26:32,270 --> 00:26:49,090
But some, so 18 of them didn't provide us, said that they cannot provide us with the technical specificities because of a non-disclosure agreement.

208
00:26:49,270 --> 00:26:58,910
16 did not invoke NDAs, but that doesn't mean that they provided us with all the documents we asked for.

209
00:26:58,910 --> 00:27:02,410
They just didn't invoke it and didn't send us.

210
00:27:02,410 --> 00:27:07,850
But the good thing is that they say we can appeal, but we didn't appeal.

211
00:27:09,170 --> 00:27:19,970
And 8 counties did not invoke NDAs, but they just said we do not have those documents.

212
00:27:19,970 --> 00:27:25,730
And as you see here, the language varies from a county to another.

213
00:27:25,730 --> 00:27:34,030
Some counties made an effort to provide us with elaborate responses with legal explanation.

214
00:27:34,290 --> 00:27:49,870
Other counties, for example, they said the company had us read up the information or we cannot, the information you're asking for or the documents requested are not public records and that's it.

215
00:27:52,550 --> 00:28:02,070
And so all counties that provided manuals, you know, remember the six manuals we received, did not invoke NDAs.

216
00:28:02,070 --> 00:28:19,550
And even though they did not deny providing documents pursuant to an NDA provision, counties varied in the number of documents they provided and they mostly did not provide us with technical specifications.

217
00:28:21,350 --> 00:28:21,650
Yeah.

218
00:28:21,650 --> 00:28:24,090
Can we go to the next slide, please?

219
00:28:24,290 --> 00:28:25,250
Thank you.

220
00:28:25,250 --> 00:28:30,710
And now, so that's about the project.

221
00:28:30,750 --> 00:28:45,530
Now to be more specific and to have preliminary comparisons between the terms of contracts from the different, the four companies the counties were dealing with.

222
00:28:45,530 --> 00:28:47,030
The next, please.

223
00:28:48,530 --> 00:28:56,010
So as you see in the first slide, I just put some different pricings.

224
00:28:56,010 --> 00:28:58,690
Each company had a different price.

225
00:28:58,690 --> 00:29:07,330
The prices varied from 450,000 to 700 and so thousands.

226
00:29:07,330 --> 00:29:09,910
Next, please.

227
00:29:11,450 --> 00:29:24,830
But the thing that mostly all counties agreed upon in all their different companies, in all their different agreements, were the company's responsibilities.

228
00:29:24,830 --> 00:29:53,430
So each company is obliged to deliver a complete voting system, having the hardware and the software, according to the agreed upon project schedule, install the hardware and the software for the county, provide a user license, assign a project manager to help with,

229
00:29:53,430 --> 00:30:04,310
to enable all services and to make sure that all deliverables have been sent in a proper way and according to the agreement.

230
00:30:05,710 --> 00:30:17,310
Also, they are bound to provide customers with all product documentations, architecture and technical specifications, et cetera.

231
00:30:17,310 --> 00:30:37,690
And they are required to provide the necessary training materials to the counties and, I mean, the election officials and provide access to training videos if they are available and or provide trainings on site if they were required.

232
00:30:38,210 --> 00:30:42,890
And also, they would assist in the acceptance testing process.

233
00:30:42,890 --> 00:31:03,210
But one vendor, though, stated an additional responsibility and it was that they will have to remove and securely destroy all voting machines currently owned by the client and the certificate of destruction shall be issued to the client.

234
00:31:03,210 --> 00:31:08,070
That was one vendor that was required to do that.

235
00:31:09,250 --> 00:31:11,950
Back to the NDA.

236
00:31:11,950 --> 00:31:44,150
So, the NDA clause in the agreement, all the companies included a confidentiality clause and all companies included a definition of either confidential information or proprietary information, which is good from a lawyer perspective because these terms are pretty vague and if you do not include them and if you don't include the definition,

237
00:31:44,150 --> 00:31:58,950
basically, if one party discloses certain information, it will be hard to prove that it was confidential information in the sense of the agreement.

238
00:32:01,430 --> 00:32:19,740
So, basically, it says that neither party shall disclose to any third party any proprietary or confidential information of the other party and the information should be used by parties with reasonable care not to allow disclosure.

239
00:32:21,340 --> 00:32:49,650
One vendor indicated that this agreement, the confidentiality agreement, is binding even after the term of the sales agreement and two vendors stated that if the customer was asked to disclose certain information, they must notify the company and request their consent and I actually saw that in many emails between me and the accountant,

240
00:32:49,650 --> 00:33:14,470
they stated in the email that we provide you these documents but I will be contacting the company and asking whether it is possible to share other information with you and they wait for their consent and then they get back to me with the documents if the company accepts.

241
00:33:15,310 --> 00:33:32,810
But one remark here is that the six counties that provided us with manuals were also bound by such confidentiality agreements but they still provided the documents, the technical specificities.

242
00:33:32,810 --> 00:33:36,570
So, to what extent are the counties respecting such NDAs?

243
00:33:36,570 --> 00:33:37,370
I don't know.

244
00:33:37,370 --> 00:33:39,190
But personally, I'm happy.

245
00:33:39,190 --> 00:33:49,110
At least we got six examples of technical specifications to revise, to review.

246
00:33:49,810 --> 00:33:52,870
Can you have the next slide, please?

247
00:33:53,510 --> 00:33:55,930
About first measure.

248
00:33:55,930 --> 00:34:07,390
So, first measure is mainly not being able to deliver product, hardware, software, etc.

249
00:34:07,390 --> 00:34:15,310
as a result of acts of God, weather, unforeseeable circumstances, fire, war, anything you can imagine.

250
00:34:15,310 --> 00:34:31,170
As long as it's not, it's beyond the control of the party that could not meet their duty or obligation to deliver such products.

251
00:34:31,170 --> 00:34:43,460
So, in the agreements, we reviewed, only two vendors had first measure clause.

252
00:34:43,460 --> 00:34:52,860
And they state that neither party shall be liable in case first measure things happen.

253
00:34:52,860 --> 00:34:58,560
But for the other two vendors, it was a bit different.

254
00:34:58,560 --> 00:35:06,240
Because one vendor did not state, did not include a first measure clause on its own.

255
00:35:06,240 --> 00:35:21,820
But under limitation of remedies, they said that the vendor assumes no liabilities, no liability for delays as a result of acts of God, weather, unforeseeable circumstances, etc.

256
00:35:21,820 --> 00:35:26,640
So, they included the first measure clause.

257
00:35:26,640 --> 00:35:38,180
They excluded themselves from liability, but the clause doesn't say anything about the customer.

258
00:35:38,360 --> 00:35:50,400
The first vendor, though, included a first measure clause under another clause, which is defects due to customer actions or omissions.

259
00:35:50,700 --> 00:36:06,060
And in this case, they actually state that the customer shall pay for any loss caused by acts of God, fire, flooding, riots, acts of war, etc.

260
00:36:06,060 --> 00:36:12,080
So, they are making the customer liable, which is a peril to the customer.

261
00:36:12,760 --> 00:36:14,720
This is about first measure.

262
00:36:14,720 --> 00:36:16,140
Next, please.

263
00:36:17,620 --> 00:36:21,000
Next is the indemnification clause.

264
00:36:21,000 --> 00:36:37,580
So, an indemnity clause is a contractual transfer of risk between two contractual parties, generally to prevent loss or compensate for a loss, which may occur as a result of specified events.

265
00:36:38,500 --> 00:37:00,200
Here, in the agreement, we had only two vendors have included indemnification clauses, but the other two vendors remained silent on the matter, which means that in case such problems occur, the issue will have to be disputed.

266
00:37:01,260 --> 00:37:03,220
Next, please.

267
00:37:04,360 --> 00:37:07,340
So, limitation of liability.

268
00:37:07,440 --> 00:37:19,280
Here, I provided three sample clauses of the limitation of liability clauses we found in the various agreements we had.

269
00:37:22,020 --> 00:37:30,260
So, two vendors limit the liability for the company only.

270
00:37:31,000 --> 00:37:35,720
And two vendors limit the liability for both parties.

271
00:37:37,420 --> 00:37:49,740
And all companies, for all companies, the liability, they state that the liability shall in no event exceed the total amount of the agreement.

272
00:37:49,740 --> 00:37:57,800
And two vendors stated that the limitation of liability clause does not limit the indemnification obligation.

273
00:38:02,340 --> 00:38:04,280
Next, please.

274
00:38:06,000 --> 00:38:08,040
About the license and use.

275
00:38:08,040 --> 00:38:19,480
So, all companies state that they have to grant the customer a non-exclusive, non-transferable license to use the system they provided them with.

276
00:38:19,480 --> 00:38:35,040
And they only can use it with a licensed software on licenses we have in the course of operating the equipment and solely for the purposes of the elections.

277
00:38:35,280 --> 00:38:43,480
Any other use would be considered breach of contract and the companies assert sole rights in the software.

278
00:38:43,850 --> 00:39:10,600
And certain companies included in their agreement that reverse engineering, so, they were more specified, specific, and they said reverse engineering, reengineering, duplicating the source code or any other thing to replicate the source code is prohibited.

279
00:39:10,600 --> 00:39:14,300
Which is under the terms of a license.

280
00:39:14,940 --> 00:39:16,820
About the warranties.

281
00:39:16,820 --> 00:39:28,200
So, the companies warrant that the software they are providing is original and will operate in substantial performance and that the hardware is free from defects.

282
00:39:29,360 --> 00:39:36,440
Three vendors warranted hardware between one year and five years.

283
00:39:36,440 --> 00:39:39,760
So, there is a variation there.

284
00:39:39,760 --> 00:39:43,710
And, of course, a vendor did not specify the duration.

285
00:39:44,900 --> 00:39:50,960
Which can be interpreted that they will be warranted for life.

286
00:39:50,960 --> 00:39:53,880
And that's not good for the companies.

287
00:39:55,020 --> 00:39:58,980
But that's just a preliminary reading.

288
00:40:00,820 --> 00:40:13,580
So, what they warrant is that the warranty is to fix or replace the defective hardware or malware.

289
00:40:14,780 --> 00:40:17,940
About the updates and upgrades.

290
00:40:17,940 --> 00:40:29,080
Here, if you follow with me the language, some vendors say that they may provide new releases, upgrades, updates, et cetera.

291
00:40:29,080 --> 00:40:35,860
And other vendors say they will make certified software upgrades available.

292
00:40:35,860 --> 00:40:45,800
So, for certain companies, they are not required or obligated to make upgrades to their systems.

293
00:40:46,060 --> 00:40:47,860
And here is a problem.

294
00:40:47,860 --> 00:41:05,620
If they're not updating their systems, they can make new systems and then the counties will have to buy the new systems because their old systems are no longer okay to run.

295
00:41:05,940 --> 00:41:11,800
So, this obligation versus possibility dilemma, there it exists.

296
00:41:12,400 --> 00:41:14,360
Next, please.

297
00:41:14,620 --> 00:41:26,680
So, here's all the information I wanted to share with you about the project, about the process, and about preliminary reading of the agreement.

298
00:41:26,680 --> 00:41:31,220
And please, if you have any questions, just give them to me.

299
00:41:31,960 --> 00:41:39,560
So, it looks like... well, before we continue, we would like to thank our speaker again for giving us this awesome talk.

300
00:41:39,940 --> 00:41:45,880
It's actually kind of interesting how... a little sketchy how it is with some of these devices and some of the manufacturers.

301
00:41:46,620 --> 00:41:48,140
A little eye-opening, I would say.

302
00:41:48,780 --> 00:41:55,500
But it looks like we have a talk in the channel that says, how many different vendors did you see in total?

303
00:41:56,480 --> 00:41:59,240
In total, we had four.

304
00:41:59,240 --> 00:42:05,460
So, as I told you, out of 67 counties, only 44 provided us