WEBVTT 00:00.000 --> 00:01.680 Thanks for coming to my talk. 00:01.680 --> 00:07.380 I'm a first-time speaker at Crypto & Privacy Village, though I've been at Defcon before last year. 00:07.800 --> 00:12.980 Regrettably, we couldn't get together this year, but I sure hope I get to see you all in Vegas next year. 00:13.520 --> 00:21.580 Today, I'll be talking about COVID-19 apps, and how Norway became worst-in-class in digital contact tracing from a privacy perspective. 00:22.760 --> 00:25.140 First off, a short introduction. 00:25.240 --> 00:27.040 My name is Ivan Arvesen. 00:27.040 --> 00:32.900 I work as a consultant at a Norwegian consultancy called Bouvet in Oslo, Norway. 00:33.540 --> 00:42.080 I mainly focus on privacy and security these days, typically via AppSec, pen testing, data PR, advisory, and so on. 00:43.660 --> 00:49.820 My roles these days tend to be tech lead, architect, advisory work, etc. 00:50.040 --> 00:55.960 And day-to-day, I work on a few projects within critical infrastructure. 00:56.620 --> 00:59.040 I'm a huge privacy geek. 00:59.040 --> 01:08.920 I've written and spoken about stuff like GDPR and privacy issues in append-only storage, browser fingerprinting, and mass surveillance of bulk metadata collection. 01:09.980 --> 01:19.360 And earlier this year, I was a part of the government-appointed expert group that were to evaluate the Norwegian solution for digital contact tracing of COVID-19. 01:20.740 --> 01:24.540 Now, before I begin, a few disclaimers. 01:24.560 --> 01:38.460 I do not represent the expert group as such in this talk, as it no longer exists, and as I can only really speak for myself, nor any of its members or any other party you mentioned here. 01:38.460 --> 01:40.360 I can only talk for myself. 01:40.580 --> 01:43.520 So this is my own professional opinion. 01:43.820 --> 01:47.220 I do not represent my employer either. 01:48.640 --> 01:57.640 Additionally, I'm an engineer, not a doctor, so take any health statements or health-related statements with a grain of salt. 01:57.640 --> 02:01.820 Same goes for legal stuff, as I'm not a lawyer. 02:02.340 --> 02:11.180 And most of the media coverage I reference here that originates in Norway has been run through Google Translate for the occasion. 02:12.400 --> 02:23.540 Additionally, some of the materials here are from or based on the final public report of our expert group, and others are my own opinion, but I'll be sure to point that out. 02:25.540 --> 02:39.480 So back in June, Amnesty International published a report naming and shaming the most dangerous contact tracing apps for privacy, where Norway appeared amongst countries like Bahrain and Kuwait. 02:39.640 --> 02:42.920 Now, you might ask, what's Norway anyway? 02:42.920 --> 02:59.700 So for those unfamiliar, Norway is basically a country in the lands of fjords, oil, hygge, never mind that the book says Danish, black metal, and blue parrots apparently. 03:00.620 --> 03:05.800 Long story short, we're a Scandinavian Nordic country, i.e. 03:05.800 --> 03:16.460 we're in the northern part of Europe, and we consistently score highly on a bunch of metrics that basically tell you we're doing pretty all right. 03:16.640 --> 03:22.600 Stuff like Democracy Index and Better Life Index, Happiness Report, and so on. 03:22.980 --> 03:32.200 So additionally, we tend to hold a high level of trust in the government and in public services in general. 03:32.200 --> 03:40.660 We're also a highly digitalized country, and we pride ourselves on knowledge and in trusting experts. 03:41.460 --> 03:49.150 So how then could this happen in Norway, a liberal western democracy and a human rights champion? 03:50.820 --> 03:54.180 Well, a bit of background. 03:54.900 --> 04:00.400 As you're all aware, in the beginning of this year, COVID-19 was spreading like wildfire throughout the world. 04:00.400 --> 04:09.640 And while it had first been reported as another exotic local virus outbreak in China, it was soon seen as a global threat. 04:09.880 --> 04:19.160 Now, pretty early on, I think February, we could read early reports on digital contact tracing in some parts of the world. 04:20.260 --> 04:32.120 For instance, we had Singapore, which used a mainly Bluetooth-based application they developed themselves, which hit upon a few snags we'll get back to. 04:33.060 --> 04:41.120 Then there were South Korea, which used a combination of cell phone data, credit card purchase history, and surveillance cameras. 04:41.420 --> 04:56.960 And then there was Israel, where basically the government would have received geolocation from phone companies and were to download information and call records from mobile phones to track and identify. 04:57.620 --> 05:04.460 So they basically put the intelligence service on surveilling the population. 05:05.360 --> 05:17.540 So these are different approaches, and depending on what vectors and sensors were used, what data were communicated where and to and so on, there were different privacy impacts, obviously. 05:18.120 --> 05:33.840 So while digital solutions can obviously potentially be a valuable tool, it might be useful in this case, it was also evident that certain governments cared more about civil liberties and privacy than others. 05:35.480 --> 05:49.140 So this is an illustration from Carl Frederik Schoellam and the CDC, illustrating contact tracing, which is a part of the track and trace strategy many countries are taking. 05:50.840 --> 06:02.620 So basically, you want to identify, or when someone is diagnosed with COVID-19, you're trying to find out who's been exposed by that person to the disease, right? 06:03.500 --> 06:10.380 So this could be household members, friends, or other that this so-called index patient has been in close contact with. 06:10.640 --> 06:23.680 Now, this is most commonly done via interviews with diagnosed, like laboratory confirmed patients, and trying to figure out where they've been at what time and with whom during their infectious period. 06:26.000 --> 06:31.400 Now, in the end of February, the coronavirus was first registered in Norway. 06:31.960 --> 06:46.780 And in early to mid-March, the government implemented what they call the most invasive measures in peacetime, shutting down schools and colleges and universities and hairdressers, shopping centers, and so on. 06:48.280 --> 07:13.460 And in March of this year, the Norwegian Public Broadcasting Service stumbled upon a public source code repo on GitHub, which revealed the existence of a digital contact tracing app by a company called Simula Research Laboratory on behalf of the Norwegian Institute of Public Health, 07:13.460 --> 07:17.200 which is a government agency under the Ministry of Health and Care Services. 07:18.440 --> 07:23.640 Simula had been chosen as a supplier, producing parts of the solution. 07:23.640 --> 07:31.000 So other parts of the government would do some integrations and extending existing services and stuff like that. 07:33.260 --> 07:52.700 And we were soon able to read about some of the controversial design choices of this app, such as centralized storage, collection of location data, producing long-term anonymized and aggregated data potentially, which made a few alarm bells go off. 07:53.940 --> 08:05.060 Because when people talk about anonymized data, what they often mean is de-identified data, which can potentially be re-identified. 08:05.700 --> 08:11.920 So it was also made clear that the source code would not be publicly available. 08:13.140 --> 08:21.560 So all of this brought about a bit of criticism and public debate, and didn't really bode well. 08:21.560 --> 08:25.200 There were many that were critical for many reasons in this. 08:26.180 --> 08:29.100 Now, this is what the final app looked like. 08:29.300 --> 08:39.280 On the left, you can see a splash screen, then there's the onboarding process, and on the far right, your run-of-the-mill multi-page privacy policy. 08:40.460 --> 08:46.720 To break it down, Smittestop was a closed source solution. 08:46.860 --> 09:02.420 It required registration and de facto identification of users, because users had to register using their phone number, which in Norway is very tightly connected to a specific person. 09:02.420 --> 09:09.160 It's almost impossible to get a phone number anonymously. 09:09.660 --> 09:16.760 So it also collects sensor data from multiple sources, both Bluetooth and location services. 09:16.840 --> 09:24.580 It uploads data from all the users, all the time, to a centralized storage that the government controls. 09:25.420 --> 09:33.720 And there's the use of a static, device-specific identifier when devices running the app communicate. 09:33.720 --> 09:38.700 So they identify themselves using the same identifier every time. 09:40.100 --> 09:50.260 Now, the degree to which, if any, there is data minimization in such a solution has been questioned by experts in the public debate from the get-go. 09:51.780 --> 10:06.370 And any privacy engineer, and indeed many other with a little amount of technical or practical understanding, will quickly see that these design choices have big privacy implications. 10:07.740 --> 10:28.300 So, comparatively, Norway was fairly early on in rolling out an app, and the app itself is arguably one of the most invasive ones on the market, at least in a European context, where there are few, if any, other countries that have the same configuration of privacy-impacting factors in their applications. 10:30.160 --> 10:42.780 So, the formal basis for processing, in the case of Smittestop, is a dedicated regulation. 10:44.060 --> 10:55.600 So, the app is voluntary to use as such, but its formal basis for processing is authorized by the Diseases Act, not consent. 10:56.180 --> 11:02.820 So, outside of this, the GDPR, or the Norwegian implementation of GDPR, obviously applies as otherwise. 11:05.160 --> 11:08.040 So, the app 11:12.220 --> 11:22.120 is different from most other COVID-19 apps, contact tracing apps, in that most other apps do contact tracing, and that's it. 11:22.360 --> 11:25.100 While Smittestop has two purposes. 11:28.320 --> 11:45.800 It's to notify users of close contact with people that have been infected, but also to collect data that can be used in evaluating interventions that aim to lower infection rates. 11:45.800 --> 11:52.420 For instance, looking at public movement patterns, as well as further research analysis. 11:52.580 --> 11:57.840 Maybe input to epidemiological models, for instance. 11:58.380 --> 12:10.780 But the current app is made in sort of an all-or-nothing fashion, in that users can choose to have their data used for all the app's purposes, or not to use the app at all. 12:11.840 --> 12:20.720 Now, it's obviously not ideal to not let users explicitly opt-in for either purpose, nor is it really even best practice. 12:21.840 --> 12:38.300 And, not to mention, a potential consequence of implementing the app this way is that users feel that they're not in control of their data, and maybe they don't fully understand what happens to their data in what case. 12:38.440 --> 12:42.220 And that may, in turn, hamper user uptake. 12:42.260 --> 12:51.280 So, you might not get as many users installing and using the app, which would lead to a lowered effect of or from the app. 12:53.960 --> 12:56.360 About location data. 12:56.560 --> 13:02.780 So, what one is interested in when performing contact tracing is really whom met whom. 13:02.960 --> 13:13.760 The identity of either party, or the location of contact, is not necessarily relevant to prove contact itself. 13:13.760 --> 13:21.020 So, you thus don't necessarily need to know who the involved parties are, or where the contact took place. 13:21.620 --> 13:38.180 But the argument made for the use of location data, in the case of Smittstop, is that they wanted to attempt to compensate for the lack of data quality as a consequence of Bluetooth API limitations at the time. 13:38.400 --> 13:55.500 Because Bluetooth wouldn't work reliably in the background on iOS, and whereas Android might just kill background processes that continuously use Bluetooth or location services for energy or privacy reasons. 13:55.500 --> 14:09.660 Now, there's also another factor here, which is that Norway, being a country where people have a lot of purchasing power, iOS has more of a market share here than other places. 14:10.880 --> 14:13.500 So, that's a factor here as well. 14:15.120 --> 14:24.580 But on the other hand, GPS or location services typically has an accuracy of 3 to 10 meters under ideal conditions, meaning outdoor usage. 14:25.380 --> 14:38.480 So, you know, the quality of location service data from indoor use is uncertain, let's say. 14:39.280 --> 14:53.240 So, a proper and transparent evaluation of all the possibilities available here, then, might include something like looking at how big of a problem the current, or then, API limitations were in practice. 14:53.240 --> 14:56.820 Like, could you get by with spotty Bluetooth data, i.e. 14:56.820 --> 15:00.240 data only from when users were actively using the application? 15:03.000 --> 15:11.600 And that's probably a no, because then you'd always only have data from when users were unlocking their phone and having the app in the foreground. 15:13.580 --> 15:19.860 Then there's the Singaporean approach, where they, in practice, implemented like a faux sleep mode. 15:21.540 --> 15:33.640 Well, this necessitated keeping the app in the foreground, but you could dim the screen when the device was rotated, like placed upside down, so as to not totally drain the battery. 15:34.400 --> 15:35.980 So that's something. 15:36.520 --> 15:41.800 And that also lets you keep getting Bluetooth data continuously. 15:42.440 --> 15:50.300 Or you could collect location data, which is personally identifiable information, which is what we did in the Norwegian case. 15:51.080 --> 15:59.660 And then you'd have to think, how do the privacy implications of the respective alternatives here size up against each other and against the issue at hand? 16:00.100 --> 16:12.700 And another thing we have to mention here too is that, though it has been mentioned in many instances, that the data is anonymous in several contexts. 16:12.980 --> 16:23.200 This is really incorrect, because by virtue of being personal and identifiable information, location data can, by definition, not be anonymous. 16:24.520 --> 16:28.720 Because location data can, in itself, reveal a person's identity. 16:29.260 --> 16:34.420 That means there is no such thing as anonymous location data on an individual basis. 16:35.200 --> 16:45.640 Whereas in aggregated datasets, you can have certain quantifiable guarantees about the degree of privacy, for instance, via k-anonymity, differential privacy. 16:46.600 --> 17:00.100 But this gets complicated pretty quickly for a variety of reasons, such as temporal correlation or re-identification by combining data sources. 17:01.760 --> 17:10.560 But in practice, location data is only really a clear functional requirement in the case of monitoring public movement. 17:10.560 --> 17:17.400 And that's the other purpose, not contact tracing, but collecting data. 17:20.920 --> 17:23.760 About centralized storage. 17:23.920 --> 17:36.520 So when we talk about centralized storage in the context of contact tracing apps, we usually mean systems that are based on collection that's continuously uploaded to a central server that holds all of everyone's data. 17:36.820 --> 17:43.980 This is in contrast with decentralized systems, where every user's data is stored on their device until it's needed. 17:44.260 --> 18:03.660 And one should also note that the most popular decentralized solutions of digital contact tracing are not distributed, and that they still use a central server as a communications channel of sorts, as opposed to purely peer-to-peer communication. 18:03.660 --> 18:23.700 So for instance, a central server might hold a list of identifiers from people who are sick in the last two weeks, and then other apps poll the server regularly and then check locally if they've been in contact with any of the publicly known infectious or infected keys. 18:26.040 --> 18:41.060 The argument made in favor for data centralization in the case of the Norwegian app is that augmentation of user data with data from other users was needed in order to perform analysis. 18:43.460 --> 18:58.460 It's also a prerequisite, of course, for the purpose of looking at movement patterns, like collecting the data, the second purpose of the app, or to do other unspecified research on it. 18:59.080 --> 19:17.100 But a centralized data store is really, in principle, a defining factor when dealing with private data in this case, because its very existence makes misuse, function creep, leakage, and so on possible in a way that a decentralized solution just plainly doesn't. 19:17.100 --> 19:21.440 As in, you can't lose or abuse data you don't have. 19:22.780 --> 19:41.460 Now, alternative sources to the aggregated long-term data they hope to collect here may already exist, such as the data that Norwegian telcos had already provided in aggregate form, and that had been used for the same purpose. 19:41.460 --> 19:56.860 So they had taken cell site data, basically, and had some sort of coarse estimation of population density at points in time. 19:57.400 --> 20:02.460 And the government had used that for this very purpose already. 20:03.640 --> 20:17.880 The downside might be that that data might not be as precise as location data that you could get directly from devices, because resolution would depend on many factors, like cell site density and so on. 20:18.080 --> 20:35.060 But the privacy cost of uploading every user's location and movement, as well as who they've met and timestamps for all these events, that's undoubtedly much larger than just uploading what data is needed when you think you need it. 20:35.060 --> 20:47.580 For instance, prompting users to upload their movements, or even just use Bluetooth data only once a person has been in contact with someone who's positively diagnosed. 20:49.080 --> 21:04.140 There's also, of course, the risk of function creep, like the data could be used for something we haven't thought about as of yet, or misuse in case of a regime change. 21:04.620 --> 21:16.240 Additionally, if one didn't think the thought oneself, there were already publicly developed decentralized protocols back then, so there were lots of materials for inspiration. 21:17.360 --> 21:30.520 Now, this is an illustration of one of those privacy-first contact tracing schemes, so-called, which Nikki Case has generously donated to the public domain. 21:30.520 --> 21:34.300 I think what's illustrated here is based on DP3T. 21:34.840 --> 21:44.260 But basically, Alice's phone broadcasts random messages every few minutes, and then when she sits next to Bob, their phones exchange messages. 21:45.080 --> 21:55.580 Both their phones remember what they have said and what they've heard, i.e., what they've transmitted, sent, and received for the past 14 days. 21:55.780 --> 22:02.520 And if Alice gets COVID, she sends her messages to a central server or a hospital. 22:02.840 --> 22:17.000 Now, you can't identify her from this list of what messages she has sent, and this can be added to the big public list of messages from everyone who's been diagnosed. 22:17.280 --> 22:31.620 So then, Bob's phone can query this central place for a list of everyone who's been... or keys from messages from everyone who's been infected for the last 14 days. 22:31.620 --> 22:38.520 And if his phone recognizes enough of these messages, then he can isolate. 22:39.220 --> 22:41.060 So that's the gist of it. 22:41.720 --> 22:58.420 Now, when it comes to data integrity and user traceability, the use and communications of static device identifiers makes it possible to track or impersonate others, to trace users in limited partial leaks, and so on. 22:58.500 --> 23:11.540 So this is a bad one, because just about every other proposed solution I've seen, like both protocol specification and in existing apps, use rolling identifiers in one form or another. 23:13.320 --> 23:31.460 So there's also the fact that data was temporarily stored unencrypted in a local database on devices before uploading device data to the central server, which made it possible to inject or modify data before uploading it. 23:33.140 --> 23:40.140 Now, that means that data integrity, at least until the point where this was fixed, cannot be guaranteed. 23:41.100 --> 23:50.740 Additionally, the application connects to a server using what looks like an everlasting connection string and no other session handling. 23:52.420 --> 23:59.800 Now, in order to use the application, users had to register using their phone number. 23:59.800 --> 24:08.020 In Norway, this is de facto identifying yourself, because it's almost impossible to get a phone number without identifying yourself. 24:08.020 --> 24:12.920 So there is a connection there, which is even public in many cases, i.e. 24:12.920 --> 24:18.020 you can search for a name or a phone number and get the match on public services here. 24:19.640 --> 24:36.280 So functionally, there's no real need to identify any involved party, because even in contact tracing, users could be notified by the application when a contact has been diagnosed with COVID by health authorities. 24:37.140 --> 24:45.300 Because one could argue that registration is a mechanism that protects against bogus uploads to some extent. 24:45.760 --> 24:58.260 But this, in addition to protection of privacy, is in a way already baked into many of the decentralized approaches that demand human intervention before any upload takes place. 24:58.660 --> 25:13.140 So for instance, I think in the case of DP3T, they actually distribute upload codes so that when you're diagnosed, the doctor gives you an upload code that you can use to upload your data, i.e. 25:13.580 --> 25:18.280 the keys from your infected period to the central datastore. 25:19.060 --> 25:32.200 And many of these other approaches also let users choose specifically what time spans to share, or alternatively edit out some time spans to not share them for any reason. 25:33.320 --> 25:47.060 Also, Smith to Stop was found to be uploading a bunch of analytics data, including potentially fingerprintable information, on just about any interaction the users do with the application, i.e. 25:47.060 --> 25:53.760 from the get-go, from the moment you open your apps, analytics are firing, without telling users this. 25:53.900 --> 26:02.380 And this was not stated in the privacy policy, and users could not choose whether they want to upload this data or not. 26:05.380 --> 26:14.600 There are also some legal implications, I think, but note I'm not a lawyer, so you'll need to hear the reflections with this in mind. 26:15.160 --> 26:30.140 So the regulation I mentioned earlier, that was the formal basis for processing of the data, mentions that health and location data collected for this purpose cannot be shared with law enforcement or other parties. 26:30.180 --> 26:33.920 But Bluetooth data, however, isn't mentioned. 26:34.280 --> 26:56.300 Now, I interpret this as saying that sharing of Bluetooth data is permitted, because this would mean that the parties that the data is shared with could be able to, for instance, build social graphs of the data subjects. 26:56.820 --> 27:09.940 Though the regulation puts in place certain limitations, like including a sunset clause, the regulation also states that it can be changed at any time by the government via a new regulation. 27:12.220 --> 27:36.720 There's also the fact that data was stored on Microsoft servers in Ireland, and Ireland, there's already precedence for delivering user data to American authorities there, which can be done via the Cloud Act, I think, where U.S. 27:36.720 --> 27:43.160 governments can demand and secretly obtain data stored on the servers of American providers, even abroad. 27:46.090 --> 28:15.950 Now, another consequence of what the Norwegian solution ended up looking like is that it would be hard, if not impossible, to at least automatically achieve data interoperability in collaboration with other European countries, because most of these have already implemented or will implement solutions based on Apple and Google's new APIs, 28:15.950 --> 28:21.910 and or that are compatible with other protocols, such as DP3T. 28:22.550 --> 28:39.810 Now, other countries' contact tracing systems will, therefore, in theory, be able to register contact events that involve citizens of other countries, and or persons using other apps even, potentially, including apps produced by other countries' health officials. 28:41.550 --> 28:46.130 And here's a slide for Miscellany. 28:47.410 --> 29:13.370 A few points about the app is that the publicly available DPIA appears to not seriously consider alternative approaches and implementation, nor does it really consider malicious use of the data by the legitimate party itself, or data breach, or data leakage, 29:13.370 --> 29:17.110 other than via security features of the mobile app. 29:17.630 --> 29:23.430 And some of the probabilities stated in the risk assessment seem optimistic. 29:25.330 --> 29:33.110 As I mentioned, using a static identifier that's never rotated is obviously a bad idea and makes it possible to track users. 29:33.630 --> 29:38.810 And there's also something to be desired in transparency here. 29:39.590 --> 29:53.030 Both the purpose of the application, which wasn't very well communicated in the app itself, as well as just how the data was planned to be anonymized and aggregated for long-term use. 29:53.030 --> 29:57.210 That should be clearly and specifically communicated to the public, I think. 29:59.370 --> 30:11.910 But the anonymization process was not finished during our expert group evaluation of the app, other than involving various forms of aggregation. 30:14.270 --> 30:23.650 So all we can really say about that is we had some meetings with some stakeholders, but no public information on that. 30:25.890 --> 30:43.490 If the code were open-sourced, the public would be able to verify the functionality of the app, as opposed to depending on security by obscurity, or as opposed to trusting, for instance, the expert group and their interpretations. 30:44.250 --> 31:06.590 Now, the fact that the functionality used to bind phone numbers to the cloud device ID is implemented using a so-called preview feature, which the supplier says one should not use to process personal data or any other data that's subject to heightened compliance requirements. 31:06.590 --> 31:08.910 That's obviously not great. 31:09.290 --> 31:14.910 This was, I think it was B2C functionality in Azure in Ireland, or some such. 31:16.050 --> 31:26.770 But there were also various logging and compliance issues, such that users weren't able to see any data about their Bluetooth contacts in the audit solution. 31:27.110 --> 31:36.770 They couldn't access logs from health authorities or view audit logs after requesting deletion of their data, which could obviously be easily decoupled. 31:40.110 --> 31:54.970 Additionally, the contact analysis code was very complicated and complex, which means it was both, you know, kind of hard to understand what was actually going on there. 31:54.970 --> 32:10.950 And it entails like low quality and a maintainability context, and had weaknesses both in implementation and in software methodology. 32:11.110 --> 32:24.190 And finally, the app also used text messages, SMS, to notify users if they had been in contact with someone infected with COVID. 32:27.980 --> 32:35.800 Now, it's not really known whether digital contact tracing is even a viable solution to the problem at hand. 32:35.800 --> 32:40.440 It's unknown what value it can bring, and even if it's feasible at all. 32:41.180 --> 32:55.220 So most scientists with direct experience, like the ones involved with the Singaporean program, they claim that digital contact tracing is at best a complementary addition to manual contact tracing. 32:56.160 --> 33:13.080 Then there's the fact that the Norwegian app is not in accordance with common European guidelines, such as the EU Commission's recommendations for these sorts of apps, the EU resolution on coordinated work against COVID-19, nor the guidelines from the European Data Protection Board. 33:14.680 --> 33:21.900 You know, surveilling movements and contacts of all users of the app is an extremely invasive measure. 33:21.900 --> 33:29.200 And as the effectiveness and usefulness of this system is not clear, the proportionality of this measure is questionable at best. 33:29.680 --> 33:44.060 One might expect some sort of explanation as to why or how this would be effective and why this outweighs the invasiveness of these actions, but this remains to be seen pretty much. 33:46.800 --> 34:13.200 Now, as I said earlier, the app is an open source, and this is because it was claimed that while open sourcing the code would, in the long run, potentially lead to better security, but in a shorter perspective, it might lead the attackers or would-be attackers to be able to exploit any vulnerabilities before anyone else could see them or fix them or so on. 34:13.200 --> 34:17.230 So that's the argument that was made to keep the app closed source, basically. 34:21.260 --> 34:40.440 And about anonymity in the long term, you know, there was a quote, a comment on our final report where we commented on the state of anonymity in the long-term data, which would be aggregated and anonymized so-called. 34:41.460 --> 34:50.960 They responded that the report also has a recommendation of anonymization of data for analysis purposes through so-called differential privacy. 34:52.060 --> 35:13.200 The Norwegian Institute of Public Health has at this point already developed an elaborate system for anonymization that is, in their view, will have an equally anonymizing effect as so-called differential privacy, but which is easier to implement, communicate, 35:13.200 --> 35:16.300 and doesn't lose data quality to speak of. 35:17.000 --> 35:30.220 Now, while that statement makes sense syntactically, I guess we'll leave evaluating the logic of it as an exercise for the listener, right? 35:30.220 --> 35:38.320 Is it possible to somehow deliver anonymization to the same extent as differential privacy, but without the formal guarantees? 35:38.760 --> 35:40.620 You decide. 35:43.840 --> 35:57.300 There are also a bunch of potential possible attacks here, like relay attacks, tracking attacks, where if many people collaborated on this, you'd basically have distributed surveillance. 35:57.580 --> 36:03.340 And this is obviously possible because of the static identifiers used by the application. 36:03.660 --> 36:06.420 Or you can map infection. 36:08.100 --> 36:19.760 And you could potentially, from the long-term data, the anonymized and aggregated data, so-called, re-identify people, potentially. 36:20.240 --> 36:34.580 And also potential risk of data theft, leak, or misuse, which this risk is obviously magnified because there's a state actor that's in control of all of this data. 36:34.820 --> 36:41.020 So the potential for abuse, if it were to be abused, is obviously much higher. 36:41.020 --> 36:49.280 Now that we've discussed this app somewhat, let's take a look at how events unfolded earlier this year. 36:51.000 --> 37:01.920 Back in early April, after the public had been informed that this app was in development, and it was going to look like what we just discussed. 37:03.280 --> 37:19.800 As a consequence of the criticism, probably, as well as in order to increase transparency and so on, the Ministry of Health and Care Services appointed a group of experts based on recommendations from an organization called IKT Norway. 37:21.080 --> 37:27.660 And none of the participants were connected to any of the other involved parties, obviously. 37:27.980 --> 37:34.920 But we were basically tasked with evaluating all of the involved components in the solution. 37:34.920 --> 37:58.200 This includes mobile apps on respective platforms, a backend encompassing many components, including analysis in order to figure out what's a contact event, and reporting code, and connected services running in the cloud, integrations, and new solutions in existing government systems. 38:02.600 --> 38:15.060 We had mostly direct access to all the source code repositories, so we basically just dove into the code and looked at it, and also had a bunch of meetings with all the stakeholders. 38:15.500 --> 38:34.780 But our mandate said that we had to deliver an open report, like a public report, where we would assess whether security and privacy are properly taken care of, and also that we communicate with them and tell them about any vulnerabilities, we found any issues, 38:34.780 --> 38:36.560 discuss that with them, and so on. 38:39.460 --> 39:00.120 So, we delivered a preliminary report after four or five days, and because of that very limited time, we had to limit ourselves to just the smartphone apps and a select part of the backend, and only the technical security aspects. 39:01.820 --> 39:16.600 And also, because of the fact that not all the parts of this entire system were finished at this point, or had even been started at that point, we refrained from commenting on privacy. 39:17.120 --> 39:21.780 We waited until we'd seen the entirety, or enough. 39:22.400 --> 39:48.360 So, deletion of data was not implemented either at this point, but long story short, we pointed to some data integrity issues and the static ID, obviously, as one of the main reasons that the apps shouldn't be launched as is, you know, since there were issues and you'd have to view data as potentially compromised or bad. 39:48.360 --> 40:00.840 So, then we recommended that maybe you could do a soft test in a couple of test municipalities, but no more than that would be responsible in our view. 40:01.640 --> 40:09.960 Still, we recommended a lot of improvements, most importantly, related to integrity and scalability of the system. 40:11.160 --> 40:20.980 What happened then was the app was launched to the entire country at the same time. 40:22.580 --> 40:27.020 Well, it was called an evaluation or kind of a trial, right? 40:27.020 --> 40:31.220 But it was really launched countrywide. 40:31.220 --> 40:35.680 Everyone could go to the app store and download this app and register and upload data. 40:36.540 --> 40:46.100 What was limited to a couple of select test municipalities was the tracing functionality. 40:46.840 --> 40:59.620 So, they only provided notification services based on the data from the system to the people in the test municipalities, whereas everyone could and would upload data. 41:00.820 --> 41:15.420 And some of the reason might be because some of the issues we reported in our preliminary report, but the app was promptly reversed engineered and inspected by a critical tech community. 41:16.260 --> 41:35.520 And there were also, as I mentioned on the previous slide, a few scalability issues that we pointed out, which led to the fact that the backend didn't quite scale so that when so many people in the country tried to download and register at the same time, 41:35.520 --> 41:39.080 the backend just couldn't handle the pressure, basically. 41:40.680 --> 41:53.100 Also, the fact that they hadn't had time to focus on energy efficiency led to a bunch of low user scores and negative reviews and so on. 41:55.760 --> 42:20.660 Shortly thereafter, Google and Apple announced their API collaboration, which would fix all the Bluetooth issues in the platform, basically, which would enable you to do Bluetooth activity in a background app, leaving no real reason to use location services for the purpose of contact tracing, 42:21.840 --> 42:27.600 if you agree that Bluetooth data is enough to do contact tracing, that is. 42:28.800 --> 42:36.180 But this new API demands that you don't use location services for access to the API. 42:38.480 --> 42:49.880 And after that, a large amount of the Norwegian tech community mobilized and published a petition. 42:50.320 --> 43:10.340 Over 300 professionals in security and privacy and tech in general basically stated that we don't have to choose between privacy and health, and that we have the knowledge and technology to implement a better solution, which was met with some scorn from, 43:10.340 --> 43:14.040 well, the supplier mainly. 43:17.520 --> 43:29.280 So when we delivered our final public report on May the 20th, this report included some findings and recommendations. 43:29.280 --> 43:44.500 Now, most of the findings I've mentioned already, but they included aggressive analytics, static identifiers and Bluetooth contact, eternal connection strings, using preview features for personal data, limitations in auditing solutions, data deletion, 43:44.500 --> 43:55.640 also deleting the audit logs, quality issues in the contact analysis code, as well as using SMS, which is an insecure channel, as a notifications channel. 43:57.640 --> 44:07.700 Now, in evaluating whether security and privacy was properly handled, our evaluation concluded no on both accounts, unfortunately. 44:09.320 --> 44:30.900 And then we delivered a few recommendations, which were basically clarifying the regulation, splitting the app in two based on purpose, so one for contact tracing and one for the data collection for those who wanted to contribute to that, to remove all data that they didn't need at any point in time, 44:30.900 --> 44:53.760 so basically work towards more of a data minimization thing, to implement differential privacy in their data aggregation plans, to consider to rewrite towards a more decentralized solution, and to potentially implement differential privacy before uploading user data, 44:53.760 --> 45:09.580 like differential privacy locally on the device, and also make as much of the source code as possible open source so that people could verify the functionality for themselves, and to regularly re-evaluate. 45:11.460 --> 45:15.920 Now, the aftermath was pretty interesting. 45:16.340 --> 45:38.160 At the same government press conference that we delivered our conclusions on, the Institute of Public Health basically just disagreed with our conclusion, and then the supplier, Simula Research Laboratory, basically just wrote a blog post publicly attacking the expert group, 45:40.100 --> 45:49.920 and questioning our integrity, claiming that our conclusions were based on personal opinions, that our recommendations were politically motivated, and so on. 45:49.920 --> 46:01.940 So here you can see the original blog post translated by me from Norwegian to English, though they've since edited it exchanging political for personal a few times. 46:02.460 --> 46:12.040 But nevertheless, questioning the motives of an impartial external group tasked with evaluating your work in this way, kind of concerning nonetheless, I think. 46:12.720 --> 46:19.840 But then again, this is another quote from some higher up at the same company. 46:20.220 --> 46:26.960 There are many countries I think should not use the Norwegian solution, precisely because they don't have a well-regulated democracy. 46:26.960 --> 46:31.080 They don't have strong privacy interests and governments to keep watch. 46:31.660 --> 46:47.240 Now, following this logic in the Norwegian app, privacy would then by definition not be handled responsibly, as any privacy guarantees would be contingent on trust, which isn't much of a guarantee. 46:47.880 --> 46:52.920 Key point here, data protection and privacy are two different things. 46:56.200 --> 47:04.480 After this, the parliament actually got involved and decided that they'd have to split the app based on purpose. 47:04.480 --> 47:10.940 So one app for contact tracing and another for data collection for those who want to contribute to that. 47:12.000 --> 47:20.240 Then the Norwegian Data Protection Authority concluded that the degree of privacy invasiveness was not justified. 47:20.240 --> 47:31.520 Now, this was based on both things like data minimization issues, but also on the current situation in Norway with a low amount of infection, I guess. 47:34.040 --> 47:36.620 The health authorities, i.e. 47:36.620 --> 47:43.700 the Institute of Public Health, then chose to stop all data collection and to delete all the data they already had. 47:44.820 --> 47:55.700 And the next day, Amnesty International stated that they found the Norwegian app to be amongst the most dangerous tracing apps for privacy, which was where we started out. 47:57.060 --> 48:04.700 Then international media picked up on this, went around New York Times, BBC, and all those places. 48:07.320 --> 48:10.080 Now, what can we learn from all of this? 48:11.460 --> 48:16.980 Well, you don't want to be in the position of trying to figure out this stuff while the world's crumbling around you, I think. 48:18.660 --> 48:25.900 But I think we need to be able to still stick to our principles, especially in times of crises. 48:26.080 --> 48:31.280 Because privacy isn't only a compliance issue nor a subset of security. 48:32.360 --> 48:44.460 And it's pretty problematic, I think, that the supplier has consistently been led to do all the communications and just refuse to listen to or even understand criticism. 48:45.740 --> 48:52.860 And basically argued that the solution works in Norway because we have such a high degree of trust in the state. 48:53.420 --> 48:55.740 Because this misses the point entirely. 48:57.960 --> 49:05.140 And as to what happens next, there were talks of experimenting with the Apple Google API. 49:05.180 --> 49:08.100 But now that everything's on ice, who knows? 49:10.000 --> 49:11.620 So quick recap. 49:13.500 --> 49:17.840 Norway is hence worst in class in contact tracing for COVID-19. 49:17.840 --> 49:19.940 And that's pretty unexpected. 49:19.940 --> 49:24.060 It's not something I would have seen coming like half a year ago even. 49:25.700 --> 49:35.500 But continuously storing where everyone is, at what point, and who they're meeting with, that's an extremely aggressive move and excessive even. 49:35.580 --> 49:48.580 Whereas other protocols and solutions make as little of an impact on privacy as possible and make users take active, explicit choices about their data, the Norwegian app Smittestop did not do that. 49:51.170 --> 50:04.890 Now, I think this highlights the need for a drastic improvement in general understanding of privacy, even amongst technologists, so that people will know how to make solutions that are private by design and we don't end up in this situation again. 50:05.390 --> 50:16.010 And there are, of course, also discussions to be had on a more principled level, which might illuminate this issue to even more people like decision makers, directors, politicians, and so on. 50:16.010 --> 50:28.130 Because privacy is a foundational human right and it's guaranteed by national constitutions and the EU and the Universal Declaration of Human Rights. 50:28.130 --> 50:41.250 And that means there is a minimum of how much of these rights that people must always have, meaning that states can only go so far in any case, even though they, of course, might get more leeway in emergencies. 50:42.090 --> 50:47.670 And that's because our principles, by virtue of being principles, always apply. 50:48.370 --> 50:54.090 Also, you know, privacy is a defining feature of liberal democracies, we should take this stuff seriously. 50:54.610 --> 50:57.230 But where then is this line? 50:57.450 --> 51:05.090 That's a hard issue and one that I don't really have a good answer to, but that discussion is needed anyways. 51:06.090 --> 51:22.570 Because, you know, I experienced, even in working with the expert group, that there are people that honestly believe that privacy costs are a pure balancing act, that any degree of invasiveness can be warranted if it leads to effectiveness, if the results are good enough, 51:22.570 --> 51:26.210 in whatever sense you're evaluating that. 51:27.250 --> 51:38.650 I tend to think the laws and regulations are pretty clear that this is not the case, and, you know, also protection against the tyranny of majority is another defining feature of liberal democracies after all. 51:42.690 --> 51:54.910 So, I think the head of Amnesty Security Lab had a pretty good quote here in that, you know, privacy doesn't need to be a causality in rolling out of these maps. 51:54.910 --> 52:08.310 And after all, we have to remember Amnesty, the DPA, Parliament, EU, Google and Apple, our expert group and 300 professionals have all warned the involved parties several times. 52:08.390 --> 52:18.010 And the fact that there's been no change until the entire app was put on hold is very strange, given the degree of trust we usually pride ourselves in placing in experts in Norway. 52:19.230 --> 52:28.590 You know, when there are privacy preserving alternatives, such as other existing protocols and solutions, they should always be explored first. 52:29.190 --> 52:44.970 Of course, it's laudable to want to solve this very real and very big problem with the means available to us, but we really can't excuse bad work and lack of understanding by claiming that the ends justify the means, in my opinion. 52:46.150 --> 52:55.830 Now, hopefully, you would rewrite the app in a more privacy preserving way and try to learn from what went wrong here in order not to make this mistake again. 52:56.930 --> 53:06.790 But the design choices were questionable or even arguable at first, but they never really changed. 53:07.270 --> 53:21.390 And now that it's put on hold, this is the worst that could happen because now we're seeing a bit of a rise again and the government is maybe having to crack down a bit and implement more safety issues, and now we have no app. 53:23.110 --> 53:26.490 So this is not a good place to be in. 53:27.690 --> 53:33.430 But as for the future, at least there's hope, because only yesterday I read this. 53:33.990 --> 53:36.570 But only time will tell where we go from here. 53:37.170 --> 53:40.190 And that's all I had to say. 53:40.530 --> 53:42.230 With that, I'd like to say thank you. 53:42.230 --> 53:44.890 It's been great fun and I appreciate you all being here. 53:45.630 --> 53:46.970 And now for some Q&A.