[00:19.400 --> 00:24.720] Hello, everybody. Good morning. My name is Julia Reinhardt. I'm a fellow in residence [00:24.720 --> 00:30.560] with the Mozilla Foundation and a privacy and data protection professional based in San Francisco. [00:30.720 --> 00:36.380] Previously, for almost 15 years, I used to work as a diplomat for my native Germany. [00:36.520 --> 00:41.000] And I'm really happy to present at DEF CON this year, even if it's safe mode. [00:42.440 --> 00:47.160] I'm here today to speak about regulatory trends for artificial intelligence, [00:47.160 --> 00:53.280] particularly those coming from Europe. I know this is not a topic otherwise featured at the [00:53.280 --> 00:58.940] Crypto & Privacy Village. And I hope you will bear with me when we venture out into a topic [00:58.940 --> 01:06.820] that is not top of mind for most of you. But I expect it to be at least as important as GDPR, [01:06.820 --> 01:10.280] the European data protection regime that took effect two years ago, [01:10.280 --> 01:14.620] and that hit many in the technology space in the US unprepared. [01:15.480 --> 01:22.720] As with any new technology, the use of AI brings both opportunities and risks. Citizens fear being [01:22.720 --> 01:29.080] left powerless in defending their rights and safety when facing informational asymmetries [01:29.080 --> 01:34.140] of algorithmic decision making. And companies are concerned by legal uncertainty. [01:35.020 --> 01:42.700] I'm usually not one to quote CEO of Tesla and SpaceX Elon Musk, but no doubt he's among those [01:42.700 --> 01:49.920] who said it the pointiest way. Mark my words, he said, AI is far more dangerous than nukes. [01:49.920 --> 01:56.100] Why do we have no regulatory oversight? So of course, he's not the only one to say that. [01:56.100 --> 02:02.060] Other corporate leaders have called for increased regulation and so have politicians and respected [02:02.060 --> 02:09.160] scholars and academic research institutes and think tanks. Reasons for this call to action [02:09.160 --> 02:15.540] is the understanding that for all its many benefits, AI also presents many risks and they [02:15.540 --> 02:22.520] include biased algorithms, privacy violations, and the potential for injuries caused by defective [02:22.520 --> 02:28.780] software. So with the increasing use of AI based solutions in areas like criminal justice, [02:28.780 --> 02:35.220] healthcare, robotics, financial services, and education, corporate interests will inevitably [02:35.220 --> 02:42.740] conflict with societal benefits. So that conflict raises the question of what systems should be put [02:42.740 --> 02:49.860] in place to mitigate potential harms. My research is focused on upcoming regulation in Europe [02:49.860 --> 02:57.300] for applications of artificial intelligence and how they could affect US companies. Not the big [02:57.300 --> 03:02.740] tech giants that we all think of that all have their offices in Brussels observing and influencing [03:02.740 --> 03:10.460] these discussions, but the small ones and that don't operate in Europe or globally yet. If you [03:10.460 --> 03:15.800] have thoughts about this as an individual, but also if you work for a company that fits into what [03:15.800 --> 03:21.840] I'm looking at, I'd love if you reached out and gave me feedback. So on my last slide, I share my [03:21.840 --> 03:27.140] website and my Twitter handle again, and I'd be really grateful to receive your messages. [03:27.850 --> 03:35.080] So you could ask, why would a US organization be affected by European regulation? They only have to [03:35.080 --> 03:42.020] follow US law. But actually, what we have seen with the European Union's General Data Protection [03:42.020 --> 03:52.120] Regulation, GDPR, is that lots of businesses based solely in the US are affected, mainly because they [03:52.120 --> 03:57.740] have European users. So they process personal data of European residents. And these residents [03:57.740 --> 04:04.020] of Europe are protected by GDPR wherever the processing of their data takes place. So I could [04:04.020 --> 04:10.700] go on forever on GDPR. It's a topic that I know well and that I've worked on for years, including [04:10.700 --> 04:17.600] when it was still being negotiated among EU member states, and I worked for one of them. But [04:17.600 --> 04:24.340] an important experience I wanted to share here is that many US-based organizations [04:24.920 --> 04:32.300] that process personal data of people around the world have decided to apply GDPR and extend all [04:32.300 --> 04:39.260] rights that go with it to their customers who are not European residents but live outside of Europe. [04:41.780 --> 04:47.580] So I've made a little slide on the advantage of GDPR compliance on a global level. [04:48.340 --> 04:54.100] To these companies, it gives them an edge in global compliance. It's easier for them [04:54.100 --> 05:00.360] in terms of handling complaints and requests. So they say, just give all of our customers [05:00.360 --> 05:06.780] all the rights that Europeans have, a very high bar. And it's still easier for the organization [05:06.780 --> 05:11.800] than to sort out the customer's location and attribute different rights according to their [05:11.800 --> 05:20.740] location. GDPR offers them a legal framework and a set of standards that is at least compared to [05:20.740 --> 05:27.600] other less spelled out legislation or the lack thereof, relatively clearly adoptable. [05:28.860 --> 05:35.800] So while my clients are mainly small and medium enterprises based in the US, [05:35.800 --> 05:42.500] with only some clients in Europe, or with the mere intention of soon expanding to Europe, [05:42.500 --> 05:49.440] this privacy management strategy has been found with bigger tech firms as well. So the image on [05:49.440 --> 05:54.880] the slide that you see shows the view from Salesforce's legal team space in San Francisco. [05:55.460 --> 06:01.460] And they are certainly part of these companies I'm talking about. Organizations appreciate that [06:01.460 --> 06:08.840] there is a standard now that is law in one part of the world that can serve as a guideline [06:08.840 --> 06:14.460] also for other parts of the world. And even if this guideline is more demanding than legislation [06:14.460 --> 06:20.540] in their markets outside of Europe, it makes life so much easier for them to have one high profile [06:20.540 --> 06:27.820] standard than have many different ones, what I call the global privacy patchwork, or none at all. [06:27.820 --> 06:36.560] Now, this was definitely a huge finding in the past two years since May 2018, when GDPR became [06:36.560 --> 06:44.120] enforceable. Also, with the US administration being relatively silent about data protection, [06:44.120 --> 06:51.740] and other countries like Japan or Israel following a rather GDPR-like model, that made the EU the [06:51.800 --> 06:58.220] de facto rule setter in the technology policy worldwide, in the important field that is data [06:58.220 --> 07:06.040] protection, as data is integral to all technology. So what I want to find out now is whether we [07:06.040 --> 07:11.840] could expect to see the same trend with upcoming EU regulation in other fields of tech policy, [07:11.840 --> 07:20.020] for example, artificial intelligence. So if we look at the timeline of policymaking on AI in [07:20.020 --> 07:26.480] Europe, and that's the top line, the dark blue one, we're pretty much in the middle of concepts [07:26.480 --> 07:33.840] gearing up to become concrete policy proposals. As you see on that timeline, after sketching first [07:33.840 --> 07:40.140] ideas and establishing a high level expert group that has issued its proposals, the European [07:40.140 --> 07:45.680] Commission issued a white paper early this year and started a consultation process in which more [07:45.680 --> 07:53.940] than 1,200 individuals, advocacy groups, researchers and companies participated, and I [07:53.940 --> 08:05.060] helped to formulate the Mozilla contribution. Most recently, in July, the experts last July, [08:05.060 --> 08:11.780] the experts invited by the European Commission have issued an assessment list for trustworthy AI [08:11.780 --> 08:19.600] that companies and developers can use to assess their algorithm use. On the international level, [08:19.600 --> 08:25.200] that's the bottom part of the timeline, many of the concepts that form part of the European draft [08:25.200 --> 08:31.180] have been formulated as principles within the OECD, the Organization for Economic Development [08:31.180 --> 08:37.460] and Cooperation and Development, that most European countries and the US are members of, [08:37.940 --> 08:44.880] adopted in May 2019. They promote uses of AI that are innovative and trustworthy, [08:44.880 --> 08:51.460] and that respect human rights and democratic values. A month later, the G20, the world's [08:51.460 --> 08:58.840] biggest economies, have drawn from them heavily when they adopted their human-centered AI principles. [08:58.840 --> 09:06.800] Of course, these are all non-binding, but still highly influential. In the US, the White House [09:06.800 --> 09:12.760] announced the American AI Initiative, which focuses on driving technological innovation and [09:12.760 --> 09:20.000] standards to protect a competitive edge on AI. So the White House last January released 10 principles [09:20.000 --> 09:25.540] for federal agencies to follow when proposing rules governing the private sector's deployment [09:25.540 --> 09:32.540] of AI technology. These are binding, and they follow many of the same ideas, but they are very [09:32.540 --> 09:39.040] light touch, with lots of exceptions, and a strong emphasis on leaving space for corporations [09:39.040 --> 09:48.760] in whatever way to innovate and compete globally. It's a more hands-off free market view, and a [09:48.760 --> 09:53.660] pretty stark renunciation of the careful approach that was initiated under the Obama administration [09:53.660 --> 09:59.900] in 2016, but that hadn't yet led to any actual US comprehensive legislation. [10:00.940 --> 10:07.020] So, back to Europe. Right now, Brussels is working on a draft policy that is expected [10:07.020 --> 10:14.060] to be tabled to member states for negotiation around the beginning of 2021, and that could, [10:14.060 --> 10:21.100] once those negotiations have been successful, become the world's first general regulation on AI. [10:23.660 --> 10:31.480] So what will European AI regulation look like, most likely? It's a three-pronged approach, [10:31.480 --> 10:37.880] with a substantial increase in public and private investments in AI to boost its uptake, [10:37.880 --> 10:43.480] but also funding for research, more cooperation between researchers across Europe, [10:43.480 --> 10:49.100] and better know-how to prepare for socio-economic changes. This is called the ecosystem of [10:49.100 --> 10:55.160] excellence on top. And I would say that the numbers floating here don't add up in [10:55.160 --> 11:03.600] any way to what is seen in the US or China, but it's a start. Combined with a European strategy [11:03.600 --> 11:11.760] for data, which aims to improve the use of data by creating an EU single market for data, because, [11:11.760 --> 11:18.220] of course, as you know, it's 27 different member states, so we're aiming for a single market in [11:18.220 --> 11:24.200] that field to facilitate access to data and computing infrastructures, which is, of course, [11:24.200 --> 11:30.060] an essential requirement for the development and use of AI applications. And last, and I've put [11:30.060 --> 11:38.660] that in the middle, with the aim of an ecosystem of trust. The Commission uses a human rights first [11:38.660 --> 11:44.560] approach and details a number of requirements that AI applications would have to fulfill [11:44.560 --> 11:51.400] in order to be considered trustworthy. This is the new framework that I will present further on. [11:51.640 --> 11:57.080] And, of course, not forget existing EU laws and regulations, whether on EU level or in the member [11:57.080 --> 12:03.920] states, that already apply to AI solutions, including rules on data protection, GDPR, [12:03.920 --> 12:10.640] non-discrimination, consumer protection, product safety, and liability. Consumers, of course, [12:10.640 --> 12:16.820] expect the same level of safety and respect of their rights, whether or not a product or a system [12:16.820 --> 12:23.520] relies on AI. So, in some cases, the AI-related aspects of these existing rules may be difficult [12:23.520 --> 12:30.340] to enforce because AI systems are opaque, unpredictable, complex, and autonomous. [12:30.780 --> 12:38.820] So, in some cases, they might need to be amended or updated. So, I'll come to the requirements on [12:38.820 --> 12:43.820] my next slide, but I need to first underline that this approach singles out high-risk [12:43.820 --> 12:51.240] applications, which would become subject to strict requirements and not those that are considered [12:51.240 --> 12:57.180] low-risk, for which no additional legal requirements would be imposed. And for these [12:57.180 --> 13:04.080] low-risk ones, the Commission is considering a voluntary labeling system that would certify [13:04.080 --> 13:10.400] compliance with parts of the requirements and allow companies to market their AI products [13:10.400 --> 13:18.540] as trustworthy. So, rather than introducing a generic AI application, this is a regulation, [13:18.540 --> 13:24.800] sorry, this is a nuanced risk-based approach, possibly one that is application and technology [13:24.800 --> 13:33.240] specific. However, this approach may also lead to uncertainty, and I personally worry that although [13:33.240 --> 13:40.580] the text includes a definition of what is meant by high-risk sectors, specifically mentioned are [13:40.580 --> 13:47.360] healthcare, transport, energy, judicial decision-making, and mass citizen surveillance, [13:47.880 --> 13:54.280] it still won't always be easy to determine. It is very subjective, and an algorithm use [13:54.280 --> 14:01.760] could be considered low-risk for someone, but actually has high-risk consequences for others. [14:01.760 --> 14:10.040] Plus, risk here is defined as a risk to an individual, which excludes a whole [14:10.040 --> 14:14.540] category of AI applications that pose major collective risks. [14:17.320 --> 14:25.060] So, developers of high-risk AI applications would be obliged to follow rules in these key aspects. [14:25.320 --> 14:32.760] First, human oversight. Some degrees of human oversight would be required, ranging from [14:32.760 --> 14:39.260] requiring human review before a decision is implemented to the possibility of human [14:39.260 --> 14:45.440] intervention in real-time or afterwards. Then there could be requirements on robustness [14:45.440 --> 14:51.540] and accuracy, the ability to react to inconsistencies, and that the application [14:51.540 --> 14:59.720] is resilient against attacks and manipulation. Training data sets need to be representative and [14:59.720 --> 15:04.100] comprehensive and comply with privacy and data protection rules. [15:05.160 --> 15:11.400] Information to be provided. People would need to be informed when they interact with an AI system [15:11.400 --> 15:16.400] and not a human, and what its capabilities and limitations are. [15:17.940 --> 15:23.100] Developers would also need to keep records of how they selected what kind of training and [15:23.100 --> 15:29.380] testing data, and how they programmed and trained the system to enable AI decisions to be traced [15:29.380 --> 15:36.980] back. And then there will probably be specific requirements for remote biometric [15:36.980 --> 15:46.120] identification, also known as facial recognition. So, no ban inevitably, but the EU insists that [15:46.120 --> 15:51.920] data protection rules and the Charter of Fundamental Rights apply, which already allow [15:51.920 --> 15:57.820] the uses of facial recognition only in cases where this action is justified and proportionate [15:58.300 --> 16:04.460] and is subject to adequate safeguards. So, there will probably be an open discussion on whether [16:04.460 --> 16:11.520] exceptions could be justified. So, as a developer, and I know that many of you are developers, [16:11.840 --> 16:18.840] think of this like some kind of FDA-style clinical testing, not for drugs, but for algorithms. [16:19.900 --> 16:26.120] So, how do I expect US companies to be affected by future regulation on AI? [16:27.820 --> 16:32.020] The EU guidelines will likely have a ripple effect in the US, [16:32.020 --> 16:37.700] since many American technology companies provide AI solutions and services to the EU. [16:37.700 --> 16:44.980] It will also impact US companies that EU investors are looking to buy, [16:44.980 --> 16:52.040] and companies that plan to expand into European markets. Of course, the additional policy and [16:52.040 --> 16:56.980] regulatory measures the EU considers will increase the cost of compliance and put the [16:56.980 --> 17:04.260] administrative burden and possible IP-related difficulties on companies that develop or [17:04.840 --> 17:12.540] deploy AI systems. And, of course, on a less business and more political level, [17:12.540 --> 17:18.540] citizens will see what rights are granted in the EU, and at least some of them could demand [17:18.540 --> 17:29.260] those rights as well. So, in the US, it's private businesses that are currently pushing harder for [17:29.260 --> 17:35.200] AI regulation than government agencies. Some of the big players, I mentioned them here, but it's [17:35.200 --> 17:41.480] also Salesforce that I mentioned earlier, have established internal principles on which they [17:41.480 --> 17:49.680] base their AI development in the future. This is not ideal to my mind, but it's a first step, [17:49.680 --> 17:56.100] because internally established principles of AI ethics are hardly enforceable, [17:56.100 --> 18:02.120] competitors are not bound by them. And also, there are so many guidelines around from a consumer [18:02.120 --> 18:09.200] point of view that I take, it's hard to really know where a company stands in terms of compliance, [18:09.200 --> 18:15.860] because everybody has their own rules. But at least it obliges the company that decides to go [18:15.860 --> 18:22.440] this way to face scrutiny about its products, but it isn't designed to really calm citizens' concerns [18:22.440 --> 18:27.260] on a broad base, because there are still players around who don't feel bound by any of those [18:27.260 --> 18:36.220] principles. And then there's self-regulation of industry. And by that, I mean that a whole [18:36.220 --> 18:41.540] industry designs and enforces new rules and standards for themselves when government rules [18:41.540 --> 18:49.480] are lacking. Usually, these would be codes of conduct, or binding rules that industry establishes [18:49.480 --> 18:56.700] for itself, for example, within federations. The hope would be that industry collective action [18:56.700 --> 19:03.940] can change the incentives in a situation where the competitive dynamic of AI development leads [19:03.940 --> 19:11.820] to a prisoner dilemma for both companies and state, wherein both are incentivized to [19:12.420 --> 19:18.500] prioritize the fast development of AI instead of the safe development of AI. [19:18.500 --> 19:23.740] But unfortunately, this hasn't been successful, or at least not impactful for now. [19:25.120 --> 19:31.420] And that's why some companies actively push government for more committed steps in the US. [19:31.420 --> 19:36.360] I've often heard people in companies say that they look to Europe for these obligations to come [19:36.360 --> 19:43.740] up. They are mostly the ones who've seen the success of GDPR and are convinced the US needed [19:43.740 --> 19:50.260] similar groundbreaking law regulating technological development, but they don't see that the current [19:50.260 --> 19:57.080] US administration is in any way interested in pushing that. The dominant view in DC for the [19:57.080 --> 20:04.920] moment is that the US needs to lead in AI development to compete with China, no matter [20:04.920 --> 20:14.940] what. But of course, we have elections in November. So please vote. At last, and last, there's the way [20:14.940 --> 20:23.180] through local and state laws. We've seen that with data protection, that when the federal level in [20:23.180 --> 20:30.020] the US doesn't move enough, the states engage more. There are some examples that state and [20:30.020 --> 20:36.180] local level have moved that are pretty promising. They don't have the reach to create general [20:36.180 --> 20:43.840] nationwide regulation, of course, but they're creating smaller scale uses, use cases for [20:43.840 --> 20:52.280] regulation, like regulatory sandboxes, but in real life, that could be useful once the federal level [20:52.280 --> 21:00.720] is ready to engage. So for instance, the California Consumer Privacy Act, CCPA, that took effect last [21:00.720 --> 21:09.620] year, showed that a US state can issue impactful legislation that addresses concerns about data [21:09.620 --> 21:14.600] protection and that significant fines can be levied against misuse of personal data. [21:14.900 --> 21:23.300] So the CCPA is far from perfect. And it started to be enforced only now, but I don't really see [21:23.300 --> 21:30.120] how it will be effectively enforced, given so many imperfections. But still, it was quite a step. [21:30.120 --> 21:35.900] And many US states have meanwhile issued their own privacy legislation. However, [21:36.600 --> 21:42.020] none of them is as far reaching as the California law. [21:43.020 --> 21:48.640] A more recent example, so more specifically from the AI field, again, from California, [21:48.640 --> 21:56.400] is the BOT, the Bolstering Online Transparency Act, which makes it unlawful for any person to [21:56.400 --> 22:01.460] use a bot to communicate or interact with another person in California online, [22:01.460 --> 22:05.710] with the intent to mislead the other person about its artificial identity. [22:06.260 --> 22:13.060] This legislation specifically applies to bots that intend to influence voters, [22:13.060 --> 22:19.940] as well as intentionally deceptive bots used to sell goods and services. It doesn't make bots [22:19.940 --> 22:27.220] illegal, but it requires them to identify themselves as non-human. So the law took effect [22:27.260 --> 22:33.360] a month ago, July 1st, only in California, and it doesn't clearly address how it applies to bots [22:33.360 --> 22:41.220] that don't originate in the Golden State, and how enforcement would work then. So one last example [22:41.220 --> 22:47.260] for local action on municipal level, this time regarding facial recognition technology, which [22:47.260 --> 22:57.120] is the most controversial area of AI use today. In this case, when so far no federal guidelines [22:57.120 --> 23:03.820] exist to limit or standardize its use, and few state rules are in place, cities feel left to [23:03.820 --> 23:11.960] decide for themselves what, if anything, to do. So the city and county of San Francisco last year [23:11.960 --> 23:18.600] passed a bill to ban facial recognition technologies for police and city agency use. [23:18.740 --> 23:25.780] So other US cities have followed suit. In this case, it is only one sector, government, [23:25.780 --> 23:32.280] that is required to refrain from the use of this technology, while the private sector and [23:32.280 --> 23:39.500] individual can still use it. But it shows an awareness that with the current state of technology [23:39.500 --> 23:46.820] development, organizations clearly aren't ready to use facial recognition in a safe, secure, and [23:46.820 --> 23:53.260] responsible manner. And that there is a need for clear legislation that, on a federal level, so far [23:53.260 --> 24:01.920] no one has managed to pass. As I mentioned before, what most technology companies are saying is that [24:01.920 --> 24:07.340] they look not to DC, but to Europe when it comes to effective regulation in that field. [24:08.480 --> 24:15.180] So what I've been doing recently is to interview companies in the Bay Area working on AI systems [24:15.180 --> 24:22.480] about the expected impact of European regulation on them, and about their preparedness and their [24:22.480 --> 24:29.900] sense of agreement with these principles. My focus, I mentioned that, is on small and medium-sized [24:29.900 --> 24:36.240] companies that are less in the focus of the media and that face the additional challenges competing [24:36.240 --> 24:43.100] with the tech giants in a field where size clearly matters. Because the more data you can gather, [24:43.100 --> 24:48.740] the better your AI works. So I'm also interested in this type of company because the general [24:48.740 --> 24:55.500] impression with GDPR was that the small players have been losing ground due to compliance costs, [24:55.500 --> 25:03.860] and that regulation blocks them more than the big ones. So far, my interviews show that most, [25:03.860 --> 25:11.520] if not all, contacts have an understanding of the importance of rule-setting around AI-based systems, [25:11.520 --> 25:16.660] and that they report having thought about limiting their own developments [25:16.660 --> 25:24.080] until political guidance is given what is admissible and what is not. All of them had [25:24.080 --> 25:30.380] experienced some kind of adaptation process to GDPR two years ago. Some of them were my clients, [25:30.380 --> 25:38.320] so I helped them with that. And they reported that having spent considerable money and time [25:38.320 --> 25:44.380] on compliance with rules that originated in Europe, but that several of them applied to [25:44.380 --> 25:51.620] all of their products and all users wherever they are located. Now, with AI regulation coming up, [25:51.620 --> 25:58.440] they expect the same to happen, but there is disagreement on whether this kind of general [25:58.440 --> 26:05.680] regulation is even possible in a broad field like AI. So they consider it too broad to fall under [26:05.680 --> 26:12.340] one rule. That includes small applications that are considered to simply make life easier, [26:12.900 --> 26:21.560] as well as large systems of mass surveillance. So importantly, many of them also said that they [26:21.560 --> 26:27.900] expected European legislation to be faster, more comprehensive, and more enforceable than any [26:27.900 --> 26:35.420] US attempt for regulation. Maybe that's a lesson learned from GDPR and doesn't mean that [26:35.420 --> 26:42.760] this would be the same with the upcoming policies in the AI sector, but it's a good indicator of the [26:42.760 --> 26:49.300] urge many players feel to prepare for rules and not to leave the space entirely to the big players [26:49.300 --> 26:57.100] and their lobbying power. So in my mind, although the EU's upcoming rules are geared primarily [26:57.100 --> 27:04.260] to European firms and will become law only potentially sometime in the future, [27:04.260 --> 27:09.580] US companies need to look at them already now. So it won't be enough to just wait and [27:09.580 --> 27:15.220] see, and it's better to prepare for the probability of AI regulation in Europe now. [27:17.600 --> 27:23.540] So again, if you have thoughts about this as an individual, but also if you work for a company [27:23.540 --> 27:28.280] that fits into what I'm looking at, I'd love if you reached out and gave me a feedback. [27:28.280 --> 27:32.920] Here's my website and Twitter handle at the bottom of the page. And if you're interested [27:32.920 --> 27:38.120] in reading more about this field and add your comments, have a look at the white paper that [27:38.120 --> 27:43.140] we're working on with the Mozilla Foundation at the link in the center of the slide. [27:43.380 --> 27:48.440] So thank you for attending. I wish I could have seen you all in person. [27:48.500 --> 27:53.880] I hope you're all safe and stay healthy. And I'm looking forward to your questions. [28:00.800 --> 28:08.600] And welcome back. We have Julia here with us. If anybody has any questions, [28:08.600 --> 28:16.000] drop them in the Q&A channel and Discord. We've got a few here. Julia, I enjoyed your talk. Is [28:16.000 --> 28:21.080] there a document with more in-depth info about the requirements of trustworthy AI in high-risk [28:21.080 --> 28:35.060] applications? Yeah, so I directed the person who asked to the assessment list that has been [28:35.060 --> 28:41.320] published by the High-Level Expert Group this July, a couple weeks ago. And I think that shows [28:41.580 --> 28:48.100] a bit of what they expect, you know, in terms of actual requirements, very concretely. But of [28:48.100 --> 28:52.940] course, I mean, remember that this hasn't been finalized yet. This is just a proposal. Well, [28:52.940 --> 28:58.200] it's actually not even a concrete proposal yet. It will become a proposal probably around the [28:58.200 --> 29:07.200] beginning of 2021. So it's all in the making. Cool. Thank you. And then here's another question. [29:07.460 --> 29:12.620] What would you like to see happen with CCPA in regards to clarifying bots, whether from [29:12.620 --> 29:18.740] California or not, on how bots identify as non-humans prior to contact? What type of [29:18.740 --> 29:26.340] enforcement or not would you like to see? Yeah, so I think there are two aspects here, [29:26.340 --> 29:33.040] bots specifically, and then CCPA. For CCPA, I think my main doubt is that the Attorney General [29:33.040 --> 29:40.920] in California won't or doesn't have the bandwidth, really, the capacity to really enforce all the [29:40.920 --> 29:50.200] cases, because it really depends on him and his team to find and to inquire and then to enforce. [29:50.340 --> 29:57.860] And I don't think this will really work, especially because the text is not that clear. [29:58.640 --> 30:04.640] And so that is my main doubt. I mean, there are lots of others. You know, CCPA is really kind of [30:05.420 --> 30:14.420] not very well designed in terms of wording and terminology. But it's been enforceable only since [30:14.420 --> 30:20.140] 1st of July this year. So we'll see how that goes. In terms of bots, I think it's most useful [30:20.140 --> 30:34.620] to look at the bot bill that's been in effect also since July 1st here in California. It's a [30:34.620 --> 30:44.440] I think, I mean, also there, the application, the actual what it applies to is a bit narrow, [30:44.440 --> 30:51.340] but it shows how this could work. You know, you really, when you're contacted by a bot, [30:51.340 --> 30:56.560] you know exactly whether you're talking to you or not. And that would be the same also in the [30:56.560 --> 31:03.580] European proposal. I mean, not exactly the same that we don't know yet, but the idea is the same. [31:03.580 --> 31:09.360] That'd be interesting. All right, we have one last question here. Do you think that AI will [31:09.360 --> 31:14.920] develop that much to actually take us over, take us over in the sense that they might steal our [31:14.920 --> 31:21.640] jobs, our houses, our budget, etc? Well, I mean, there, of course, [31:21.640 --> 31:28.620] the eternal question is, what do you define as AI? And if we take a broad look at it, [31:28.620 --> 31:36.340] your wording of the question is, sounds a bit dramatic, but I mean, in a way, [31:36.340 --> 31:42.780] in some sectors, you could be right, because in the end, it's really important for humans [31:43.580 --> 31:52.740] to control what they're creating and to actually have a last say in what the technology is supposed [31:52.740 --> 31:59.920] to do. And so I think if we, I think the awareness is there to find rules, to make that, [31:59.920 --> 32:09.040] to make sure that this happens. So I do think that in some sectors, there's a potential for [32:09.040 --> 32:15.500] actual harm, yes. And for dramatic harm. I mean, this is in the very high risk sectors, of course. [32:15.500 --> 32:20.640] But even in the less high risk sectors, I think that, you know, democracy, fundamental rights, [32:20.640 --> 32:23.580] are already at stake. Yes. [32:24.500 --> 32:32.120] Wow. Well, thank you for taking the time to join us today. If you have any more questions, [32:32.120 --> 32:38.100] put them in the Discord and Julia will be able to answer them there. Thank you. [32:38.460 --> 32:39.000] Thank you. [32:39.000 --> 32:41.100] Our next talk up will be...