WEBVTT 00:00.000 --> 00:04.380 All right, it looks like we are resetting the stream. 00:04.600 --> 00:09.980 One quick second, let me get a heads up by our recording folks. 00:09.980 --> 00:10.740 Let's see. 00:13.690 --> 00:17.490 Let me just double-check while we get this, get this sorted. 00:17.710 --> 00:22.910 Yeah, and sometimes I would announce a class like welcome to Chem 107 or something, just something weird. 00:23.230 --> 00:28.830 I like to have fun with my students, but when I was teaching, it was good. 00:30.470 --> 00:33.950 I don't think I was ever bold enough to try anything like that. 00:36.010 --> 00:38.550 Or maybe I just had more pity for my students. 00:38.550 --> 00:40.130 I don't know, one of those two. 00:42.730 --> 00:50.010 Well, I lucked out and taught a, what do you call it, a simple class on... 00:50.610 --> 00:52.710 it was we were building an electric vehicle. 00:53.750 --> 00:54.950 Oh, hi everybody. 00:54.950 --> 01:02.310 For those of you that are just tuning in, we're holding out for one quick second while we get our recording AV equipment going. 01:02.970 --> 01:03.690 Mr. 01:03.690 --> 01:10.370 The Fixer, let me know when things are ready to go and we will commence. 01:13.750 --> 01:18.330 Let's see, it looks like... all right, so looks like things are ready. 01:18.330 --> 01:20.290 So let us begin. 01:20.490 --> 01:23.490 We're introducing our speakers. 01:23.490 --> 01:29.770 So everybody, welcome to the very first talk of Ethics Village on day two. 01:29.910 --> 01:39.110 So today we have Larry Lewis and we have Diane Bavacek, and they were going to be talking a little bit about killer robots, so reconsidered. 01:39.110 --> 01:41.130 So a little bit about Larry. 01:41.130 --> 01:47.190 Larry is the director of the Center for Autonomy and Artificial Intelligence at CNA. 01:47.190 --> 02:00.310 His areas of expertise include military applications of artificial intelligence and autonomy, reducing civilian casualties, identifying lessons from current operations, security systems, and counterterrorism. 02:00.550 --> 02:10.870 So Larry spearheaded the first database approach to protecting civilians in conflict, analyzing military operational data in conjunction with open source data. 02:10.870 --> 02:17.250 He has worked extensively to help reduce, to help militaries reduce civilian casualties in their operations. 02:17.250 --> 02:25.750 Larry has also served as a senior advisor to the Department of State's Assistant Secretary for Democracy, Human Rights, and Labor, and was on the U.S. 02:25.750 --> 02:31.410 delegation to the United Nations regarding lethal autonomous weapon systems or laws. 02:31.590 --> 02:35.690 Larry currently holds a PhD in physical chemistry from Rice University. 02:36.510 --> 02:51.850 And our other speaker, Diane Bavacek, is an analyst at CNA, where she researches military operations and policy with a focus on emerging technologies, including autonomy, artificial intelligence, unmanned systems, and the cyberspace domain. 02:51.850 --> 02:59.350 She deployed abroad on the USS Vinson during Operation Iraqi Freedom and Operation Inherent Resolve. 02:59.350 --> 03:03.850 Diane currently has a PhD in mathematics from the University of Michigan. 03:04.230 --> 03:12.110 And without further ado, welcome again, y'all, and thank you again for doing this talk for Ethics Village 2020. 03:12.110 --> 03:17.510 Let me queue up those slides and I will pass it off to you. 03:17.830 --> 03:22.050 So I'm gonna go mute. 03:23.070 --> 03:24.070 Great. 03:24.070 --> 03:27.810 So before we get to the slides, just a few introductory comments. 03:27.810 --> 03:28.850 So thank you. 03:28.970 --> 03:30.630 Thank you for joining us. 03:30.630 --> 03:34.670 Diane and I are really excited to talk to you guys. 03:35.050 --> 03:36.550 Just a few things. 03:36.550 --> 03:51.250 If this is your first acquaintance with CNA, we're a nonprofit research organization and we provide objective and independent analysis and perspectives to the U.S. 03:51.250 --> 03:52.350 government, including the U.S. 03:52.350 --> 03:55.910 military, but also other parts of the government. 03:56.970 --> 04:08.270 And several years ago, CNA established a Center for Autonomy and Artificial Intelligence because those technologies are so important to the U.S. 04:08.270 --> 04:10.910 and national security priorities. 04:10.910 --> 04:17.090 So and a key focus for us at CNA is AI safety. 04:17.110 --> 04:19.910 So we're going to be talking about that today. 04:20.110 --> 04:26.870 This presentation incorporates a number of different areas of CNA research. 04:26.870 --> 04:34.970 At the same time, this presentation reflects our opinion and not that of CNA or the U.S. 04:34.970 --> 04:35.600 government. 04:36.520 --> 04:40.340 Finally, we really want your participation today. 04:40.380 --> 04:43.280 So we want this to be participatory. 04:43.280 --> 04:49.540 So you can send us a question in the Discord general text chat. 04:49.540 --> 04:50.800 We're monitoring it. 04:50.800 --> 04:52.360 I have it open right here. 04:52.940 --> 04:56.400 So feel free, please, to ask questions. 04:56.400 --> 04:57.720 We'd love to answer them. 04:57.720 --> 05:03.000 We're aiming to finish this presentation and the general question and answer in an hour. 05:03.000 --> 05:07.920 But we are also happy to stick around longer if we haven't gotten your questions. 05:07.920 --> 05:12.760 So feel free to stick around if we haven't answered it, and we will gladly get to it. 05:12.760 --> 05:14.920 So with that, we'll go ahead and get started. 05:14.920 --> 05:16.980 And Diane is going to start us off. 05:17.820 --> 05:18.640 Thanks, Larry. 05:18.640 --> 05:20.160 And let me echo what Larry said. 05:20.160 --> 05:24.300 Thanks to the Ethics Village for hosting us. 05:24.880 --> 05:27.220 Steve, you can go ahead to the next slide, please. 05:27.580 --> 05:28.380 Thanks. 05:28.940 --> 05:31.720 So what are we going to talk to you about today? 05:32.580 --> 05:35.440 So here's kind of a general outline. 05:35.840 --> 05:38.860 Militaries are turning to artificial intelligence. 05:39.040 --> 05:39.780 And the U.S. 05:39.780 --> 05:46.520 Department of Defense is certainly one of those that sees moving forward with AI as an imperative. 05:47.480 --> 05:56.020 And the DoD is going to need the tech sector, the help of the tech sector, the help of AI researchers in order to be competitive here. 05:56.860 --> 06:02.040 There are risks associated with incorporating AI into military applications. 06:02.440 --> 06:07.420 A lot of risks and challenges that are really important and crucial getting right. 06:07.420 --> 06:10.420 But there are also a lot of misconceptions that come up in this conversation. 06:10.420 --> 06:13.520 And so we're going to talk both about challenges and the misconceptions. 06:14.140 --> 06:21.380 And DoD practices and policies that are in place right now that address some of these risks. 06:21.900 --> 06:36.000 And again, to go back to the role of the tech sector and AI researchers here, addressing some of the challenges associated with AI in this setting is something that we think the tech sector can really help DoD to do. 06:36.580 --> 06:45.200 In addition to risks, though, there are actually benefits, opportunities associated with integrating AI into military applications. 06:45.200 --> 06:54.740 So there's a real potential for benefits like fewer civilian casualties or protecting hospitals, different humanitarian benefits that we'll talk through. 06:55.100 --> 07:05.660 And again, to bring back to what is the role of tech sector here, we've seen through history that outside influences can motivate DoD to try harder in these areas. 07:06.260 --> 07:13.860 So what we have here is a new paradigm with AI coming into applications in the military and more broadly. 07:14.000 --> 07:22.780 And we think that you should see this as an opportunity to help and influence the way that AI is used in warfighting. 07:22.840 --> 07:24.060 So this is our aim. 07:24.160 --> 07:30.720 The plan for this talk is that I'm going to talk through the first few points and then Larry will talk through the later points. 07:30.720 --> 07:36.180 And then we will spend some time going through the questions that we see in the chat. 07:37.920 --> 07:39.800 All right, go to the next slide, please. 07:42.960 --> 07:45.960 So what are we talking about here, first of all? 07:45.960 --> 07:48.720 How are we thinking about the definition of AI? 07:48.760 --> 07:51.680 So to be consistent with how the U.S. 07:51.680 --> 07:54.060 government uses this term, we're thinking of this broadly. 07:54.060 --> 08:03.200 So one definition you can think of for AI is a set of techniques, including machine learning, that is designed to approximate a cognitive task. 08:06.680 --> 08:16.340 So one kind of piece that I want to get at, at this point, is to talk about how people group AI into kind of two broad bins. 08:16.340 --> 08:19.700 So people talk about artificial general intelligence. 08:19.900 --> 08:24.880 And the idea there is kind of thinking of the Terminator, sort of, right? 08:25.160 --> 08:29.000 Is artificial intelligence that can do what a human can do? 08:29.000 --> 08:33.140 That can understand or learn any cognitive task that a human can do. 08:33.480 --> 08:45.680 So artificial general intelligence is not the state of technology today, and it's not expected to come to fruition in the foreseeable future at this point. 08:45.760 --> 08:47.940 So this is not going to be the focus of our talk today. 08:47.940 --> 08:52.540 What we're going to focus on is the other category, which is narrow artificial intelligence. 08:52.900 --> 08:59.800 So narrow AI is AI that's focused on a specific task, something like image classification, right? 08:59.800 --> 09:03.480 So it carries out a specific task within a specific context. 09:05.300 --> 09:09.140 Why is AI relevant to warfighting? 09:09.160 --> 09:13.200 Well, one reason is that AI helps you make better decisions. 09:13.440 --> 09:21.040 So if you might want the best possible decision-making tools to be used in areas of your life that you care a lot about, right? 09:21.040 --> 09:31.020 Maybe the person managing your investments or a doctor giving you diagnosis, if you want them to have access to the best technologies to help them with decision-making, right? 09:31.020 --> 09:31.820 It's the same idea. 09:31.820 --> 09:35.580 This is why military commanders, this is why U.S. 09:35.580 --> 09:40.660 government leadership wants to draw from this technology. 09:41.720 --> 09:51.660 Also in warfighting, AI can be useful if your military is facing a threat that is vast or numerous or very dynamic. 09:51.660 --> 09:57.160 So think about something like a drone swarm attacking a naval ship, right? 09:57.160 --> 10:01.900 So that's the kind of setting that DoD cares very much about, being able to combat that threat. 10:01.900 --> 10:06.040 And so this is another place where AI can be really critical. 10:06.480 --> 10:16.360 So for all of these reasons, DoD and other militaries around the world are moving forward with exploring and investing in artificial intelligence. 10:17.460 --> 10:20.860 What type of military applications are we talking about here? 10:21.020 --> 10:22.740 So a few examples, right? 10:22.740 --> 10:25.100 And I've listed a few bins here in the slide. 10:25.440 --> 10:30.440 So AI can help with by providing decision aids on the battlefield. 10:30.460 --> 10:36.660 So things like detecting anomalies in behavior, the behavior of say people on the ground or shipping traffic. 10:37.340 --> 10:40.680 Sensor data fusion is another application. 10:41.480 --> 10:48.440 Human machine teaming is also a key area of AI applications. 10:48.440 --> 10:58.820 So think about maybe an unmanned ground system that is assisting a soldier by carrying a lot of heavy gear or by providing surveillance where the soldier is patrolling. 10:59.460 --> 11:05.020 And business applications are also something that are going to be important to DoD going forward. 11:05.340 --> 11:14.460 AI uses in say preventative maintenance, network defense, or even to figure out how to target recruiting efforts. 11:16.940 --> 11:31.760 But an AI is going to, you know, there are important challenges of artificial intelligence that comes into those type of applications, but that's not the most contentious type of applications that we're talking about here. 11:31.880 --> 11:41.420 So DoD and other militaries are also considering how you might integrate AI into autonomous systems or weapon systems, right? 11:41.420 --> 11:46.560 So autonomous systems being systems that operate in some sense on their own, right? 11:46.560 --> 11:50.940 And you can combine that with a lethality component too. 11:50.940 --> 11:58.820 And so here's a case where you really see that it's critical that we make sure to get it right moving forward with this technology. 12:00.180 --> 12:01.400 Next slide. 12:03.100 --> 12:05.960 Okay, so DoD and the U.S. 12:05.960 --> 12:12.380 government more broadly are thinking about how to use AI in national security. 12:12.500 --> 12:21.340 So they're already moving ahead with creating strategies and guidance, making some organizational changes, and investing in research and development. 12:21.340 --> 12:28.980 And so I've listed some examples of these types of developments in the slide. 12:28.980 --> 12:38.000 The one that I'll anchor on here is this first sub-bullet, which is the recently established DoD AI strategy. 12:38.740 --> 12:46.680 And this is a document that... this and most of the other documents that I reference here are things that are available open source. 12:46.680 --> 12:50.800 So you can just Google and find at least an unclassified summary of the strategy. 12:51.580 --> 12:59.460 So a couple of things that I'll just list off here, just to anchor on this piece, a couple of things that they say in the strategy, right? 12:59.460 --> 13:04.380 So they emphasize that the women and men in the U.S. 13:04.380 --> 13:06.920 armed forces are the real source of strength. 13:06.920 --> 13:13.260 And the intent is for AI to empower but not replace those who serve. 13:14.240 --> 13:25.020 Another thing that the strategy says is that AI is poised to change the character of the future battlefield and the pace of the threats that we must face. 13:25.880 --> 13:39.820 And so I just want to point out that you can see from that line that the DoD is... the context that DoD is thinking about this in is one in which they expect that adversaries will also be using this technology. 13:40.440 --> 13:44.380 So that's kind of the landscape that we're thinking of here. 13:45.640 --> 13:47.180 Go ahead to the next slide. 13:48.140 --> 13:48.980 Thanks. 13:49.880 --> 13:57.160 So AI really represents a paradigm in terms of the relationship between DoD and the private sector. 13:57.160 --> 14:19.380 That's a new and interesting dynamic, which is that in the past, we're used to thinking of DoD as having unique needs with research and development and that they very much bankroll the research in order to develop platforms they need, like military aircraft or develop weapon systems technologies. 14:20.000 --> 14:30.040 Even early versions of the internet were something that DoD bankrolled because it was something that they needed and the developments weren't available elsewhere. 14:30.100 --> 14:33.080 So that's the dynamic that DoD is used to having. 14:33.080 --> 14:44.460 But bringing AI into the picture is a big change because obviously AI is something that has many uses outside of just military applications. 14:44.460 --> 14:51.940 And because of that, tech sector is already moving forward rapidly with research and development. 14:52.320 --> 15:03.240 So the graph here is one where we kind of try to get at this snapshot of the difference between U.S. 15:03.240 --> 15:14.840 government and tech sector spending here, which is that you can see the lower bars in the graph are the budget that U.S. 15:14.840 --> 15:21.580 government overall, information technology research and development, including AI. 15:22.040 --> 15:33.440 And the taller light purple bars are we just took just five top tech companies and it is their summed research and development investments. 15:33.440 --> 15:38.600 And so you can see that just looking at that snapshot, DoD and the U.S. 15:38.600 --> 15:42.860 government overall are going to be dwarfed by this type of research and development spending. 15:42.860 --> 15:53.520 And so this just illustrates that DoD has a real dependency on what's happening in the private sector, what's happening with external research and development in AI. 15:55.680 --> 15:57.420 All right, next slide, please. 15:59.380 --> 16:01.480 Okay, so now let's talk about risks. 16:02.660 --> 16:14.020 Actually, let's try, hopefully this will work, to see if you guys can, this is one of the areas where we really want to try to have as much of a dialogue as we can with you, the listeners. 16:14.020 --> 16:18.180 So I'm going to talk through sort of the landscape of risks. 16:18.180 --> 16:28.280 As I'm doing that, we would love it if you would type into the Discord chat, what are your biggest concerns here with regard to military applications of AI, right? 16:28.280 --> 16:30.860 Or what are questions or thoughts that you have on this? 16:31.420 --> 16:38.140 And after I talk through some of these risks, we'll see if Larry can read off some of what you all wrote in. 16:39.180 --> 16:52.400 Okay, so AI, also thinking of AI potential use in lethal autonomy, these are really controversial ideas in the public sphere. 16:52.680 --> 17:06.760 I don't know if how many of you were at DEFCON and came to Ethics Village sessions last year, you may have seen a talk given by a group that's a proponent of preemptively banning lethal autonomous weapon systems. 17:07.900 --> 17:15.220 So there's a real kind of an active debate and dialogue about some of these questions. 17:15.220 --> 17:19.320 And so what are the concerns that people are voicing here? 17:20.200 --> 17:23.720 All right, well, one concern is that AI will destroy the world. 17:23.780 --> 17:34.900 So that's one that we've seen, for example, Elon Musk say quite publicly, even as I'll mention, even as, of course, Elon Musk has simultaneously been investing in AI research. 17:35.260 --> 17:41.060 But AI and lethal autonomy are inherently unlawful or unethical. 17:41.060 --> 17:47.420 This is an assumption that is made by the proponents of banning these types of technologies. 17:48.160 --> 18:01.820 There are questions about accountability, concerns about discrimination, which means the ability of a weapon system to tell a combatant from a civilian. 18:02.480 --> 18:09.760 Of course, biases are also an important challenge and concern when it comes to AI. 18:10.860 --> 18:12.740 Let's go to the next slide. 18:15.940 --> 18:31.260 So embedded in these concerns are some real and critical challenges that are actually going to be they appear they're going to be surmountable, but they're important challenges to think about. 18:31.260 --> 18:34.640 But also embedded in these are some real misconceptions. 18:35.340 --> 18:39.820 Okay, so just to talk through some of this, AI will destroy the world. 18:40.220 --> 18:48.360 Okay, so that concern is really rooted in worries about artificial general intelligence. 18:48.360 --> 18:50.720 And that's not what we're talking about here. 18:51.340 --> 18:57.900 But what about questions of lawfulness, ethics, and accountability? 18:57.900 --> 19:00.120 So those next two points. 19:01.080 --> 19:08.800 So these concerns are rooted in a misunderstanding of how DoD operates. 19:08.800 --> 19:17.800 So DoD follows many domestic and international laws, follows many ethical principles. 19:17.800 --> 19:24.380 And these laws and principles don't depend on the type of technology that you're using. 19:24.380 --> 19:29.060 So the laws of war apply no matter what weapon you're using. 19:29.520 --> 19:32.880 So similarly, accountability. 19:32.880 --> 19:37.140 Accountability is also baked into how DoD operates. 19:37.140 --> 19:43.180 From acquisition, test and evaluation standards, to the chain of command. 19:43.180 --> 19:48.620 And also, this doesn't depend on the type of technology that you're talking about. 19:50.180 --> 20:03.320 Although certainly things like test and evaluation specific processes will have to be updated for new technologies as they come through, including for artificial intelligence. 20:04.460 --> 20:15.780 So that said, there definitely, not to discount, there will absolutely be challenges in developing military applications of AI. 20:16.040 --> 20:20.360 So in particular, you want to make sure that it really works. 20:21.660 --> 20:30.840 So integrated into that is making sure that systems can be discriminant, that systems will, or use of systems will address bias. 20:30.940 --> 20:36.900 Basically, you want to make sure that systems will operate as intended in any other ways. 20:37.600 --> 20:41.040 And so just a couple quick comments here. 20:41.040 --> 20:44.480 We say that these concerns are surmountable. 20:44.480 --> 21:02.840 So some of the way to think about that here is that the goal is not necessarily to, say, develop some kind of AI-enabled weapon system that is able to discriminate civilians from combatants in all possible situations. 21:03.260 --> 21:12.180 So the point is that you would only approve a system for use in settings in which you knew it was going to be able to function properly. 21:12.180 --> 21:15.040 So that's kind of the aim and the goal here. 21:15.040 --> 21:18.860 This isn't going to be something that works in all settings. 21:18.860 --> 21:27.180 Of course, that's true of legacy non-AI-enabled systems and weapons as well. 21:28.480 --> 21:37.340 Also, we're not necessarily aiming for, say, systems that are learning in real time, especially weapon systems. 21:37.340 --> 21:52.540 So it's more likely to be thinking about at this stage, right, maybe systems that will be collecting data in real time, but that maybe it would not go ahead and update how it's making decisions until those updates can be validated. 21:53.000 --> 21:56.960 So keep some of those important caveats in mind here. 21:58.720 --> 22:04.060 Larry, did anybody write anything good in the chat about concerns and risks? 22:06.360 --> 22:08.460 Yes, definitely. 22:08.460 --> 22:19.620 So a few things here are... there's a link to a report that is a really good report from early 2019. 22:19.620 --> 22:23.600 And it talks about different risks of AI. 22:23.600 --> 22:29.700 And I think it's actually good to kind of frame this. 22:29.700 --> 22:33.560 So we are talking specifically about military applications of AI. 22:33.560 --> 22:41.880 So the report that we do recommend you look at talks about broader risks of AI. 22:41.880 --> 22:44.360 And it talks about three different kinds of risks. 22:44.360 --> 22:46.100 One is digital security. 22:46.100 --> 22:50.340 So digital security, physical security, political security. 22:50.340 --> 22:56.120 So digital security goes to cyber attacks and the kinds of infiltration I can do. 22:56.200 --> 23:06.100 Physical security, they give the example of the potential for using an autonomous self-driving car as a weapon. 23:06.100 --> 23:09.420 So like a terrorist attack that you can use that. 23:09.420 --> 23:12.820 And then third, they talk about political security. 23:13.020 --> 23:18.660 And that includes misuse of surveillance, propaganda, misinformation. 23:18.660 --> 23:21.620 And these are all really great points. 23:22.660 --> 23:31.160 So the fact that we don't have those on there is because we're kind of talking about the more narrow military applications of AI. 23:31.160 --> 23:35.320 But it's important to acknowledge that there are broader risks as well. 23:35.320 --> 23:37.120 Larry, what is that? 23:37.120 --> 23:38.780 What report is it that you're referring to? 23:38.780 --> 23:40.120 Or who made it? 23:40.140 --> 23:41.960 Yeah, there's a link. 23:42.240 --> 23:47.280 And see, actually, who did it? 23:48.100 --> 23:50.460 I've looked at the link at this before. 23:50.460 --> 23:54.760 There are many authors from different organizations. 23:54.800 --> 24:04.460 So University of Oxford, Future of Humanity Institute, CNAS was also a contributor, OpenAI. 24:04.480 --> 24:09.320 So it's a multi-author, multi-organization report. 24:09.320 --> 24:13.480 But the link is in the general text channel. 24:13.480 --> 24:15.960 So definitely, it's good to check out. 24:17.140 --> 24:17.900 Yeah. 24:18.740 --> 24:28.020 So another comment is about bias and how bias seems like a really big deal, which is it absolutely is. 24:28.320 --> 24:35.140 And I guess one comment to make about that is that humans are biased, right? 24:35.980 --> 24:38.840 Humans have lots of cognitive biases. 24:38.840 --> 24:48.140 And so AI applications that are built by humans that leverage data that are developed by humans should also be expected to be biased. 24:48.140 --> 24:49.480 So that is absolutely true. 24:49.480 --> 25:00.500 The one thing to me that's a key difference is that with AI applications, you can explicitly examine those biases and try to correct for them. 25:00.500 --> 25:04.260 So that's actually a valuable thing that we can do. 25:04.260 --> 25:07.140 However, too often, we don't do that. 25:07.140 --> 25:14.560 So we need to be very deliberate about trying to characterize and correct those biases. 25:15.400 --> 25:16.220 Let's see. 25:16.220 --> 25:27.280 Another comment is majority of DoD spending on AI goes to warfighting and lethality, not logistics, humanitarian assistance, predictive maintenance, logistics. 25:27.740 --> 25:29.540 Thank you for that comment. 25:29.540 --> 25:33.600 We're actually going to... that is a big topic of our talk. 25:33.600 --> 25:37.740 So please keep on listening and participating. 25:38.780 --> 25:43.800 Do AI weapon systems fall under the Geneva Convention? 25:43.800 --> 25:48.760 Um, so if it's used for armed conflict, then it should. 25:49.020 --> 25:56.460 And I think you're going to you're going to continue to talk about this, kind of the legal aspect of this. 25:57.500 --> 26:01.180 bleed over from DoD and the intelligence community. 26:02.060 --> 26:05.560 And that's, that's also it. 26:05.560 --> 26:22.040 That is a tricky issue today, because because of the um, like, we have some some US activities that do kind of combine the DoD and intelligence community. 26:22.040 --> 26:25.340 So that is something that kind of needs to be watched. 26:25.340 --> 26:30.060 And then how do we ethically import bias into AI? 26:30.060 --> 26:38.960 And again, I think, going back to the earlier comment about bias, bias is really a significant issue we need to be careful about. 26:39.500 --> 26:44.720 And I think, you know, yes, the Microsoft chatbot is a great example. 26:45.440 --> 27:01.880 And how, again, we can't just do something and then put it out in the world and then assume it's going to work well, we need to monitor it, we need to be explicitly monitoring for bias and intervening as well. 27:01.980 --> 27:10.420 So that kind of gets it what I said a minute ago about making sure to validate as as you're learning, right, we're gonna we're gonna have to validate that learning. 27:12.750 --> 27:14.470 So great comments. 27:15.890 --> 27:16.730 Okay, great. 27:16.730 --> 27:19.110 Well, thank you guys all and keep them coming. 27:19.110 --> 27:25.150 Um, we will we will come back to looking at the at the comments in a little bit. 27:25.750 --> 27:28.270 I'll go go ahead for the next slide, please. 27:29.750 --> 27:31.490 Okay, I'm gonna say a few words now. 27:31.490 --> 27:31.810 Okay, right. 27:31.810 --> 27:38.670 So, so there are misconceptions out there, but also a lot of very real challenges and risks. 27:39.170 --> 27:44.550 And DOD is doing some things now that address some of these risks. 27:44.550 --> 27:52.950 And so any, any, any discussion of this topic, right needs to have have this information as part of the context. 27:52.970 --> 28:05.430 So to run through some of that, right, some some of the things that DOD does that address these risks are just general DOD practices, and some of them are things specifically being developed for AI in a related context. 28:05.430 --> 28:08.150 So, so first, I'll talk about the general practices. 28:08.290 --> 28:12.230 Okay, so, so legal considerations, right. 28:12.230 --> 28:26.930 So, so, to kind of reiterate something I mentioned before, DOD incorporates legal considerations into, you know, pretty much everything it does development certification, use of any systems. 28:28.130 --> 28:39.970 So, so, and these laws, you know, laws of war, they apply no matter what type of technology you're using, right, things like proportionality, that, that, that's a principle that does not depend on the weapon that you're using. 28:39.970 --> 28:44.510 And so, so these things generally apply independent of the technology. 28:45.930 --> 28:51.110 Practices also help to mitigate some of these risks, right. 28:51.110 --> 28:55.730 So test and evaluation processes need to make sure that systems function as intended. 28:56.250 --> 29:05.510 Whenever you have a new system, also training will be established for that and operating procedures for using any kind of system, especially weapon systems. 29:06.650 --> 29:12.330 Policy is also a really important part of how DOD operates. 29:12.330 --> 29:21.270 So one example I'll give here is a 2016 executive order that's focused on minimizing civilian casualty. 29:21.690 --> 29:25.470 So this executive order says a lot of really great things. 29:25.470 --> 29:31.170 And I want to just highlight one, one piece of it, which I think is especially interesting in this context. 29:31.170 --> 29:35.230 So, so the, the EO says that the U.S. 29:35.230 --> 29:46.270 is going to develop, acquire, and field weapon systems and other technological capabilities that further enable the discriminant use of force in different operational contexts. 29:46.670 --> 29:48.410 So what are we saying there, right? 29:48.410 --> 29:56.710 We're saying that if there's technology that comes along that will allow us to be more discriminant in war fighting, you know, we're going to look at using that. 29:56.970 --> 30:08.990 And so, so keep that in mind because what Larry's going to talk about in a, in a minute or two here is, is the, you know, the very real possibility that AI can, can help us be more discriminant in war fighting. 30:11.170 --> 30:13.130 See, please go to the next slide. 30:13.670 --> 30:20.130 So now I'll talk just briefly about some AI specific steps that the DoD is taking. 30:20.130 --> 30:23.410 I mentioned the AI strategy earlier. 30:23.490 --> 30:29.570 DoD also very recently adopted some ethics, AI ethics principles. 30:29.570 --> 30:34.330 These came from the work of the Defense Innovation Board, right? 30:34.330 --> 30:46.210 And so these are, these are principles that, that include, you know, appropriate levels of human judgment, the intent that that will allow it if they exhibit unintended escalatory behavior. 30:46.210 --> 30:50.210 So this is, this is really the intent of DoD moving forward. 30:50.910 --> 31:09.010 So another, another related policy dimension, right, is a DoD has a directive on lethal autonomous weapon systems, which was actually from all the way back in 2012, which again says that appropriate levels of human judgment will always be used when it comes to lethal autonomy. 31:09.750 --> 31:20.210 And, and it also puts out that in many cases, a lethal autonomous weapon system would have to undergo a senior level review before it would ever be used by DoD forces. 31:20.570 --> 31:23.730 Test and evaluation requirements are being looked at, right? 31:23.730 --> 31:29.370 So all of these steps are good and they do start to get at a lot of the risks that we talked about. 31:29.770 --> 31:37.870 But as, as Larry alluded to, with talking about bias a minute ago, right, we, we would put out there that more is needed here. 31:37.870 --> 31:48.450 And in particular, what we don't see right now and really, really is necessary is a comprehensive approach to identifying and systematically addressing risks. 31:49.750 --> 31:59.410 So I've, I've talked here about how AI represents a new dynamic between DoD and the tech sector, and how there are a lot of challenges out there. 31:59.410 --> 32:03.190 And we're already working to address some of those challenges. 32:03.610 --> 32:08.010 I want to turn things over now to Larry, and we'll hear a little bit about the opportunities. 32:11.910 --> 32:12.550 Great. 32:12.550 --> 32:14.250 So if you can go to the next slide, please. 32:15.650 --> 32:16.170 Okay. 32:16.170 --> 32:29.510 So we've heard about that there are risks of military use of AI, how those risks can be different than those that are sometimes commonly discussed, and that we need to be more deliberate in addressing them. 32:29.510 --> 32:33.170 At the same time, we'll highlight that there are opportunities. 32:33.410 --> 32:35.770 I'll talk about three things. 32:35.790 --> 32:40.270 One is there is potential for good with the use of technology. 32:40.270 --> 32:46.370 AI and autonomous systems can be used deliberately for humanitarian benefits. 32:46.370 --> 32:52.830 Second of all, history shows that outside influence can help, can help the U.S. 32:52.830 --> 32:57.550 military work harder on focusing on humanitarian issues. 32:57.550 --> 33:11.270 And finally, times, things are, times are changing, and there are new opportunities where the tech sector can have an influence on national security policy that it has not had in the past. 33:11.770 --> 33:15.130 So if you can go to the next slide, please. 33:17.830 --> 33:18.390 Okay. 33:18.390 --> 33:27.110 So, so an argument is often made how advanced technology has humanitarian benefits. 33:27.110 --> 33:37.150 So an example that I hear a lot is, well, we have precision-guided munitions, they're much better for civilians on the battlefield than dumb bombs. 33:37.150 --> 33:37.430 Okay. 33:37.430 --> 33:39.870 And that is, that is generally true. 33:39.870 --> 33:47.810 At the same time, that what that really is, is a humanitarian side benefit of a military aim, right? 33:47.810 --> 33:53.570 So precision-guided munitions were not created to help innocent civilians. 33:53.570 --> 34:01.470 They were created to help the military to hit targets more effectively, and they happen to have a humanitarian side benefit. 34:01.590 --> 34:09.910 But technology can also be used deliberately to help address pressing humanitarian challenges and war. 34:10.050 --> 34:33.630 And some of those opportunities include reducing civilian casualties, you know, avoiding tragic attacks on hospitals and other protected sites, and also deconfliction of humanitarian aid on the battlefield so that those, you know, so that World Food Program trucks aren't targeted in military attacks. 34:33.630 --> 34:35.110 Next slide. 34:36.470 --> 34:39.290 So here's an example. 34:39.290 --> 34:43.810 So protecting hospitals in conflict. 34:43.810 --> 34:51.710 Medical care in conflict was the very first issue taking up as a measure of international humanitarian law. 34:51.710 --> 35:00.210 We talked about the Geneva Conventions, and there are other legal agreements that governments have agreed to over the years. 35:00.210 --> 35:03.650 Medical care in conflict was the very first issue taken up. 35:03.650 --> 35:14.850 And of course, the origin of ICRC was centered around providing medical care in conflict, making sure it was not interrupted by a war. 35:16.030 --> 35:31.990 So it was the first thing, and yet attacks on hospitals continue 150 years later after ICRC was created, after the first international law was created to help conflict be more humane. 35:31.990 --> 35:43.890 The picture here is an MSF hospital in Yemen that was destroyed by an airstrike by the Saudi-led coalition in August 2016. 35:44.010 --> 35:48.750 This kind of thing, this kind of attack, happens too many times. 35:49.170 --> 35:52.370 And you see the international community trying to do things. 35:52.370 --> 36:02.190 I think of the United Nations, they had a Security Council Resolution 2286 that calls the international community to action. 36:02.290 --> 36:04.410 We need to do more. 36:04.930 --> 36:07.750 And yet nothing practical has been done. 36:07.850 --> 36:11.190 So we have a really pressing problem with protecting hospitals. 36:11.790 --> 36:14.190 So to go to the next slide. 36:16.070 --> 36:18.330 There are things we can do. 36:18.330 --> 36:21.130 So we're a research organization. 36:21.370 --> 36:27.830 And in my grad school, I have a PhD in physical chemistry. 36:27.830 --> 36:40.690 So I had a laboratory with a laser and with liquid nitrogen kind of fuming to make things really cold and a vacuum chamber, and it smelled horrible. 36:41.150 --> 36:44.850 And that was kind of the environment that I worked in. 36:44.870 --> 36:48.670 Now at CNA, the world is our laboratory. 36:48.670 --> 37:03.370 So we go out and we go to places like Yemen and Afghanistan and Iraq and Syria, and we try to get the data to help us to better understand these kinds of problems. 37:03.960 --> 37:11.930 So one of the things I did is look at attacks on hospitals and looking at the data and say, okay, why do these things happen? 37:11.930 --> 37:14.070 What goes wrong when they're attacked? 37:14.070 --> 37:22.430 And our analysis shows that there are two kinds of failures that both have to happen simultaneously for a hospital to be attacked. 37:22.570 --> 37:26.690 One is a failure in deconfliction. 37:26.790 --> 37:37.350 And that is, if you look at modern militaries, they try to maintain a list of locations that are supposed to be protected and not attacked, and they call this a no-strike list. 37:37.350 --> 37:44.710 So they're supposed to be looking at lists when they are about to attack someone. 37:45.010 --> 37:51.110 And an attack on a medical facility means that this deconfliction step has failed. 37:51.630 --> 37:53.870 But that's not the only thing they have to do. 37:53.870 --> 37:57.910 They're also supposed to try to and avoid attacks on medical facilities. 37:57.910 --> 38:11.590 And one way is that, again, through the Geneva Conventions, identification of hospitals can be facilitated by the hospitals displaying a Red Cross or a Red Crescent. 38:11.590 --> 38:16.490 So an attack means that that identification process has also failed. 38:16.530 --> 38:22.010 So if you can fix either of these problems, you can stop the attack. 38:22.010 --> 38:32.310 And it'd be nice to work on both, but if you can make one of these things successful instead of a failure, you prevented a tragic attack on a hospital. 38:32.310 --> 38:33.980 So if you go to the next slide... 38:36.670 --> 38:42.650 technology can help reduce attacks on hospitals in various ways. 38:42.650 --> 38:44.870 So one example is Australia. 38:44.870 --> 38:47.230 They're using machine learning right now. 38:47.230 --> 38:55.650 They're exploring machine learning applications to help automatic recognition of protected symbols, like the Red Cross and the Red Crescent. 38:56.090 --> 38:58.390 So it doesn't automate the attack. 38:58.390 --> 39:00.410 Basically, it's a safety net. 39:00.410 --> 39:11.850 So if you're looking to make an attack and the machine learning algorithm senses that there's a Red Cross or a Red Crescent in the target area, it says, hey, stop. 39:12.590 --> 39:17.610 You need to stop. 39:18.310 --> 39:24.290 There are also opportunities to use technology to strengthen the deconfliction process. 39:24.290 --> 39:26.930 For example, White Flag is being developed. 39:26.930 --> 39:29.350 It's a protocol developed by the Netherlands. 39:29.610 --> 39:34.670 And that is an encrypted technology to strengthen this deconfliction process. 39:34.710 --> 39:51.690 And that's really important because what we see as we look at the data on this is that often the information that would stop an attack is known somewhere, but there is an information flow problem and it's not reaching the person that is making the decision whether to attack or not. 39:51.690 --> 39:57.630 So if we can improve that, then we can also help reduce attacks on hospitals. 39:57.770 --> 40:01.090 And the protection of hospitals is just one area. 40:01.090 --> 40:07.730 It's an important area, but just one area where technology can be used specifically for humanitarian benefit. 40:07.730 --> 40:14.930 We've also looked at how to reduce civilian casualties, how to detect civilian casualties, which is also an issue. 40:14.930 --> 40:20.650 Oftentimes you'll hear you'll hear militaries say, you know, they did a strike. 40:20.650 --> 40:25.010 There are accusations about civilian casualties and they say, nope, we didn't do it. 40:25.670 --> 40:32.970 And what the data says is oftentimes they did, but they're just not very good at detecting civilian casualties. 40:33.450 --> 40:38.690 So, and if you can't detect them, then you don't know you have a problem to solve. 40:38.690 --> 40:43.450 So this is also a really important issue that can be helped by technology. 40:43.450 --> 40:47.790 There's also anticipating and avoiding reverberating long-term effects of attacks. 40:47.790 --> 40:53.730 So essential services, that's something that militaries around the world really do poorly. 40:53.730 --> 40:55.350 And that could also be helped. 40:55.350 --> 40:59.770 And then there's also traceability and accountability for violations. 41:00.110 --> 41:10.600 The New York Times had a really powerful article on how Russia and the Syrian regime were specifically targeting hospitals in Syria. 41:10.930 --> 41:23.730 And we can potentially use technology to not only, not only avoid things, but also provide more accountability, provide basically the proof that says, Hey, this is the smoking gun. 41:23.730 --> 41:28.430 You guys did this and you can't just argue, you know, fog of war. 41:29.550 --> 41:36.870 So these are just some examples of how technology can be used for, for good and for humanitarian benefits. 41:37.270 --> 41:38.730 So the next slide, please. 41:39.130 --> 41:54.730 Hey, Larry, can I jump in with a good point that somebody made in one of the chats about what you're talking about here, which is that, you know, if you have these, these algorithms that are designed to detect red crosses and then not shoot at that thing, 41:54.730 --> 41:54.890 right? 41:54.890 --> 42:01.530 The question is, well, doesn't that mean that all of the bad guys are just going to start putting red crosses on, on their, on their stuff? 42:03.790 --> 42:04.510 Sure. 42:04.510 --> 42:06.330 And I mean, yeah. 42:06.330 --> 42:17.110 So, so there, there, there are states and actors that try to follow the law, the law of war, internationally humanitarian law, like the genomic conventions. 42:17.110 --> 42:18.790 And then there are some that don't. 42:18.790 --> 42:23.150 That's actually why I included that last piece there, traceability and accountability. 42:23.910 --> 42:48.170 Because with, with, you know, with technology, you can also better document how actors, and that may be a state like Russia, or it may be a non-state armed group, but if they are intentionally flouting requirements of the law, you now have documented this and you can put more pressure on them to, 42:48.170 --> 42:49.670 to behave differently. 42:51.910 --> 42:57.090 And try to use different ways to, to, to positively influence them. 42:57.090 --> 42:57.290 Yeah. 42:57.290 --> 43:10.870 And, and one of the things I think you're getting at when you're talking about flouting the law is that it is, goes against international humanitarian law to put those red cross symbols, red crescent symbols on something that is not actually a protected site. 43:10.870 --> 43:17.290 So, so some bad actors, right, would be doing that even, even without AI technology. 43:17.290 --> 43:18.450 Right. 43:18.570 --> 43:19.190 Right. 43:19.190 --> 43:21.850 Yeah, certainly, certainly they can, they can do that now. 43:21.850 --> 43:23.410 And sometimes they do. 43:23.950 --> 43:29.490 That is, that is not, that is not an excuse to, to just be reckless. 43:29.570 --> 43:30.590 Right. 43:31.630 --> 43:35.850 So, so we need, still need to, to have, to strengthen our protection. 43:36.450 --> 43:43.510 But at the same time, this is also an opportunity to avoid the bad behavior that we see today. 43:46.410 --> 43:47.350 Okay. 43:47.430 --> 43:52.490 So finally, I mean, so the military cares about humanitarian issues. 43:52.490 --> 43:58.030 But at the same time, there's, there's not a lot of, of strong leadership on this issue. 43:58.030 --> 44:03.170 And that's, that's really because of a combination of formal responsibilities. 44:03.170 --> 44:14.810 If you look organizationally at DOD, you know, they have, they have people in charge of lots of different things, you know, developing weapons systems, developing, you know, lethal capabilities. 44:15.070 --> 44:22.350 There's really not that same kind of leadership on, hey, how can we, how can we get better at humanitarian issues? 44:22.890 --> 44:31.890 At the same time, there, there, there has been, there have been leaders who have really focused on this issue, but then they rotate out. 44:31.890 --> 44:35.150 And so it does, long lasting focus. 44:35.590 --> 44:38.150 So, so what, what can we do? 44:38.150 --> 44:44.970 History shows that non-governmental influences can strongly encourage progress in this area. 44:44.970 --> 44:49.170 And typically that has been through media, you know, media accounts. 44:49.170 --> 45:10.070 And I think of the uncounted article, for example, in 2017, by Asma Khan was one great example of this, how one powerful, well-researched article basically influenced DOD to, to be more introspective and to do kind of internal reviews that actually make real changes. 45:10.110 --> 45:15.230 And civil society, you know, NGOs, academia can have a role. 45:15.230 --> 45:17.810 And then of course the UN and ICRC. 45:17.870 --> 45:26.750 So they have all, at different times, had important contributions to help the military to do better. 45:26.930 --> 45:31.690 And I listed, listed three specific examples. 45:31.690 --> 45:46.630 You know, there are, these are actually examples, real examples where outside influences encourage the military to try harder, both in Afghanistan and that really, they really turned things around and did things differently. 45:46.630 --> 45:48.970 And the civilian casualty numbers went down. 45:49.390 --> 46:01.510 Drone strikes and counter-terrorism, you know, leading to the presidential policy directive, the presidential policy guidance issued by President Obama for counter-terrorism operations. 46:01.570 --> 46:14.310 And then the campaign in Iraq and Syria, again, there were things like, you know, counted article and also air wars that contributed to greater scrutiny and led to differences in how the U.S. 46:14.310 --> 46:16.050 military has done things. 46:16.050 --> 46:17.590 If you go to the next slide. 46:18.830 --> 46:24.730 So, so bottom line, there, there's a, there's an opportunity here. 46:24.730 --> 46:32.790 So if you combine kind of the things we've been talking about, the community needs the tech sector as they have not before. 46:32.790 --> 46:51.010 So, you know, looking at that graph we had earlier, you know, the, just the top five tech companies in the U.S., their research and development, you know, now 15 times larger than the entire tech development budget of the United States government. 46:51.290 --> 46:53.910 And that ratio is increasing over time. 46:53.910 --> 46:55.170 So the U.S. 46:55.170 --> 47:01.210 government sees it needs the tech sector to be competitive. 47:01.410 --> 47:06.310 And you see this in DoD wanting to work with the tech sector and they have a number of initiatives. 47:06.310 --> 47:07.590 There's the defense innovation board. 47:07.590 --> 47:12.870 And of course they developed the AI principles that DoD then accepted. 47:12.930 --> 47:22.450 You have the defense innovation unit, defense digital service, and then there are other initiatives that DoD is trying to engage more with the tech sector. 47:22.550 --> 47:27.310 So we just wanted to highlight this is an opportunity for the tech sector, right? 47:27.310 --> 47:31.130 So this is an opportunity to engage and have influence. 47:31.430 --> 47:36.090 So as DoD says, Hey, we want to talk, we want to work together. 47:36.590 --> 47:45.330 There are opportunities for tech companies and people working in the tech sector to influence the areas for engagement and for cooperation. 47:45.470 --> 47:51.510 And also voice concerns about how these issues can be addressed. 47:51.510 --> 48:05.430 And then, you know, I mentioned, you know, right now there's academia, there's a civil society, there's a UN that are engaging now with DoD on discussions and humanitarian issues. 48:05.430 --> 48:08.290 I was in such a discussion yesterday, right? 48:08.290 --> 48:13.770 There was nobody in the tech sector in that discussion, but there could be, right? 48:13.770 --> 48:16.990 As a stakeholder and as a potential solution. 48:17.570 --> 48:19.670 So this is an opportunity. 48:19.670 --> 48:23.670 And then I also just wanted to highlight, this is not just for the US, right? 48:23.670 --> 48:27.230 So I mentioned Australia doing work. 48:27.810 --> 48:35.710 Also the Netherlands are working on this protocol, white flag, white flag is an open source solution. 48:35.990 --> 48:51.550 So it's also, it doesn't have to be a proprietary solution for the US or it could be other open source solutions that help address humanitarian issues that basically could be offered to all parties to conflict. 48:52.390 --> 48:56.250 So I think that's, if you go to the next slide. 48:56.990 --> 49:02.730 So now we'd love to have a discussion and we also have contact information here. 49:02.990 --> 49:18.410 So we'd love to continue to answer questions or things you'd like to raise, but also in the future, we'd love to hear from you and we can continue those discussions. 49:18.450 --> 49:21.670 So open to any questions you may have. 49:21.710 --> 49:22.870 Yeah, absolutely. 49:22.870 --> 49:25.810 I'll certainly echo what Larry has said there. 49:26.410 --> 49:30.290 And so I've been reading through the comments that people have been posting. 49:30.290 --> 49:36.570 So I'm just going to go ahead and kind of read off the things that I've seen there. 49:37.050 --> 49:45.250 So one of the things that kind of the main thing that a couple of different folks have raised are concerns around proliferation, right? 49:45.250 --> 49:56.510 So one person raised a worry, and this is very timely, a worry about the trickle down of military technology being used in domestic law enforcement, right? 49:56.510 --> 50:00.350 Or being used abroad to stifle dissent. 50:00.850 --> 50:15.390 There was also some concerns brought up about, well, if other, say, bad actor nations get these technologies, that's something that's going to serve as a force multiplier for those bad bad actors, dictators or whatever. 50:15.390 --> 50:26.590 And so a mention of, let's see, so the person wrote in that the best way to keep control of technology is to not develop it. 50:26.590 --> 50:29.970 And there's another mention of mutually assured destruction. 50:30.970 --> 50:36.850 So there's kind of a lot baked up in those proliferation concerns. 50:36.850 --> 50:52.350 I might start to address those by saying that certainly thinking about this technology and how we want to move forward with thinking about exporting or sharing it is absolutely something that DOD and the U.S. 50:52.350 --> 50:56.970 government is taking very seriously and really thinking about right now. 50:58.090 --> 51:07.650 Another thing I'll... and so, again, it's going to be one of these contexts where there's some hard challenges, you know, wrapped up in there. 51:07.650 --> 51:20.970 But at the same time, right, keep in mind that, you know, as we talked about with that earlier slide with the graph on it, most of the R&D that's going on in this space is not happening by state actors. 51:20.970 --> 51:22.550 It's not happening by the U.S. 51:22.550 --> 51:23.830 government or DOD. 51:23.830 --> 51:30.670 So that's a reality to keep in mind, thinking about proliferation also. 51:31.790 --> 51:53.270 And one other thing I'll mention, just thinking about the domestic law enforcement context, is just be aware that a lot of the rules and laws that DOD has to follow and what it makes are... so domestic law enforcement is not subject to those laws. 51:53.270 --> 52:04.770 So , again, I think there are things to worry about here, but I'm not sure if DOD's development of these technologies... that may not be the biggest concern in some of these cases. 52:05.590 --> 52:08.970 Larry, would you add anything on any of those topics? 52:10.350 --> 52:15.170 So, yeah, so great questions. 52:15.170 --> 52:27.430 One thing I wanted to mention, especially for the export and proliferation piece, because I've spent a couple years at state where this was a really key concern. 52:27.430 --> 52:34.730 And one thing that became very apparent to me, in the case of Yemen, so, you know, the U.S. 52:34.730 --> 52:42.990 has provided a lot of arms, you know, weapons systems, intelligence to the Saudi-led coalition in Yemen. 52:42.990 --> 52:48.890 And, you know, there were lots of discussions about, is it legal to do that? 52:48.890 --> 52:54.570 And I think I saw one report kind of summarized it as lawful, but awful. 52:54.650 --> 52:59.610 So the laws and policies that the U.S. 52:59.610 --> 53:08.830 applies to security assistance, providing stuff to other countries, it considers some things. 53:08.830 --> 53:15.390 It considers things like diversion, which is, are you going to give, we want to safeguard that if we give this, if the U.S. 53:15.390 --> 53:18.770 gives this to another country, they won't give it to somebody else. 53:18.770 --> 53:23.910 And there's also something called Leahy law that guards against gross violations of human rights. 53:23.910 --> 53:29.270 So if they're doing like war crimes, really horrible things, then that can stop transfers. 53:29.270 --> 53:38.190 But the more general civilian protection considerations that we're talking about really are not well incorporated in security assistance. 53:38.190 --> 53:44.610 And so that is something that the state department is really trying to think about doing. 53:44.610 --> 53:50.790 We're talking to them about, you know, new policies and new ways to think about this. 53:51.030 --> 54:03.230 But one thing that can help is if you can provide equipment, including weapons, that have some of these safeguards, right? 54:03.230 --> 54:08.590 So right now, the Saudi-led coalition, how many hospitals have they struck? 54:08.590 --> 54:09.810 I've lost count. 54:09.810 --> 54:11.950 It's so many, so many, right? 54:12.770 --> 54:18.010 And I mean, you can certainly say, well, we're not going to give them anything, right? 54:18.010 --> 54:22.150 But they've also shown that they'll just buy from the Russians. 54:22.150 --> 54:26.810 So it'd be great to stop them, okay? 54:26.810 --> 54:28.630 I'm in favor of that. 54:28.930 --> 54:42.690 But if war is going to continue, are there things that can be done to make war more humane and safer for the civilians that are in the conflict areas? 54:42.690 --> 54:46.890 And so the answer is yes, and that technology can help. 54:48.990 --> 54:54.750 And that can certainly be the case for exports as well. 54:55.490 --> 55:05.090 And as I'm reading the chats coming in, I'll just say that folks are kind of doubling down on concerns about domestic law enforcement again, right? 55:05.090 --> 55:19.670 And so predator drones at protests, you know, an interest in, you know, can we have voting where we select that we're gonna not fund these types of technologies? 55:19.670 --> 55:23.150 Maybe that's in a military or in a law enforcement context. 55:25.550 --> 55:30.690 So let's see, and a note that killer robots are perfect for war crimes. 55:30.690 --> 55:32.330 So back to the armed conflict case. 55:32.330 --> 55:45.470 So I'll say again, with respect to the domestic law enforcement front, right, we're just seeing that, you know, some of this is not, maybe not necessarily an issue limited to AI developments, right? 55:45.470 --> 55:46.770 We're seeing it now. 55:47.030 --> 55:51.650 It's a real problem in domestic law enforcement. 55:51.650 --> 56:03.230 And so, you know, it's important that we fix the laws and fix the practices that are in place now, even before we have sort of new classes of weapons that the military is using. 56:03.930 --> 56:22.950 Um, let's see one, another comment that somebody wrote into the chat was, was concerned about AI vulnerabilities, right, being especially problematic in this context, they mentioned adversarial attacks, data poisoning, right. 56:22.950 --> 56:31.690 So these are, so these are getting technical challenges, right, that are going to be hard challenges to to solve, right. 56:31.690 --> 56:39.430 So this goes back to, there are a lot of challenges and just getting getting the technology right to actually make these applications work. 56:40.010 --> 56:47.610 And so I guess one thing I would say here is is that, that, you know, yeah, these are hard problems. 56:47.610 --> 56:53.730 I think we, you know, we, DoD will need the tech sector, private industries help to get good solutions here. 56:53.730 --> 57:10.210 And in cases where we can't be confident in these technologies, especially in in lethal, or weapons contexts, those are cases where we need to not not deploy systems, right, unless we can, unless we can be confident with how they're going to operate. 57:13.710 --> 57:22.350 Can I make one comment about the, you know, I do hear this a lot about, you know, killer robots are perfect for war crimes, right. 57:22.350 --> 57:27.050 And so I think it's useful kind of just to build a framework here. 57:27.050 --> 57:32.150 So there are two kinds of parties that are involved in conflict. 57:32.150 --> 57:34.950 Some can be states, or it can be armed groups. 57:35.570 --> 57:40.390 One is parties that want to follow the law. 57:40.490 --> 57:40.810 Okay. 57:40.810 --> 57:44.530 And then there are some that don't, they show impunity. 57:44.850 --> 57:49.050 And so I think technology applies differently to the two of them. 57:49.050 --> 58:05.030 So those that that wants to apply the law, then this kind of technology can be useful for, for helping, helping them to better comply with the law and their legal obligations. 58:05.030 --> 58:11.350 But even beyond the law, because in the US, we talked about there are legal things we're required to do. 58:11.350 --> 58:16.510 But then we also want to try even harder than what we're required to do, right. 58:16.510 --> 58:19.990 The law is just the basic requirements. 58:20.630 --> 58:25.670 Diane mentioned the, the US civilian casualty executive order in 2016. 58:26.370 --> 58:29.210 That goes beyond what is required by the law. 58:29.210 --> 58:33.910 It says, you know, there are things we're required to do, but we're going to do more because it's important. 58:34.290 --> 58:43.030 So, so for the group of states or parties that want to comply with the law, you can help comply with the law. 58:43.030 --> 58:44.490 And then you can even go further. 58:44.490 --> 58:46.450 And we talked about some ways to do that. 58:46.450 --> 58:50.250 Then there's others that don't comply with the law, right. 58:50.250 --> 58:52.210 And there's the Russians and the Syrians. 58:52.210 --> 58:56.570 And, you know, there are different, different, you know, different terrorist groups out there. 58:56.570 --> 58:57.770 They just don't care. 58:57.770 --> 59:01.570 They exploit the law for their own benefits. 59:02.230 --> 59:13.790 And in those cases, technology can also be used for accountability and to traceability and to try to highlight some of the things that they do. 59:13.810 --> 59:29.650 Honestly, you know, there were, there were things I remember in the counter ISIS conflict in, in Iraq and Syria, ISIS did horrible, horrible things, and most people have no idea. 59:29.650 --> 59:30.450 Right. 59:30.490 --> 59:46.170 And so, and just better, better showing those things can help in themselves to influence those groups, just like the New York Times article, talking about the Russians and delivery striking hospitals. 59:46.170 --> 59:46.730 Okay. 59:46.730 --> 59:48.790 They, they did that quietly. 59:48.990 --> 59:53.390 When the New York Times came out, Russia started to squirm, right? 59:53.390 --> 59:55.630 Because they don't want to be known for that. 59:55.630 --> 59:58.890 They want to do it for various interests. 59:58.890 --> 01:00:07.310 But when you start putting the spotlight on them, that can help, that can help influence them in themselves. 01:00:07.310 --> 01:00:15.250 We should also be thinking about how can we better use that information to really deter them from doing those terrible things. 01:00:15.250 --> 01:00:19.030 So just wanted to kind of give that overall framework for war crimes. 01:00:20.190 --> 01:00:29.970 So Larry, another comment that somebody wrote in that I think is a really interesting one is a question of whether, right, so DOD has these AI policies. 01:00:30.570 --> 01:00:32.770 Should, should those be codified in law? 01:00:32.770 --> 01:00:36.610 Because it's, it's really easy to change norms and guidelines. 01:00:36.830 --> 01:00:38.690 What would you say to that? 01:00:43.620 --> 01:00:57.700 So, so I remember actually having a conversation with Eric Schmidt about, he's part of the, the National Security Commission for AI. 01:00:57.700 --> 01:01:02.820 And so there was this discussion about, you know, law versus policy. 01:01:02.820 --> 01:01:19.240 And, you know, the tricky thing is that it is so early that oftentimes we don't even really know what we're talking about, you know, and, and these kinds of conversations are so valuable because we're, we're, we're trying to figure out what is really important. 01:01:19.240 --> 01:01:22.660 And that, that's one reason why we've been asking these questions too, right? 01:01:22.660 --> 01:01:24.060 You know, what do you think is important? 01:01:24.060 --> 01:01:26.380 Because we want to hear from, from all of you. 01:01:27.040 --> 01:01:33.760 And, but, but trying to make a law too soon has risks, right? 01:01:33.760 --> 01:01:38.020 Because then you've sort of codified your limited understanding. 01:01:38.380 --> 01:01:43.720 And it's really, it's harder to change law than it is to change policy. 01:01:43.720 --> 01:01:51.160 So, so in these early stages and really, you know, DoD has not done much. 01:01:51.160 --> 01:01:52.560 They've done a lot of talking. 01:01:52.560 --> 01:01:54.780 There really has not been a lot. 01:01:55.240 --> 01:02:02.500 Diane mentioned the, the DoD directive on, on autonomous weapons that was created in 2012. 01:02:02.500 --> 01:02:10.080 And it calls for the senior level review for additional scrutiny for systems that would meet certain requirements. 01:02:10.080 --> 01:02:14.060 They haven't done a single one because they're just not there yet. 01:02:14.100 --> 01:02:21.800 So I think... And just, they haven't done a single one because, because nobody's made it, nobody's tried, nobody's asked, right? 01:02:21.800 --> 01:02:23.560 Just to, just to clarify that. 01:02:23.680 --> 01:02:24.460 Yeah. 01:02:26.320 --> 01:02:26.980 Yeah. 01:02:27.020 --> 01:02:38.060 So I think, you know, it's, it's kind of easy to say, you know, yeah, there's so much that can be done, but in reality, militaries move slowly. 01:02:38.060 --> 01:02:39.760 They're built to move slowly. 01:02:39.760 --> 01:02:46.240 They're these huge bureaucratic structures that are designed to reduce risk. 01:02:46.240 --> 01:02:47.860 And so they move really slowly. 01:02:47.860 --> 01:02:52.900 And so I think at this point, we, we don't necessarily need a law. 01:02:52.900 --> 01:02:56.800 We need to actually figure out what the important things are. 01:02:57.200 --> 01:02:59.380 And we're not, we're not even there yet. 01:02:59.380 --> 01:03:07.260 So, so that's why we really appreciate this discussion today to try to help figure out what are the things we really need to be worried about. 01:03:08.060 --> 01:03:08.860 Yeah, thanks. 01:03:08.860 --> 01:03:22.500 And, and I'm tracking, just looking through the chat here, I think there's, there's, there's one more comment that I'm seeing that I don't think we've, we've addressed yet that I'll read off here, which is just that I think the person who recommended this, 01:03:22.500 --> 01:03:29.680 this report that you talked about earlier, so this, so they, they raised a few issues from that report. 01:03:30.000 --> 01:03:33.000 This was the vulnerabilities, AI vulnerabilities issue. 01:03:33.000 --> 01:03:41.760 They also raised the point that there's a dynamic where offense is easier than defense in some of the space. 01:03:41.760 --> 01:03:53.880 And so I think that, that one thing I would, you know, I would say two things there, right, which is that, you know, one is that I don't, I don't think that's special of the AI. 01:03:54.140 --> 01:04:07.360 And I would also say that, you know, we did some research a little while back looking at lethal autonomy, you know, potential dynamics or uses within, hypothetical uses within the military. 01:04:07.700 --> 01:04:26.740 And, you know, back to this, a drone swarm attacking a ship example, I would say just don't discount that, that, you know, certainly it's right that in a lot of contexts, the offense is easier than defense, but there are still really important defensive contexts where technologies like this can be really helpful. 01:04:26.740 --> 01:04:31.060 And, and those are cases where we're talking about, you know, protecting the lives of our own soldiers. 01:04:31.060 --> 01:04:39.140 So they're ones that we, you know, I think as a country and certainly our military regards very highly and is very important. 01:04:40.280 --> 01:04:45.620 If I can just kind of add to that, the, the context thing is really important. 01:04:45.620 --> 01:04:52.720 So, you know, I was, I was part of the US delegation to the UN and in some of these talks on lethal autonomous systems. 01:04:52.840 --> 01:04:59.020 And one thing that was often lost is that the context really is important. 01:04:59.020 --> 01:05:13.620 So, so you have some, some people would talk about, well, how will you use a lethal autonomous weapons system in an urban environment against, you know, against groups that aren't wearing uniforms and they're, you know, hiding within the population. 01:05:14.160 --> 01:05:20.960 And then, you know, basically the answer of, you know, the US and Australia and the UK was like, we won't. 01:05:21.220 --> 01:05:31.360 And what, what we really, what they kind of want this stuff for is, is for missile defense in, in a maritime setting, right? 01:05:31.360 --> 01:05:43.000 So there's no civilians anywhere around but they have hypersonic missiles coming in and they want, they want to have capability that helps them to, to better defend against those kinds of things. 01:05:43.000 --> 01:05:54.980 So, so just, just recognizing that there are some contexts where you can use some of this technology with a much lower risk to civilians than others. 01:05:55.300 --> 01:06:00.640 And just having a more nuanced conversation where we recognize that. 01:06:00.660 --> 01:06:03.460 And, and, you know, militaries need to recognize that too, right? 01:06:03.460 --> 01:06:14.300 They do internal reviews where they try to recognize risk and the context is really important for those, those decisions on whether they think it's okay or not to use them. 01:06:14.900 --> 01:06:15.940 Yeah, thanks. 01:06:15.940 --> 01:06:21.140 And so one more person wrote in a good comment that, that I'll read off here, right? 01:06:21.140 --> 01:06:23.320 Which is the concerns about, right? 01:06:23.320 --> 01:06:27.440 AI needs big data a lot of times to, to work. 01:06:27.440 --> 01:06:30.500 And so what about data security there? 01:06:31.760 --> 01:06:34.260 So do you have any, any comments on that, Larry? 01:06:34.340 --> 01:06:43.940 So I think this is, this is, I mean, we talked about, there are some risks that, that, you know, DoD is like, is really handling well. 01:06:43.940 --> 01:06:47.200 And then there are some that I think they're behind. 01:06:47.200 --> 01:06:49.220 And I think this is one of them. 01:06:49.620 --> 01:06:54.680 And it's, you know, we talked about AI and applications. 01:06:54.680 --> 01:07:03.700 I think this is one reason why, you know, DoD is not as, as making as much progress as people may think. 01:07:04.100 --> 01:07:10.200 Because there are all these enablers that they need to have enabled to use AI effectively. 01:07:10.200 --> 01:07:14.000 And the data situation with DoD is really problematic. 01:07:14.560 --> 01:07:17.580 They, they, they have lots of different networks. 01:07:17.860 --> 01:07:20.620 You know, they're not necessarily secure. 01:07:20.620 --> 01:07:22.300 They don't necessarily talk to each other. 01:07:22.300 --> 01:07:28.440 Well, they're, they're not always formatted in a consistent way or quality control. 01:07:28.440 --> 01:07:34.260 And so that makes it really difficult to actually use that data in a, in a constructive way. 01:07:34.820 --> 01:07:40.000 So, so DoD is going to have to get a handle on that. 01:07:40.000 --> 01:07:45.100 And part of, part of that is going to be trying to make data more secure. 01:07:45.100 --> 01:07:58.160 I will say in the few serious AI applications that, that I'm aware of, there has been kind of this recognition, both of they need to be careful about data poisoning. 01:07:58.560 --> 01:08:06.840 And, and that has implications both on storage, but also on who you get to do your, your data processing, right? 01:08:06.840 --> 01:08:15.480 You can't just necessarily outsource this to some commercial organization that may, may or may not be the most trustworthy. 01:08:16.040 --> 01:08:17.500 And then the storage. 01:08:17.500 --> 01:08:19.020 So it's a great question. 01:08:19.020 --> 01:08:19.700 I think. 01:08:21.860 --> 01:08:22.740 Yeah. 01:08:23.340 --> 01:08:23.620 Okay. 01:08:23.620 --> 01:08:25.980 Well, thank you guys so much. 01:08:25.980 --> 01:08:30.320 We really appreciate hearing your, your thoughts and comments. 01:08:30.320 --> 01:08:34.680 I'll reiterate what Larry said a minute ago, which is the, please feel free to reach out to us. 01:08:34.680 --> 01:08:39.960 You can either do it through discord or through our email addresses that we put up a few minutes ago. 01:08:39.980 --> 01:08:53.840 We would love to hear, you know, if there's, if there's more that you want to talk about that we didn't get to, again, we would, you know, we, we don't have all the answers, but we are really interested in hearing your thoughts and concerns, even if they're not things that are, 01:08:53.840 --> 01:08:55.480 can be perfectly answered today. 01:08:58.000 --> 01:09:01.760 So, so thank you again, Ethics Village for hosting us. 01:09:02.400 --> 01:09:06.280 And I really appreciate everybody's, everybody's participation here. 01:09:08.520 --> 01:09:09.480 Awesome. 01:09:09.480 --> 01:09:15.880 We'd like to, again, thank you to our speakers, Dan and Larry for our very first talk of day two. 01:09:16.160 --> 01:09:22.040 For those of you that are here for the rest of the few hours, stay tuned for our next few talks. 01:09:22.040 --> 01:09:22.980 Our next talk will be from our guest speaker, Dan. 01:09:22.980 --> 01:09:22.980 So, Dan, thank you so much for