WEBVTT 00:00.000 --> 00:04.740 Right, so along with me, for the last talk, we have Andy Dennis. 00:04.840 --> 00:08.960 Andy is a VP of consulting at Modus Create. 00:12.630 --> 00:20.510 He has over 20 years of experience in security, software engineering, DevOps, consulting, system migration, and modernization. 00:20.670 --> 00:27.190 Andy is a published author and online tutor for Goldsmiths University of London. 00:27.490 --> 00:31.770 Andy has also spoken at and also helped organize B-Sides events in New England. 00:31.770 --> 00:41.170 And he has also spoken in multiple other conferences such as B-Sides CT, B-Sides Orlando, and Defcon Village in the previous editions as well. 00:41.550 --> 00:48.730 The topic Andy will be covering is the Perils of Generative AI Implications for Open Source Intelligence Research. 00:48.790 --> 00:50.690 So please give it up for Andy. 01:03.880 --> 01:09.640 Also, a little request that if you could cover up the seats ahead so that it would be better because... 01:09.640 --> 01:13.860 I mean, we have a lot of seats that are empty in between, so if you could cover those up. 01:14.060 --> 01:18.340 And please turn your phones into silence so that the speaker isn't disturbed in between. 01:18.340 --> 01:19.220 And have a good time. 01:25.520 --> 01:29.120 So they just did the introduction for me, so this slide's kind of redundant. 01:29.540 --> 01:35.340 But I will tell you, we are recruiting, so go to this website if you're a Java engineer, if you work in DevOps. 01:36.220 --> 01:42.440 If you're really good with GitHub and GitHub Actions, we're definitely looking for folks right now, so please do take a look. 01:42.780 --> 01:49.680 And yeah, my previous talk, if you're interested, was on using passive DNS for gathering business intelligence, and that was from 2021. 01:49.700 --> 01:51.660 So feel free to check that out. 01:51.660 --> 01:54.080 So then, let's get started, I guess. 01:55.820 --> 02:03.600 So we're going to start with a brief explanation of generative AI and open-source intelligence. 02:03.600 --> 02:05.600 Sorry, I'm just going to move this mic so it's a little easier. 02:05.960 --> 02:12.540 So for those of you new to the topic, I'm going to give a brief overview of what this is and also a little background on OSINT. 02:12.800 --> 02:19.200 I've included a timeline on the next few slides, which just gives a brief overview of some of the major milestones in the AI space. 02:19.260 --> 02:23.600 We won't be going into deep mathematical details here, so you don't have to run for the exit yet. 02:24.060 --> 02:30.860 But in brief, generative AI is a category of artificial intelligence that's capable of generating new content, such as text, images, and audios. 02:30.860 --> 02:32.540 And I'm sure all of you are familiar with that. 02:33.480 --> 02:37.320 Many of you will have kicked the tires on tools such as ChatGBT and MidJourney. 02:37.580 --> 02:43.880 This approach is built upon years of research and development in the AI space and is really coming to fruition at this point in time. 02:43.880 --> 02:49.300 Let's see what ChatGBT says when we ask it to summarize what generative AI is. 02:51.440 --> 02:55.580 I have gone too many slides forwards by the looks of it. 02:56.280 --> 02:57.120 There we go. 02:57.120 --> 02:57.820 Sorry. 02:57.960 --> 03:06.860 So generative AI is a subset of artificial intelligence that utilizes machine learning models to create new data samples that mimic the properties of some source data set. 03:07.040 --> 03:11.860 This could involve generating anything from written text, images, music, or even video content. 03:11.860 --> 03:19.020 This is achieved through algorithms such as GANs, where two neural networks, a generator and a discriminator, are trained in tandem. 03:19.020 --> 03:25.160 The generator aims to produce realistic data, while the discriminator's job is to distinguish between real and generated data. 03:25.160 --> 03:30.860 The iterative competition between the two leads to the production of increasingly accurate and realistic synthetic data. 03:30.860 --> 03:37.320 Other popular generators include VAEs and transformer-based models like GPT-3 and GPT-4. 03:37.320 --> 03:42.180 So, hopefully that's nice and clear now what that means. 03:43.180 --> 03:53.860 Basically, these tools and models are trained on large data sets and apply machine learning techniques to generate new content such as image from a text prompt or poem in the style of, say, a long-dead author. 03:56.240 --> 04:06.260 This ability to mimic styles and generate novel content allows users to create realistic images, compose music, and increasingly explore problem spaces in various scientific fields. 04:06.260 --> 04:12.740 Despite all the fun and serious use of the technology, concerns around ethics and the ability for gross misuse have been a key concern. 04:12.940 --> 04:16.180 I'm sure you've been following anything in the news recently. 04:16.720 --> 04:20.640 You know, AI alignment and ethics is a major topic. 04:21.300 --> 04:25.820 Fraudulent data, such as deepfakes, will become increasingly easy to generate for anyone with a laptop. 04:26.300 --> 04:30.220 And separating the real video from the AI-generated one will get harder over time. 04:30.500 --> 04:33.240 That's a very, very basic overview of Gen-AI. 04:33.240 --> 04:40.320 To sum it up, it's a way of taking large amounts of data and training software to be able to create a new, compelling content based off of this. 04:40.340 --> 04:44.900 And later in the talk, we're going to pull some of this apart and delve into it in more detail. 04:46.820 --> 04:51.300 So, we did a quick overview of Gen-AI, but what about OSINT? 04:51.300 --> 04:53.660 So, OSINT stands for Open Source Intelligence. 04:53.660 --> 05:02.580 It's a set of methods, techniques, and tools to collect information from publicly available sources, such as blogs, social media, data sets, and reports, and so on. 05:02.760 --> 05:04.580 It's not just hackers using this. 05:04.580 --> 05:13.520 Journalists use it, business intelligence analysts, law enforcement, cyber security professionals, and a bunch of other folks use it for a variety of purposes. 05:13.800 --> 05:21.100 This data can be collected through a range of techniques, such as web scraping tools, natural language processing of data, machine learning. 05:21.320 --> 05:27.480 The Recon Village website includes some links to these various tools, and a bunch of presentations from over the years. 05:27.480 --> 05:35.840 And on this slide and next, you can see some of the various products and projects and open source GitHub repos that contribute to this space. 05:35.840 --> 05:37.180 And so, please go check them out. 05:37.180 --> 05:40.800 And I think maybe Amass was already covered earlier today in one of the talks. 05:43.140 --> 05:48.800 So, now a key aspect of OSINT is the analysis of the data we've collected with the various tools. 05:48.800 --> 05:53.140 Once we have the data, we can use it to study trends, draw conclusions, and make decisions. 05:53.780 --> 05:55.920 This is incredibly useful, of course. 05:55.980 --> 06:00.120 For example, if we're trying to trace an individual or discover some information about a competitor. 06:00.120 --> 06:06.020 But despite this usefulness, we need to be cognizant of privacy concerns and ensure we are operating in an ethical fashion. 06:06.680 --> 06:11.720 A key problem arises, though, when considering using OSINT to inform decisions and drive actions. 06:11.740 --> 06:13.340 What if the data is fake? 06:13.900 --> 06:17.600 What if it was generated by AI methods, like I previously mentioned? 06:19.960 --> 06:21.360 So, a bit more background. 06:21.360 --> 06:23.000 Let's dig a little deeper. 06:23.880 --> 06:33.320 So, now we're going to dive into JANA and OSINT a little bit more, but we just did a speed run through the basics there. 06:33.460 --> 06:42.680 So, let's cover some of the techniques used by large language models and other generative platforms that were hinted at in the chat GPT's definition previously. 06:42.940 --> 06:48.700 So, key machine learning techniques listed on these slides include GANs, VAEs, and transformer models. 06:48.920 --> 06:51.760 Chat GPT previously explained what a GAN was. 06:51.760 --> 06:57.380 A VAE is a variational autoencoder, and it's a form of a neural network. 06:57.380 --> 06:59.260 They use probabilistic models. 06:59.320 --> 07:05.840 So, a VAE is a type of generative model that uses a variational interface to encode and decode data. 07:05.840 --> 07:11.860 They represent the data in a probabilistic latent space, enabling them to generate new data that's similar to the training data. 07:11.860 --> 07:13.700 And then we get to transformers. 07:13.880 --> 07:22.260 So, transformers, which is a type of technology used by chat GPT, are also a neural network-based approach that underpins large language models. 07:22.560 --> 07:26.320 You may have heard the term transformers a lot recently due to LLM's popularity. 07:26.320 --> 07:29.020 So, that's kind of just a basic description of it there. 07:29.020 --> 07:32.920 And you can essentially think of them as taking one input and transforming it into another. 07:33.300 --> 07:40.580 They're a mechanism for learning context through tracking relationships and data without needing recurrent layers, enabling parallel processing. 07:41.120 --> 07:44.140 A couple of other phrases you may have heard are weights and biases. 07:44.140 --> 07:55.580 So, in a neural network, weights are the parameters that are learned during the training to scale the input of data and connections between the neurons in the neural network and determining the strength of the signal passed through the network. 07:55.580 --> 08:06.160 And biases are additional parameters in a neural network that allow for the shifting, shifting the activation function to the left or right, helping the network to fit the data more flexibly. 08:06.240 --> 08:10.260 These models use a variety of approaches, such as games and competition, 08:13.540 --> 08:16.960 which help to improve constantly the quality of the output. 08:17.480 --> 08:19.900 As each technique gets better, so do the outputs. 08:19.900 --> 08:26.080 So, effectively, as each of these techniques get better, the images are going to get better, the text output is going to get better, the video output is going to get better. 08:26.600 --> 08:28.880 You really don't have to understand all the math here. 08:29.240 --> 08:35.920 There are people much smarter than me who I work with who really get that side of things, and that's not my background as a mathematician. 08:35.920 --> 08:39.880 But having a general understanding of the techniques is useful in understanding space. 08:40.520 --> 08:51.820 So, as a result, AI researchers have been able to improve the outputs to start mimicking real-life photography, sound like the voice of a living individual, or create text that reads like a famous author. 08:52.140 --> 08:55.300 And we'll be seeing a few examples of these later in the talk. 08:56.620 --> 09:00.840 So, a little bit on the pace of improvements between 2022 and 2023. 09:00.980 --> 09:08.180 As I just started when discussing the previous slide, as the outputs get better, they get closer to mimicking the real, effectively. 09:08.220 --> 09:15.220 And what's driving us closer in on the simulacrum being indistinguishable from the real is the pace of technological change. 09:15.220 --> 09:22.620 So, this diagram shows that between 2020 and 2023, we went from GPT 3.0 to GPT 3.4. 09:22.700 --> 09:25.940 The change in quality of the outputs in this period took leaps and bounds. 09:26.220 --> 09:36.520 We saw this on the image side of things too, an explosion of vendors releasing new products and new open source projects and code that might have been leaked and shared. 09:36.820 --> 09:41.280 And it's all in the image space, the text space, the data processing space, the sound space. 09:41.280 --> 09:43.780 So , it's really quite across the spectrum. 09:44.280 --> 09:52.280 And I think this is going to become obviously important in relation to OSINT, as some examples that are coming up will demonstrate. 09:56.450 --> 10:02.030 So, open source intelligence and its importance to security, the security space. 10:02.790 --> 10:08.470 So, that was a little bit more about some of the techniques that are used by Gen AI and the pace of change around these. 10:08.870 --> 10:14.670 But let's switch back to discuss open sources intelligence and the importance in the security space. 10:14.670 --> 10:17.990 And as we can see here, there are four key areas that come to mind. 10:18.890 --> 10:27.550 So, we have threat intelligence, which is basically detecting cyber threats and patterns and using that information to harden systems, processes, and policies that drive that. 10:27.750 --> 10:34.330 Incident response, so when something bad happens, we can use OSINT to understand the nature of the attack and the potential culprits. 10:34.890 --> 10:41.670 And the security awareness size, that's basically monitoring OSINT sources, which allows us to get ahead of events. 10:41.870 --> 10:43.750 So, understand threat actors. 10:43.910 --> 10:53.850 And then digital forensics, OSINT can add another tool to our kit when we're digging into a breach, building on what we may have discovered during an incident response event. 10:54.330 --> 11:02.350 So, a major source of OSINT in these cases are social media platforms, forums, and other venues online where people interact. 11:05.400 --> 11:09.600 So, social media, as I'm sure many of you are aware, is a goldmine when it comes to OSINT. 11:09.600 --> 11:13.660 I've included a few examples of the role it plays on this slide. 11:13.940 --> 11:21.540 Using social media, we can gather multiple types of information from blog posts, Facebook postings, photos, tweets, documents, and videos. 11:22.160 --> 11:27.080 With this information, we can do some interesting things, such as real-time monitoring and alerting. 11:27.320 --> 11:40.640 A good example is the current war in Ukraine, for example, where researchers have used OSINT, including geotagging, to calculate Russian battlefield losses and the locations where these battlefield losses have happened. 11:41.120 --> 11:45.100 Of course, we can also use information from social media for verification and fact-checking. 11:45.100 --> 11:54.260 For example, by cross-referencing information from multiple sources, we can help to verify if an event took place, such as, was that person at that particular place at that point in time? 11:55.260 --> 12:00.020 I hope these few examples here give you an idea of how social media is an important component of OSINT. 12:00.020 --> 12:04.000 Obviously, this isn't exhaustive, it's just a quick overview. 12:04.220 --> 12:07.500 But let's now consider the intersection of this with generative AI. 12:12.070 --> 12:18.350 In this section of the talk, we're now going to look at some problem areas where OSINT is impacted by various types of AI. 12:18.410 --> 12:28.730 These include fake profiles on social media, misinformation spread through bot farms, election interference potential, and the impact on tracing missing people. 12:33.180 --> 12:35.600 The first we'll discuss is bots. 12:35.600 --> 12:41.300 Bots and troll farms were a huge controversy in the 2016 election, as I'm sure many of you will remember. 12:41.620 --> 12:44.580 Researchers uncovered thousands of fake accounts on Twitter. 12:44.880 --> 12:52.660 Many were easy to detect, and a number of OSINT researchers did a very good job at identifying these bot farms, and some of them were obviously brought down as well. 12:53.640 --> 12:55.340 The issues, though... 12:55.340 --> 12:59.560 These issues, sorry, weren't put to rest in the previous decade. 12:59.560 --> 13:06.360 The paper linked on this slide is from NIH, the National Institute of Health, and here's some of the key pointers that came out of that. 13:06.360 --> 13:11.220 The pandemic saw activity with bots spreading misinformation around COVID-19. 13:11.220 --> 13:19.180 About a third, so 33%, of the top sharers of content from low-credibility sources were likely due to bot activity. 13:19.180 --> 13:27.760 And this key point, you know, bots are becoming increasingly difficult to separate from human users, so detecting them will need a higher level of precision as we move forwards. 13:28.280 --> 13:44.540 So, between ChatGBT and its variants, API access given to large language models and similar generative AI tools, and the ability to generate an almost infinite amount of fake photos, names, histories via gen-AI techniques, you know, raiding through this to detect a bot from the troll, 13:44.540 --> 13:47.620 from the genuine account, is going to become increasingly difficult. 13:47.620 --> 13:53.240 So, let's take a look at an example of a generative AI used in the social media space to demonstrate this. 13:55.740 --> 14:00.540 So, LinkedIn is a prime target for running bots and fake accounts. 14:00.640 --> 14:01.140 Why? 14:01.140 --> 14:05.120 Well, it's used by all levels of a company's team, from the CEO down to the engineers. 14:05.220 --> 14:12.160 Individuals working for sensitive companies, the military, or have access to intellectual property are present on these platforms. 14:12.880 --> 14:17.980 Therefore, it's just a goldmine for a hostile actor to target individuals who may have access to something they want. 14:18.780 --> 14:25.440 NPR wrote a very accessible article on this topic and provided a great graphic, which I've kindly borrowed up here. 14:25.800 --> 14:28.160 For now, at least, you can spot a generative AI. 14:28.160 --> 14:40.360 So, if you look at this photo for some oddities, you can see that, for example, the earring's missing, there's a weird-looking earlobe, maybe the hair merges slightly into the background here. 14:41.180 --> 14:46.280 In this case, we can't see the hands, but that's often an area where gen AI has struggled with. 14:46.340 --> 14:50.720 I've also seen images that look incredibly realistic, apart from having, say, too many teeth. 14:51.140 --> 15:02.780 Hopefully, in the future, we'll be able to use AI tools to detect these anomalies, although we're likely to enter into a bit of an arms race here, with hostile actors trying to outwit detection tools, much like we've seen in the antivirus space. 15:05.400 --> 15:13.860 So, we've seen there's an issue with the proliferation of bots, and then a problem with fake social media accounts posing as real individuals. 15:13.860 --> 15:17.580 This obviously has some concerns when it comes to the election space as well. 15:17.580 --> 15:24.660 So, you know, looking back at recent history, there was concerns around bots, fake ads, and similar with the UK's referendum on leaving the EU. 15:24.860 --> 15:38.480 Moving forward seven years, with the tools accessible today, it becomes much easier to generate vast amounts of realistic-looking accounts, fake videos, fraudulent photographs, in order to try and impact the outcome of an election. 15:39.120 --> 15:56.180 The Guardian newspaper recently published an article where they interviewed René Durest of the Stanford Internet Observatory, and this quote stands out, you know, text and images, however, have the potential for higher actual impact in an election scenario because they can be more subtle and longer-lasting. 15:56.180 --> 15:59.900 So, I'd recommend reading that article if you're interested in this topic. 15:59.900 --> 16:01.500 It's a good overview. 16:02.840 --> 16:19.860 So, back in 2016 through 2019 period, I was interested in this topic, although the technologies weren't as sophisticated or as accessible as they are now, and while at grad school I'd proposed the idea of sort of the equivalent to a CIA triad that maps the attack surface for an election, 16:19.860 --> 16:21.800 and this little diagram here shows us. 16:21.940 --> 16:28.220 Looking back at this from 2023, number five in this list, which is the information-based attacks, 16:31.660 --> 16:37.020 you know, they're especially vulnerable to the intersection of Gen AI and social media in the context of an election. 16:37.140 --> 16:49.160 We can imagine all sorts of examples where a politician from either party is accused of doing something, and there's apparent fake footage of this event, and then bots spread this information at astonishing speed due to network effects. 16:49.200 --> 16:55.500 So, as we move towards the 2024 presidential election, you know, we're going to have to be on our guard from being manipulated. 16:57.620 --> 17:03.040 So, I mentioned missing persons previously, so let's move on to a topic of missing persons. 17:03.040 --> 17:10.140 I know this is close to the heart of many of the people in the audience, and the Recon Village has hosted a number of talks on this subject. 17:10.240 --> 17:15.280 OSINT is used in a variety of fashions to help track down people who are missing or being trafficked. 17:15.700 --> 17:23.100 Trace Labs are perhaps the most famous in this space, and are a non-profit that helps to organize online search parties to trace individuals. 17:23.120 --> 17:32.560 They even have a VM packed with tools to help in this space, and perhaps soon they'll have some generative AI tools as well to help. 17:33.620 --> 17:53.480 But as I alluded to a moment ago, there's a lot of Recon Village participants who are also active in this space, and Ray Baker's video on using OSINT to aid in human trafficking cases is well worth watching, just to sort of get an idea of what people are trying to do using open source intelligence to effectively save people's lives. 17:54.460 --> 17:58.260 However, we have to now consider the new world they're operating in. 17:58.260 --> 18:02.580 What happens when bad actors can generate thousands of fake images to cover their tracks? 18:02.580 --> 18:05.580 How do we sift through these to find the real images from the fake? 18:05.580 --> 18:09.520 What about bad actors generating thousands of fake abuse images and showing them? 18:09.520 --> 18:11.400 How do we detect the person is real? 18:11.400 --> 18:18.920 So I think there was somebody recently, if I remember correctly from the news, who created lots of fake child porn images and was arrested for that. 18:18.920 --> 18:23.140 Now if you can imagine someone doing that, they can flood the market. 18:23.140 --> 18:25.080 They can effectively poison the well. 18:25.480 --> 18:34.940 It makes it very difficult for the FBI or others to then track down, is that a real individual versus just a horrifically nasty image that someone's generating. 18:34.940 --> 18:39.960 So some real perils here, with people poisoning that well to cover up crimes. 18:40.240 --> 18:44.480 And what else do we need to be concerned about then in that space? 18:50.710 --> 18:56.570 So issues of data validity and reliability with large language model generated text content. 18:56.570 --> 18:59.450 This is probably one of the first perils we need to consider. 18:59.530 --> 19:02.150 It's the data validity and reliability. 19:02.150 --> 19:04.610 And we have four key concerns here. 19:04.610 --> 19:06.070 So the first is bias. 19:06.070 --> 19:12.470 So the training data used in large language models can include biased information leading to outputs that reflect these biases. 19:12.490 --> 19:16.970 This impacts the validity and reliability of the generated content. 19:16.970 --> 19:23.610 So just for context, bias here is different to how I was using bias previously when we were talking about large language models. 19:24.610 --> 19:25.870 Contextual understanding. 19:25.870 --> 19:32.070 So despite their advanced capabilities, LLMs can still struggle with nuanced context. 19:32.070 --> 19:37.530 So idioms or specific cultural references which can result in misleading content effectively. 19:37.910 --> 19:39.710 And then there's factual inaccuracy. 19:39.710 --> 19:44.150 So I'm sure if any of you tried ChatGPT out, you found that it's not always correct. 19:44.150 --> 19:54.150 You know, large language models don't verify the factual correctness of their output which potentially leads to generation of misleading or false information compromising the validity of the content. 19:54.210 --> 20:04.730 There have been cases in the news recently, I think, of a lawyer who generated some sort of legal paper using ChatGPT and didn't actually check the content of it and apparently it wasn't correct. 20:04.730 --> 20:07.230 So it's evidently a problem. 20:07.570 --> 20:10.870 Then we have the consistency issue which is... 20:10.870 --> 20:23.250 And if you tried asking ChatGPT the same question twice, you know, you might not get consistent answers to the same or similar queries across different instances or different even sessions within ChatGPT. 20:23.250 --> 20:28.850 And then, of course, if you take that and apply it to BARD or some of the others, you might get some completely different answer. 20:28.850 --> 20:34.970 And that's due to sort of the inherent randomness in the generation process which can affect the reliability of the information. 20:35.410 --> 20:44.430 So when conducting OSINT exercises, we need to consider if the information we're using has been purposefully or accidentally tainted. 20:47.830 --> 20:50.430 So moving into 2024. 20:50.970 --> 20:55.190 It doesn't look like we can escape the topic of politics, unfortunately, today. 20:55.190 --> 20:59.490 But with the election of 2024, there's, you know, more perils out there. 20:59.490 --> 21:15.630 The types of issues we need to be cognizant of include fake photos of events in candidates and fake news articles, bots using Gen-AI to create images and content to pose as individuals, LLM-generated content designed to be incendiary or spread conspiracy theories. 21:15.830 --> 21:18.130 You know, LLMs lower that language barrier. 21:18.130 --> 21:31.130 So if you're part of the internet troll farm in Russia and you want to generate content that reads like an American Reddit and not somebody, you know, who's been sitting there for 12 hours a day just plugging away on Twitter, it gives you a bit of an edge there. 21:31.830 --> 21:36.030 You know, we can do things like fake constituent sentiment. 21:36.150 --> 21:38.390 We can generate content much quicker. 21:38.570 --> 21:48.130 And I've added some links here for those of you who want to dig into this topic further, including another guide, an article, and then there's a paper down here as well on faking constituent sentiment. 21:49.130 --> 21:57.490 You know, so in summary, from an OS perspective, we can see how validating an event happened or information is accurate is going to increasingly become a sort of uphill battle. 22:00.800 --> 22:07.940 So talking of fake news articles, you know, ChatGPT allows bad actors the ability to generate readable content very quickly. 22:07.940 --> 22:13.460 This can lead to a proliferation of fake news articles being spread across the internet via social media. 22:13.700 --> 22:21.320 Here's a great example on this slide from The Guardian called Breaking News from CelebritiesDeaths.com. 22:21.320 --> 22:22.640 The president is dead. 22:22.740 --> 22:27.740 I'm sure we can all think of a few jokes about Joe Biden, but I promise you he's not actually dead. 22:28.600 --> 22:37.020 In the broader context, you know, AI-driven news is the target of a variety of companies as well looking to cash in on the space. 22:37.040 --> 22:44.820 So the New York Times recently reported on this, and we've got another example, you know, Google tests AI tool that is able to write news articles. 22:44.820 --> 22:56.860 So as well as bad actors out there writing news articles designed to sort of taint something, we also have major media companies who are using these tools to generate content evidently to cut costs, right? 22:56.860 --> 23:01.260 Which brings a whole set of risks if that content is incorrect. 23:02.080 --> 23:03.900 You know, there's some news group... 23:03.900 --> 23:07.640 Sorry, there's some groups out there trying to counter this. 23:07.640 --> 23:14.900 So NewsGuard, for example, is spearheading the effort into making AI news safe, and there's a link here which you can go read about. 23:14.900 --> 23:18.900 I'm sure there's plenty of others out there who are also concerned about this. 23:20.260 --> 23:22.440 So I thought this was a good one. 23:22.840 --> 23:26.020 So remember how we discussed elections and fake photos of politicians? 23:26.020 --> 23:27.680 Well, here we have an example. 23:28.160 --> 23:31.920 Imagine if this type of thing ended up in a fake news article being spread as well. 23:33.220 --> 23:37.540 So apparel of the generative AI and ever since, insecting is the type of thing we see here. 23:37.540 --> 23:43.980 And the BBC ran a good little article on this topic, which I've linked and which I borrowed the pictures from. 23:44.160 --> 23:51.480 They also used a couple of examples of photos that were circulating showing Trump being arrested, which is the two I've used here. 23:51.540 --> 23:54.120 And these and others like them, of course, are all fake. 23:54.120 --> 24:06.040 For the eagle-eyed on there who saw these a few weeks back, it becomes increasingly obvious they were fake as tools like Mid Journey often struggle with generating text within the image. 24:06.360 --> 24:10.200 So the police badges or the background signs might often give things away. 24:11.020 --> 24:18.880 Whatever jokes you've heard about Trump's tiny hands, this photo on the left is a dead giveaway as that tiny hand just doesn't look very real. 24:23.100 --> 24:26.560 So the problem of source attribution and provenance. 24:26.560 --> 24:31.780 So another problem we face is really where on earth did this data come from? 24:31.780 --> 24:42.500 As generative AI models create content based on their training data, it's difficult to attribute specific outputs to individual data sources, complicating copyright and plagiarism issues. 24:42.520 --> 24:44.660 We also need to consider authenticity. 24:44.660 --> 24:55.380 So the ability of AI to create realistic content like deep fakes determining the authenticity of a piece of content becomes challenging, increasing a risk of misinformation and fraud. 24:55.700 --> 24:57.180 Then we need to consider ownership. 24:57.180 --> 25:00.400 It's unclear who owns the rights to AI-generated content. 25:00.400 --> 25:04.640 The AI developers, the owners of the training data, the AI itself? 25:04.640 --> 25:07.360 I mean, that all leads to various legal headaches. 25:07.380 --> 25:21.040 And then finally, you know, considering traceability, given the large scale and complex nature of the training data, tracking the provenance, which is essentially the origin and the history of a specific AI-generated output is almost impossible, making it difficult to verify or challenge the output. 25:21.040 --> 25:23.000 So this has a huge impact on OSINT. 25:23.100 --> 25:34.520 Just on the topic of authenticity and ownership and similar, you know, it's very difficult now when students submit papers to know whether they are actually written by them or by AI. 25:34.760 --> 25:42.580 There's tools like Turnitin which check for plagiarism, but I'm sure in the future it's going to become increasingly difficult to know who actually wrote the paper. 25:42.580 --> 25:46.960 It's an interesting time to be in higher ed, if any of you out there are. 25:48.200 --> 25:51.380 So challenges in discerning human versus gen-AI content. 25:51.380 --> 25:56.560 So all of this may have got you thinking, there's going to be some serious challenges, and I think that yes, there are. 25:56.560 --> 25:58.140 And yes, you're right, I know about that. 25:58.140 --> 26:00.600 I've touched on that across the last few slides. 26:00.700 --> 26:09.820 One significant challenge is discerning human versus gen-AI content is the increasing ability of AI models to produce highly realistic and natural-looking photos and text. 26:09.820 --> 26:13.880 We saw the Trump example, but don't expect it to always be like that. 26:13.880 --> 26:17.540 I think you'll probably start to see all of those anomalies vanish. 26:17.940 --> 26:33.740 So now, couple up the sophistication of AI models, as the models become more... as they just become better over time, and they're trained on vast data sets, they gain a better understanding of human language, behaviour and artistic styles, which means we might see fewer of those tiny hand anomalies. 26:34.740 --> 26:37.900 This in turn leads to a technology that is continuously evolving. 26:37.900 --> 26:42.260 New breakthroughs in gen-AI can quickly outpace existing methods for detecting them. 26:42.460 --> 26:55.480 So, finally, just to hammer it home, with this proliferation of AI-generated content, there's growing concern about its potential malicious use, including spreading disinformation, generating fake news, or creating deceptive social media profiles. 26:55.920 --> 27:08.780 And on that social media profile, I can just say from myself and some of my team over here, we've had to deal with it in LinkedIn, where we get people saying they work for Modus, and I'm like, we don't have an office in Minneapolis, so who are you, you know? 27:09.160 --> 27:11.520 And then you're playing whack-a-mole constantly. 27:12.140 --> 27:15.400 So, yeah, I mean, it's already a problem, it's just, I think it's going to get worse. 27:17.060 --> 27:20.060 So, what tools are people using to generate this content? 27:20.060 --> 27:21.580 Let's take a look at a few of them. 27:21.580 --> 27:24.880 I'm sure you're probably familiar with many of them, so we'll start with images. 27:25.220 --> 27:28.780 So I included some graphics throughout these slides, which were generated with Midjourney. 27:28.780 --> 27:32.400 They were very easy to create, and Midjourney isn't the only tool on the market. 27:32.740 --> 27:35.540 We've got others out there, including Crayon, Dali. 27:35.540 --> 27:45.540 And when I wrote the talk, I'd said, you know, I think one of the 2,600 covers from last year used Crayon to generate some of the images, and I found the copy of the magazine, which is in there. 27:45.540 --> 27:57.500 And if you take a look at it, and I'm happy to show it after, with, like, the images now, it's just night and day how easy it is to generate very realistic images compared to what we had even, you know, two years ago, a year ago. 27:59.300 --> 28:01.100 So, moving on to text. 28:02.020 --> 28:05.060 So, you know, the infamous LMLs, right? 28:05.420 --> 28:13.520 ChatGBT is the obvious one, but we've got BARD, we've got LLAMA, we've got Bing Chat, and there's this paper linked on here from Schick and all discusses. 28:13.660 --> 28:16.960 You know, we can start thinking about how do we plug LLMs into other tools? 28:16.960 --> 28:19.160 So, like, APIs to pull data on the fly. 28:19.160 --> 28:20.020 So, think about that. 28:20.020 --> 28:23.800 You start to open up these tools, and they can interact with other third-party services. 28:24.020 --> 28:26.600 That puts us in a very interesting space. 28:29.680 --> 28:30.400 Sound. 28:30.980 --> 28:34.120 So, that could be spoken word, or it could be music. 28:34.120 --> 28:39.740 There's a number of tools and platforms out there, such as Soundraw, Voicebox, and Valli. 28:39.960 --> 28:44.240 We've already seen some amusing uses for these types of platforms. 28:44.400 --> 28:48.980 So, the story at the top here, that heart on my sleeve's long was completely fake. 28:48.980 --> 28:59.820 I think the record companies were trying to constantly take it down, but it was just an amalgamation of The Weeknd and Drake, and it sounded pretty realistic, to be honest. 29:00.260 --> 29:08.880 You know, and we can also do some other interesting things in this space, such as convert text to speech, speech to text, and realistically mimic existing voices. 29:11.960 --> 29:17.420 So, you know, the above leads us to ask what the potential is for AI-driven influence operations. 29:17.420 --> 29:20.060 What if we combine the outputs from all of these tools? 29:20.360 --> 29:35.120 So, an AI-driven influence operation, you know, has the potential to amplify disinformation campaigns on a massive scale, because it can use these generative AI models, you know, or the malicious actors can, sorry, to spread convincing fake content, including text, 29:35.120 --> 29:38.160 images, and videos, which can mislead and deceive the public. 29:39.260 --> 29:42.700 This, in turn, can lead to what we call targeted manipulation. 29:42.860 --> 29:55.360 We can see that AI-powered influence operations can analyze vast amounts of data about individuals, such as social media posts, browsing behavior, and personal preferences, to tailor and target and manipulate content specifically for them. 29:55.360 --> 30:03.620 This personalized approach increases the effectiveness of the influence campaign, making it harder for the individuals to recognize the manipulation. 30:04.140 --> 30:07.020 The outcome of that can be polarization and division. 30:07.020 --> 30:14.900 So, influence operations can exploit social and political fault lines to intensify that polarization, and so discord among different groups. 30:14.900 --> 30:23.040 And by strategically disseminating divisive content, AI can exacerbate existing tensions and create more fragmented society. 30:24.020 --> 30:27.220 And bad actors can use automated social bots to handle this. 30:27.220 --> 30:35.060 So, AI can be used to create and deploy automated social bots that can engage with real users and spread propaganda, fake news, misleading information. 30:36.160 --> 30:41.260 Whatever you can think of in theory, they can probably do it if it's text-based and it's a one-on-one interaction. 30:41.600 --> 30:50.020 These AI-driven bots can make content appear to be more popular or generate artificial trends, manipulating public perception and influencing public discourse. 30:50.220 --> 30:55.740 So, I think I've beaten this horse enough about bots and similar, but how do we mitigate this? 31:01.240 --> 31:04.520 So, the first thing we can look at doing is training and workshops. 31:04.520 --> 31:12.740 So, conducting regular training workshops and webinars specifically tailored towards open-source intelligence researchers can help enhance their AI literacy. 31:12.820 --> 31:22.820 These workshops can cover fundamentals of AI, explain how AI is used in generating content, and demonstrate techniques to identify AI content. 31:22.840 --> 31:33.300 And since this field is moving so quickly, by the time I've given this talk, it's probably already obsolete, it's important that we stay on top of this and that there's a continuous training process. 31:34.300 --> 31:41.180 Conferences and venues such as the Recon Village at DEF CON are great venues to share this information with the OSINT research community. 31:41.240 --> 31:50.940 We're still in very early days, so I'm sure we're going to learn a lot over the next year and by the time the Recon Village comes back to DEF CON next year, the talk will be even more informative. 31:52.000 --> 31:54.240 We can collaborate with AI experts. 31:54.240 --> 32:05.420 So , facilitating collaboration between OSINT researchers and AI experts can foster a deeper understanding of AI technologies and their potential applications in content generation. 32:05.420 --> 32:13.600 Regular interactions with AI specialists can help researchers stay informed about the latest advancements in AI and how they may impact OSINT investigations. 32:13.800 --> 32:27.060 And this kind of reminds me of the opening talk at the AppSec Village, where essentially one of the main points of it was about cross-team collaboration and expertise, that sort of cross-disciplinary approach to things, and I think we're going to have to adopt that in this space, 32:27.060 --> 32:31.160 by working closely with those really smart folks in the AI space. 32:33.220 --> 32:40.860 Another area, you know, which is theoretically possible, and a topic everyone loves, is legislation regulations. 32:41.160 --> 32:47.760 You know, I think we can expect to see calls for transparency and regulations for gen AI usage in social media. 32:48.300 --> 32:49.060 Will this work? 32:49.060 --> 32:50.320 I don't know. 32:50.320 --> 32:51.860 I couldn't tell you. 32:53.000 --> 32:56.220 Blumenthal and Hawley, the Senators, think so. 32:56.220 --> 33:03.560 The No Section 230 Immunity for AI Act calls to remove most immunity under the Communications Decency Act. 33:03.560 --> 33:14.640 So Section 230, if you're not familiar, of the CDA of 1996 provides broad protection to what they call interactive computer services, which really does sound a phrase straight out of the 90s. 33:14.640 --> 33:26.280 And by holding that they can't be treated as the publisher or speaker of information provided by users, so effectively that immunity would say that AI, you know, doesn't... 33:27.020 --> 33:31.340 It loses that immunity effectively if you're using it to generate content. 33:33.400 --> 33:35.560 So validation and tooling. 33:36.320 --> 33:39.820 Can we use tooling, including AI, to detect AI? 33:39.900 --> 33:44.980 This may be one avenue that OSN researchers can use to filter the signal from the noise. 33:44.980 --> 33:46.500 So a few ideas here. 33:46.500 --> 33:55.900 So improve access to AI tools, so providing OSN researchers with access to AI analysis tools can enable you to better understand the capabilities and limitations of AI in generating content. 33:56.520 --> 34:06.020 And by experimenting with AI tools themselves, researchers can gain first-hand experience in discerning human versus AI-created content. 34:06.240 --> 34:08.420 Then we have detection techniques. 34:08.420 --> 34:13.360 So employing AI tools to detect and flag AI-generated content. 34:13.360 --> 34:20.500 I've seen some attempts here, and I can imagine in the higher ed space, as I mentioned before, in stopping plagiarism, it could be useful. 34:20.500 --> 34:22.300 Then we have validation. 34:22.300 --> 34:26.220 So developing more robust data validation and verification techniques. 34:26.600 --> 34:27.520 Watermarks. 34:27.520 --> 34:32.640 So digitally, cryptographically watermark content so it can be verified against a third-party list. 34:32.640 --> 34:36.700 This could be a good use for the blockchain-style technologies, for example. 34:37.920 --> 34:45.260 And then I think when it comes to other approaches, we're probably going to learn them as we go. 34:45.860 --> 34:51.460 So that's some ideas on how we tackle the issues, and people much smarter than me are definitely on this case. 34:51.460 --> 34:54.340 So let's take a look at the future. 34:59.720 --> 35:15.260 So unless the technology hits some sort of wall or barrier, which is certainly possible, I guess, then I just think the performance will continue to grow, at least to a point where the problems it solves will be very difficult for us to tell real content from fake content. 35:15.940 --> 35:25.400 We shouldn't be alarmists in space, and I know some of the debates are quite hot right now about AI alignment and similar, but I definitely don't think we can be complacent either. 35:25.400 --> 35:26.760 We sort of want to be empirical about this. 35:26.760 --> 35:27.720 We've got to collect our data. 35:27.720 --> 35:38.300 We've got to understand where the trends are and then work and operate against them as opposed to sort of imagining where things might go in a very negative fashion. 35:38.300 --> 35:54.440 I think, yeah, being empirical is important, but, you know, coupled with the commercial efforts by Meta, Google, OpenAI, and similar, I just think we're going to see a proliferation of open source and free LLMs, commercial options, so it's very quickly sort of the technology's out of the box, 35:54.440 --> 35:55.300 effectively. 35:55.820 --> 36:02.640 From nowhere, potential trends we should consider, therefore, that's realistic fake video generated on the fly. 36:02.640 --> 36:12.000 So right now, images are a very popular sound, but I think video is certainly going to be something that we can expect to see increasingly appear. 36:12.140 --> 36:20.840 Improvements in mimicking human voices, increasing difficulties in detecting fake images, and the weird artifacts I talked about, sort of uncanny hands and ears. 36:21.340 --> 36:26.640 We should also consider how OSINT will adopt techniques to improve gathering and analyzing data. 36:26.980 --> 36:31.160 And there's an open source library, which I've linked to here, which, you know, is worth checking out. 36:31.380 --> 36:37.360 There's tools for handling ETL processes, for consuming large amounts of data and doing ETL, and there's a link here. 36:37.480 --> 36:45.200 And I think processing samples of text to find similarities with other text online, so you can detect the author, I think that's going to be an area. 36:45.220 --> 36:49.080 And there's already tools out there, but I think they're going to become more sophisticated. 36:50.400 --> 36:55.300 And there's lots of opportunities for building OSINT tools on top of Gen AI models. 36:55.300 --> 37:00.660 We just have to collectively think about how we would approach doing that and build it. 37:03.040 --> 37:09.360 So I'm going to wrap up, because I know we're almost at time, and it's Saturday night at DEF CON, so I'm sure everyone has party plans. 37:10.880 --> 37:13.740 So in conclusion, I hope you enjoyed the talk. 37:13.740 --> 37:16.640 Let's do just a quick, brief recap. 37:17.260 --> 37:23.180 Gen AI is here to stay for the foreseeable future, and it looks like it will continue to improve. 37:23.400 --> 37:26.040 OSINT practitioners will need to adapt to this world. 37:26.060 --> 37:29.040 We're going to need better tools to detect fake content. 37:29.320 --> 37:34.840 Creators of original, genuine content will need to find ways to better distinguish their material from fake content. 37:34.940 --> 37:39.520 And the OSINT world can leverage Gen AI tools as well. 37:39.680 --> 37:43.500 And here's a nice quote from William Gibson's Johnny Mnemonic. 37:43.740 --> 37:51.140 It's impossible to move, to live, to operate at any level without leaving traces, bits, seemingly meaningless fragments of personal information. 37:51.180 --> 37:55.460 So in the new world of Gen AI, we're going to need to separate the winter meats from the humans. 37:56.600 --> 37:58.660 And with that, I'd like to say thank you. 37:58.660 --> 38:03.120 And finally, could we all give the guys from the Recon Village a round of applause as well? 38:03.120 --> 38:04.080 Thank you.