1
00:00:00,000 --> 00:00:04,740
Right, so along with me, for the last talk, we have Andy Dennis.

2
00:00:04,840 --> 00:00:08,960
Andy is a VP of consulting at Modus Create.

3
00:00:12,630 --> 00:00:20,510
He has over 20 years of experience in security, software engineering, DevOps, consulting, system migration, and modernization.

4
00:00:20,670 --> 00:00:27,190
Andy is a published author and online tutor for Goldsmiths University of London.

5
00:00:27,490 --> 00:00:31,770
Andy has also spoken at and also helped organize B-Sides events in New England.

6
00:00:31,770 --> 00:00:41,170
And he has also spoken in multiple other conferences such as B-Sides CT, B-Sides Orlando, and Defcon Village in the previous editions as well.

7
00:00:41,550 --> 00:00:48,730
The topic Andy will be covering is the Perils of Generative AI Implications for Open Source Intelligence Research.

8
00:00:48,790 --> 00:00:50,690
So please give it up for Andy.

9
00:01:03,880 --> 00:01:09,640
Also, a little request that if you could cover up the seats ahead so that it would be better because...

10
00:01:09,640 --> 00:01:13,860
I mean, we have a lot of seats that are empty in between, so if you could cover those up.

11
00:01:14,060 --> 00:01:18,340
And please turn your phones into silence so that the speaker isn't disturbed in between.

12
00:01:18,340 --> 00:01:19,220
And have a good time.

13
00:01:25,520 --> 00:01:29,120
So they just did the introduction for me, so this slide's kind of redundant.

14
00:01:29,540 --> 00:01:35,340
But I will tell you, we are recruiting, so go to this website if you're a Java engineer, if you work in DevOps.

15
00:01:36,220 --> 00:01:42,440
If you're really good with GitHub and GitHub Actions, we're definitely looking for folks right now, so please do take a look.

16
00:01:42,780 --> 00:01:49,680
And yeah, my previous talk, if you're interested, was on using passive DNS for gathering business intelligence, and that was from 2021.

17
00:01:49,700 --> 00:01:51,660
So feel free to check that out.

18
00:01:51,660 --> 00:01:54,080
So then, let's get started, I guess.

19
00:01:55,820 --> 00:02:03,600
So we're going to start with a brief explanation of generative AI and open-source intelligence.

20
00:02:03,600 --> 00:02:05,600
Sorry, I'm just going to move this mic so it's a little easier.

21
00:02:05,960 --> 00:02:12,540
So for those of you new to the topic, I'm going to give a brief overview of what this is and also a little background on OSINT.

22
00:02:12,800 --> 00:02:19,200
I've included a timeline on the next few slides, which just gives a brief overview of some of the major milestones in the AI space.

23
00:02:19,260 --> 00:02:23,600
We won't be going into deep mathematical details here, so you don't have to run for the exit yet.

24
00:02:24,060 --> 00:02:30,860
But in brief, generative AI is a category of artificial intelligence that's capable of generating new content, such as text, images, and audios.

25
00:02:30,860 --> 00:02:32,540
And I'm sure all of you are familiar with that.

26
00:02:33,480 --> 00:02:37,320
Many of you will have kicked the tires on tools such as ChatGBT and MidJourney.

27
00:02:37,580 --> 00:02:43,880
This approach is built upon years of research and development in the AI space and is really coming to fruition at this point in time.

28
00:02:43,880 --> 00:02:49,300
Let's see what ChatGBT says when we ask it to summarize what generative AI is.

29
00:02:51,440 --> 00:02:55,580
I have gone too many slides forwards by the looks of it.

30
00:02:56,280 --> 00:02:57,120
There we go.

31
00:02:57,120 --> 00:02:57,820
Sorry.

32
00:02:57,960 --> 00:03:06,860
So generative AI is a subset of artificial intelligence that utilizes machine learning models to create new data samples that mimic the properties of some source data set.

33
00:03:07,040 --> 00:03:11,860
This could involve generating anything from written text, images, music, or even video content.

34
00:03:11,860 --> 00:03:19,020
This is achieved through algorithms such as GANs, where two neural networks, a generator and a discriminator, are trained in tandem.

35
00:03:19,020 --> 00:03:25,160
The generator aims to produce realistic data, while the discriminator's job is to distinguish between real and generated data.

36
00:03:25,160 --> 00:03:30,860
The iterative competition between the two leads to the production of increasingly accurate and realistic synthetic data.

37
00:03:30,860 --> 00:03:37,320
Other popular generators include VAEs and transformer-based models like GPT-3 and GPT-4.

38
00:03:37,320 --> 00:03:42,180
So, hopefully that's nice and clear now what that means.

39
00:03:43,180 --> 00:03:53,860
Basically, these tools and models are trained on large data sets and apply machine learning techniques to generate new content such as image from a text prompt or poem in the style of, say, a long-dead author.

40
00:03:56,240 --> 00:04:06,260
This ability to mimic styles and generate novel content allows users to create realistic images, compose music, and increasingly explore problem spaces in various scientific fields.

41
00:04:06,260 --> 00:04:12,740
Despite all the fun and serious use of the technology, concerns around ethics and the ability for gross misuse have been a key concern.

42
00:04:12,940 --> 00:04:16,180
I'm sure you've been following anything in the news recently.

43
00:04:16,720 --> 00:04:20,640
You know, AI alignment and ethics is a major topic.

44
00:04:21,300 --> 00:04:25,820
Fraudulent data, such as deepfakes, will become increasingly easy to generate for anyone with a laptop.

45
00:04:26,300 --> 00:04:30,220
And separating the real video from the AI-generated one will get harder over time.

46
00:04:30,500 --> 00:04:33,240
That's a very, very basic overview of Gen-AI.

47
00:04:33,240 --> 00:04:40,320
To sum it up, it's a way of taking large amounts of data and training software to be able to create a new, compelling content based off of this.

48
00:04:40,340 --> 00:04:44,900
And later in the talk, we're going to pull some of this apart and delve into it in more detail.

49
00:04:46,820 --> 00:04:51,300
So, we did a quick overview of Gen-AI, but what about OSINT?

50
00:04:51,300 --> 00:04:53,660
So, OSINT stands for Open Source Intelligence.

51
00:04:53,660 --> 00:05:02,580
It's a set of methods, techniques, and tools to collect information from publicly available sources, such as blogs, social media, data sets, and reports, and so on.

52
00:05:02,760 --> 00:05:04,580
It's not just hackers using this.

53
00:05:04,580 --> 00:05:13,520
Journalists use it, business intelligence analysts, law enforcement, cyber security professionals, and a bunch of other folks use it for a variety of purposes.

54
00:05:13,800 --> 00:05:21,100
This data can be collected through a range of techniques, such as web scraping tools, natural language processing of data, machine learning.

55
00:05:21,320 --> 00:05:27,480
The Recon Village website includes some links to these various tools, and a bunch of presentations from over the years.

56
00:05:27,480 --> 00:05:35,840
And on this slide and next, you can see some of the various products and projects and open source GitHub repos that contribute to this space.

57
00:05:35,840 --> 00:05:37,180
And so, please go check them out.

58
00:05:37,180 --> 00:05:40,800
And I think maybe Amass was already covered earlier today in one of the talks.

59
00:05:43,140 --> 00:05:48,800
So, now a key aspect of OSINT is the analysis of the data we've collected with the various tools.

60
00:05:48,800 --> 00:05:53,140
Once we have the data, we can use it to study trends, draw conclusions, and make decisions.

61
00:05:53,780 --> 00:05:55,920
This is incredibly useful, of course.

62
00:05:55,980 --> 00:06:00,120
For example, if we're trying to trace an individual or discover some information about a competitor.

63
00:06:00,120 --> 00:06:06,020
But despite this usefulness, we need to be cognizant of privacy concerns and ensure we are operating in an ethical fashion.

64
00:06:06,680 --> 00:06:11,720
A key problem arises, though, when considering using OSINT to inform decisions and drive actions.

65
00:06:11,740 --> 00:06:13,340
What if the data is fake?

66
00:06:13,900 --> 00:06:17,600
What if it was generated by AI methods, like I previously mentioned?

67
00:06:19,960 --> 00:06:21,360
So, a bit more background.

68
00:06:21,360 --> 00:06:23,000
Let's dig a little deeper.

69
00:06:23,880 --> 00:06:33,320
So, now we're going to dive into JANA and OSINT a little bit more, but we just did a speed run through the basics there.

70
00:06:33,460 --> 00:06:42,680
So, let's cover some of the techniques used by large language models and other generative platforms that were hinted at in the chat GPT's definition previously.

71
00:06:42,940 --> 00:06:48,700
So, key machine learning techniques listed on these slides include GANs, VAEs, and transformer models.

72
00:06:48,920 --> 00:06:51,760
Chat GPT previously explained what a GAN was.

73
00:06:51,760 --> 00:06:57,380
A VAE is a variational autoencoder, and it's a form of a neural network.

74
00:06:57,380 --> 00:06:59,260
They use probabilistic models.

75
00:06:59,320 --> 00:07:05,840
So, a VAE is a type of generative model that uses a variational interface to encode and decode data.

76
00:07:05,840 --> 00:07:11,860
They represent the data in a probabilistic latent space, enabling them to generate new data that's similar to the training data.

77
00:07:11,860 --> 00:07:13,700
And then we get to transformers.

78
00:07:13,880 --> 00:07:22,260
So, transformers, which is a type of technology used by chat GPT, are also a neural network-based approach that underpins large language models.

79
00:07:22,560 --> 00:07:26,320
You may have heard the term transformers a lot recently due to LLM's popularity.

80
00:07:26,320 --> 00:07:29,020
So, that's kind of just a basic description of it there.

81
00:07:29,020 --> 00:07:32,920
And you can essentially think of them as taking one input and transforming it into another.

82
00:07:33,300 --> 00:07:40,580
They're a mechanism for learning context through tracking relationships and data without needing recurrent layers, enabling parallel processing.

83
00:07:41,120 --> 00:07:44,140
A couple of other phrases you may have heard are weights and biases.

84
00:07:44,140 --> 00:07:55,580
So, in a neural network, weights are the parameters that are learned during the training to scale the input of data and connections between the neurons in the neural network and determining the strength of the signal passed through the network.

85
00:07:55,580 --> 00:08:06,160
And biases are additional parameters in a neural network that allow for the shifting, shifting the activation function to the left or right, helping the network to fit the data more flexibly.

86
00:08:06,240 --> 00:08:10,260
These models use a variety of approaches, such as games and competition,

87
00:08:13,540 --> 00:08:16,960
which help to improve constantly the quality of the output.

88
00:08:17,480 --> 00:08:19,900
As each technique gets better, so do the outputs.

89
00:08:19,900 --> 00:08:26,080
So, effectively, as each of these techniques get better, the images are going to get better, the text output is going to get better, the video output is going to get better.

90
00:08:26,600 --> 00:08:28,880
You really don't have to understand all the math here.

91
00:08:29,240 --> 00:08:35,920
There are people much smarter than me who I work with who really get that side of things, and that's not my background as a mathematician.

92
00:08:35,920 --> 00:08:39,880
But having a general understanding of the techniques is useful in understanding space.

93
00:08:40,520 --> 00:08:51,820
So, as a result, AI researchers have been able to improve the outputs to start mimicking real-life photography, sound like the voice of a living individual, or create text that reads like a famous author.

94
00:08:52,140 --> 00:08:55,300
And we'll be seeing a few examples of these later in the talk.

95
00:08:56,620 --> 00:09:00,840
So, a little bit on the pace of improvements between 2022 and 2023.

96
00:09:00,980 --> 00:09:08,180
As I just started when discussing the previous slide, as the outputs get better, they get closer to mimicking the real, effectively.

97
00:09:08,220 --> 00:09:15,220
And what's driving us closer in on the simulacrum being indistinguishable from the real is the pace of technological change.

98
00:09:15,220 --> 00:09:22,620
So, this diagram shows that between 2020 and 2023, we went from GPT 3.0 to GPT 3.4.

99
00:09:22,700 --> 00:09:25,940
The change in quality of the outputs in this period took leaps and bounds.

100
00:09:26,220 --> 00:09:36,520
We saw this on the image side of things too, an explosion of vendors releasing new products and new open source projects and code that might have been leaked and shared.

101
00:09:36,820 --> 00:09:41,280
And it's all in the image space, the text space, the data processing space, the sound space.

102
00:09:41,280 --> 00:09:43,780
So , it's really quite across the spectrum.

103
00:09:44,280 --> 00:09:52,280
And I think this is going to become obviously important in relation to OSINT, as some examples that are coming up will demonstrate.

104
00:09:56,450 --> 00:10:02,030
So, open source intelligence and its importance to security, the security space.

105
00:10:02,790 --> 00:10:08,470
So, that was a little bit more about some of the techniques that are used by Gen AI and the pace of change around these.

106
00:10:08,870 --> 00:10:14,670
But let's switch back to discuss open sources intelligence and the importance in the security space.

107
00:10:14,670 --> 00:10:17,990
And as we can see here, there are four key areas that come to mind.

108
00:10:18,890 --> 00:10:27,550
So, we have threat intelligence, which is basically detecting cyber threats and patterns and using that information to harden systems, processes, and policies that drive that.

109
00:10:27,750 --> 00:10:34,330
Incident response, so when something bad happens, we can use OSINT to understand the nature of the attack and the potential culprits.

110
00:10:34,890 --> 00:10:41,670
And the security awareness size, that's basically monitoring OSINT sources, which allows us to get ahead of events.

111
00:10:41,870 --> 00:10:43,750
So, understand threat actors.

112
00:10:43,910 --> 00:10:53,850
And then digital forensics, OSINT can add another tool to our kit when we're digging into a breach, building on what we may have discovered during an incident response event.

113
00:10:54,330 --> 00:11:02,350
So, a major source of OSINT in these cases are social media platforms, forums, and other venues online where people interact.

114
00:11:05,400 --> 00:11:09,600
So, social media, as I'm sure many of you are aware, is a goldmine when it comes to OSINT.

115
00:11:09,600 --> 00:11:13,660
I've included a few examples of the role it plays on this slide.

116
00:11:13,940 --> 00:11:21,540
Using social media, we can gather multiple types of information from blog posts, Facebook postings, photos, tweets, documents, and videos.

117
00:11:22,160 --> 00:11:27,080
With this information, we can do some interesting things, such as real-time monitoring and alerting.

118
00:11:27,320 --> 00:11:40,640
A good example is the current war in Ukraine, for example, where researchers have used OSINT, including geotagging, to calculate Russian battlefield losses and the locations where these battlefield losses have happened.

119
00:11:41,120 --> 00:11:45,100
Of course, we can also use information from social media for verification and fact-checking.

120
00:11:45,100 --> 00:11:54,260
For example, by cross-referencing information from multiple sources, we can help to verify if an event took place, such as, was that person at that particular place at that point in time?

121
00:11:55,260 --> 00:12:00,020
I hope these few examples here give you an idea of how social media is an important component of OSINT.

122
00:12:00,020 --> 00:12:04,000
Obviously, this isn't exhaustive, it's just a quick overview.

123
00:12:04,220 --> 00:12:07,500
But let's now consider the intersection of this with generative AI.

124
00:12:12,070 --> 00:12:18,350
In this section of the talk, we're now going to look at some problem areas where OSINT is impacted by various types of AI.

125
00:12:18,410 --> 00:12:28,730
These include fake profiles on social media, misinformation spread through bot farms, election interference potential, and the impact on tracing missing people.

126
00:12:33,180 --> 00:12:35,600
The first we'll discuss is bots.

127
00:12:35,600 --> 00:12:41,300
Bots and troll farms were a huge controversy in the 2016 election, as I'm sure many of you will remember.

128
00:12:41,620 --> 00:12:44,580
Researchers uncovered thousands of fake accounts on Twitter.

129
00:12:44,880 --> 00:12:52,660
Many were easy to detect, and a number of OSINT researchers did a very good job at identifying these bot farms, and some of them were obviously brought down as well.

130
00:12:53,640 --> 00:12:55,340
The issues, though...

131
00:12:55,340 --> 00:12:59,560
These issues, sorry, weren't put to rest in the previous decade.

132
00:12:59,560 --> 00:13:06,360
The paper linked on this slide is from NIH, the National Institute of Health, and here's some of the key pointers that came out of that.

133
00:13:06,360 --> 00:13:11,220
The pandemic saw activity with bots spreading misinformation around COVID-19.

134
00:13:11,220 --> 00:13:19,180
About a third, so 33%, of the top sharers of content from low-credibility sources were likely due to bot activity.

135
00:13:19,180 --> 00:13:27,760
And this key point, you know, bots are becoming increasingly difficult to separate from human users, so detecting them will need a higher level of precision as we move forwards.

136
00:13:28,280 --> 00:13:44,540
So, between ChatGBT and its variants, API access given to large language models and similar generative AI tools, and the ability to generate an almost infinite amount of fake photos, names, histories via gen-AI techniques, you know, raiding through this to detect a bot from the troll,

137
00:13:44,540 --> 00:13:47,620
from the genuine account, is going to become increasingly difficult.

138
00:13:47,620 --> 00:13:53,240
So, let's take a look at an example of a generative AI used in the social media space to demonstrate this.

139
00:13:55,740 --> 00:14:00,540
So, LinkedIn is a prime target for running bots and fake accounts.

140
00:14:00,640 --> 00:14:01,140
Why?

141
00:14:01,140 --> 00:14:05,120
Well, it's used by all levels of a company's team, from the CEO down to the engineers.

142
00:14:05,220 --> 00:14:12,160
Individuals working for sensitive companies, the military, or have access to intellectual property are present on these platforms.

143
00:14:12,880 --> 00:14:17,980
Therefore, it's just a goldmine for a hostile actor to target individuals who may have access to something they want.

144
00:14:18,780 --> 00:14:25,440
NPR wrote a very accessible article on this topic and provided a great graphic, which I've kindly borrowed up here.

145
00:14:25,800 --> 00:14:28,160
For now, at least, you can spot a generative AI.

146
00:14:28,160 --> 00:14:40,360
So, if you look at this photo for some oddities, you can see that, for example, the earring's missing, there's a weird-looking earlobe, maybe the hair merges slightly into the background here.

147
00:14:41,180 --> 00:14:46,280
In this case, we can't see the hands, but that's often an area where gen AI has struggled with.

148
00:14:46,340 --> 00:14:50,720
I've also seen images that look incredibly realistic, apart from having, say, too many teeth.

149
00:14:51,140 --> 00:15:02,780
Hopefully, in the future, we'll be able to use AI tools to detect these anomalies, although we're likely to enter into a bit of an arms race here, with hostile actors trying to outwit detection tools, much like we've seen in the antivirus space.

150
00:15:05,400 --> 00:15:13,860
So, we've seen there's an issue with the proliferation of bots, and then a problem with fake social media accounts posing as real individuals.

151
00:15:13,860 --> 00:15:17,580
This obviously has some concerns when it comes to the election space as well.

152
00:15:17,580 --> 00:15:24,660
So, you know, looking back at recent history, there was concerns around bots, fake ads, and similar with the UK's referendum on leaving the EU.

153
00:15:24,860 --> 00:15:38,480
Moving forward seven years, with the tools accessible today, it becomes much easier to generate vast amounts of realistic-looking accounts, fake videos, fraudulent photographs, in order to try and impact the outcome of an election.

154
00:15:39,120 --> 00:15:56,180
The Guardian newspaper recently published an article where they interviewed René Durest of the Stanford Internet Observatory, and this quote stands out, you know, text and images, however, have the potential for higher actual impact in an election scenario because they can be more subtle and longer-lasting.

155
00:15:56,180 --> 00:15:59,900
So, I'd recommend reading that article if you're interested in this topic.

156
00:15:59,900 --> 00:16:01,500
It's a good overview.

157
00:16:02,840 --> 00:16:19,860
So, back in 2016 through 2019 period, I was interested in this topic, although the technologies weren't as sophisticated or as accessible as they are now, and while at grad school I'd proposed the idea of sort of the equivalent to a CIA triad that maps the attack surface for an election,

158
00:16:19,860 --> 00:16:21,800
and this little diagram here shows us.

159
00:16:21,940 --> 00:16:28,220
Looking back at this from 2023, number five in this list, which is the information-based attacks,

160
00:16:31,660 --> 00:16:37,020
you know, they're especially vulnerable to the intersection of Gen AI and social media in the context of an election.

161
00:16:37,140 --> 00:16:49,160
We can imagine all sorts of examples where a politician from either party is accused of doing something, and there's apparent fake footage of this event, and then bots spread this information at astonishing speed due to network effects.

162
00:16:49,200 --> 00:16:55,500
So, as we move towards the 2024 presidential election, you know, we're going to have to be on our guard from being manipulated.

163
00:16:57,620 --> 00:17:03,040
So, I mentioned missing persons previously, so let's move on to a topic of missing persons.

164
00:17:03,040 --> 00:17:10,140
I know this is close to the heart of many of the people in the audience, and the Recon Village has hosted a number of talks on this subject.

165
00:17:10,240 --> 00:17:15,280
OSINT is used in a variety of fashions to help track down people who are missing or being trafficked.

166
00:17:15,700 --> 00:17:23,100
Trace Labs are perhaps the most famous in this space, and are a non-profit that helps to organize online search parties to trace individuals.

167
00:17:23,120 --> 00:17:32,560
They even have a VM packed with tools to help in this space, and perhaps soon they'll have some generative AI tools as well to help.

168
00:17:33,620 --> 00:17:53,480
But as I alluded to a moment ago, there's a lot of Recon Village participants who are also active in this space, and Ray Baker's video on using OSINT to aid in human trafficking cases is well worth watching, just to sort of get an idea of what people are trying to do using open source intelligence to effectively save people's lives.

169
00:17:54,460 --> 00:17:58,260
However, we have to now consider the new world they're operating in.

170
00:17:58,260 --> 00:18:02,580
What happens when bad actors can generate thousands of fake images to cover their tracks?

171
00:18:02,580 --> 00:18:05,580
How do we sift through these to find the real images from the fake?

172
00:18:05,580 --> 00:18:09,520
What about bad actors generating thousands of fake abuse images and showing them?

173
00:18:09,520 --> 00:18:11,400
How do we detect the person is real?

174
00:18:11,400 --> 00:18:18,920
So I think there was somebody recently, if I remember correctly from the news, who created lots of fake child porn images and was arrested for that.

175
00:18:18,920 --> 00:18:23,140
Now if you can imagine someone doing that, they can flood the market.

176
00:18:23,140 --> 00:18:25,080
They can effectively poison the well.

177
00:18:25,480 --> 00:18:34,940
It makes it very difficult for the FBI or others to then track down, is that a real individual versus just a horrifically nasty image that someone's generating.

178
00:18:34,940 --> 00:18:39,960
So some real perils here, with people poisoning that well to cover up crimes.

179
00:18:40,240 --> 00:18:44,480
And what else do we need to be concerned about then in that space?

180
00:18:50,710 --> 00:18:56,570
So issues of data validity and reliability with large language model generated text content.

181
00:18:56,570 --> 00:18:59,450
This is probably one of the first perils we need to consider.

182
00:18:59,530 --> 00:19:02,150
It's the data validity and reliability.

183
00:19:02,150 --> 00:19:04,610
And we have four key concerns here.

184
00:19:04,610 --> 00:19:06,070
So the first is bias.

185
00:19:06,070 --> 00:19:12,470
So the training data used in large language models can include biased information leading to outputs that reflect these biases.

186
00:19:12,490 --> 00:19:16,970
This impacts the validity and reliability of the generated content.

187
00:19:16,970 --> 00:19:23,610
So just for context, bias here is different to how I was using bias previously when we were talking about large language models.

188
00:19:24,610 --> 00:19:25,870
Contextual understanding.

189
00:19:25,870 --> 00:19:32,070
So despite their advanced capabilities, LLMs can still struggle with nuanced context.

190
00:19:32,070 --> 00:19:37,530
So idioms or specific cultural references which can result in misleading content effectively.

191
00:19:37,910 --> 00:19:39,710
And then there's factual inaccuracy.

192
00:19:39,710 --> 00:19:44,150
So I'm sure if any of you tried ChatGPT out, you found that it's not always correct.

193
00:19:44,150 --> 00:19:54,150
You know, large language models don't verify the factual correctness of their output which potentially leads to generation of misleading or false information compromising the validity of the content.

194
00:19:54,210 --> 00:20:04,730
There have been cases in the news recently, I think, of a lawyer who generated some sort of legal paper using ChatGPT and didn't actually check the content of it and apparently it wasn't correct.

195
00:20:04,730 --> 00:20:07,230
So it's evidently a problem.

196
00:20:07,570 --> 00:20:10,870
Then we have the consistency issue which is...

197
00:20:10,870 --> 00:20:23,250
And if you tried asking ChatGPT the same question twice, you know, you might not get consistent answers to the same or similar queries across different instances or different even sessions within ChatGPT.

198
00:20:23,250 --> 00:20:28,850
And then, of course, if you take that and apply it to BARD or some of the others, you might get some completely different answer.

199
00:20:28,850 --> 00:20:34,970
And that's due to sort of the inherent randomness in the generation process which can affect the reliability of the information.

200
00:20:35,410 --> 00:20:44,430
So when conducting OSINT exercises, we need to consider if the information we're using has been purposefully or accidentally tainted.

201
00:20:47,830 --> 00:20:50,430
So moving into 2024.

202
00:20:50,970 --> 00:20:55,190
It doesn't look like we can escape the topic of politics, unfortunately, today.

203
00:20:55,190 --> 00:20:59,490
But with the election of 2024, there's, you know, more perils out there.

204
00:20:59,490 --> 00:21:15,630
The types of issues we need to be cognizant of include fake photos of events in candidates and fake news articles, bots using Gen-AI to create images and content to pose as individuals, LLM-generated content designed to be incendiary or spread conspiracy theories.

205
00:21:15,830 --> 00:21:18,130
You know, LLMs lower that language barrier.

206
00:21:18,130 --> 00:21:31,130
So if you're part of the internet troll farm in Russia and you want to generate content that reads like an American Reddit and not somebody, you know, who's been sitting there for 12 hours a day just plugging away on Twitter, it gives you a bit of an edge there.

207
00:21:31,830 --> 00:21:36,030
You know, we can do things like fake constituent sentiment.

208
00:21:36,150 --> 00:21:38,390
We can generate content much quicker.

209
00:21:38,570 --> 00:21:48,130
And I've added some links here for those of you who want to dig into this topic further, including another guide, an article, and then there's a paper down here as well on faking constituent sentiment.

210
00:21:49,130 --> 00:21:57,490
You know, so in summary, from an OS perspective, we can see how validating an event happened or information is accurate is going to increasingly become a sort of uphill battle.

211
00:22:00,800 --> 00:22:07,940
So talking of fake news articles, you know, ChatGPT allows bad actors the ability to generate readable content very quickly.

212
00:22:07,940 --> 00:22:13,460
This can lead to a proliferation of fake news articles being spread across the internet via social media.

213
00:22:13,700 --> 00:22:21,320
Here's a great example on this slide from The Guardian called Breaking News from CelebritiesDeaths.com.

214
00:22:21,320 --> 00:22:22,640
The president is dead.

215
00:22:22,740 --> 00:22:27,740
I'm sure we can all think of a few jokes about Joe Biden, but I promise you he's not actually dead.

216
00:22:28,600 --> 00:22:37,020
In the broader context, you know, AI-driven news is the target of a variety of companies as well looking to cash in on the space.

217
00:22:37,040 --> 00:22:44,820
So the New York Times recently reported on this, and we've got another example, you know, Google tests AI tool that is able to write news articles.

218
00:22:44,820 --> 00:22:56,860
So as well as bad actors out there writing news articles designed to sort of taint something, we also have major media companies who are using these tools to generate content evidently to cut costs, right?

219
00:22:56,860 --> 00:23:01,260
Which brings a whole set of risks if that content is incorrect.

220
00:23:02,080 --> 00:23:03,900
You know, there's some news group...

221
00:23:03,900 --> 00:23:07,640
Sorry, there's some groups out there trying to counter this.

222
00:23:07,640 --> 00:23:14,900
So NewsGuard, for example, is spearheading the effort into making AI news safe, and there's a link here which you can go read about.

223
00:23:14,900 --> 00:23:18,900
I'm sure there's plenty of others out there who are also concerned about this.

224
00:23:20,260 --> 00:23:22,440
So I thought this was a good one.

225
00:23:22,840 --> 00:23:26,020
So remember how we discussed elections and fake photos of politicians?

226
00:23:26,020 --> 00:23:27,680
Well, here we have an example.

227
00:23:28,160 --> 00:23:31,920
Imagine if this type of thing ended up in a fake news article being spread as well.

228
00:23:33,220 --> 00:23:37,540
So apparel of the generative AI and ever since, insecting is the type of thing we see here.

229
00:23:37,540 --> 00:23:43,980
And the BBC ran a good little article on this topic, which I've linked and which I borrowed the pictures from.

230
00:23:44,160 --> 00:23:51,480
They also used a couple of examples of photos that were circulating showing Trump being arrested, which is the two I've used here.

231
00:23:51,540 --> 00:23:54,120
And these and others like them, of course, are all fake.

232
00:23:54,120 --> 00:24:06,040
For the eagle-eyed on there who saw these a few weeks back, it becomes increasingly obvious they were fake as tools like Mid Journey often struggle with generating text within the image.

233
00:24:06,360 --> 00:24:10,200
So the police badges or the background signs might often give things away.

234
00:24:11,020 --> 00:24:18,880
Whatever jokes you've heard about Trump's tiny hands, this photo on the left is a dead giveaway as that tiny hand just doesn't look very real.

235
00:24:23,100 --> 00:24:26,560
So the problem of source attribution and provenance.

236
00:24:26,560 --> 00:24:31,780
So another problem we face is really where on earth did this data come from?

237
00:24:31,780 --> 00:24:42,500
As generative AI models create content based on their training data, it's difficult to attribute specific outputs to individual data sources, complicating copyright and plagiarism issues.

238
00:24:42,520 --> 00:24:44,660
We also need to consider authenticity.

239
00:24:44,660 --> 00:24:55,380
So the ability of AI to create realistic content like deep fakes determining the authenticity of a piece of content becomes challenging, increasing a risk of misinformation and fraud.

240
00:24:55,700 --> 00:24:57,180
Then we need to consider ownership.

241
00:24:57,180 --> 00:25:00,400
It's unclear who owns the rights to AI-generated content.

242
00:25:00,400 --> 00:25:04,640
The AI developers, the owners of the training data, the AI itself?

243
00:25:04,640 --> 00:25:07,360
I mean, that all leads to various legal headaches.

244
00:25:07,380 --> 00:25:21,040
And then finally, you know, considering traceability, given the large scale and complex nature of the training data, tracking the provenance, which is essentially the origin and the history of a specific AI-generated output is almost impossible, making it difficult to verify or challenge the output.

245
00:25:21,040 --> 00:25:23,000
So this has a huge impact on OSINT.

246
00:25:23,100 --> 00:25:34,520
Just on the topic of authenticity and ownership and similar, you know, it's very difficult now when students submit papers to know whether they are actually written by them or by AI.

247
00:25:34,760 --> 00:25:42,580
There's tools like Turnitin which check for plagiarism, but I'm sure in the future it's going to become increasingly difficult to know who actually wrote the paper.

248
00:25:42,580 --> 00:25:46,960
It's an interesting time to be in higher ed, if any of you out there are.

249
00:25:48,200 --> 00:25:51,380
So challenges in discerning human versus gen-AI content.

250
00:25:51,380 --> 00:25:56,560
So all of this may have got you thinking, there's going to be some serious challenges, and I think that yes, there are.

251
00:25:56,560 --> 00:25:58,140
And yes, you're right, I know about that.

252
00:25:58,140 --> 00:26:00,600
I've touched on that across the last few slides.

253
00:26:00,700 --> 00:26:09,820
One significant challenge is discerning human versus gen-AI content is the increasing ability of AI models to produce highly realistic and natural-looking photos and text.

254
00:26:09,820 --> 00:26:13,880
We saw the Trump example, but don't expect it to always be like that.

255
00:26:13,880 --> 00:26:17,540
I think you'll probably start to see all of those anomalies vanish.

256
00:26:17,940 --> 00:26:33,740
So now, couple up the sophistication of AI models, as the models become more... as they just become better over time, and they're trained on vast data sets, they gain a better understanding of human language, behaviour and artistic styles, which means we might see fewer of those tiny hand anomalies.

257
00:26:34,740 --> 00:26:37,900
This in turn leads to a technology that is continuously evolving.

258
00:26:37,900 --> 00:26:42,260
New breakthroughs in gen-AI can quickly outpace existing methods for detecting them.

259
00:26:42,460 --> 00:26:55,480
So, finally, just to hammer it home, with this proliferation of AI-generated content, there's growing concern about its potential malicious use, including spreading disinformation, generating fake news, or creating deceptive social media profiles.

260
00:26:55,920 --> 00:27:08,780
And on that social media profile, I can just say from myself and some of my team over here, we've had to deal with it in LinkedIn, where we get people saying they work for Modus, and I'm like, we don't have an office in Minneapolis, so who are you, you know?

261
00:27:09,160 --> 00:27:11,520
And then you're playing whack-a-mole constantly.

262
00:27:12,140 --> 00:27:15,400
So, yeah, I mean, it's already a problem, it's just, I think it's going to get worse.

263
00:27:17,060 --> 00:27:20,060
So, what tools are people using to generate this content?

264
00:27:20,060 --> 00:27:21,580
Let's take a look at a few of them.

265
00:27:21,580 --> 00:27:24,880
I'm sure you're probably familiar with many of them, so we'll start with images.

266
00:27:25,220 --> 00:27:28,780
So I included some graphics throughout these slides, which were generated with Midjourney.

267
00:27:28,780 --> 00:27:32,400
They were very easy to create, and Midjourney isn't the only tool on the market.

268
00:27:32,740 --> 00:27:35,540
We've got others out there, including Crayon, Dali.

269
00:27:35,540 --> 00:27:45,540
And when I wrote the talk, I'd said, you know, I think one of the 2,600 covers from last year used Crayon to generate some of the images, and I found the copy of the magazine, which is in there.

270
00:27:45,540 --> 00:27:57,500
And if you take a look at it, and I'm happy to show it after, with, like, the images now, it's just night and day how easy it is to generate very realistic images compared to what we had even, you know, two years ago, a year ago.

271
00:27:59,300 --> 00:28:01,100
So, moving on to text.

272
00:28:02,020 --> 00:28:05,060
So, you know, the infamous LMLs, right?

273
00:28:05,420 --> 00:28:13,520
ChatGBT is the obvious one, but we've got BARD, we've got LLAMA, we've got Bing Chat, and there's this paper linked on here from Schick and all discusses.

274
00:28:13,660 --> 00:28:16,960
You know, we can start thinking about how do we plug LLMs into other tools?

275
00:28:16,960 --> 00:28:19,160
So, like, APIs to pull data on the fly.

276
00:28:19,160 --> 00:28:20,020
So, think about that.

277
00:28:20,020 --> 00:28:23,800
You start to open up these tools, and they can interact with other third-party services.

278
00:28:24,020 --> 00:28:26,600
That puts us in a very interesting space.

279
00:28:29,680 --> 00:28:30,400
Sound.

280
00:28:30,980 --> 00:28:34,120
So, that could be spoken word, or it could be music.

281
00:28:34,120 --> 00:28:39,740
There's a number of tools and platforms out there, such as Soundraw, Voicebox, and Valli.

282
00:28:39,960 --> 00:28:44,240
We've already seen some amusing uses for these types of platforms.

283
00:28:44,400 --> 00:28:48,980
So, the story at the top here, that heart on my sleeve's long was completely fake.

284
00:28:48,980 --> 00:28:59,820
I think the record companies were trying to constantly take it down, but it was just an amalgamation of The Weeknd and Drake, and it sounded pretty realistic, to be honest.

285
00:29:00,260 --> 00:29:08,880
You know, and we can also do some other interesting things in this space, such as convert text to speech, speech to text, and realistically mimic existing voices.

286
00:29:11,960 --> 00:29:17,420
So, you know, the above leads us to ask what the potential is for AI-driven influence operations.

287
00:29:17,420 --> 00:29:20,060
What if we combine the outputs from all of these tools?

288
00:29:20,360 --> 00:29:35,120
So, an AI-driven influence operation, you know, has the potential to amplify disinformation campaigns on a massive scale, because it can use these generative AI models, you know, or the malicious actors can, sorry, to spread convincing fake content, including text,

289
00:29:35,120 --> 00:29:38,160
images, and videos, which can mislead and deceive the public.

290
00:29:39,260 --> 00:29:42,700
This, in turn, can lead to what we call targeted manipulation.

291
00:29:42,860 --> 00:29:55,360
We can see that AI-powered influence operations can analyze vast amounts of data about individuals, such as social media posts, browsing behavior, and personal preferences, to tailor and target and manipulate content specifically for them.

292
00:29:55,360 --> 00:30:03,620
This personalized approach increases the effectiveness of the influence campaign, making it harder for the individuals to recognize the manipulation.

293
00:30:04,140 --> 00:30:07,020
The outcome of that can be polarization and division.

294
00:30:07,020 --> 00:30:14,900
So, influence operations can exploit social and political fault lines to intensify that polarization, and so discord among different groups.

295
00:30:14,900 --> 00:30:23,040
And by strategically disseminating divisive content, AI can exacerbate existing tensions and create more fragmented society.

296
00:30:24,020 --> 00:30:27,220
And bad actors can use automated social bots to handle this.

297
00:30:27,220 --> 00:30:35,060
So, AI can be used to create and deploy automated social bots that can engage with real users and spread propaganda, fake news, misleading information.

298
00:30:36,160 --> 00:30:41,260
Whatever you can think of in theory, they can probably do it if it's text-based and it's a one-on-one interaction.

299
00:30:41,600 --> 00:30:50,020
These AI-driven bots can make content appear to be more popular or generate artificial trends, manipulating public perception and influencing public discourse.

300
00:30:50,220 --> 00:30:55,740
So, I think I've beaten this horse enough about bots and similar, but how do we mitigate this?

301
00:31:01,240 --> 00:31:04,520
So, the first thing we can look at doing is training and workshops.

302
00:31:04,520 --> 00:31:12,740
So, conducting regular training workshops and webinars specifically tailored towards open-source intelligence researchers can help enhance their AI literacy.

303
00:31:12,820 --> 00:31:22,820
These workshops can cover fundamentals of AI, explain how AI is used in generating content, and demonstrate techniques to identify AI content.

304
00:31:22,840 --> 00:31:33,300
And since this field is moving so quickly, by the time I've given this talk, it's probably already obsolete, it's important that we stay on top of this and that there's a continuous training process.

305
00:31:34,300 --> 00:31:41,180
Conferences and venues such as the Recon Village at DEF CON are great venues to share this information with the OSINT research community.

306
00:31:41,240 --> 00:31:50,940
We're still in very early days, so I'm sure we're going to learn a lot over the next year and by the time the Recon Village comes back to DEF CON next year, the talk will be even more informative.

307
00:31:52,000 --> 00:31:54,240
We can collaborate with AI experts.

308
00:31:54,240 --> 00:32:05,420
So , facilitating collaboration between OSINT researchers and AI experts can foster a deeper understanding of AI technologies and their potential applications in content generation.

309
00:32:05,420 --> 00:32:13,600
Regular interactions with AI specialists can help researchers stay informed about the latest advancements in AI and how they may impact OSINT investigations.

310
00:32:13,800 --> 00:32:27,060
And this kind of reminds me of the opening talk at the AppSec Village, where essentially one of the main points of it was about cross-team collaboration and expertise, that sort of cross-disciplinary approach to things, and I think we're going to have to adopt that in this space,

311
00:32:27,060 --> 00:32:31,160
by working closely with those really smart folks in the AI space.

312
00:32:33,220 --> 00:32:40,860
Another area, you know, which is theoretically possible, and a topic everyone loves, is legislation regulations.

313
00:32:41,160 --> 00:32:47,760
You know, I think we can expect to see calls for transparency and regulations for gen AI usage in social media.

314
00:32:48,300 --> 00:32:49,060
Will this work?

315
00:32:49,060 --> 00:32:50,320
I don't know.

316
00:32:50,320 --> 00:32:51,860
I couldn't tell you.

317
00:32:53,000 --> 00:32:56,220
Blumenthal and Hawley, the Senators, think so.

318
00:32:56,220 --> 00:33:03,560
The No Section 230 Immunity for AI Act calls to remove most immunity under the Communications Decency Act.

319
00:33:03,560 --> 00:33:14,640
So Section 230, if you're not familiar, of the CDA of 1996 provides broad protection to what they call interactive computer services, which really does sound a phrase straight out of the 90s.

320
00:33:14,640 --> 00:33:26,280
And by holding that they can't be treated as the publisher or speaker of information provided by users, so effectively that immunity would say that AI, you know, doesn't...

321
00:33:27,020 --> 00:33:31,340
It loses that immunity effectively if you're using it to generate content.

322
00:33:33,400 --> 00:33:35,560
So validation and tooling.

323
00:33:36,320 --> 00:33:39,820
Can we use tooling, including AI, to detect AI?

324
00:33:39,900 --> 00:33:44,980
This may be one avenue that OSN researchers can use to filter the signal from the noise.

325
00:33:44,980 --> 00:33:46,500
So a few ideas here.

326
00:33:46,500 --> 00:33:55,900
So improve access to AI tools, so providing OSN researchers with access to AI analysis tools can enable you to better understand the capabilities and limitations of AI in generating content.

327
00:33:56,520 --> 00:34:06,020
And by experimenting with AI tools themselves, researchers can gain first-hand experience in discerning human versus AI-created content.

328
00:34:06,240 --> 00:34:08,420
Then we have detection techniques.

329
00:34:08,420 --> 00:34:13,360
So employing AI tools to detect and flag AI-generated content.

330
00:34:13,360 --> 00:34:20,500
I've seen some attempts here, and I can imagine in the higher ed space, as I mentioned before, in stopping plagiarism, it could be useful.

331
00:34:20,500 --> 00:34:22,300
Then we have validation.

332
00:34:22,300 --> 00:34:26,220
So developing more robust data validation and verification techniques.

333
00:34:26,600 --> 00:34:27,520
Watermarks.

334
00:34:27,520 --> 00:34:32,640
So digitally, cryptographically watermark content so it can be verified against a third-party list.

335
00:34:32,640 --> 00:34:36,700
This could be a good use for the blockchain-style technologies, for example.

336
00:34:37,920 --> 00:34:45,260
And then I think when it comes to other approaches, we're probably going to learn them as we go.

337
00:34:45,860 --> 00:34:51,460
So that's some ideas on how we tackle the issues, and people much smarter than me are definitely on this case.

338
00:34:51,460 --> 00:34:54,340
So let's take a look at the future.

339
00:34:59,720 --> 00:35:15,260
So unless the technology hits some sort of wall or barrier, which is certainly possible, I guess, then I just think the performance will continue to grow, at least to a point where the problems it solves will be very difficult for us to tell real content from fake content.

340
00:35:15,940 --> 00:35:25,400
We shouldn't be alarmists in space, and I know some of the debates are quite hot right now about AI alignment and similar, but I definitely don't think we can be complacent either.

341
00:35:25,400 --> 00:35:26,760
We sort of want to be empirical about this.

342
00:35:26,760 --> 00:35:27,720
We've got to collect our data.

343
00:35:27,720 --> 00:35:38,300
We've got to understand where the trends are and then work and operate against them as opposed to sort of imagining where things might go in a very negative fashion.

344
00:35:38,300 --> 00:35:54,440
I think, yeah, being empirical is important, but, you know, coupled with the commercial efforts by Meta, Google, OpenAI, and similar, I just think we're going to see a proliferation of open source and free LLMs, commercial options, so it's very quickly sort of the technology's out of the box,

345
00:35:54,440 --> 00:35:55,300
effectively.

346
00:35:55,820 --> 00:36:02,640
From nowhere, potential trends we should consider, therefore, that's realistic fake video generated on the fly.

347
00:36:02,640 --> 00:36:12,000
So right now, images are a very popular sound, but I think video is certainly going to be something that we can expect to see increasingly appear.

348
00:36:12,140 --> 00:36:20,840
Improvements in mimicking human voices, increasing difficulties in detecting fake images, and the weird artifacts I talked about, sort of uncanny hands and ears.

349
00:36:21,340 --> 00:36:26,640
We should also consider how OSINT will adopt techniques to improve gathering and analyzing data.

350
00:36:26,980 --> 00:36:31,160
And there's an open source library, which I've linked to here, which, you know, is worth checking out.

351
00:36:31,380 --> 00:36:37,360
There's tools for handling ETL processes, for consuming large amounts of data and doing ETL, and there's a link here.

352
00:36:37,480 --> 00:36:45,200
And I think processing samples of text to find similarities with other text online, so you can detect the author, I think that's going to be an area.

353
00:36:45,220 --> 00:36:49,080
And there's already tools out there, but I think they're going to become more sophisticated.

354
00:36:50,400 --> 00:36:55,300
And there's lots of opportunities for building OSINT tools on top of Gen AI models.

355
00:36:55,300 --> 00:37:00,660
We just have to collectively think about how we would approach doing that and build it.

356
00:37:03,040 --> 00:37:09,360
So I'm going to wrap up, because I know we're almost at time, and it's Saturday night at DEF CON, so I'm sure everyone has party plans.

357
00:37:10,880 --> 00:37:13,740
So in conclusion, I hope you enjoyed the talk.

358
00:37:13,740 --> 00:37:16,640
Let's do just a quick, brief recap.

359
00:37:17,260 --> 00:37:23,180
Gen AI is here to stay for the foreseeable future, and it looks like it will continue to improve.

360
00:37:23,400 --> 00:37:26,040
OSINT practitioners will need to adapt to this world.

361
00:37:26,060 --> 00:37:29,040
We're going to need better tools to detect fake content.

362
00:37:29,320 --> 00:37:34,840
Creators of original, genuine content will need to find ways to better distinguish their material from fake content.

363
00:37:34,940 --> 00:37:39,520
And the OSINT world can leverage Gen AI tools as well.

364
00:37:39,680 --> 00:37:43,500
And here's a nice quote from William Gibson's Johnny Mnemonic.

365
00:37:43,740 --> 00:37:51,140
It's impossible to move, to live, to operate at any level without leaving traces, bits, seemingly meaningless fragments of personal information.

366
00:37:51,180 --> 00:37:55,460
So in the new world of Gen AI, we're going to need to separate the winter meats from the humans.

367
00:37:56,600 --> 00:37:58,660
And with that, I'd like to say thank you.

368
00:37:58,660 --> 00:38:03,120
And finally, could we all give the guys from the Recon Village a round of applause as well?

369
00:38:03,120 --> 00:38:04,080
Thank you.