Right, so along with me, for the last talk, we have Andy Dennis.
Andy is a VP of consulting at Modus Create.
He has over 20 years of experience in security, software engineering,
DevOps, consulting, system migration, and modernization.
Andy is a published author and online tutor
for Goldsmiths University of London.
Andy has also spoken at and also helped organize B-Sides events in New England.
And he has also spoken in multiple other conferences such as B-Sides CT,
B-Sides Orlando, and Defcon Village in the previous editions as well.
The topic Andy will be covering is the Perils of Generative AI
Implications for Open Source Intelligence Research.
So please give it up for Andy.
Also, a little request that if you could cover up the seats ahead
so that it would be better because...
I mean, we have a lot of seats that are empty in between,
so if you could cover those up.
And please turn your phones into silence
so that the speaker isn't disturbed in between.
And have a good time.
So they just did the introduction for me,
so this slide's kind of redundant.
But I will tell you, we are recruiting,
so go to this website if you're a Java engineer,
if you work in DevOps.
If you're really good with GitHub and GitHub Actions,
we're definitely looking for folks right now,
so please do take a look.
And yeah, my previous talk, if you're interested,
was on using passive DNS for gathering business intelligence,
and that was from 2021.
So feel free to check that out.
So then, let's get started, I guess.
So we're going to start with a brief explanation
of generative AI and open-source intelligence.
Sorry, I'm just going to move this mic so it's a little easier.
So for those of you new to the topic,
I'm going to give a brief overview of what this is
and also a little background on OSINT.
I've included a timeline on the next few slides,
which just gives a brief overview of some of the major milestones
in the AI space.
We won't be going into deep mathematical details here,
so you don't have to run for the exit yet.
But in brief, generative AI is a category of artificial intelligence
that's capable of generating new content,
such as text, images, and audios.
And I'm sure all of you are familiar with that.
Many of you will have kicked the tires on tools
such as ChatGBT and MidJourney.
This approach is built upon years of research and development
in the AI space and is really coming to fruition
at this point in time.
Let's see what ChatGBT says when we ask it to summarize
what generative AI is.
I have gone too many slides forwards by the looks of it.
There we go. Sorry.
So generative AI is a subset of artificial intelligence
that utilizes machine learning models to create new data samples
that mimic the properties of some source data set.
This could involve generating anything from written text, images, music,
or even video content.
This is achieved through algorithms such as GANs,
where two neural networks, a generator and a discriminator,
are trained in tandem.
The generator aims to produce realistic data,
while the discriminator's job is to distinguish
between real and generated data.
The iterative competition between the two
leads to the production of increasingly accurate
and realistic synthetic data.
Other popular generators include VAEs and transformer-based models
like GPT-3 and GPT-4.
So, hopefully that's nice and clear now what that means.
Basically, these tools and models are trained on large data sets
and apply machine learning techniques to generate new content
such as image from a text prompt
or poem in the style of, say, a long-dead author.
This ability to mimic styles and generate novel content
allows users to create realistic images,
compose music, and increasingly explore problem spaces
in various scientific fields.
Despite all the fun and serious use of the technology,
concerns around ethics and the ability for gross misuse
have been a key concern.
I'm sure you've been following anything in the news recently.
You know, AI alignment and ethics is a major topic.
Fraudulent data, such as deepfakes,
will become increasingly easy to generate
for anyone with a laptop.
And separating the real video from the AI-generated one
will get harder over time.
That's a very, very basic overview of Gen-AI.
To sum it up, it's a way of taking large amounts of data
and training software to be able to create
a new, compelling content based off of this.
And later in the talk, we're going to pull some of this apart
and delve into it in more detail.
So, we did a quick overview of Gen-AI,
but what about OSINT?
So, OSINT stands for Open Source Intelligence.
It's a set of methods, techniques, and tools
to collect information from publicly available sources,
such as blogs, social media, data sets, and reports, and so on.
It's not just hackers using this.
Journalists use it, business intelligence analysts,
law enforcement, cyber security professionals,
and a bunch of other folks use it for a variety of purposes.
This data can be collected through a range of techniques,
such as web scraping tools, natural language processing of data,
machine learning.
The Recon Village website includes some links to these various tools,
and a bunch of presentations from over the years.
And on this slide and next, you can see some of the various products
and projects and open source GitHub repos that contribute to this space.
And so, please go check them out.
And I think maybe Amass was already covered earlier today in one of the talks.
So, now a key aspect of OSINT is the analysis of the data we've collected
with the various tools.
Once we have the data, we can use it to study trends,
draw conclusions, and make decisions.
This is incredibly useful, of course.
For example, if we're trying to trace an individual
or discover some information about a competitor.
But despite this usefulness, we need to be cognizant of privacy concerns
and ensure we are operating in an ethical fashion.
A key problem arises, though, when considering using OSINT
to inform decisions and drive actions.
What if the data is fake?
What if it was generated by AI methods, like I previously mentioned?
So, a bit more background. Let's dig a little deeper.
So, now we're going to dive into JANA and OSINT a little bit more,
but we just did a speed run through the basics there.
So, let's cover some of the techniques used by large language models
and other generative platforms that were hinted at
in the chat GPT's definition previously.
So, key machine learning techniques listed on these slides
include GANs, VAEs, and transformer models.
Chat GPT previously explained what a GAN was.
A VAE is a variational autoencoder, and it's a form of a neural network.
They use probabilistic models.
So, a VAE is a type of generative model that uses a variational interface
to encode and decode data.
They represent the data in a probabilistic latent space,
enabling them to generate new data that's similar to the training data.
And then we get to transformers.
So, transformers, which is a type of technology used by chat GPT,
are also a neural network-based approach that underpins large language models.
You may have heard the term transformers a lot recently due to LLM's popularity.
So, that's kind of just a basic description of it there.
And you can essentially think of them as taking one input and transforming it into another.
They're a mechanism for learning context through tracking relationships and data
without needing recurrent layers, enabling parallel processing.
A couple of other phrases you may have heard are weights and biases.
So, in a neural network, weights are the parameters that are learned during the training
to scale the input of data and connections between the neurons in the neural network
and determining the strength of the signal passed through the network.
And biases are additional parameters in a neural network that allow for the shifting,
shifting the activation function to the left or right,
helping the network to fit the data more flexibly.
These models use a variety of approaches, such as games and competition,
which help to improve constantly the quality of the output.
As each technique gets better, so do the outputs.
So, effectively, as each of these techniques get better, the images are going to get better,
the text output is going to get better, the video output is going to get better.
You really don't have to understand all the math here.
There are people much smarter than me who I work with who really get that side of things,
and that's not my background as a mathematician.
But having a general understanding of the techniques is useful in understanding space.
So, as a result, AI researchers have been able to improve the outputs
to start mimicking real-life photography, sound like the voice of a living individual,
or create text that reads like a famous author.
And we'll be seeing a few examples of these later in the talk.
So, a little bit on the pace of improvements between 2022 and 2023.
As I just started when discussing the previous slide,
as the outputs get better, they get closer to mimicking the real, effectively.
And what's driving us closer in on the simulacrum being indistinguishable from the real
is the pace of technological change.
So, this diagram shows that between 2020 and 2023, we went from GPT 3.0 to GPT 3.4.
The change in quality of the outputs in this period took leaps and bounds.
We saw this on the image side of things too,
an explosion of vendors releasing new products and new open source projects
and code that might have been leaked and shared.
And it's all in the image space, the text space, the data processing space, the sound space.
So, it's really quite across the spectrum.
And I think this is going to become obviously important in relation to OSINT,
as some examples that are coming up will demonstrate.
So, open source intelligence and its importance to security, the security space.
So, that was a little bit more about some of the techniques that are used by Gen AI
and the pace of change around these.
But let's switch back to discuss open sources intelligence
and the importance in the security space.
And as we can see here, there are four key areas that come to mind.
So, we have threat intelligence, which is basically detecting cyber threats and patterns
and using that information to harden systems, processes, and policies that drive that.
Incident response, so when something bad happens, we can use OSINT
to understand the nature of the attack and the potential culprits.
And the security awareness size, that's basically monitoring OSINT sources,
which allows us to get ahead of events.
So, understand threat actors.
And then digital forensics, OSINT can add another tool to our kit
when we're digging into a breach,
building on what we may have discovered during an incident response event.
So, a major source of OSINT in these cases are social media platforms,
forums, and other venues online where people interact.
So, social media, as I'm sure many of you are aware,
is a goldmine when it comes to OSINT.
I've included a few examples of the role it plays on this slide.
Using social media, we can gather multiple types of information
from blog posts, Facebook postings, photos, tweets, documents, and videos.
With this information, we can do some interesting things,
such as real-time monitoring and alerting.
A good example is the current war in Ukraine, for example,
where researchers have used OSINT, including geotagging,
to calculate Russian battlefield losses
and the locations where these battlefield losses have happened.
Of course, we can also use information from social media for verification and fact-checking.
For example, by cross-referencing information from multiple sources,
we can help to verify if an event took place,
such as, was that person at that particular place at that point in time?
I hope these few examples here give you an idea of how social media is an important component of OSINT.
Obviously, this isn't exhaustive, it's just a quick overview.
But let's now consider the intersection of this with generative AI.
In this section of the talk, we're now going to look at some problem areas
where OSINT is impacted by various types of AI.
These include fake profiles on social media,
misinformation spread through bot farms,
election interference potential,
and the impact on tracing missing people.
The first we'll discuss is bots.
Bots and troll farms were a huge controversy in the 2016 election,
as I'm sure many of you will remember.
Researchers uncovered thousands of fake accounts on Twitter.
Many were easy to detect,
and a number of OSINT researchers did a very good job at identifying these bot farms,
and some of them were obviously brought down as well.
The issues, though...
These issues, sorry, weren't put to rest in the previous decade.
The paper linked on this slide is from NIH, the National Institute of Health,
and here's some of the key pointers that came out of that.
The pandemic saw activity with bots spreading misinformation around COVID-19.
About a third, so 33%, of the top sharers of content
from low-credibility sources were likely due to bot activity.
And this key point, you know,
bots are becoming increasingly difficult to separate from human users,
so detecting them will need a higher level of precision as we move forwards.
So, between ChatGBT and its variants,
API access given to large language models and similar generative AI tools,
and the ability to generate an almost infinite amount of fake photos, names,
histories via gen-AI techniques,
you know, raiding through this to detect a bot from the troll,
from the genuine account, is going to become increasingly difficult.
So, let's take a look at an example of a generative AI
used in the social media space to demonstrate this.
So, LinkedIn is a prime target for running bots and fake accounts.
Why? Well, it's used by all levels of a company's team,
from the CEO down to the engineers.
Individuals working for sensitive companies, the military,
or have access to intellectual property are present on these platforms.
Therefore, it's just a goldmine for a hostile actor to target individuals
who may have access to something they want.
NPR wrote a very accessible article on this topic
and provided a great graphic, which I've kindly borrowed up here.
For now, at least, you can spot a generative AI.
So, if you look at this photo for some oddities,
you can see that, for example, the earring's missing,
there's a weird-looking earlobe,
maybe the hair merges slightly into the background here.
In this case, we can't see the hands,
but that's often an area where gen AI has struggled with.
I've also seen images that look incredibly realistic,
apart from having, say, too many teeth.
Hopefully, in the future, we'll be able to use AI tools to detect these anomalies,
although we're likely to enter into a bit of an arms race here,
with hostile actors trying to outwit detection tools,
much like we've seen in the antivirus space.
So, we've seen there's an issue with the proliferation of bots,
and then a problem with fake social media accounts posing as real individuals.
This obviously has some concerns when it comes to the election space as well.
So, you know, looking back at recent history,
there was concerns around bots, fake ads,
and similar with the UK's referendum on leaving the EU.
Moving forward seven years, with the tools accessible today,
it becomes much easier to generate vast amounts of realistic-looking accounts,
fake videos, fraudulent photographs,
in order to try and impact the outcome of an election.
The Guardian newspaper recently published an article
where they interviewed René Durest of the Stanford Internet Observatory,
and this quote stands out, you know,
text and images, however, have the potential for higher actual impact in an election scenario
because they can be more subtle and longer-lasting.
So, I'd recommend reading that article if you're interested in this topic.
It's a good overview.
So, back in 2016 through 2019 period,
I was interested in this topic,
although the technologies weren't as sophisticated or as accessible as they are now,
and while at grad school I'd proposed the idea of sort of the equivalent to a CIA triad
that maps the attack surface for an election,
and this little diagram here shows us.
Looking back at this from 2023, number five in this list,
which is the information-based attacks,
you know, they're especially vulnerable to the intersection of Gen AI and social media
in the context of an election.
We can imagine all sorts of examples where a politician from either party
is accused of doing something,
and there's apparent fake footage of this event,
and then bots spread this information at astonishing speed due to network effects.
So, as we move towards the 2024 presidential election,
you know, we're going to have to be on our guard from being manipulated.
So, I mentioned missing persons previously,
so let's move on to a topic of missing persons.
I know this is close to the heart of many of the people in the audience,
and the Recon Village has hosted a number of talks on this subject.
OSINT is used in a variety of fashions to help track down people who are missing or being trafficked.
Trace Labs are perhaps the most famous in this space,
and are a non-profit that helps to organize online search parties to trace individuals.
They even have a VM packed with tools to help in this space,
and perhaps soon they'll have some generative AI tools as well to help.
But as I alluded to a moment ago,
there's a lot of Recon Village participants who are also active in this space,
and Ray Baker's video on using OSINT to aid in human trafficking cases is well worth watching,
just to sort of get an idea of what people are trying to do
using open source intelligence to effectively save people's lives.
However, we have to now consider the new world they're operating in.
What happens when bad actors can generate thousands of fake images to cover their tracks?
How do we sift through these to find the real images from the fake?
What about bad actors generating thousands of fake abuse images and showing them?
How do we detect the person is real?
So I think there was somebody recently, if I remember correctly from the news,
who created lots of fake child porn images and was arrested for that.
Now if you can imagine someone doing that, they can flood the market.
They can effectively poison the well.
It makes it very difficult for the FBI or others to then track down,
is that a real individual versus just a horrifically nasty image that someone's generating.
So some real perils here, with people poisoning that well to cover up crimes.
And what else do we need to be concerned about then in that space?
So issues of data validity and reliability with large language model generated text content.
This is probably one of the first perils we need to consider.
It's the data validity and reliability.
And we have four key concerns here.
So the first is bias.
So the training data used in large language models can include biased information
leading to outputs that reflect these biases.
This impacts the validity and reliability of the generated content.
So just for context, bias here is different to how I was using bias previously
when we were talking about large language models.
Contextual understanding.
So despite their advanced capabilities, LLMs can still struggle with nuanced context.
So idioms or specific cultural references which can result in misleading content effectively.
And then there's factual inaccuracy.
So I'm sure if any of you tried ChatGPT out, you found that it's not always correct.
You know, large language models don't verify the factual correctness of their output
which potentially leads to generation of misleading or false information
compromising the validity of the content.
There have been cases in the news recently, I think, of a lawyer who generated
some sort of legal paper using ChatGPT and didn't actually check the content of it
and apparently it wasn't correct.
So it's evidently a problem.
Then we have the consistency issue which is...
And if you tried asking ChatGPT the same question twice, you know,
you might not get consistent answers to the same or similar queries across different instances
or different even sessions within ChatGPT.
And then, of course, if you take that and apply it to BARD or some of the others,
you might get some completely different answer.
And that's due to sort of the inherent randomness in the generation process
which can affect the reliability of the information.
So when conducting OSINT exercises, we need to consider if the information we're using
has been purposefully or accidentally tainted.
So moving into 2024.
It doesn't look like we can escape the topic of politics, unfortunately, today.
But with the election of 2024, there's, you know, more perils out there.
The types of issues we need to be cognizant of include
fake photos of events in candidates and fake news articles,
bots using Gen-AI to create images and content to pose as individuals,
LLM-generated content designed to be incendiary or spread conspiracy theories.
You know, LLMs lower that language barrier.
So if you're part of the internet troll farm in Russia and you want to generate content
that reads like an American Reddit and not somebody, you know,
who's been sitting there for 12 hours a day just plugging away on Twitter,
it gives you a bit of an edge there.
You know, we can do things like fake constituent sentiment.
We can generate content much quicker.
And I've added some links here for those of you who want to dig into this topic further,
including another guide, an article, and then there's a paper down here as well
on faking constituent sentiment.
You know, so in summary, from an OS perspective,
we can see how validating an event happened or information is accurate
is going to increasingly become a sort of uphill battle.
So talking of fake news articles, you know,
ChatGPT allows bad actors the ability to generate readable content very quickly.
This can lead to a proliferation of fake news articles
being spread across the internet via social media.
Here's a great example on this slide from The Guardian
called Breaking News from CelebritiesDeaths.com.
The president is dead.
I'm sure we can all think of a few jokes about Joe Biden,
but I promise you he's not actually dead.
In the broader context, you know, AI-driven news
is the target of a variety of companies as well looking to cash in on the space.
So the New York Times recently reported on this,
and we've got another example, you know,
Google tests AI tool that is able to write news articles.
So as well as bad actors out there writing news articles
designed to sort of taint something,
we also have major media companies who are using these tools
to generate content evidently to cut costs, right?
Which brings a whole set of risks if that content is incorrect.
You know, there's some news group...
Sorry, there's some groups out there trying to counter this.
So NewsGuard, for example, is spearheading the effort
into making AI news safe,
and there's a link here which you can go read about.
I'm sure there's plenty of others out there who are also concerned about this.
So I thought this was a good one.
So remember how we discussed elections and fake photos of politicians?
Well, here we have an example.
Imagine if this type of thing ended up in a fake news article being spread as well.
So apparel of the generative AI and ever since,
insecting is the type of thing we see here.
And the BBC ran a good little article on this topic,
which I've linked and which I borrowed the pictures from.
They also used a couple of examples of photos that were circulating
showing Trump being arrested, which is the two I've used here.
And these and others like them, of course, are all fake.
For the eagle-eyed on there who saw these a few weeks back,
it becomes increasingly obvious they were fake
as tools like Mid Journey often struggle with generating text within the image.
So the police badges or the background signs might often give things away.
Whatever jokes you've heard about Trump's tiny hands,
this photo on the left is a dead giveaway
as that tiny hand just doesn't look very real.
So the problem of source attribution and provenance.
So another problem we face is really where on earth did this data come from?
As generative AI models create content based on their training data,
it's difficult to attribute specific outputs to individual data sources,
complicating copyright and plagiarism issues.
We also need to consider authenticity.
So the ability of AI to create realistic content like deep fakes
determining the authenticity of a piece of content becomes challenging,
increasing a risk of misinformation and fraud.
Then we need to consider ownership.
It's unclear who owns the rights to AI-generated content.
The AI developers, the owners of the training data, the AI itself?
I mean, that all leads to various legal headaches.
And then finally, you know, considering traceability,
given the large scale and complex nature of the training data,
tracking the provenance, which is essentially the origin and the history
of a specific AI-generated output is almost impossible,
making it difficult to verify or challenge the output.
So this has a huge impact on OSINT.
Just on the topic of authenticity and ownership and similar,
you know, it's very difficult now when students submit papers
to know whether they are actually written by them or by AI.
There's tools like Turnitin which check for plagiarism,
but I'm sure in the future it's going to become increasingly difficult
to know who actually wrote the paper.
It's an interesting time to be in higher ed, if any of you out there are.
So challenges in discerning human versus gen-AI content.
So all of this may have got you thinking,
there's going to be some serious challenges, and I think that yes, there are.
And yes, you're right, I know about that.
I've touched on that across the last few slides.
One significant challenge is discerning human versus gen-AI content
is the increasing ability of AI models to produce highly realistic
and natural-looking photos and text.
We saw the Trump example, but don't expect it to always be like that.
I think you'll probably start to see all of those anomalies vanish.
So now, couple up the sophistication of AI models,
as the models become more... as they just become better over time,
and they're trained on vast data sets, they gain a better understanding
of human language, behaviour and artistic styles,
which means we might see fewer of those tiny hand anomalies.
This in turn leads to a technology that is continuously evolving.
New breakthroughs in gen-AI can quickly outpace existing methods for detecting them.
So, finally, just to hammer it home,
with this proliferation of AI-generated content,
there's growing concern about its potential malicious use,
including spreading disinformation, generating fake news,
or creating deceptive social media profiles.
And on that social media profile, I can just say from myself
and some of my team over here, we've had to deal with it in LinkedIn,
where we get people saying they work for Modus,
and I'm like, we don't have an office in Minneapolis,
so who are you, you know?
And then you're playing whack-a-mole constantly.
So, yeah, I mean, it's already a problem,
it's just, I think it's going to get worse.
So, what tools are people using to generate this content?
Let's take a look at a few of them.
I'm sure you're probably familiar with many of them,
so we'll start with images.
So I included some graphics throughout these slides,
which were generated with Midjourney.
They were very easy to create,
and Midjourney isn't the only tool on the market.
We've got others out there, including Crayon, Dali.
And when I wrote the talk, I'd said, you know,
I think one of the 2,600 covers from last year
used Crayon to generate some of the images,
and I found the copy of the magazine, which is in there.
And if you take a look at it, and I'm happy to show it after,
with, like, the images now, it's just night and day
how easy it is to generate very realistic images
compared to what we had even, you know, two years ago, a year ago.
So, moving on to text.
So, you know, the infamous LMLs, right?
ChatGBT is the obvious one, but we've got BARD, we've got LLAMA,
we've got Bing Chat, and there's this paper linked on here
from Schick and all discusses.
You know, we can start thinking about how do we plug LLMs into other tools?
So, like, APIs to pull data on the fly.
So, think about that.
You start to open up these tools,
and they can interact with other third-party services.
That puts us in a very interesting space.
Sound.
So, that could be spoken word, or it could be music.
There's a number of tools and platforms out there,
such as Soundraw, Voicebox, and Valli.
We've already seen some amusing uses for these types of platforms.
So, the story at the top here,
that heart on my sleeve's long was completely fake.
I think the record companies were trying to constantly take it down,
but it was just an amalgamation of The Weeknd and Drake,
and it sounded pretty realistic, to be honest.
You know, and we can also do some other interesting things in this space,
such as convert text to speech, speech to text,
and realistically mimic existing voices.
So, you know, the above leads us to ask
what the potential is for AI-driven influence operations.
What if we combine the outputs from all of these tools?
So, an AI-driven influence operation, you know,
has the potential to amplify disinformation campaigns on a massive scale,
because it can use these generative AI models,
you know, or the malicious actors can, sorry,
to spread convincing fake content, including text, images, and videos,
which can mislead and deceive the public.
This, in turn, can lead to what we call targeted manipulation.
We can see that AI-powered influence operations can analyze
vast amounts of data about individuals, such as social media posts,
browsing behavior, and personal preferences,
to tailor and target and manipulate content specifically for them.
This personalized approach increases the effectiveness
of the influence campaign, making it harder for the individuals
to recognize the manipulation.
The outcome of that can be polarization and division.
So, influence operations can exploit social and political fault lines
to intensify that polarization, and so discord among different groups.
And by strategically disseminating divisive content,
AI can exacerbate existing tensions and create more fragmented society.
And bad actors can use automated social bots to handle this.
So, AI can be used to create and deploy automated social bots
that can engage with real users and spread propaganda,
fake news, misleading information.
Whatever you can think of in theory, they can probably do it
if it's text-based and it's a one-on-one interaction.
These AI-driven bots can make content appear to be more popular
or generate artificial trends, manipulating public perception
and influencing public discourse.
So, I think I've beaten this horse enough about bots and similar,
but how do we mitigate this?
So, the first thing we can look at doing is training and workshops.
So, conducting regular training workshops and webinars
specifically tailored towards open-source intelligence researchers
can help enhance their AI literacy.
These workshops can cover fundamentals of AI,
explain how AI is used in generating content,
and demonstrate techniques to identify AI content.
And since this field is moving so quickly,
by the time I've given this talk, it's probably already obsolete,
it's important that we stay on top of this
and that there's a continuous training process.
Conferences and venues such as the Recon Village at DEF CON
are great venues to share this information
with the OSINT research community.
We're still in very early days,
so I'm sure we're going to learn a lot over the next year
and by the time the Recon Village comes back to DEF CON next year,
the talk will be even more informative.
We can collaborate with AI experts.
So, facilitating collaboration between OSINT researchers
and AI experts can foster a deeper understanding
of AI technologies and their potential applications
in content generation.
Regular interactions with AI specialists
can help researchers stay informed about the latest advancements in AI
and how they may impact OSINT investigations.
And this kind of reminds me of the opening talk
at the AppSec Village,
where essentially one of the main points of it
was about cross-team collaboration and expertise,
that sort of cross-disciplinary approach to things,
and I think we're going to have to adopt that in this space,
by working closely with those really smart folks in the AI space.
Another area, you know, which is theoretically possible,
and a topic everyone loves, is legislation regulations.
You know, I think we can expect to see calls for transparency
and regulations for gen AI usage in social media.
Will this work? I don't know. I couldn't tell you.
Blumenthal and Hawley, the Senators, think so.
The No Section 230 Immunity for AI Act
calls to remove most immunity under the Communications Decency Act.
So Section 230, if you're not familiar, of the CDA of 1996
provides broad protection to what they call
interactive computer services,
which really does sound a phrase straight out of the 90s.
And by holding that they can't be treated as the publisher
or speaker of information provided by users,
so effectively that immunity would say that
AI, you know, doesn't...
It loses that immunity effectively
if you're using it to generate content.
So validation and tooling.
Can we use tooling, including AI, to detect AI?
This may be one avenue that OSN researchers can use
to filter the signal from the noise.
So a few ideas here.
So improve access to AI tools,
so providing OSN researchers with access to AI analysis tools
can enable you to better understand the capabilities
and limitations of AI in generating content.
And by experimenting with AI tools themselves,
researchers can gain first-hand experience
in discerning human versus AI-created content.
Then we have detection techniques.
So employing AI tools to detect and flag AI-generated content.
I've seen some attempts here,
and I can imagine in the higher ed space,
as I mentioned before, in stopping plagiarism,
it could be useful.
Then we have validation.
So developing more robust data validation
and verification techniques.
Watermarks.
So digitally, cryptographically watermark content
so it can be verified against a third-party list.
This could be a good use for the blockchain-style technologies,
for example.
And then I think when it comes to other approaches,
we're probably going to learn them as we go.
So that's some ideas on how we tackle the issues,
and people much smarter than me are definitely on this case.
So let's take a look at the future.
So unless the technology hits some sort of wall or barrier,
which is certainly possible, I guess,
then I just think the performance will continue to grow,
at least to a point where the problems it solves
will be very difficult for us to tell real content from fake content.
We shouldn't be alarmists in space,
and I know some of the debates are quite hot right now
about AI alignment and similar,
but I definitely don't think we can be complacent either.
We sort of want to be empirical about this.
We've got to collect our data.
We've got to understand where the trends are
and then work and operate against them
as opposed to sort of imagining
where things might go in a very negative fashion.
I think, yeah, being empirical is important,
but, you know, coupled with the commercial efforts by Meta,
Google, OpenAI, and similar,
I just think we're going to see a proliferation of open source
and free LLMs, commercial options,
so it's very quickly sort of the technology's out of the box, effectively.
From nowhere, potential trends we should consider, therefore,
that's realistic fake video generated on the fly.
So right now, images are a very popular sound,
but I think video is certainly going to be something
that we can expect to see increasingly appear.
Improvements in mimicking human voices,
increasing difficulties in detecting fake images,
and the weird artifacts I talked about,
sort of uncanny hands and ears.
We should also consider how OSINT will adopt techniques
to improve gathering and analyzing data.
And there's an open source library, which I've linked to here,
which, you know, is worth checking out.
There's tools for handling ETL processes,
for consuming large amounts of data and doing ETL,
and there's a link here.
And I think processing samples of text
to find similarities with other text online,
so you can detect the author, I think that's going to be an area.
And there's already tools out there,
but I think they're going to become more sophisticated.
And there's lots of opportunities for building OSINT tools
on top of Gen AI models.
We just have to collectively think about
how we would approach doing that and build it.
So I'm going to wrap up, because I know we're almost at time,
and it's Saturday night at DEF CON,
so I'm sure everyone has party plans.
So in conclusion, I hope you enjoyed the talk.
Let's do just a quick, brief recap.
Gen AI is here to stay for the foreseeable future,
and it looks like it will continue to improve.
OSINT practitioners will need to adapt to this world.
We're going to need better tools to detect fake content.
Creators of original, genuine content
will need to find ways to better distinguish their material
from fake content.
And the OSINT world can leverage Gen AI tools as well.
And here's a nice quote from William Gibson's
Johnny Mnemonic.
It's impossible to move, to live, to operate at any level
without leaving traces, bits, seemingly meaningless fragments
of personal information.
So in the new world of Gen AI, we're going to need to separate
the winter meats from the humans.
And with that, I'd like to say thank you.
And finally, could we all give the guys from the Recon Village
a round of applause as well?
Thank you.