Alright, so before we start, a little request to please keep your phones on silent.
And there will be a Q&A round, and if there are further questions after this talk is completed,
we have a speaker's lounge right next door. So you can meet up with the speaker,
and you can have your questions addressed over there.
Also, if you like this particular talk, please tweet us at DefConVillage...
at ReconVillage on Twitter and on LinkedIn,
and we'll be handing out special gifts and prizes for some random people.
Alright, so the next speaker that we have is Patrick Warren,
and just a little introduction about him.
He is an Associate Professor in the Department of Economics at Clemson University.
He co-directs the Media Forensic Hub at Clemson University,
where he does a bunch of work using OSEAN techniques, mostly in the context of social media.
And the topic that he will be covering is,
Oh, the Places You Will Go Using Media Variants to Trace the Organization and Behavior
of a Coordinated Inauthentic Influence Operation.
So, yes, please welcome Patrick Warren.
Thank you so much. I did not coordinate with the speaker before.
We both randomly picked, oh, the places you'll go,
but I guarantee to you this is all out of my own brain
and not GPT created.
I'm an academic.
I'm going to tell you about a project that we've been working on at the Hub
with a big team of undergrads, where we teach them some basic open source techniques
and then turn them loose on a problem that we find interesting and we think they'll find interesting.
And they often will come up with new and interesting ways to look at stuff
that we wouldn't have thought of at all.
And so, if you like this, if you find this interesting,
you can go to our website, just Google Media Forensics Hub Clemson,
and you can find a bunch of reports that our students have created.
And I also want to thank Three Gimbals.
They do a bunch of open source stuff for money.
And they've partnered with our students quite a bit
and they sponsored us to come out here, so thanks to them.
So, pictures. Oh, the places you'll go.
So, here's what we're going to do today. We're talking about Dragon Bridge.
So, Dragon Bridge is a Chinese originating disinformation actor.
You may have heard them called Spamouflage Dragon or Dragon Bridge.
For folks who have worked in the kind of coordinated
and authentic state-backed information operations space,
they're probably the most prolific Chinese actor in that space.
If you know anything about Chinese info ops, you probably know about them.
But just in case, I'm going to tell you about them a little bit.
And then the guts of the talk, the main point,
is this idea of using media variances data.
So, I'm going to talk a lot about what I mean by that.
And then I'm going to compare using media to using text and why it might be better
or some reasons it might be better.
And then maybe, if we have time, I'll take the whole thing to 11
and think about rather than using media or using text as data,
can we use theories as data?
So, hopefully we'll get there.
So, Spamouflage Dragon or Dragon Bridge.
We have a report. You can Google it. Google the five-year span.
This is kind of a longitudinal investigation of this actor and how they work.
They've been around for at least five years.
Bandian has reported on them. Graphic has reported on them.
Google has talked about them a lot. A lot of people have talked about them.
But I think the best way to understand them
is as a persistent Chinese influence operation
that uses tons and tons of accounts that are immediately obviously fake
and yet still manage to have...
And I'll argue about this during Q&A if you want.
That's not really what this talks about.
But still, despite having accounts that are obviously fake,
can still have an enormous impact.
And we can think about why that might be.
That's not what this talks about.
What this talks about is using one of the techniques
that Dragon Bridge uses to have the impact that they do
in order to unpack something about the way that organization operates.
So, just in case you don't know anything about Dragon Bridge at all,
here's what they like to talk about.
They like to talk about how the USA sucks.
They like to talk about Guo, the Chinese dissident billionaire
who's now in trouble with the DOJ.
Miles Guo, Miles Kwok, he goes by all kinds of different names.
They hate him and have hated him for a long time.
But then they like to talk about Xinjiang,
they like to talk about Hong Kong,
they like to talk about Taiwan recently.
One of their very favorite topics are what we call
problematic expat Chinese people.
So basically, anyone who's saying bad things about the Chinese government
while overseas, they hate those people.
And how do they try to influence conversations around these topics?
Well, they do all kinds of different stuff.
Original cartoons, original video, multilingual recycling,
hashtag flooding, calls to action,
lots of baby accounts, lots of account spoofing,
lots of hacked accounts,
little bit of cross-platform stuff,
little bit of GAN faces.
We heard a little bit about the way you can do GAN faces better now
in the sort of generative AI world.
But we're going to concentrate on these things.
These are techniques that they've used since their inception.
And this is basically creating from scratch novel media.
Now, it's not great.
I'll show you what it looks like.
This is what it looks like.
It might be hard for the folks in the back.
There's plenty of seats if you want to come up where you can see.
It's basically really, really mediocre hand-drawn or computer-drawn cartoons
or videos where they superimpose AI announcers in the background.
Often they'll do these cartoons in lots of different languages.
They'll just recycle the same cartoon
and put the little captions in Chinese and in English and in French.
I don't really know.
Well, I have some ideas about what they might do in French.
But this is what we're talking about.
Think about hundreds and hundreds or even up to a thousand such media objects
that have been created by this campaign.
Okay, so now to the guts.
So now you know a little bit about Dragon Bridge and what they're up to.
I want to talk about how we might be able to use this media against them, in a sense,
or certainly to learn about them.
And so here's the process that I'm going to talk you through today, just in one slide.
So what do we start with? We start with a seed set.
We start with a couple hundred examples of cartoons
that we think were created by Dragon Bridge.
And again, they don't hide who they are really at all.
It's pretty easy to find cartoons that you can attribute at a very high confidence to this operator.
And so cartoons or YouTube videos are still from YouTube videos, or the whole video, we can think about that.
But we need a little seed set of images or videos that we are pretty sure are coming from this actor.
Then we need to turn that seed into a field.
And the way to do that is reverse image search.
There's lots of ways that you can automate this reverse image search. I won't talk through all of them.
You can roll your own. There's search API ways to do this. You can actually pay Google to do it.
It's not just Google. Of course, there's many, many reverse image search engines that you can use.
You really want to cast just as broad of a net as you can.
Because what we're looking for is all of the different variants of this image that pop up all over the internet.
As many as we can find in as many places as we find.
And I'll tell you, for this actor, you're going to find them everywhere.
So if you pick a topic where you know this actor is active,
and you search on that topic on basically any place where someone can post images on the internet,
you're going to find images from this actor.
I mean, in a Hindi website that helps Indian high school students cheat on their homework,
if one of the things you can do is post the solution as an image,
you're going to find a picture of Miles Guo on that website ripping off ants.
So they are literally everywhere.
And so once you plug these seed images into these reverse image search,
you're going to get lots and lots and lots and lots of these.
And you're going to find lots of them in social media platforms.
And of course, every social media platform you can imagine.
I've just shown some here, but all of them.
Many variants.
But why do we care?
These are just more examples of the same thing.
Well, there's a couple reasons we care.
The first reason we care is that those variants can become new seeds.
How do they become new seeds?
Well, if I have one account that's posted ten variants of one of the seeds,
and then they've posted some other image that isn't one of the seeds,
well, now I've probably got a new seed.
That's just the way these accounts work.
So I can go from 200 seeds to 400 seeds to 800 seeds,
and then for each of the seed images, I can find lots and lots and lots of variants.
Why do I care about variants?
Because the variants will sometimes get posted in places where the metadata remain intact.
So you may start with images on Twitter,
but from a metadata perspective, images on Twitter are pretty weak sauce, right?
I can see the dimensions and things like that,
but most of the metadata are gone by the time they show up on Twitter.
But if instead they posted on IndianHomeworkHelp.com,
and they haven't been careful enough, they may have left a trace that I can see.
And not only might they have left a trace that I can see,
I may find those same traces on some other websites,
but on some other variants of that image, I see different traces,
and now, look, I've learned something about the way they post their images.
Because however they're posting them here is not the same way they're posting them there,
because the traces look different in these two cases.
So both of these things are valuable.
Both running it through the social media profiles to find more seeds,
and then extracting the metadata not just from each seed,
but from this big old scope worth of variants.
So what are the payoffs you'd get out of this?
So let's say I do this. Imagine I do it. I've left some of the details, obviously, vague.
What can I get out of it? Well, profiles. Profiles are actually valuable, right?
So I start with these sets of images, and now I have images, and I also have profiles,
social media profiles across lots of places.
I might be able to get profile information. I may learn something about how those profiles got created.
For example, in one of the cases that we tracked, which had to do with Xinjiang cotton,
we found that a ton of the profiles that they had posted with on Twitter,
they had actually set up with hacked accounts from the same hack release.
That's useful to know.
I may also learn something about the software that they're using.
So here I have Photoshop, Adobe Photoshop.
Why do I have Adobe Photoshop here?
Common image editing software.
And metadata about Photoshop is often left in images that get edited with Photoshop.
That can include things like what version of Photoshop got used.
Is it a Windows version or a Mac version? I might find out that.
What year of the Photoshop was used? I might find out that.
Was there different directories that were used to make that Photoshop image?
Sometimes that gets left, and so I've learned directory structure now, at least for that variant.
I have a signature here because some idiots put their name in the directory structure,
which is crazy, but happens, and has happened here.
Hardware specs, sometimes that gets captured in the metadata.
And of course the Holy Grail, sometimes they leave the GPS information in the metadata,
and then we're like, oh well, this was in Yulin. That's kind of nice to know.
Also, I can use these individual bits of data about the variants to do a couple things.
I can cluster them together.
So if I see combinations of the variants being used on these sites and not these sites,
or by these accounts and not these accounts,
I know that the guy that's posting on IndianHomeworkHelp.com
is probably the same guy that's posting on DWNews.com,
because the images have identical metadata.
And in fact, that person is posting the same set of images in both those places,
but it's different than the set of images or the metadata that are getting posted on Quora, or Pinterest, or wherever.
And so now I can cluster users or accounts, and sometimes I can even learn something about the structure.
And what do I mean by the structure here?
So if you know anything about Chinese information operations, you probably also know something about Russian information operations.
If you know anything about Russian information operations, probably what you know is the Internet Research Agency.
And they're famous for having a centralized media shop.
What do we mean? We mean that they had some guys that made all their media,
and then they used those media on a lot of different accounts all over the place, but it was centralized.
So the media was created, at least back in 2016-2017, was all created in sort of the same place.
But if I see that there are 17 different versions of Adobe being used to create these images,
then they're not all happening in the same shop.
This has been farmed out.
And is it being farmed out to individuals, or contractors, like Fiverr contractors, or is it getting farmed out to marketing shops?
The patterns can tell us something about it.
So, application. So that's the idea of theory.
In practice, this is what it looked like in this...
It's called All the Places You'll Glow, because it was the title of the report,
where we looked at this campaign that focused on these three guys.
Steve Bannon, Lieberman, and Miles Guo.
We started with 236 seed images that we found, mostly on mainstream social media, Twitter.
Once we run them through some very simple search functions,
we end up with 2,700 variants of these 236 images.
Again, this is a huge underestimate. There are many, many more variants out there.
But this is what was still on... this was just using Google or a semi-search.
And then, once we see those variants, we can then infer some social media profiles that are probably central to this campaign,
because they posted many of those variants.
Once they posted many of those variants, I was able to extract 200 new seeds,
so images that were not in my original set of images,
because those images were also posted by those accounts that were central to this campaign.
And from those 200 new seeds, I ended up with 800 new variants.
So I ended up with, I don't know, about 3,500 variants that were all basically pictures of these three guys,
Lieberman, Steve Bannon, and Miles Guo, in all kinds of different languages.
And just to give you a sense of where these were, this is just the top-level domains where we found these images.
And here's the order they're in. So Twitter, number one.
Reddit. Does anybody know what this is?
No fair if you're with the team.
Nyonga Money. It's like a Nigerian crypto forum.
It is the number three place that these things showed up.
OK. Who's surprised by this one?
Yeah, your mom is seeing a lot of pictures of Miles Guo.
In fact, they were so successful on Pinterest that Pinterest shut down the ability to search for the term Lieberman,
the Chinese virologist that they targeted here.
You can't search for Lieberman on Pinterest right now.
They've shut down the topic.
I said I wasn't going to argue about this, but this is one of the ways that they're successful in ways that people don't quite appreciate.
They can affect the topic without having accounts that convince anyone,
if they can manage to flood out the target so much that people can't talk about it.
And Pinterest has helped them do that.
Flickr. I love Flickr. Why do I love Flickr?
Metadata intact on Flickr.
Metadata intact on Pinterest.
Metadata intact on the Nigerian cryptocurrency, but not on Twitter, not on Reddit, by default, anyway.
OK, so there's a lot, 180 of these, that we found substantial numbers of these images.
Substantial variance in the software being used to post these images. I've sort of previewed that already.
I mean, Adobe Photoshop 2020 was the number one, but there's things I had never heard.
PicsArt, I didn't know PicsArt. MeToo is a Chinese, I think, web-based photo editing thing.
There was MeToo, but tons of variance, including tons of different systems.
So, Windows, cloud-based, Mac. This was not, not, not, not a centralized art shop, for sure.
Unless you're having everybody bring in their system to do it, right?
Very different structure.
Multigo, had to use Multigo a little bit for the OSINT class.
This is just a little, when I was talking about clustering them together, this is just an illustration of that.
I can connect images by devices, or images by software, or images by accounts, and you can get fun little clusters.
And sometimes, those clusters include, as I mentioned, our kind of fun gold standard,
which is, I could find a bunch of images that were all posted, and oops, a couple times, they forgot to clean the GPS off them, and that's where they were.
I could show you the house they were in. It was not in an industrial area.
This had been outsourced, for sure.
This is an example in a different context, but I was talking about the file path,
so we did some work looking at some metadata on videos released around the Russian invasion of Ukraine.
These are, like, fact checks that had been faked.
So these were, like, imagine that you saw a fact check that had a burning Russian piece of machinery,
and then a picture of a non-burning piece of machinery, and they said,
look, this has been photoshopped by the Ukrainians to try to convince you that they're blowing up our stuff, but it's not true.
But then you dig in the metadata, and you see that, oh, look, whoever created this fact check also created the burning image.
So they have created fake news and then fact-checked it to convince you that the Ukrainians are lying.
But they forgot to clean up their metadata, and it turns out it was in a directory called
Gerasimov-Constantin Working Folder.
Probably not every variant of that media had this metadata still intact,
but if you search hard and far and long enough, you might find a variant that does.
So just because you found it once doesn't mean you're done.
You want to find as many variants as you can, because it just takes one slip.
OK, could we do all this with text?
So instead of media, instead of a book with text, could we do all this?
I think we could do some of this, still. This process isn't crazy.
You could start with some text, and these actors use long text, too.
Duplicate those texts, search for those texts, find other accounts that use the exact same text,
pull them together, identify new texts by looking at the text that accounts that post a lot of these seed texts also post.
You could do this, but the metadata is not going to work very well.
I mean, I guess you could use font or something, specific word choices, maybe,
but metadata, I mean, they're just going to exactly duplicate. It's not as obvious.
And the other thing is that the way people use social media tends to differ a lot by platform.
So it would be rare, for instance, that the screed that I point on Reddit, I would post on Twitter.
It won't fit on Twitter. And so, people reuse images, at least these guys,
and I think many people who work in this domain, reuse images in a way that they don't reuse text.
And so, you can't do this for text. It doesn't work quite as well.
But here's the problem, or a problem with this approach.
This works because media is expensive. I reuse media because it's expensive.
I've got to hire somebody else to draw a new picture. But you know, let's go back to the talk two times ago.
Media is not as expensive as it used to be. This guy said, if I were an artist, I'd be scared to death,
because my bad-ass tiger here was created with a generative AI, and I could create three tigers with turtlenecks tomorrow,
and I could create thousands of tigers with turtlenecks tomorrow,
and reverse image search isn't going to tie those tigers with turtlenecks together, probably,
at least not traditional reverse image search,
because they don't look enough alike to kind of get matched up by reverse image search.
And so, although these are clearly all variants of the theme of tigers with turtlenecks,
less bad-ass than the neon tiger, of course, it's hard to reverse image search for a theme.
And so, when I said that maybe we could take this to 11, I think there is still a path forward for this idea of using media variants,
but that path is going to use AI to fight AI.
So, maybe reverse image search can't say those are all tigers with turtlenecks,
but computer vision can say those are all tigers with turtlenecks,
or at least it can get pretty close to saying those are all tigers with turtlenecks.
I've been talking about pictures here, it's not just pictures, there's videos, too, but we heard about Whisper this morning.
We can do this with images, too, right? You combine computer vision with some screenshots out of a video,
and now we've got the transcript, and at least we know at least every 10 seconds what was on the screen.
We can get a pretty good sense of what the video is about, and so a video about a tiger and the turtleneck,
we might be able to match it to other videos about tigers and turtlenecks, or about how Taiwan is really part of China.
And so, we may be able to find variants of videos or media in the same way that we find variants of exact images.
It's just going to be a little harder.
The idea I guess I want to leave you with here is that maybe tigers with turtlenecks is not actually what we want to match on.
What we want to match on is cute animals with turtlenecks, or something.
What we really want to do is we want to stick media items together in the same way that the influence actor would.
And so, if I could guess or have some sense of what sort of target they might have,
then I could use this same process where I look for images or videos or whatever that fit that target, and run the exact same idea.
Or, if I have seeds of accounts, I can extract their target.
Not at the level of tigers with turtlenecks, but at a more general level.
This is hard. This is really hard.
We're trying to teach GPT-4 right now to extract what you might call narratives.
We like to call theories, because we think it's more general than narratives.
You might be able to extract theories from tweets. You might be able to extract theories from images and videos.
And then you can run all of this at the level of the theory.
It's pretty abstract, but it may be where we need to go.
I'm going to leave it there, but I'm happy to field questions about this, or about the Hub, or anything.
And I don't know how you run questions. Just stand up and talk?
Yeah, please.
You could imagine that if I'm afraid of this sort of move by the detector,
then I take my account and I seed it every once in a while with images that are purposely made to lead the LLM down the wrong track.
That's a risk, for sure. That's a risk even in the approach that I'm using right now.
If they sprinkle throughout the social media accounts some images that aren't part of the campaign at all,
then it's hard for me... I'm going to match on those.
My solution right now is, in order to consider it part of a campaign, it's going to have to be used by multiple accounts that are part of the campaign.
Now, they may do that. They may stick false flag images throughout multiple accounts.
But then, is that any different than it being part of the campaign?
I mean, so now I've found that probably this dude who's running these ten accounts, he uses the same false flag images.
Now I've got a trace. It's not a trace that's related to the product of the campaign, but rather it's a trace related to the countermeasures he's taking.
But whatever, that's good. I'll count that as a trace.
So I don't think that's quite as worrisome as you might be worried.
Well, catch me in the speaker's lounge if you have any more questions. I'm happy to talk. Thank you.