start	end	text
3150	8290	Hi, I'm Leo, the founder and CEO over at Graphistry, and I'm going to be talking today about Project
8290	13630	Domino, which is not even at Graphistry, but a community open citizen data science effort
13630	18610	against COVID misinformation. And before we go into it, I do want to say thank you to
18610	24890	the DEF CON staff, AI Village staff, and the Project Domino volunteers, and also AI Village
24890	30630	in particular, Sven. This is a crazy time for all of us, and so to be able to put this together,
31570	36510	a big thank you for that. I'm honored to be giving this as a keynote. I'm hoping folks
36510	41530	here will benefit from hearing what we've been learning, and also if you are impacted somehow
41530	49070	by COVID and want a way to give back, this might actually be an opportunity for you as well.
49090	55790	With that, I want to start with an example. This is about three months ago. It might feel like
55790	61530	three years for some of us. And around the time when the Seattle breakout was happening,
61530	65130	our friends were kind of dealing with the hospital situations over there.
65570	72390	I think it was already like people were actually dying already in Seattle. Governor Kevin Stitt
72390	77850	from Oklahoma posted this strange tweet, eating with my kids and all my fellow Oklahomans at the
77850	83710	collective OKC. It's packed tonight, support local, Oklahoma proud, basically. And then he clarified
83710	89870	next day in a press release, we encourage Oklahomans to go out to restaurants. So for us,
89870	97910	with ties to the medical system, our head kind of was exploding. Like, what's up? Fast forward to a
97910	106290	couple weeks ago, and unfortunately for Governor Kevin Stitt and his family, he tested positive
106290	111610	for COVID. And unfortunately for his constituents who are relying on him, they have about a thousand
111610	116370	per day infections at this point, because kind of what we feared would happen is exactly what
116370	125710	happens. And now Oklahomans are dying. So it's... I do want to do kind of some disclosures and stuff
125710	130090	in a bit. But to us, it's not about a left or right thing. It's like we're, you know,
130090	136790	this is a medical health emergency. We need to do stuff. And that's kind of a good note,
136790	144750	kind of where this project came from. We're kind of looking at kind of what can we do.
144930	150510	And there's this interesting statement by the... from the World Health Organization saying,
150510	155910	we're not just fighting an epidemic, we're fighting an infodemic. And kind of the reasoning
155910	163510	that we kind of believe in here is until there's a medical cure, a pill, a vaccine,
163510	172470	been tested, it's safe, kind of the only thing we really have is behavior change. So for that,
172470	178810	unfortunately, you either need to get top-down policy enforcement or bottom-up
179510	184450	kind of citizen voluntary compliance. And unfortunately, we have folks standing in the
184450	189450	way of that. And, you know, for Seattle, for Oklahoma, for all of us at home,
189450	194450	we're seeing kind of the effects of failures to win that fight.
195390	201270	Some quick disclosures here. We are going to talk about real people. We've published... actually,
201270	206110	we haven't directly published. We publish through partners. So we empower researchers to do through
206110	210250	our tech. So they've published on some of this stuff before. And based on some of the results
210250	218070	today, I expect to see some accounts going out and hopefully some takedowns. Likewise,
218070	224650	this is a volunteer organization. So the views expressed by us, me, this talk, they're not
224650	231890	necessarily representative of the employers of the volunteers. Likewise, you should expect some
231890	237230	bias here. So a lot of us, actually, some people on the team were even affected by COVID and didn't
237230	242290	have their lives impacted by that. Family members do. Their work and work life has been impacted.
242410	247490	Misinformation is a broader topic. I'm personally in a place some folks may remember called the
247490	252790	Soviet Union, where propaganda was a thing and surveillance was a thing. So conflicting thoughts
252790	258390	on that. But I do want to point out from the team, I do especially want to call out some of the kind
258390	264370	of folks on the leadership team. Sean Griffin's been involved in pandemic response. Actually,
264370	271010	he helped both Obama and the Trump administration with pandemic planning. Anita was over at the NSF
271010	276870	and looking at a lot of digital crime, blockchain, stuff like that. She's at UIUC now.
276870	282850	Cody Webb has been this research powerhouse for our team and researching misinformation
283530	289410	forever. I shared the names of some of the engineers helping on our backbone.
290650	295850	The first example today will be actually Jeff Goldberg over at Social Forensics and David
295850	301470	Knickerbocker, an independent researcher, has been helping us a bunch on. And I think, oh,
301470	306650	yeah, and then tomorrow, if this is interesting to you, kind of the same type of stuff we're doing
306650	312170	for COVID. Andrea's going to be talking over at the Biohacking Village about imagine you're a
312170	317590	cancer patient and you go to Facebook and then you see all the same misinformation giving you
317590	325510	false hope or misguiding you. So it's important stuff. Okay. So today, I'm going to talk about
325510	329950	basically three things. First, I just want to take you down the rabbit hole through an example
329950	336850	from anti-vaccination misinformation. Second, I want to then give you more of a security data
336850	340170	science perspective and how we're actually going about it, trying to figure out scale and
340170	345450	autoresponse and all that stuff. And then briefly, but incredibly important,
345450	351790	why we're doing this as Open Citizen Data Science and why I'm actually pretty hopeful about where
351790	356470	this is going long term. And maybe this could kind of give you a chance or give you ways to
356470	367810	help out the community. Okay. So let's dive in. So we were doing an investigation and this name
367810	373810	popped up, Sherry Tenpenny, and we decided to see where we go for the purposes of this talk.
374830	381450	And for some folks, if you're from Australia, this may be a familiar name. She's Wikipedia
381450	388890	famous or infamous at this point. So her 2015 lecture was canceled in Australia because
388890	396510	apparently she is involved in a lot of vaccine misinformation and private organizations in
396510	406000	Australia wanted no part of that. And so they didn't support her. For our purpose and purposes,
407840	412680	I do want to point out there's kind of an ecosystem with different incentives here.
412680	419420	And so if we look at somebody like Dr. Sherry, we see that she does teach courses,
419420	425180	like hence the book tour. She sells books. She trains folks. She has about 50, 60,000 followers
425180	431400	on Twitter, not shown. She also has a big presence on Facebook and also on newsletters. So stuff that
431400	435520	you don't even see. And she gets paid to do it. Now it's kind of like her
436220	440240	beliefs and her livelihood are the same thing. So she's very invested in this.
442660	448420	But once we start looking at it, there are kind of a few things that looks a little strange.
448740	456180	So she talks about Nazis in her profile, like that's the first thing, that's the first word
456180	462520	she shows you. And so that kind of is kind of like a whistle to one side of the house.
462660	467160	And then talks about abuses by the state. So that's kind of like a different kind of thing.
467160	472000	And so it's kind of interesting to ask, like, who is the community around her?
472180	483220	So let's take a look. So what I did here is loaded in busy Dr. T. And so we're looking at her,
483220	489040	and what we see in colors around her are basically, she's only following a couple hundred
489040	493840	people, but she has 50,000 people following her. And so when we look at a sample of that,
493840	499980	what we see up top is a bunch of folks, essentially public, basically anti-vax,
500980	504960	natural medicine, a lot of kind of medically-minded folks that you'd sort of expect
504960	510440	to be here. But then the surprise is once we start seeing the other colors.
510440	516320	And so the tools here just kind of automatically colored and laid it out and everything,
516320	520800	like it's kind of just traditional graph analytics. So we see Donald Trump,
520800	527620	we see Michelle Malkin, we see Candace. Basically, like in the dark green,
527620	531800	we're seeing these public figures, kind of national conservative public figures.
532640	538720	On the right, we're seeing, but a bit on the right, and then actually, especially down here,
538720	544360	we're seeing a mixture of QAnon, which is a conspiracy group we're hearing about a lot
544360	551380	nowadays. We're seeing a lot of MAGA and Trump supporters. So not super surprising given we
551380	556820	had Trump. And so it's kind of interesting that we have on the top, we have like, who do you expect
556820	562340	with anti-vax? But on the bottom, we have like the whole MAGA thing. We have conspiracy theorists
562340	567660	from QAnon. They're actually even like UFO stuff going on, on the bottom. It's kind of like
567660	572840	interesting to see that mix. So I want to point out another phenomenon here.
573260	577980	When we're looking at each community, like the question is like, how are they interacting? Is it
577980	583480	like they all really love our Dr. Busy T? Like she just wrote this wonderful book. What we actually
583480	589720	see is like, for example, from the QAnon zone, they're reaching into the medical zone. So they're
589720	595560	following folks there. Conversely, if we pick somebody big in the kind of this top region,
595560	600420	sometimes it's localized, but other times you see it's reaching into the QAnon zone.
600820	605560	Kind of no matter which cluster I do, you're going to see those reach outs.
607120	615040	Excuse me. So that's actually kind of strange, where there's no reason to believe one conspiracy
615460	620640	has anything to do with another. And so we're actually kind of almost like in this other world.
621520	628520	And what we're seeing is the networks do cross over. And so the first thing is what we're seeing
628520	635700	is the question is why? And that's basically to support influence. Sometimes that could be direct
635700	639520	where you're just retweeting from each other's. Other times it might be to trick the Twitter
639520	644460	algorithm. So for example, here is actually a Fox News reporter. So it's someone on TV.
645560	652660	They have 40,000 followers, which is reasonable for a personality, but they're following 30,000
652660	657620	people back. So that probably suggests an inauthentic and automatic action of following
657620	662200	back somehow. We don't know what order they happened in, but in general, that's prohibited
662200	669040	by Twitter's terms of service. But they get clicks. And so even though it violates terms
669040	675940	of service, there they are. And that's basically the welcome to the misinformation network.
677360	682920	And I realized that I glossed over this, but the reason we care about Dr. T and that she's
682920	689820	talking about this stuff is that she and her community are already pushing messages against
690040	695520	a COVID vaccine, even though one hasn't even passed trials yet. She's against it before it's
695520	701100	even been tested, just by principle. And so Twitter's been supporting her in doing that,
701100	707440	essentially. And I'll get to that in a second. So I do want to take one more, I want to go a
707440	714120	little bit deeper into our Dr. T. And so in particular, I want to understand the communities,
714120	719300	like what they're doing. So here what we're going to do, and I want to see if there's kind of extra
719300	722900	coordination structure. And so we're getting a bit more on the data science side. So what's
722900	728340	going on here is we took the accounts, we actually took a look at about 100 million COVID-related
728340	734120	tweets to see if there's kind of organized behavior within it. For each account, we just
734120	740080	tried to look at for known sort of misinformation topics, what's sort of the signature, like, you
740080	745380	know, how many times did it like activate on different topics. And for each account, we now
745380	750640	have that sort of like misinformation fingerprint, essentially. And then we ran something called
750640	757040	UMAP. I think it's a universal manifold approximation or something like that. But basically
757040	764100	tries to smooth out and assume structure behind that. And then we took a look. So let's try it
764100	771820	out. So yeah, so what's going on here is we're taking a look at those accounts. And we only
771820	778080	are showing from all the accounts, we're looking at the 5000 or so that we actually saw active
778080	784160	across multiple misinformation topics. And so these are kind of the really the ones that have
784160	788960	kind of that behavior going across topics like are just like what's going on.
791020	796320	And we ran UMAP. But then what we also did, you're seeing here, I'm drawing these little edges.
796320	802660	We also ran a afterwards k-nearest neighbor and put them on screen. So that whenever we see
802660	809500	accounts, kind of tweeting on similar topics and similar intensities. So it's really, for example,
809500	813860	imagine two bots program to kind of talk about the same thing, we're controlled by the same person to
813860	820160	kind of, you know, retweet someone else. That's when we're going to draw that edge. And so again,
820160	824100	they're not necessarily following, they're actually just on certain topics, they're actually pushing
824100	829960	those topics in the same way. So it's entirely content based. And what we're seeing here is
829960	836240	some interesting structure pops out for across COVID Twitter. And then the second thing I did
836240	845800	here is coloring, you're seeing some of these are a bit pinkish. So if we jump, let's take a look at
845800	852180	this one. So for this kind of group of activity, we're seeing a lot of this q&a and stuff. And so
852180	859420	what I'm doing here is if in those tweets talking about COVID, they also talk about q&a. And we have
859520	864760	a couple words on that, we call that out too. And so what we're seeing is like, you know, for this
864760	871380	general topic, a bunch of accounts never explicitly mentioned q&a. But it's definitely maybe it's like
871380	876700	two coordinators, or who knows, or like something like that. But we're actually seeing a lot of q&a
876700	883260	influence across these different organized groups. And actually, in particular, almost all of these
883260	888520	have a bit of q&a. So you're seeing it's not all white, it's all a little off white. And then what's
888520	893520	more, you're seeing some of them are you're seeing these pockets of heaviness. And we calculated
893520	900240	out about 15% of these accounts spread throughout our q&a. And so at least for COVID Twitter,
900240	903520	that was actually kind of interesting. And for us, that was a surprise.
903800	910600	So that kind of put q&a on in terms of medical misinformation on our radar. So that's not good.
912920	925360	Okay, so jumping ahead. So okay, so we're looking at our busy Dr. T. And clearly blatant activity
925360	930840	going on here. The last thing I want to point out was, then we look on the right here, and Twitter
930840	935520	is saying, hey, you should probably now that you're here, you should also check out this
937340	944320	Judy Mikovits. This PhD sounds authentic, right? Sounds smart. You may have also known about this
944920	952140	Judy as a video going around called Plandemic, which, you know, got pulled down as pretty bad
952140	957220	misinformation. And so that was pretty shocking to see, like, I was like, oh, that name sounds
957220	961460	familiar. So that was a little crazy. That was like the number one recommendation of Twitter
961460	966540	as soon as you go to this web page is to learn about Plandemic. So and this is like, you know,
966540	973220	way after the fact. And so kind of in summary, what we're seeing in this, in this basic, pretty
973220	979980	basic, honestly, analysis, very blatant anti-vaccine misinformation, a lot of it about
979980	985360	COVID in particular, we're seeing it's not just anti-vaxxers, but we're actually seeing a lot of
985360	992200	mis, cross-community support by misinformers. It's pretty blatant. It's also it's pretty,
992200	997500	pretty obvious how they abuse, like, the very few terms of service Twitter does have.
997500	1006780	So that's a little strange. It is a little disappointing to see Fox, the Republican Party,
1006780	1013500	MAGA, and a lot of them kind of involved in this anti-medical misinformation. And again,
1013500	1020740	it's so blatant. And then on the platform side, it's when we saw that overlap and like,
1020740	1023680	oh, wait a minute, Twitter makes money off of some of this stuff.
1024740	1029540	They're clearly looking the other way for the terms of service violation. And then like,
1029540	1034480	what was the the the Plandemic promotion? Just that was just that blew my mind. So
1035080	1041140	this is not, this is not a healthy situation. So from a data perspective,
1043500	1047480	or actually security analytics perspective, I do want to step back.
1047960	1054180	If we think about attackers, defenders, ultimately, the targets are individuals and organizations
1054180	1060420	going on a radicalization journey. And so every time Twitter recommends you do another crazy thing
1060420	1068120	that just takes you deep, deeper. Interesting extra level on the threat model here is that
1068120	1074380	folks are essentially influenced based on their life conditions. So maybe they're going through a
1074380	1078740	hard time, maybe they're elderly and don't really know what's going on. And also by their community,
1078740	1083100	you know, if your if your pastor or somebody or your family friends tells you something,
1083100	1089020	you're more inclined to believe it. And so being able to infect communities with this stuff is
1089020	1092060	having mastery over that and platform level support, it's not good.
1092680	1099880	For actors, it's a lot of what you'd expect. I think that the surprising thing for for threat
1099880	1106820	actors here is we actually consider the, at this point, the, the platforms to be part of the
1106820	1114280	threat. In particular, we we break down the platforms as the social network system. So
1114860	1118700	Facebook owns a bunch of social network, Google owns a bunch of social networks,
1118880	1122800	a few are just big on their own, right, like Reddit, Reddit and Twitter. And, you know,
1122800	1128540	Twitter is a multi billion dollar profitable company. But here we are. A little less obvious
1128540	1134040	to folks is messaging. And again, actually, a lot of the same companies like Google, Microsoft,
1134040	1140700	Yahoo, own essentially the messaging channels, those are completely unmonitored. And so a lot
1140700	1145800	of times the radicalization actually hops over to newsletters. So that's stuff basically dark data.
1146680	1155360	And also the local news, I think like Fox and folks like that. A little surprising is there's
1155360	1161400	kind of a monopoly effect where because you do have regional different ways of having monopolies
1161400	1166840	on people's attention. That's all the reason these companies got so big. Same reason we're
1166840	1170980	basically at the whimsy of these companies. And so when you kind of calculate it out,
1170980	1179280	the 80% essentially is just about 13 of these groups. And so if we have a fickle Silicon Valley
1179280	1184860	CEO, if we have employees who just go along for the ride, if we have advertisers keep feeding
1184860	1194160	them money, you know, so that's kind of from the threat model perspective. Shifting gears a bit to
1194160	1200220	more like kind of data like security, data engineering, data science. In order to do the
1200220	1205420	data science, I did want to talk a little bit about this. We end up having to build a lot to
1205420	1209620	work at kind of the speeds and scales we're targeting. So if you want to have an instant
1209620	1213780	response pipeline, that means you have to kind of have streaming, you know, can you take hundreds
1213780	1217820	or thousands of things a second. So our V1, you're kind of seeing the performance of that
1217820	1222540	where you're architecting to a V2 so that we can actually go to arbitrary scale, just have
1222540	1228620	essentially budget limited. You are noticing we're doing a lot of graph technology. And so basically
1228620	1232860	that's a nice way to deal with a lot of heterogeneous data and a lot of like linkage of
1232860	1237280	data, even actually in a machine learning world where instead of just direct links, you might have
1237280	1242400	vectors, feature vectors. Two other interesting things about the stack I did want to point out
1242400	1248060	here. One is we do do end-to-end GPU computing just to be able to, you know, you can actually,
1248060	1251540	if you want to have a billion node graph in memory, you actually could do that stuff with,
1251540	1257560	for example, a Rapids AI stack. Likewise, neural nets in this space, graph neural nets are getting
1257560	1262800	interesting. We actually do more today on the traditional ones, but we're looking at those.
1262940	1266160	And the last part, and I think most important, is kind of division of team.
1266580	1272300	We have basically heroes for data engineers. And then a big part of this is figuring out ways to
1272300	1276800	get out of the way, just given it is an open volunteer effort. And so we actually do a lot
1276800	1282480	on the self-service. And that's a big theme for look about powering. So think notebooks,
1282480	1289120	streamlet dashboards, graphistry, do visual analytics and automatic ML.
1289120	1294680	Just increasing the ability of people just to go and handle the stuff.
1297040	1305480	From the data science side, I kind of wanted a fun example here. For example, we had a
1305480	1310740	news organization ask us about Obamagate. And so one of our analysts, Cody, did a good job of using
1311270	1318480	hashtag analysis to connect the dots to a month back to some QAnon stuff. And it was kind of
1318480	1324620	impressive because the Trump tweet made no sense at all, even when I read it today. All these kind
1324620	1334100	of code words. But once we moved to more modern NLP techniques and go away from exact match of
1334100	1342340	terms. So Cody had to look for Obamagate. But with NLP, we can do things like look for the word
1342340	1348780	Obama and the word conspiracy. But not the word, but just the fuzzy topic. And when you have those
1348780	1356660	things, then we can kind of more loosely kind of walk back. And so that means we can do kind of
1356660	1363880	traditional analysis more easily. And so we've been building out a lot around that. I want to
1363880	1370980	modeling from basically what everyone does now. And then the other thing I want to point out here
1370980	1375060	is this is also opening up types of analysis that just were straight up, like, you couldn't even
1375060	1383140	think about. So maybe one example could be for looking at masks in a community where you might
1383140	1389360	ask, like, which messages are against mask use? That's kind of a strange... that's kind of a
1389360	1395160	gets into semantics and meaning a bit. And that's hard to do just by keywords. And then
1395160	1398900	could you summarize it? Could we actually have a demographic breakdown? There are just certain
1398900	1403840	things that with traditional NLP, you couldn't have done. But over the years, over the last,
1403840	1408640	like, five years, and actually even two years, it's just been really nuts what's been going on.
1413160	1416840	Then kind of the last piece I'm going to share here is kind of just where we are,
1416840	1423120	where we're going. You're kind of getting a sense that we can now bring in a researcher on a topic,
1423120	1428200	and if we have the data, or we help them pull in the data, and kind of do this kind of analysis.
1428520	1431860	Next level of empowerment we're looking at is, like, as we're building this out,
1431860	1435780	is can we actually partner with less technical organizations, and if we have to kind of do this
1435780	1440160	to protect their community? Imagine, like, a local civic leader talking about the
1440160	1445700	cancer community with Andrea, with kind of at-risk patients. And there we're looking at,
1445700	1452340	are there folks who are not even maliciously misinforming in their community, but people
1452340	1456280	are just confused? Or are there people kind of targeting them? Can we start mapping out and
1456280	1462240	building tools and stuff around that? So that's kind of the phase shift we're starting to go into,
1462240	1466880	so that's pretty exciting. You're kind of seeing, as you build out that state of capability,
1466880	1473180	all this stuff unlocks. And then, finally, I just kind of want to close out and tie together
1473180	1477400	some of the stuff here. So the beginning, I was kind of sharing that investigation.
1478300	1481980	Hopefully, what you got across is, like, yeah, the analysis is cool, and we're able to, like,
1482380	1488440	do these giant analyses. But, like, ultimately, a lot of the important stuff is so in-your-face
1488440	1495160	and so huge numbers that, like, again, that analogy of fishing with dynamite. And it's,
1495160	1502200	you know, those 13 companies, they shouldn't be gifting, essentially, millions or billions of
1502200	1509000	dollars in free targeted ads to these groups and targeted recommendations. It's kind of that stuff
1509000	1516680	that I think is more systemic. I am disappointed to see kind of these otherwise legitimate groups
1516680	1522000	appear to be corrupted by misinformation. So call out, essentially, we're seeing MAGA is just so
1522000	1526740	ever-present here. We're seeing social media companies, regular media companies. So there
1526740	1531780	are things you could do if you're an employee, if you are anywhere near the advertising spend of
1531780	1538100	your company. All that stuff's important. And then kind of turning it around,
1539180	1545760	that's all kind of social policy. A lot of folks tuning in are technologists of different stripe.
1546940	1554440	And the bad news is we're seeing these monopolistic media companies are not able to
1554440	1558280	self-regulate. Like, the left hand and the right hand are always at each other, and it's kind of
1558280	1565380	brought us to where we are. Likewise, they've kind of gutted investigative and journalist teams
1566000	1570120	just because their ad companies sucked all the money. And likewise, we're seeing private
1570120	1577020	companies are sort of compromised as they're kind of trying to deal with revenue. So that's bad.
1577160	1583320	What's good is we actually have an open source community. We, I mean, just like the technology
1583320	1589140	community, that is huge. And now at the same time, data science is, you know, our Python that's almost
1589140	1598480	like built into the DNA. So I see strong potential and gathering potential for that. I encourage that.
1598880	1605340	You're seeing some of that happen here with Project Domino. And so on that point, if you are
1605340	1611380	into this kind of stuff and you could help out with coding, head into our Slack. Likewise, as we
1611380	1618400	launch in the coming months, these kind of more self-serve and tiers for folks sitting on top,
1618400	1625300	reach out for partnering. And likewise, while we are looking at doing sustainable approaches to our
1625300	1632520	growth, especially as sort of a bootstrap phase that we're in for the next year or so,
1632520	1636100	we're definitely, you know, grants, whatever, that stuff does help with the core team
1636100	1640500	do what they need to do. Likewise, you know, serverless compute time, anything like that.
1641380	1647680	With that, thank you for your time. And hopefully this changed a little bit about either how
1647680	1655020	misinformation is working in practice in the COVID era. And also, even more importantly,
1655020	1659900	that we actually could do things and we actually detect it and then can actually build tools for it.
1660280	1661280	Thank you.