All right. Hello, everybody. My name is Master Chen, and you are watching or listening to
Twitter Word Frequency. Welcome to my talk. All right. So, let's start with who I am,
or who am I. I'm a prior B-Side speaker, where I spoke about being a con man and
working in Vegas surveillance. I spoke at DEF CON on VoIP security, as well as DC Sky Talks
on automate your stocking, which we'll kind of cover a little bit about today.
And most recently, the Recon Village in the last few years, where we talked about
stalker in a haystack, URL shortened by any other name, which was last year,
and of course, this year and this talk. It's virtual now.
You can find me at ChenBox. That is my handle on most social media. So, feel free to follow.
All right. So, let's get into this, and let's start from
where we are at this point in time. And I think that'll help us understand
where my mindset is when it comes to this talk. So, of course, I started with automate your
stocking, where I was digging into my Twitter research. This is bad. And we learned that
during the DC Sky Talks a couple of years ago. So, the following year, I did stalker in a haystack,
where we figure out how to not be stalked, or at least we can detect cyber stalking.
That was good. And now I've gotten recently into political sports betting, or, you know,
market speculation, which is part of the story. And profiling. I mean, profiling like social media
profiling on different accounts. And of course, I do have a degree in psychology. So, all this
wraps up into where we are at this point. So, these five pieces are what kind of has
gotten around this research here. So, about this talk. I'm gonna start with
the original goal and how I came up with what I was researching at that moment.
Why that failed and why we move on to what it is currently now as I'm presenting this.
And what I would like my current research to be in the future.
So, before we get into all of that, though, let's talk about caveats and disclaimers and warnings.
Oh, my. I'm bad at that. Anyway, first caveat or first disclaimer is for those who have followed
my work before, you know that I've said I anal or I anus, which is I am not a lawyer and I am
not a stalker. I'm gonna add something new to that, because I'm very new to the whole data
science thing. I've only recently gotten into using Pandas and Scikit-learn and NumPy and all
these different data analytics tools. So, I've added INADs. I am not a data scientist. I'm not
a lawyer, I'm not a stalker, and I am definitely not a data scientist, but I'll try my best.
So, this project is a work in progress, and I'm hoping that after I explain kind of what we're
doing here today, that maybe I can get some help from the community to further this project,
or at least now that I've defined the basics, I can continue and make this something very strong.
This is a neutral tool. I'm not building what I'm building today for stalking purposes,
you know, this is just a tool. You use it how you want, but of course,
as always, I'm not going to be held responsible for what you do with my research.
And I did not stand on the shoulders of giants for this research. What I mean by that is,
I know that there's a lot of research already in the space when it comes to
frequency analysis, word frequency analysis, you know, sentiment analysis, NLP,
not neuro-linguistic programming, but natural language processing.
I know that there's all that research out there, and I barely used any of it. So,
I would like to integrate that later, but I just want to let you know that for this
particular talk, that was really not used. So, it is definitely a work in progress.
All right. So, again, for those of you who know me, I'm not a sports guy. I've only recently
gotten into hockey, and I've always been a UFC fan, but other than that, I'm not a sports person.
So, I've never been into the sports book, but one thing I know about being in Vegas for my
entire life is that there's a sports book for every market. It doesn't matter what the market
or what the speculation is, there's a sports book for it or a betting market for it. And that's
where we start our story. So, if anybody's ever heard of predicted.org, it's a market for
political sports betting. So, you can bet on who you think the next president's going to be,
not just in America, but globally. Pick a country, pick a state, pick a, you know,
impeachment hearings. Who will run the White House? Who's going to be the VP pick? All of
these different markets are on predicted.org. And so, you can make a fair amount of money
if you know what you're doing, or you can just have a casual bet and just be right with a few
dollars and cents. And so, and specifically, we're going to be talking about President Donald
Trump's tweets as that is kind of a famous thing here and how it pertains to the political sports
betting market. Okay. And well, let's get into that now, because right now we're talking about
piecount.com, which is a tweet counter that's out there on the internet. And it keeps track
on a weekly basis of the president's tweets, as well as the VP, the White House, and the POTUS
accounts. And this is all geared towards the tweet market that predicted offers or used to offer,
and we'll get into that in a second. But we're talking about Twitter, you know, Twitter markets.
Again, this is on a weekly basis, the question or the market is, how many tweets will the president
tweet out for this particular week, starting from Wednesday of this week and going on to
Wednesday of the next week, it could be anywhere from, you know, 50, which is, which is highly
unlikely, all the way up to, you know, 300, 400, you know, and, and that's, that's what I was
analyzing beforehand. Now, I wanted to know if the profits could be based off of analytics. And,
you know, could the, could I anticipate what he would be tweeting based off of current events,
news, drama, anything that's out there in the global space. And the goal, the goal with this
original research was to get the president to pay for my energy bill. So if I made enough money
on this predicted.org market, could I pay the energy, my energy bill at the house
with, with Donald Trump's tweets, that's, that's where I was going with this whole thing.
So again, prior research is the frequency of the tweets, how often he would tweet in that week. And,
and I was capturing, as you'll see here, in the CSV file on the right side of the screen here,
well, we see 68 in the week of, of April of 2019. We see 137, 140, 141. So these are all
a rolling tally or running tally of, of his tweets. And of course, you graph it using,
using Jupyter, a Jupyter notebook, you know, you, you graph it and see the frequency here.
And then you ask yourself, you know, was this peak here due to impeachment drama? Was this due
to some sort of scandal or emergency in the Middle East? What caused these peaks of tweets?
Was it just somebody getting on the president's nerves? These were all the questions that I was
asking at the time, but there's a problem. Predicted.org actually stopped their, their
tweet markets. So while, while Predicted is still around, they no longer offer you the ability to,
to bet on, on, on tweets. And so all of this research has kind of just stopped, you know,
so they, they changed their offerings. They, they started to offer other markets that
just weren't working fast enough. It, you know, these other markets aren't
resolving on a weekly basis. And so my profits slowed to a halt.
And so because of that, this whole project that I was working on, as far as tweet frequency,
it, it kind of took a back, a back burner and I, I kind of forgot about it for a couple of months.
Um, but, um, it got resurrected. And so, uh, the project lives and, and I'll tell you why.
So I tweet on a regular basis and I tweet very vague, vague things. Um, and, and that's on purpose,
I would say. Um, it could be a Zen of the day quote. It could be, uh, something obscure or
just something that's on my mind, right? Because that's exactly what Twitter is used for. You,
and then you just let it out there. It's not always political. It's not always socially driven.
It's just a thought that's on your mind. Well, I had one of my followers, um, kind of get a little
bit offended or had a little bit of ruffled feathers here, um, with one of my tweets.
And so, uh, he kind of got mad at me and, you know, I wasn't being political, but the
vagueness of my tweet allowed, uh, this particular person to,
uh, project their political ideals on, on my tweet. So, um, I thought about that for a while
and I could have gotten mad and, and, and, you know, gotten back and forth with him,
um, but I didn't. And instead, like, I don't get mad, but I will turn you into research.
And, and that's what's bringing us here today. So again, the target projected their beliefs on me
and the meaning of my tweet, even though there was no correlation there, there was nothing to,
to draw conclusions from. Okay. And so blindsided by rage, meaning that I had no idea that I,
that this was going to be happening. Um, but I asked myself, could I have seen this coming?
Could I have predicted that maybe what I tweeted, um, would result in a certain outrage from not
just a certain demographic, but from maybe specific people. And this is still the work in
progress. So, um, who are we targeting, um, with, with what we're working with today? Well, as,
as stated in the last slide, we're working with political opponents or, uh, we're talking about
political opponents and not because we're trying to be political, but just because, um, this is a
use case, uh, for, for what we're doing here. This is a scenario. Um, corporate marks, we could
use what we're going to do today for, you know, spear phishing or, uh, maybe password lists or
some sort of targeted, um, execution of, of an attack. Um, cyber bullies, you know, there are
plenty out there and wouldn't you want a leg up if you knew the type of vocabulary that they were
using, uh, the, uh, way in which they tweet, the sentiment behind their tweets, and maybe you could,
uh, you know, work for a better outcome, uh, against your cyber bullies because yes, they are
definitely out there. Now, sadly, I have to address this. Um, this also has some stalker interests.
Um, but remember what I said before, and in my previous research, stalker interest, this is bad.
It can be used for stalking. I do not condone it. I, I think we've, we've crossed that already. Yes.
Okay. So what are we talking about? Well, everybody knows Twitter is a gold mine for OSINT and recon.
Okay. Um, and, and, and it's not a secret. It's, it's very well known and it's, it's that way by
design. Um, not everybody has a private account and, and it kind of ruins the fun if you are on
a private account. So, um, Twitter, we know as a publicly shared sentiment. If you're going to put
it on Twitter, you have to expect that, uh, it can be analyzed. It can be scraped. It could be mined.
It could be, um, used for research such as today's research. So what we're doing though is we're
making an at-a-glance picture of what this profile, a target profile, um, is about, what it's all about.
So, um, you know, the term at-a-glance, what I'm talking about here is instead of, instead of
having to scroll through an entire timeline to understand what this particular profile is all
about, what if we could just take a look at a dashboard or, or just one screen and understand
the sentiment of that, uh, Twitter profile? That's, that's something that, uh, I think would be very
valuable. Why? Why are we doing this? Well, um, because I can, because we can. We're hackers, uh,
and, and this is what we do. Um, we look at something, we look at a problem, we look at
maybe a scenario that has recently happened to us, and we say, okay, I know how to fix that, or
I know what I can do about that. So yes, because I can, because we can. Remember, we're hackers.
All right, and we also want to do this because, um, this gives us an insight into tendency of
the profile. Okay. Uh, again, and the goal is here to, to find maybe quicker red flags, maybe catch
that outrage, um, before, uh, it bubbles up and, and, and blindsides us. Okay. And of course, uh, as with
anything, we could use this for later weaponization. Uh, we could turn this into something that could
be an offensive tool if needed, of course. Uh, we're talking about password lists, uh, further social
profiling, uh, maybe psychological profiling, uh, and troll bots, you know, um, maybe they could
have a conversation with themselves. Now, how are we going to do this? Well, we're going to look at a
couple of things. Uh, we're going to track a small sample size of the, uh, recent tweets in the
timeline and an ongoing sample, of course, as we add that programmatically. And we're going to be
doing our analysis on, uh, word frequency, word choice, uh, retweet frequency, um, and hashtag usage.
That's actually, uh, of course, as we know, it's pretty important. Okay. All right. So let's take a
look at the code. But before we do a couple of notes here, um, and you can play the game at home
actually. So now by the time that you're watching this, um, this should be available and public for
you and your consumption. So you can follow along with what I'm doing here, uh, in the next few
slides. So, uh, I'll go ahead and give you a minute to, to take that, uh, link down. Um, and we'll go
from there. Actually, there's a couple more, uh, points here. So, uh, it's just basically my GitHub
handle and, uh, you know, the, the name of the project, Twitter word frequency. Um, now we scraped, I
scraped, um, this, uh, information with Python and it's all Python. All of this is Python. Um, but I
scraped with, uh, an actual Python file. Um, and then I use a Jupyter notebook for more, um, analysis.
And there's a reason for that as well. And we'll see that in the next couple of slides. Uh, in the
background here, you'll see your code snippet of the actual script, uh, which pulls the timeline.
Um, now there's a couple of things here. It either, um, checks to see if the timeline has already been
saved on your machine. And if it hasn't been saved on your machine, it then goes out, uh, using the
Twitter API, um, to grab, uh, the latest, uh, timeline, uh, of your subject or of your target. Okay.
Now in the foreground here, you're going to be seeing, um, uh, a snippet from the Jupyter notebook.
And I used the Jupyter notebook because, um, maybe I wanted to run a little snippet of the code
without running the entire script. So a lot of the code is going to be the same. Um, but with, uh,
with Jupyter notebook, you know, and I'm sure everybody who's in data science already remember
I'm not, um, you, you already understand that you can take these pieces of, uh, or snippets of code
and just run those individually and change things as new data becomes available to you.
All right. So let's talk now about the analysis. Okay. So this first analysis is actually,
um, myself. This is my own, um, analysis on, on my handle at Chenbox. Um, so on the left side,
uh, you'll see a couple of the words here that I've used on a regular basis, of course,
with, with the most current recent events, you see mask is used there six times, uh, Twitter.
Okay. Cause I've, I've had opinions about Twitter on Twitter. Okay. So that's, that's there. Um,
now you'll notice that these red blotched out, uh, pieces are actually, um, these are Twitter
accounts. Okay. Um, and, and how often I've referenced these Twitter accounts. Uh, so this
can be a way that we can, again, we're talking about at a glance, um, we can see, um, who we're
looking at, um, who, uh, we're associating with, uh, quite quickly and right, right up front.
Okay. Um, on the, in the middle, on the, on the top side, you'll see, um, I have a red dot right
next to RT, RT, medium course retweet. And you'll see that my retweet frequency is about 54 times
in a 200 tweets sample. That's what that means. Remember this is out of, out of 200 tweets, um,
200, uh, tweets, uh, 54 of those were retweets of other people. And of course, uh, all these
Twitter accounts, um, that are blotched out. Um, I think it would be safe to assume that if I'm
not mentioning somebody, um, I'm definitely retweeting them. Um, at the bottom, you'll see,
uh, 2638, which is the word count associated with the last 200 tweets sample of my own personal
account. Uh, I'm using mine as a base, as a baseline. And on the right side, um, you'll see,
uh, hashtags and of course it's frequency. And at the bottom is the, um, association with the
histograms, right? So it's a histogram, uh, associated with, uh, with the hashtags, uh,
there at the top. And there's, there's my handle. So you know that this particular, uh, slide was
for, for my, uh, own profile analysis. Uh, okay. So, um, remember that person that I said got kind
of angry with my vague tweet? Well, this is, uh, this is the analysis slide of, of his account.
So, um, you'll see that there's a couple of differences obviously, um, because we're not
exactly the same. Um, on the left side, we'll see, uh, a couple, he's, he's mentioning a couple of
people who, um, are blotched out, um, not as frequently as maybe I do, but you know, these,
these profiles are in, in the scan that I've seen. Uh, you'll see his retweet frequency is 178.
So, so out of 200, out of a 200 tweet sample, we're talking about 178 of those, uh, being retweets,
retweets. Um, and, um, I also hit the, uh, a red dot on the word I, because I was just kind of curious
as to see, um, how often somebody is, um, is using I, uh, in a sentence. I'm sure that there's a
psychological profile, uh, based off of, um, the use of that word, right? And at the bottom, we see,
um, the word frequency, 37, uh, 88. I know that there's math that we can do on, uh, this word pro,
or on this, um, on this word count, uh, as far as, uh, calculating, you know, the types of words that
we're using, um, how big they are, how small they are, et cetera. And, and another interesting thing
about, uh, this particular analysis is, uh, the histogram on the right side, um, indicates that, um,
this particular individual only, uh, tweet, or only uses a hashtag, uh, once, like, uh, one and done.
So, um, I found that to be interesting. Uh, again, uh, this is for the, uh, at my target. Um, this is not
the actual Twitter account, obviously, um, but I, I thought I had to identify it somehow.
Now, lastly, I'd like to, uh, you know, I used myself as a baseline. Uh, this is the target's,
um, dashboard analysis. And, uh, this next one is, uh, actually, uh, President Donald Trump. And we'll
see, of course, a couple of, uh, of, of key features here. Uh, on the left, we don't see
the word I, uh, actually we do, but it's no more than 20 times. Now, uh, for those of us who, who
might think that, uh, President Trump is a little bit of a narcissist, uh, we might be kind of
surprised, uh, to see that he's only tweeted the word I about 20 times, uh, in a, in a, uh, 200
tweet sample. At the bottom there, we see 3,400 words. So these, these are all, you know, kind of
interesting, the same stats. Um, he's retweeted only 96 times, um, in, in the past 200 tweet
sample. Of course, uh, this piece is not so, um, surprising because we know that he uses a lot of
his own, um, words, uh, in, in his tweets. Um, but, uh, back to the left side really quick. Um, the
only blotched out account there, um, I don't know if anybody wants to take a quick guess at what that
is. I know I can't, uh, wait necessarily for your answer as, as, uh, this is the closest that we can
get to, um, audience participation at the moment. Um, but, uh, yes, if, uh, if you guessed that that
was his own account, you are correct. So he's, what I'm saying is he's mentioned himself in his
own tweets about 46 times. Uh, so that might be a, uh, an indicator of narcissism, maybe tweeting in
the third person. We'll have to analyze further for that. Uh, and on the right side, we'll see,
um, again, our histogram and, and hashtag matching. So, um, uh, President Trump has maybe tweeted a
couple of things with hashtags, but of course the recurring one is MAGA. Is anybody surprised?
I don't think anybody is. I'm not. Uh, and actually that would have been three if he didn't, uh,
add emojis to that, uh, third MAGA there at the, at the bottom. Uh, so again, my point is, um, all
of this was programmatically, programmatically, excuse me, um, all this was programmatically,
um, scraped and, and put together so that we can at a glance, take a look at what, um, a target is
talking about. Uh, and, and this is done, I understand that, you know, this could be easily
done through APIs and, uh, other research. Uh, this is my own, um, basic whiteboard. This is my
own starting from scratch, um, analysis of these targets. Myself, uh, President Trump because of
the sports, uh, betting. And of course, uh, somebody who got really mad at me for, um,
something that was really vague. Uh, okay. So now that that's, uh, an overview of the data
really quickly, let's take a look at the insights here. Uh, so some of the insights, um, it seems
like, of course, um, retweets indicate shared sentiment. Well, duh, but the question is, uh,
from who, who are we sharing a sentiment with? Uh, this might give us insights, of course, into,
um, who a person is associating with or how frequently or, um, how, how closely related
these ideals may be that they share, depending on the retweet, of course. Um, and of course,
we see narcissism in hashtags. So, uh, you may see, uh, a lot of, um, a lot of your, your own,
actually, okay, so there's a couple of things here. Um, I did forget to, uh, include, um,
uh, a screenshot that I have of my target, uh, tweeting himself, like, his own handle, um,
in, uh, in the hashtags. So, it's hashtag his own handle, um, and I, I found that to be very
narcissistic, um, kind of interesting, um, but that's something that I wanted to, uh, piece in
there. So, I do apologize for not having that screen cap in there, uh, but I found that to be
interesting, and that's why I put that in, in, in here in the insights. Um, and, uh, repeating
hashtags to me, of course, again, this is, this is kind of obvious when it comes to somebody who's
a regular Twitter user. Um, it depicts brand or focus, you know, and of course, we see that with,
with, with repeating hashtags. Um, now, I've noticed that word frequency, um, the word
frequency count significantly drops off after stop words. Um, so, you know, I have a rudimentary,
uh, introduction to, uh, natural language processing, so I understand a little bit about
tokenization and, um, what stop words are. Stop words being, like, uh, in the very small, um, words
that kind of glue the English letter, uh, language together, right? Um, and so, again, after, after
those words are scraped off the top, the, the frequency of those words, of other words, really
drop. And, of course, uh, for those who are linguists, may already know that. Um, now, um,
obviously, with what we've, uh, with what I've presented today, we know that descriptive
statistics are as far as we can get, um, without any sort of, uh, further, um, analysis with better
tools. Better tools being, of course, uh, neurolinguistic, uh, natural language processing,
maybe some other, uh, types of, uh, data science tools and, and, uh, techniques. And so, that is
where this work in progress is going to be heading next. Okay, and so, speaking of which, uh, where
do we go from here? Okay, well, uh, of course, I want to clean up the data collection, data
collection methods right now. It is pretty rough, and I'd like to turn that into something that is,
uh, uh, easier to read and, and followed very, uh, very easily. Now, better data visualization.
Um, I quickly grabbed screenshots together, um, but, of course, I'd like to incorporate word
clouds and other charting, maybe better charting, right? Um, dashboard of data. I'd like to clean
that up so that, again, the, the whole goal is an at-a-glance, uh, for, um, for a Twitter analysis.
Okay, uh, now, what can we do with this? Well, we could talk about, uh, profile scoring. So, for
instance, um, leanings, whether it's a sentimental leaning, a, uh, political leaning, a corporate
leaning, uh, I'd like to see if we can, uh, we can understand that, um, at, at a quick look. Um,
the potential for extremism, um, if, if somebody is getting mad at you for, uh, a tweet that was
vague enough for them to project, um, meaning onto it, um, then, then what is the potential for
somebody to go kind of a little bit more off the rails? That's something else that I'd like to look
at. Um, and, of course, um, ongoing, uh, me being a psychology buff, um, you know, ongoing psychological
profiles and red flag notifications. If there's a red flag with anything that we see here in, in
their timeline, um, can we detect that? And, of course, um, I am, I'm almost certain that there's
already research out there. I'd like to continue, uh, looking at that myself and, and, and building
from there. So, features to incorporate. I've said it a lot already. Um, natural language processing.
I think that'll be a very, uh, a very good thing to add in here. Uh, sentiment analysis, also part of,
part of that space. Uh, a mirror bot. Uh, real quickly, what I want to talk about there is, is, um,
if we know how our target speaks, can we make a bot that kind of talks to themselves? I think that
would be kind of funny to watch. Um, hopefully it doesn't get anybody banned, but I think it'll be,
again, it'll be funny to see, just as a funny project. This is all just, it's just jokes, folks.
It's just jokes. Um, now, uh, I want to build off of the prior research that, uh, I, myself and, and
the community has done, uh, including my, my stalker research and, and anti-stalking research
and anything else that the community has done, I'd like to incorporate in what I'm doing now.
Um, and even though we've, even though today I've only shown, uh, three profiles, I'd like to do a
quick, uh, mass script, a mass scripted analysis of maybe everybody that is following me or
everybody that I'm following, uh, and just see, um, what kind of results, um, are in their timeline.
Remember, these are public timelines. And of course, um, I don't want to hit any rate limits,
so we'll, we'll do it slowly. Uh, some, some future questions that are on my mind when it
comes to, uh, this is, uh, can I get a target to talk to themselves, uh, and agree or disagree?
I think that'd be funny. That's, that's what I was mentioning with the, the mirror bot. Um, can a
person have a conversation with themselves and disagree on their own maybe extremist points or
non-extremist points, you know? Uh, next question is, uh, you know, can outrage be predicted? You
know, that's something that I, that's something that I've brought up, um, uh, prior or earlier in,
in this talk as well. Um, could that be predicted based off of what is in the current events in the
news, uh, and what's going on globally, uh, at the moment? Uh, next one. Of course, uh, any
questions that, uh, the audience may have for me, I would definitely want to put that into this
project. So, uh, to summarize, who said nothing came, uh, nothing good came out of sports betting?
I think, uh, I think there's, uh, some benefit here. I'm glad that I went down this rabbit hole.
Uh, now sharing vague sentiments can, can still make frenemies. Uh, you know, apparently I've,
I was not expecting that even with, uh, today's climate, but hey, it is what it is. Um, now
remember, don't get mad. Uh, you know, don't get mad. Don't feed the outrage. Don't get,
don't, you know, don't send the rage back. Just get productive. Take what you have and
turn it into research. That's been, that's been my guiding light for the past five years,
maybe longer. Um, start somewhere. Like I said, this project is definitely a work in progress,
um, but I'm going to start somewhere. And of course we built, uh, okay. So fork me,
there's the link again, the GitHub link. If you didn't get it before, um, Twitter word frequency
and, Oh, sorry. I'll give you a couple of seconds on that one. A couple more and resources. So the
top link is actually, uh, another, uh, uh, it's college speech on, um, natural language processing
and the, and how that goes. Um, and then of course, um, Oh, these next two, uh, these next
two weeks are very good. I'll let you check that one out. Uh, because, uh, those are actually,
uh, kind of in line with what I'm trying to do, but on a massive scale for myself,
like I'm trying to do this on a massive scale. Uh, what these last two weeks are doing specifically
for Donald Trump's, uh, tweet account. Uh, thank you very much. And that concludes my talk.