1
00:00:01,180 --> 00:00:02,760
How's everyone doing?

2
00:00:03,360 --> 00:00:04,400
Good.

3
00:00:04,600 --> 00:00:08,860
Just curious if this is, who here, is this your first DEF CON?

4
00:00:09,500 --> 00:00:10,540
Oh.

5
00:00:11,680 --> 00:00:13,700
Oh, look around.

6
00:00:14,040 --> 00:00:17,740
For those of you who this is your first DEF CON, welcome.

7
00:00:17,760 --> 00:00:18,660
Welcome.

8
00:00:20,000 --> 00:00:32,040
Each and every year, well except for last year when we run talks, we ask this question and the numbers are usually anywhere between like 70 to 80 percent.

9
00:00:33,080 --> 00:00:35,460
So the numbers are pretty consistent.

10
00:00:35,960 --> 00:00:40,200
So for those of you, if you're, you know, this is your first DEF CON, welcome.

11
00:00:42,640 --> 00:00:53,660
It can be a little overwhelming, let's be honest about it, but my biggest advice is take every opportunity that you can.

12
00:00:54,200 --> 00:00:58,440
This is an opportunity for you to learn as much as you can.

13
00:00:58,460 --> 00:01:04,100
And also do yourself a favor, learn from other people, including strangers here as well too.

14
00:01:04,160 --> 00:01:10,460
You may never, never know that person that you speak to, that could be a lifelong connection.

15
00:01:10,460 --> 00:01:19,360
Actually speaking of, speaking of, the next speaker and talk about lifelong connection, is how I got to meet you.

16
00:01:19,760 --> 00:01:22,180
And uh, you'll be on in two minutes.

17
00:01:23,220 --> 00:01:31,400
So, make sure you enjoy DEF CON and uh, learn as much as you can.

18
00:02:07,140 --> 00:02:09,300
Sacrificial lamb.

19
00:02:39,120 --> 00:02:40,480
All right.

20
00:02:41,120 --> 00:02:43,040
Let's get started.

21
00:02:43,040 --> 00:02:47,200
Hello everyone, good, now uh, yes, uh, good afternoon everyone.

22
00:02:47,200 --> 00:02:48,620
Good afternoon.

23
00:02:49,220 --> 00:02:54,200
Welcome to DEF CON 31 and welcome to Talks.

24
00:02:54,200 --> 00:02:57,300
Uh, presented by the Packet Hacking Village.

25
00:03:00,280 --> 00:03:04,540
Since 2013, been a long time.

26
00:03:05,260 --> 00:03:10,620
Uh, the Packet Hacking Village has presented a track of talks.

27
00:03:11,120 --> 00:03:26,200
So, and the whole premise is, people who go to a talk, uh, and you walk away, you should be able to take away something, uh, immediately to use after the talk.

28
00:03:26,200 --> 00:03:29,520
So that's the whole premise of a Packet Hacking Village talk.

29
00:03:29,600 --> 00:03:34,220
And each and every year they're well attended and uh, first want to say thank you.

30
00:03:34,220 --> 00:03:41,200
Thank you for, for being here for, actually our final Packet Hacking Village talk for today.

31
00:03:41,520 --> 00:03:55,940
And it is my honor and privilege to introduce someone who has given a few talks in the past and quality, world class talk to the Packet Hacking Village for years, Mike Rago and Chet Hosmer.

32
00:03:56,700 --> 00:04:04,100
Um, so, I'll let Mike give, uh, a little background about, about him.

33
00:04:04,180 --> 00:04:08,260
Uh, and his talk is on open source, uh, uh, intelligence.

34
00:04:09,080 --> 00:04:14,600
Uh, open, open source intelligence and, uh, physical, physical threat intelligence.

35
00:04:14,740 --> 00:04:18,600
So, my pleasure to introduce an old friend, Mike Rago.

36
00:04:19,920 --> 00:04:20,920
Thanks so much.

37
00:04:22,060 --> 00:04:26,080
Thanks Ming, really appreciate it and super excited to be back.

38
00:04:26,080 --> 00:04:32,200
Uh, thanks for having me and huge shout outs to the Packet Hacking Village, uh, for everything they do over the years.

39
00:04:32,300 --> 00:04:38,720
My name is Mike Rago, um, my, uh, good friend Chet Hosmer, uh, couldn't be here today.

40
00:04:38,720 --> 00:04:43,260
Uh, some family related stuff, we wish him well, send our thoughts and prayers.

41
00:04:43,500 --> 00:04:48,040
Chet and I, uh, co-present quite a bit at a variety of conferences around the world.

42
00:04:48,040 --> 00:04:55,880
So, uh, we share a lot of the same, uh, information in terms of our research and, uh, look forward to sharing that with you.

43
00:04:56,020 --> 00:05:03,180
This particular research and title of this presentation is OSINT for Physical Threat Intelligence.

44
00:05:03,200 --> 00:05:11,420
We know that a lot of people, uh, in the security community today are leveraging OSINT intelligence in a plethora of ways.

45
00:05:11,420 --> 00:05:21,700
Uh , whether it be for a person of interest or an organization of interest, whether it be for pen testing and going out there and gathering any public information they can find.

46
00:05:21,700 --> 00:05:35,740
Our particular focus of research for the last few years is physical threat intelligence and we'll talk a little bit more about what that means and what it applies to in the real world and how organizations and people use this today.

47
00:05:40,200 --> 00:05:44,120
So, Chet and I, as I mentioned, have collaborated for many years.

48
00:05:44,120 --> 00:05:45,800
We've written books together.

49
00:05:45,880 --> 00:05:51,620
And we both have been professors with the University of Arizona, aside from our full-time jobs.

50
00:05:51,620 --> 00:05:56,240
And it's been a great research ground for a lot of the things we do over the years.

51
00:05:56,280 --> 00:06:03,880
Furthermore, we've presented at a variety of conferences, including DEF CON, HAC CON in Norway, and many other conferences.

52
00:06:04,640 --> 00:06:14,540
In addition, a lot of this research we collaborate on allows us to work with a lot of organizations and understand a lot of their needs.

53
00:06:14,540 --> 00:06:24,360
So as we go through this, there will be a lot of applied aspects and real world scenarios in how we use this physical threat intelligence for situational awareness.

54
00:06:27,340 --> 00:06:30,040
Chet has authored even more books than I have.

55
00:06:30,040 --> 00:06:33,720
And again, just a variety of research over the years.

56
00:06:33,720 --> 00:06:39,800
In addition, he does a lot of coding and runs pythonforensics.org.

57
00:06:39,800 --> 00:06:44,180
So Chet does a lot of research and a lot around Python.

58
00:06:49,340 --> 00:06:54,800
So I think most of us here are familiar with open source intelligence, or OSINT.

59
00:06:54,800 --> 00:07:03,980
Again, what we're trying to do here is apply that wealth of information to meaningful intelligence that we can apply in a physical way.

60
00:07:03,980 --> 00:07:07,740
Whether that be concerns around business risk.

61
00:07:07,740 --> 00:07:14,880
Are we building a store in a foreign country, and are we concerned about someone targeting that store?

62
00:07:14,880 --> 00:07:23,100
Either while it's being built, or the first day it opens with some kind of flash mob that destroys the store, or things like that.

63
00:07:23,100 --> 00:07:31,200
Or maybe it applies to an organization that provides help for people around the world.

64
00:07:31,200 --> 00:07:36,140
Our thoughts and prayers go out to Maui and everything that's occurred there.

65
00:07:36,140 --> 00:07:44,580
But there's a wealth of data and pictures and information people are posting that can be leveraged to determine where to mobilize help next.

66
00:07:44,580 --> 00:07:47,680
And really provide that help to people that are in need.

67
00:07:47,680 --> 00:07:51,400
So this data can be applied in a variety of a lot of ways.

68
00:07:51,420 --> 00:07:57,460
Furthermore, it can be very cutting edge in terms of collecting it in a streaming format or live.

69
00:07:57,460 --> 00:07:59,520
So it can be very actionable.

70
00:07:59,520 --> 00:08:05,460
We'll talk about the overwhelming wealth of data out there and how to better curate that.

71
00:08:05,460 --> 00:08:12,960
So you don't spend 99% of your time trying to find those needles in the haystack that are most helpful for situational awareness.

72
00:08:12,960 --> 00:08:17,860
But actually flip that model upside down to pre-curate that data.

73
00:08:17,860 --> 00:08:21,840
And make it far more actionable in a very timely manner.

74
00:08:21,840 --> 00:08:33,840
As we mentioned here in the slide, we also want that to be a key decision making piece of information that we can use.

75
00:08:33,840 --> 00:08:39,800
But it doesn't displace information you may receive via private channels or other channels you have.

76
00:08:39,800 --> 00:08:43,500
And how you collect that intelligence and use it for situational awareness.

77
00:08:43,500 --> 00:08:46,700
Bottom line, it's meant to complement that data.

78
00:08:48,160 --> 00:08:54,400
The applied aspects of it could be related to threat assessments and business risk.

79
00:08:54,400 --> 00:08:58,760
Could also be related, as I mentioned, to disaster response.

80
00:08:58,920 --> 00:09:04,920
In this particular talk, we expose a lot of the research we've done around the Ukrainian war.

81
00:09:04,920 --> 00:09:13,400
To actually identify different attack patterns, paths that troops are moving, what things are being destroyed.

82
00:09:13,400 --> 00:09:20,820
How to profile tanks and other equipment out in the Ukraine and other areas.

83
00:09:21,080 --> 00:09:27,040
And it also can be used in more geopolitical type situations as well.

84
00:09:27,240 --> 00:09:29,640
So this data is becoming more and more helpful.

85
00:09:29,960 --> 00:09:34,880
But again, one of the big concerns may be around misinformation and disinformation.

86
00:09:34,880 --> 00:09:36,580
And we'll talk about that too.

87
00:09:39,060 --> 00:09:43,060
Lastly, before we get into how we actually perform this.

88
00:09:43,060 --> 00:09:48,040
And some of the Python scripts we've created for the collections in weeding through that data.

89
00:09:48,040 --> 00:09:51,520
Is, as I mentioned, can we trust this data?

90
00:09:51,860 --> 00:09:57,620
One of the things that we do is by collecting it in a live and streaming format.

91
00:09:57,620 --> 00:10:03,780
But furthermore, pinpointing a geo and collecting what we call ground truth.

92
00:10:03,780 --> 00:10:09,400
For a bounding box or a radius from which those posts are emanating from.

93
00:10:09,400 --> 00:10:11,940
We're not collecting data from around the world.

94
00:10:11,940 --> 00:10:14,140
We don't want data from around the world.

95
00:10:14,140 --> 00:10:17,220
It's littered with misinformation and disinformation.

96
00:10:17,500 --> 00:10:21,540
So you can leverage a technique that we'll talk about more here shortly.

97
00:10:21,540 --> 00:10:28,100
Where you can create a radius or a bounding box around a city such as Kiev within the Ukraine.

98
00:10:28,100 --> 00:10:33,700
You could do it around an area that you know has been impacted by an unfortunate disaster.

99
00:10:33,700 --> 00:10:40,460
This will give you much more legitimate posts that are, again, originating from that area.

100
00:10:40,460 --> 00:10:42,220
Giving you that ground truth.

101
00:10:42,220 --> 00:10:47,340
And we have found, bottom line, that that data is increasingly helpful.

102
00:10:47,340 --> 00:10:51,280
And weeds out more than 99% of the misinformation.

103
00:10:51,280 --> 00:10:57,500
So in getting to what we mentioned earlier about being proactive and pre-curating that data.

104
00:10:57,500 --> 00:11:00,520
So you don't have a huge data warehouse of useless data.

105
00:11:00,520 --> 00:11:04,960
This is one of the preliminary things you can do so you don't end up in that scenario.

106
00:11:07,530 --> 00:11:13,250
So let's talk about the tradecraft and then we'll talk about our research and a lot of the interesting findings.

107
00:11:17,350 --> 00:11:21,890
So conducting this involves a lot of data sources.

108
00:11:21,890 --> 00:11:25,410
It may be social media, it may be other sources as well.

109
00:11:25,410 --> 00:11:33,150
And understanding the chemistry of these different social networks and other mediums that provide this information.

110
00:11:33,190 --> 00:11:36,210
For example, if you're going to take a look at Discord.

111
00:11:36,410 --> 00:11:41,950
On Discord there are a variety of servers and channels that are focused on the Ukrainian war.

112
00:11:41,950 --> 00:11:48,630
That are focused specifically on nothing but tanks and other things that people are passionate about that they're posting.

113
00:11:48,630 --> 00:11:51,690
A lot of those posts are also non-biased.

114
00:11:51,690 --> 00:11:55,690
So there are pro-Ukrainian and pro-Russian posts going on.

115
00:11:55,690 --> 00:11:58,550
But again, we want to collect ground truth.

116
00:11:58,550 --> 00:12:04,790
So we want stuff originating from that area, not across lines, not from other portions of the world.

117
00:12:05,450 --> 00:12:13,370
In addition, that helps with countering all that counterintelligence, misinformation and disinformation.

118
00:12:13,370 --> 00:12:24,530
So when we perform these analysis techniques, we also are profiling and understanding which of these accounts over time are actually spreading misinformation.

119
00:12:24,610 --> 00:12:35,090
We'll talk about how we profile those so that we understand the top posters or people that may be posting legitimate information from that ground truth.

120
00:12:35,090 --> 00:12:40,370
And people who are basically muddying the waters with that misinformation.

121
00:12:40,370 --> 00:12:47,940
So in this case, we want less data, not more.

122
00:12:47,940 --> 00:12:53,940
Most people will get on social media, leverage a hashtag or a keyword and just start collecting a whole ton of data.

123
00:12:54,060 --> 00:13:05,340
If you talk to anyone that basically takes that and tries to automate that, there's just so much data collected, you end up spending too much time analyzing all that data.

124
00:13:05,340 --> 00:13:08,660
You can actually get ahead of the curve in a variety of ways.

125
00:13:10,800 --> 00:13:16,740
Also, you know, it was a long time ago when I started out as a UNIX system administrator.

126
00:13:16,760 --> 00:13:22,540
You know, my boss told me, listen, there's a lot of things to manage here on the network at the NASDAQ stock market.

127
00:13:22,640 --> 00:13:25,140
You know, laziness breeds efficiency.

128
00:13:25,220 --> 00:13:27,520
Focus on the most important things.

129
00:13:27,720 --> 00:13:32,340
You know, try to focus on, you know, less data, not more.

130
00:13:32,340 --> 00:13:34,200
And how we pre-curate that.

131
00:13:34,200 --> 00:13:35,700
So we'll talk about that, too.

132
00:13:35,900 --> 00:13:38,360
Also, we want to respect the terms of service.

133
00:13:38,360 --> 00:13:42,140
These vary based on the website, based on the social network.

134
00:13:42,140 --> 00:13:44,660
Make sure you're familiar with what those are.

135
00:13:44,660 --> 00:13:52,600
Some of them have very important privacy policies where you're not supposed to expose the accounts from which this data may be coming from.

136
00:13:52,600 --> 00:13:56,040
Also, scraping and other things sometimes are not allowed.

137
00:13:57,140 --> 00:14:02,600
But as you interact, for example, with these social networks, there are exposed APIs.

138
00:14:02,600 --> 00:14:06,760
For example, with Twitter, those have changed quite a bit recently.

139
00:14:06,760 --> 00:14:13,380
And now also there are tiers you have to pay for to get some of that data that otherwise with a free account you wouldn't get.

140
00:14:13,380 --> 00:14:19,760
So understanding that, too, is going to help enrich your data and also ensure that you're within the terms of service.

141
00:14:23,560 --> 00:14:34,920
So as we collect based on a different methodology rather than keyword or hashtag, we actually collect by geo.

142
00:14:34,920 --> 00:14:37,560
Again, we get that ground truth.

143
00:14:37,560 --> 00:14:42,800
Data that's emanating from within that particular radius or bounding box.

144
00:14:42,960 --> 00:14:46,460
In addition, it increases the authenticity of it.

145
00:14:46,460 --> 00:14:51,760
Because we're collecting not only posts, but we want those images, videos, and even emojis.

146
00:14:51,760 --> 00:14:59,760
And we pump those emojis actually through some ML that actually helps us in an automated way understand the sentiment, too.

147
00:14:59,760 --> 00:15:04,360
Which also helps us determine whether or not this information can be trusted.

148
00:15:08,780 --> 00:15:14,040
So we created a Python script that allows us to perform this collection.

149
00:15:14,560 --> 00:15:23,680
This allows us to either pick from a particular city or location and select the bounding box or radius for that location.

150
00:15:23,680 --> 00:15:31,280
Or you can actually put in the latitude and longitude and then define the radius or bounding box for what you want to collect.

151
00:15:31,480 --> 00:15:38,300
That fundamentally performs the collection by basically eliminating all the rest of the data from around the world.

152
00:15:38,440 --> 00:15:40,780
Are we going to miss a few important things?

153
00:15:40,780 --> 00:15:41,140
Yeah.

154
00:15:41,140 --> 00:15:48,300
But what I will say, that's a lot easier than collecting data from all around the world and trying to find those needles in the haystack.

155
00:15:48,300 --> 00:15:53,160
Instead, we're weeding out all those needles and we're getting roughly 90% of them.

156
00:15:53,160 --> 00:16:01,060
And that is far more actionable and timely in terms of leveraging this data for all of those applied aspects we mentioned earlier.

157
00:16:02,820 --> 00:16:08,560
We can additionally put in keywords and hashtags, even in other languages.

158
00:16:08,560 --> 00:16:11,480
And this will help further curate a lot of that data.

159
00:16:11,480 --> 00:16:16,840
There may be stuff within a radius around Kiev that have nothing to do with the war.

160
00:16:16,840 --> 00:16:22,960
So if we're looking to collect stuff related to the Ukrainian-Russian war, there may be stuff that's irrelevant.

161
00:16:28,280 --> 00:16:32,700
As we perform the collection, then, we're collecting each post.

162
00:16:32,700 --> 00:16:36,420
It could be a tweet, it could be a post, it depends on the social network.

163
00:16:36,420 --> 00:16:40,520
And collecting those images and collecting those videos.

164
00:16:40,700 --> 00:16:45,800
We'll talk next how that stuff is really important for cross-correlating that data.

165
00:16:46,200 --> 00:16:50,260
We also take a look at the accounts that are posting this information.

166
00:16:50,260 --> 00:17:02,620
So as we categorize them and determine who are the top people that are posting legitimate and very helpful information, we can actually focus on those and eliminate a lot of the other accounts.

167
00:17:02,620 --> 00:17:15,740
What we have found is, of the data that normally emanates from a bounding box, that there are about 10 to 20 people that dominate those posts with a wealth of positive, helpful information.

168
00:17:15,740 --> 00:17:21,200
But there are also about 10 to 20 that are littering it with misinformation.

169
00:17:21,560 --> 00:17:31,120
But again, knowing these from the get-go, as you're collecting this data, and weeding that out, again, helps curate the data in a very automated way.

170
00:17:37,910 --> 00:17:42,790
So we've gone, we've collected the data, we've done this in a very automated fashion.

171
00:17:42,790 --> 00:17:46,310
Again, leveraging geolocation information.

172
00:17:46,310 --> 00:17:56,630
And leveraging this in a chronological way, so that we have a timeline as well of the events, and we can use this for predictive analysis.

173
00:17:56,770 --> 00:17:59,370
So let's talk about how we leverage this data.

174
00:18:00,730 --> 00:18:08,150
Now, as we look through the data, we can see all the posts, all the tweets, and furthermore, any of the information and referring links.

175
00:18:08,150 --> 00:18:13,710
If it may point to Twitter, if it may point to Discord, if it may point to Telegram.

176
00:18:14,570 --> 00:18:21,330
And as we look through the data that we've collected, we start to find some really interesting information.

177
00:18:21,450 --> 00:18:30,190
If you're using some form of computer vision in your favorite cloud vendor, you can automate the analysis of these images, and we've done that.

178
00:18:30,450 --> 00:18:40,090
So if you're using optical character recognition, you may start to notice here on the screen that there are serial numbers on these military vehicles.

179
00:18:40,090 --> 00:18:50,490
And by knowing we collected that information for that vehicle, at that location, at that point in time, we know that that vehicle was located there.

180
00:18:50,570 --> 00:19:09,750
But if we're monitoring multiple locations around the Ukraine, for example, Kyiv, that they targeted first, then moved on to Kyiv, if we saw that military vehicle in Kyiv, and then later Kyiv, we're starting to see a motion on the map, right, of where this equipment is going,

181
00:19:09,750 --> 00:19:13,170
where the troops are going, and potentially where they may go next.

182
00:19:13,170 --> 00:19:20,530
Really interesting sliver of information I'll show you later, that was something that we found that we completely did not expect.

183
00:19:20,830 --> 00:19:25,030
So there's a lot of related data here across these devices.

184
00:19:25,030 --> 00:19:38,530
If you're using optical character recognition, whether it be an image or a video, it'll go ahead and scan all those images, and actually gave us chronologically, in time and location-wise, where that equipment was.

185
00:19:41,540 --> 00:19:47,700
And you may or may not have heard the terminology then, chrono-locationally.

186
00:19:47,700 --> 00:19:52,200
So we're taking a look at it chronologically, but also locationally as well.

187
00:19:52,200 --> 00:19:56,240
Kind of a new term that's been floating around, chrono-locationally.

188
00:19:59,510 --> 00:20:12,450
So to recap before we delve deeper into the data, is we're performing live streaming collection, secondly, by geo, to help pre-curate that data.

189
00:20:12,530 --> 00:20:17,990
We're not running a vacuum and collecting data from all around the world based on a keyword or hashtag.

190
00:20:18,230 --> 00:20:20,910
We're not doing that from the get-go.

191
00:20:20,910 --> 00:20:25,170
We're saying we want ground truth from this radius, from this location.

192
00:20:26,910 --> 00:20:32,490
In addition, since we're getting the location information, we can plop this on a map.

193
00:20:32,530 --> 00:20:44,210
One of the frequently asked questions we get is, how are you getting the location information then if that privacy is shut down or people don't have that enabled on their mobile devices when it's posted?

194
00:20:44,210 --> 00:20:48,950
It comes from the APIs and how these social networks are collecting the data.

195
00:20:48,950 --> 00:20:52,910
They know that it's been posted from within that bounding box.

196
00:20:52,910 --> 00:21:03,710
They won't say that that person is specifically on this street, on this corner in Kiev, or San Francisco, or Chicago, or wherever, but it's definitely within that region.

197
00:21:03,710 --> 00:21:08,010
And in this case, we don't care exactly where it's pinpointed on the map.

198
00:21:08,010 --> 00:21:13,150
We can build a one-mile radius around a region and get all that ground truth from there.

199
00:21:13,150 --> 00:21:20,950
And that's good enough for making these very important decisions in terms of mobilizing troops or mobilizing help in the event of a disaster.

200
00:21:21,950 --> 00:21:24,390
So again, we're not collecting this from the device.

201
00:21:24,390 --> 00:21:33,090
This comes directly from the social network, telling us that this person with this device posted from within that region or that bounding box.

202
00:21:37,870 --> 00:21:42,310
This takes us to another piece of really interesting information.

203
00:21:42,710 --> 00:21:45,590
A lot of activity going on in the war.

204
00:21:46,930 --> 00:21:55,590
Initially, the knee-jerk reaction is, let's collect everything we can around tanks and troops and what's being bombed and targeted.

205
00:21:55,590 --> 00:21:58,430
And that's great, and that's absolutely important.

206
00:21:59,010 --> 00:22:08,570
But when we started to kind of take a look at the data in a different way, leveraging transforms, we actually discovered a completely different narrative going on.

207
00:22:08,570 --> 00:22:11,370
And that narrative involved places of worship.

208
00:22:11,850 --> 00:22:20,930
So we found that churches, synagogues, mosques across the Ukraine were being destroyed as troops moved through those areas.

209
00:22:21,150 --> 00:22:35,550
So if we set aside all the data around what we knew about the troop movements and military equipment and things like that, we also started to see some very important trends of all of these places of worship being destroyed.

210
00:22:35,550 --> 00:22:55,370
We would not have seen that had it not been for some of the automation we'll talk about next, which is why now with the invent of AI, ChatGPT, Copilot, and those things, that you can now have a right-hand virtual person, if you will, to have you or have them look at it in a completely different way.

211
00:22:55,370 --> 00:22:58,070
And this is how we found this very important artifact.

212
00:22:58,490 --> 00:22:59,990
Why is that important?

213
00:23:01,070 --> 00:23:06,850
Well, as we started to map this out, you could use something as simple as Google Maps if you wanted to.

214
00:23:06,850 --> 00:23:09,670
You could go in here and plot these on a map.

215
00:23:09,870 --> 00:23:16,350
And in addition, with the automation, you can also include the initial post or tweet or whatever.

216
00:23:16,370 --> 00:23:18,550
And in addition, that image.

217
00:23:18,550 --> 00:23:29,930
So now as you start to bring this intelligence together, if you want to look at it on a map, you can start to see, in this case, all the places of worship that unfortunately have been destroyed during the war.

218
00:23:30,690 --> 00:23:42,930
And if you start to now look at that chronolocationally, or a chronological timeline by location, there's some very important predictions you can make here about where they might target next.

219
00:23:42,930 --> 00:23:46,230
We did that, and we found out we were 100% spot on.

220
00:23:46,230 --> 00:23:56,570
Not by tracking the tanks, not by tracking the troops, but by tracking the churches, synagogues, and mosques, all these places of worship that were actually being destroyed.

221
00:24:00,520 --> 00:24:04,200
So we take all that data and then plot that on a map.

222
00:24:04,360 --> 00:24:09,380
The captured image, the tweet, or the post, if it's from another social network.

223
00:24:10,740 --> 00:24:22,740
And what we found is, as we start to go through this, we then said, okay, we have another whole, you know, lump of data here around troop movement and tanks.

224
00:24:22,740 --> 00:24:25,300
We'll plot all that on a map, too.

225
00:24:25,300 --> 00:24:33,380
And you can do layers within Google Maps to help layer that or look at that data individually by type.

226
00:24:34,140 --> 00:24:36,840
So here, we mapped all the tanks.

227
00:24:36,840 --> 00:24:42,480
These were some of the tanks with the same serial numbers that we saw at different locations at different points in time.

228
00:24:42,480 --> 00:24:45,580
So what's going on with this particular one?

229
00:24:45,660 --> 00:24:50,240
The first one, you'll see the tank here, just outside Kiev.

230
00:24:50,440 --> 00:24:55,020
But if you then look at the next tank, it's heading towards the area of Donetsk.

231
00:24:55,020 --> 00:25:05,420
What we learned later is that attacks from the Russians then occurred, you know, from the Donetsk region, from the water, from the west side of the country.

232
00:25:05,420 --> 00:25:14,580
And basically what we're able to predict is that a lot of this military, once it had targeted Kiev, started to move in the direction of Donetsk.

233
00:25:14,580 --> 00:25:19,780
And we also started to see movement from the water hitting land targeting Donetsk.

234
00:25:19,780 --> 00:25:25,740
Bottom line, all of this military equipment and troops were converging on Donetsk.

235
00:25:27,500 --> 00:25:35,680
This tank that we show here is that piece of military equipment we showed earlier with that serial number B23.

236
00:25:35,720 --> 00:25:42,980
And we showed it first next to Kiev, then moving towards Donetsk, and then further, you know, closer to Donetsk.

237
00:25:48,450 --> 00:25:53,190
So you could leverage Twitter for this and some other social networks.

238
00:25:53,190 --> 00:25:59,690
If you go, for example, to Discord, there are some amazing servers out there with a ton of information around the war.

239
00:25:59,690 --> 00:26:06,250
As I mentioned, some people are just passionate about planes or tanks or you name it.

240
00:26:06,250 --> 00:26:15,130
So there is a wealth of data out there that we leverage with this to cross correlate the data we're getting from multiple social networks.

241
00:26:15,130 --> 00:26:19,850
Again, whether it be leveraging the APIs, we're doing this within the terms of service.

242
00:26:23,300 --> 00:26:41,920
What was really cool about that, then, is as we start to merge that data on a Google map, we can basically enable all of those layers to now look at this in the context of everything going on, whether it be military equipment, troops, churches and places of worship being destroyed,

243
00:26:41,920 --> 00:26:45,560
and many of the other activities by different categorizations.

244
00:26:46,160 --> 00:26:55,260
This definitely starts to build out a storyline chronologically to help understand what might happen next, as we showed with that piece of military equipment.

245
00:26:58,190 --> 00:27:01,710
There's a lot of different ways you can dashboard this.

246
00:27:01,770 --> 00:27:04,050
And this can be very helpful for an analyst.

247
00:27:04,050 --> 00:27:09,390
There are lots of either free tools or free trial tools you could use.

248
00:27:09,390 --> 00:27:12,310
In this particular case, I used Power BI.

249
00:27:12,310 --> 00:27:19,050
We took all the data, and with an Azure Logic app, fed this automatically into Power BI.

250
00:27:19,870 --> 00:27:24,170
Initially, I created all these dashboards for all this data.

251
00:27:24,170 --> 00:27:25,570
Why is this important?

252
00:27:25,570 --> 00:27:28,610
Well, over on the far right, you'll see a filter.

253
00:27:28,690 --> 00:27:31,530
This filter allows you to search against the data.

254
00:27:31,530 --> 00:27:46,690
You can either automate that, or if an analyst needs to investigate something, it gives you full sorting capability built right into the tool.

255
00:27:48,170 --> 00:27:53,570
In addition to this, you can also filter the tweets, filter things by emojis.

256
00:27:53,570 --> 00:27:55,990
There's a wealth of things you can do with this.

257
00:27:56,450 --> 00:28:03,730
Now, with things like Copilot or other things, is the ability to say, Hey, I want to incorporate some prompt engineering.

258
00:28:03,730 --> 00:28:05,650
I want to ask it some questions.

259
00:28:05,650 --> 00:28:12,270
And when I do, I want to say, Hey, build me a dashboard based on the intelligence you're seeing.

260
00:28:12,450 --> 00:28:22,070
And it'll build another tab with that additional intelligence from a completely different viewpoint, giving you other analytics and data points you didn't even think to look for.

261
00:28:22,610 --> 00:28:35,110
So, when you start to map it out in this way, and a lot of these tools even have a free trial, you can go in there and leverage these to get some really cool, actionable, and very short timeline type data.

262
00:28:39,130 --> 00:28:43,770
There's also a plethora of widgets built into a lot of these tools.

263
00:28:43,770 --> 00:28:56,870
What I had used within Power BI allowed me to not only do a lot of the mapping with ArcGIS, which is fed by Ursa, I believe, but also other forms of widgets like Word Cloud.

264
00:28:56,870 --> 00:29:04,410
This became really informative, too, because based on the data, there was a whole bunch of different keywords that popped up.

265
00:29:04,570 --> 00:29:06,830
So, I can see a variety of things there.

266
00:29:06,830 --> 00:29:13,970
The, obviously a popular used word in Ukrainian and Russian, and then you start to look at some of this other data.

267
00:29:16,610 --> 00:29:24,910
When I started to take a look at the top ten words, one of the ones that popped up was Oblast, which I believe is a region within Kiev.

268
00:29:25,090 --> 00:29:36,150
I had no idea that things were going on at that point in time within Kiev, but this actually told me that, through the Word Cloud, through the widget, through all this active data.

269
00:29:36,150 --> 00:29:49,450
So, again, another data point I would not have looked at by doing human analysis of all of this, but leveraging the widget and the automation in your favorite dashboarding, whether it be Grafana, whether it be Power BI, whether it be whatever, name your tool.

270
00:30:00,160 --> 00:30:04,080
So, then we start to get into extrapolation and prediction.

271
00:30:04,100 --> 00:30:18,300
We've already got some really interesting data points that we didn't even think to look for, but with the automation, with a lot of different perspectives, with the AI, with chat GBT, with prompt engineering, we got a lot of additional, very helpful data,

272
00:30:18,300 --> 00:30:20,320
and got that in a very timely manner.

273
00:30:20,320 --> 00:30:27,520
The data is automatically fading into it, and analysts can be right there doing searches, creating reports, and getting that data on demand.

274
00:30:29,480 --> 00:30:35,740
So, as I mentioned, we're able to track some of the Russian troop movements based on the data we collected.

275
00:30:36,600 --> 00:30:48,960
And, in addition, we also found certain areas, places of worship that were being destroyed, location, the time at which things were occurring, and other interesting data points.

276
00:30:50,640 --> 00:30:59,780
The question here, then, was, if I look specifically at these places of worship being destroyed, can I predict where the troops might be next?

277
00:30:59,780 --> 00:31:10,700
So, if I'm on the defense, or counter, and I need to figure out where to mobilize my teams, we were able to actually use this data to do a variety of prediction.

278
00:31:10,700 --> 00:31:23,160
Up at the top, you can see Chekerev, and, before that, we actually had Kikarev, where the troops were moving based on the chronological timeline, and we could see they were moving closer and closer to Kiev.

279
00:31:23,160 --> 00:31:27,740
It was pretty much expected they were going to target Kiev, but these data points proved that.

280
00:31:27,740 --> 00:31:34,820
And it gave us some really interesting data points to actually say, yep, they're on the move, they're all converging on Kiev.

281
00:31:35,400 --> 00:31:46,360
Then you'll see a splinter of, you know, some troops in a completely different direction, up one of the main highways, in the eastern direction.

282
00:31:46,360 --> 00:31:53,400
And we showed you earlier where some of the other troops splintered off and started going towards Donetsk, where they also converged.

283
00:31:53,540 --> 00:32:01,720
So a lot of this data definitely can be used for predictive extrapolation type of analysis, and we proved it.

284
00:32:01,980 --> 00:32:07,100
But we would not have known to look at just places of worship to get that data.

285
00:32:07,100 --> 00:32:16,400
So the automation, the prompt engineering, and a lot of these things are what actually got our laser focus on some of these other areas we weren't focused on.

286
00:32:27,730 --> 00:32:32,330
So a lot of us are starting to mess around with things like chat GPT.

287
00:32:32,650 --> 00:32:46,450
And what's kind of nice about it is, a lot of vendors have taken this and basically isolated it in a tenant for you, so that you're running it only against your data, and not exposing that data or that analysis to the external world.

288
00:32:47,210 --> 00:32:56,410
This prompt engineering, as I mentioned, can allow you to put in a variety of different prompts to look at just this data in a lot of unique ways.

289
00:32:56,670 --> 00:33:00,430
Tell me, what are the top trends we're seeing at this point in time?

290
00:33:00,430 --> 00:33:01,190
What are they?

291
00:33:01,190 --> 00:33:02,730
Are they places of worship?

292
00:33:02,730 --> 00:33:04,350
Are they troop movements?

293
00:33:04,350 --> 00:33:14,570
Are they posts from people that are suffering damage and need help on the ground within this geolocation, within this bounding box?

294
00:33:15,870 --> 00:33:27,110
Through this, we basically said, let me go ahead and put in a variety of questions, and then once I got this honed in on what I want, I basically can say, build me a dashboard.

295
00:33:27,210 --> 00:33:35,950
So whereas the previous dashboard I had was built manually by myself, I can now have the AI and the automation build it for me.

296
00:33:35,950 --> 00:33:38,770
And when it did, it looked completely different.

297
00:33:38,790 --> 00:33:39,910
And that was really cool.

298
00:33:39,910 --> 00:33:46,030
So we have that as a second tab, looking at all of this data in a different transform in a different way.

299
00:33:47,830 --> 00:33:52,870
Bottom line, it helped us identify a plethora of things we had missed.

300
00:33:57,710 --> 00:34:00,070
So how can we use this methodology?

301
00:34:01,030 --> 00:34:05,090
It can be applied to not only war, but disaster response.

302
00:34:05,090 --> 00:34:10,290
There are a lot of organizations that want to know, hey, we're having all of these California rains.

303
00:34:10,290 --> 00:34:12,390
There are dams building up with water.

304
00:34:12,390 --> 00:34:17,530
People are starting to post that the water is going over the dam and destroying some homes.

305
00:34:17,530 --> 00:34:24,270
Also help us find people in need within that geobounding box of that region that we weren't focused on.

306
00:34:24,270 --> 00:34:34,850
Someone just posted something about their home being destroyed and there's a bunch of flooding from a river that overflowed or the things that were going on in Sacramento.

307
00:34:35,010 --> 00:34:42,990
So this will obviously give you a variety of really interesting data points to help with the situational awareness and mobilization of help for these individuals.

308
00:34:44,230 --> 00:34:48,210
Another thing that we frequently do is the account profiling.

309
00:34:48,210 --> 00:35:02,470
As I mentioned, we basically whittle this down so that 50% of the data we're getting are from the top 10 to 20 people that are posting on that social network versus the whole gamut of that entire region.

310
00:35:02,470 --> 00:35:19,110
That provides us about 50% of the data and also helps us further weed out misinformation and disinformation because we also profile those accounts that are spreading misinformation or disinformation and profile those top 10 accounts as well so we can say exclude that data.

311
00:35:19,110 --> 00:35:21,490
I don't even want that included with it.

312
00:35:26,160 --> 00:35:28,760
So I hope you found this talk informative.

313
00:35:28,760 --> 00:35:32,000
I'll hang out for a little bit if you have any questions about this.

314
00:35:32,000 --> 00:35:33,660
Happy to chat about it.

315
00:35:33,660 --> 00:35:35,020
Hope you enjoyed it.

316
00:35:35,020 --> 00:35:36,420
We spent a few years.

317
00:35:36,420 --> 00:35:42,440
If you're interested in a copy of the research report, it's available at our site listed there.

318
00:35:42,780 --> 00:35:43,820
SilentSignals.com.

319
00:35:43,820 --> 00:35:45,320
You can download it for free.

320
00:35:45,420 --> 00:35:48,820
And see a deeper analysis around the research report.

321
00:35:48,820 --> 00:35:49,600
It's free.

322
00:35:49,820 --> 00:35:54,820
And we're also looking for people at any time that may want to get involved with the research.

323
00:35:54,920 --> 00:35:57,460
So if you are interested, we have an email there.

324
00:35:57,460 --> 00:36:02,880
And we also have our Twitter account where we post some of the latest information around some of our research.

325
00:36:02,880 --> 00:36:04,340
So thanks so much for your time today.

326
00:36:04,340 --> 00:36:05,220
I hope you enjoyed it.

327
00:36:07,760 --> 00:36:09,360
Mike Rago, everyone.

328
00:36:09,420 --> 00:36:10,160
Hi.

329
00:36:11,420 --> 00:36:15,620
So if you have a question for Mike, yeah, we can hang out here.

330
00:36:15,620 --> 00:36:16,700
We got time.

331
00:36:17,580 --> 00:36:19,520
I don't think you need to use a microphone.

332
00:36:19,520 --> 00:36:21,840
But yeah, thank you so much for being here.

333
00:36:21,980 --> 00:36:27,260
Tomorrow we will have two more talks presented by the Packet Hacking Village in the afternoon.

334
00:36:27,560 --> 00:36:29,920
I'll ask him a question.

335
00:36:29,920 --> 00:36:30,920
Thank you again.

336
00:36:31,820 --> 00:36:33,160
I've got a question.

337
00:36:33,160 --> 00:36:37,860
What level of Twitter API were you using to gather your data?

338
00:36:39,420 --> 00:36:40,520
Can you clarify?

339
00:36:40,520 --> 00:36:40,820
I'm sorry.

340
00:36:40,820 --> 00:36:42,300
I caught the part around Twitter API.

341
00:36:42,300 --> 00:36:43,460
A free version.

342
00:36:43,460 --> 00:36:44,540
It's limited.

343
00:36:44,540 --> 00:36:48,320
There's like a $5,000 a month Firehose version.

344
00:36:48,920 --> 00:36:49,540
Yeah.

345
00:36:49,700 --> 00:36:57,160
The last that we were, you know, doing our, you know, updates related to it, there were multiple tiers.

346
00:36:57,160 --> 00:37:04,160
So the free versus the other three tiers, there's like, don't quote me on this, and it may have changed again.

347
00:37:04,160 --> 00:37:08,740
But there was basic, I forget the middle one, and then like enterprise.

348
00:37:08,780 --> 00:37:12,580
The basic, I believe, the last I checked was $100 a month.

349
00:37:12,580 --> 00:37:12,880
Yeah.

350
00:37:12,880 --> 00:37:14,740
And that's what we've used.

351
00:37:14,740 --> 00:37:16,760
And we've gotten like a wealth of data.

352
00:37:16,760 --> 00:37:17,280
Yep.

353
00:37:17,280 --> 00:37:17,880
Yep.

354
00:37:17,940 --> 00:37:21,160
But we did have to move from free to a paid version.

355
00:37:21,160 --> 00:37:23,880
But the basic did give us a wealth of data.

356
00:37:23,880 --> 00:37:31,600
If you go on the Tweepy API website, it will list out the different tiers for you in different tabs.

357
00:37:31,600 --> 00:37:34,460
So you can see exactly what APIs are available.

358
00:37:34,940 --> 00:37:35,980
Good question.

359
00:37:38,640 --> 00:37:40,400
Well, I'll hang out for a few more minutes.

360
00:37:40,400 --> 00:37:43,660
If you have any questions, feel free to use the mic.

361
00:37:43,660 --> 00:37:46,960
Or if you want to come up and chat privately, that's fine, too.

362
00:37:47,080 --> 00:37:48,140
And thanks again.

363
00:37:48,140 --> 00:37:49,280
Hope you enjoy the show.