WEBVTT 00:01.180 --> 00:02.760 How's everyone doing? 00:03.360 --> 00:04.400 Good. 00:04.600 --> 00:08.860 Just curious if this is, who here, is this your first DEF CON? 00:09.500 --> 00:10.540 Oh. 00:11.680 --> 00:13.700 Oh, look around. 00:14.040 --> 00:17.740 For those of you who this is your first DEF CON, welcome. 00:17.760 --> 00:18.660 Welcome. 00:20.000 --> 00:32.040 Each and every year, well except for last year when we run talks, we ask this question and the numbers are usually anywhere between like 70 to 80 percent. 00:33.080 --> 00:35.460 So the numbers are pretty consistent. 00:35.960 --> 00:40.200 So for those of you, if you're, you know, this is your first DEF CON, welcome. 00:42.640 --> 00:53.660 It can be a little overwhelming, let's be honest about it, but my biggest advice is take every opportunity that you can. 00:54.200 --> 00:58.440 This is an opportunity for you to learn as much as you can. 00:58.460 --> 01:04.100 And also do yourself a favor, learn from other people, including strangers here as well too. 01:04.160 --> 01:10.460 You may never, never know that person that you speak to, that could be a lifelong connection. 01:10.460 --> 01:19.360 Actually speaking of, speaking of, the next speaker and talk about lifelong connection, is how I got to meet you. 01:19.760 --> 01:22.180 And uh, you'll be on in two minutes. 01:23.220 --> 01:31.400 So, make sure you enjoy DEF CON and uh, learn as much as you can. 02:07.140 --> 02:09.300 Sacrificial lamb. 02:39.120 --> 02:40.480 All right. 02:41.120 --> 02:43.040 Let's get started. 02:43.040 --> 02:47.200 Hello everyone, good, now uh, yes, uh, good afternoon everyone. 02:47.200 --> 02:48.620 Good afternoon. 02:49.220 --> 02:54.200 Welcome to DEF CON 31 and welcome to Talks. 02:54.200 --> 02:57.300 Uh, presented by the Packet Hacking Village. 03:00.280 --> 03:04.540 Since 2013, been a long time. 03:05.260 --> 03:10.620 Uh, the Packet Hacking Village has presented a track of talks. 03:11.120 --> 03:26.200 So, and the whole premise is, people who go to a talk, uh, and you walk away, you should be able to take away something, uh, immediately to use after the talk. 03:26.200 --> 03:29.520 So that's the whole premise of a Packet Hacking Village talk. 03:29.600 --> 03:34.220 And each and every year they're well attended and uh, first want to say thank you. 03:34.220 --> 03:41.200 Thank you for, for being here for, actually our final Packet Hacking Village talk for today. 03:41.520 --> 03:55.940 And it is my honor and privilege to introduce someone who has given a few talks in the past and quality, world class talk to the Packet Hacking Village for years, Mike Rago and Chet Hosmer. 03:56.700 --> 04:04.100 Um, so, I'll let Mike give, uh, a little background about, about him. 04:04.180 --> 04:08.260 Uh, and his talk is on open source, uh, uh, intelligence. 04:09.080 --> 04:14.600 Uh, open, open source intelligence and, uh, physical, physical threat intelligence. 04:14.740 --> 04:18.600 So, my pleasure to introduce an old friend, Mike Rago. 04:19.920 --> 04:20.920 Thanks so much. 04:22.060 --> 04:26.080 Thanks Ming, really appreciate it and super excited to be back. 04:26.080 --> 04:32.200 Uh, thanks for having me and huge shout outs to the Packet Hacking Village, uh, for everything they do over the years. 04:32.300 --> 04:38.720 My name is Mike Rago, um, my, uh, good friend Chet Hosmer, uh, couldn't be here today. 04:38.720 --> 04:43.260 Uh, some family related stuff, we wish him well, send our thoughts and prayers. 04:43.500 --> 04:48.040 Chet and I, uh, co-present quite a bit at a variety of conferences around the world. 04:48.040 --> 04:55.880 So, uh, we share a lot of the same, uh, information in terms of our research and, uh, look forward to sharing that with you. 04:56.020 --> 05:03.180 This particular research and title of this presentation is OSINT for Physical Threat Intelligence. 05:03.200 --> 05:11.420 We know that a lot of people, uh, in the security community today are leveraging OSINT intelligence in a plethora of ways. 05:11.420 --> 05:21.700 Uh , whether it be for a person of interest or an organization of interest, whether it be for pen testing and going out there and gathering any public information they can find. 05:21.700 --> 05:35.740 Our particular focus of research for the last few years is physical threat intelligence and we'll talk a little bit more about what that means and what it applies to in the real world and how organizations and people use this today. 05:40.200 --> 05:44.120 So, Chet and I, as I mentioned, have collaborated for many years. 05:44.120 --> 05:45.800 We've written books together. 05:45.880 --> 05:51.620 And we both have been professors with the University of Arizona, aside from our full-time jobs. 05:51.620 --> 05:56.240 And it's been a great research ground for a lot of the things we do over the years. 05:56.280 --> 06:03.880 Furthermore, we've presented at a variety of conferences, including DEF CON, HAC CON in Norway, and many other conferences. 06:04.640 --> 06:14.540 In addition, a lot of this research we collaborate on allows us to work with a lot of organizations and understand a lot of their needs. 06:14.540 --> 06:24.360 So as we go through this, there will be a lot of applied aspects and real world scenarios in how we use this physical threat intelligence for situational awareness. 06:27.340 --> 06:30.040 Chet has authored even more books than I have. 06:30.040 --> 06:33.720 And again, just a variety of research over the years. 06:33.720 --> 06:39.800 In addition, he does a lot of coding and runs pythonforensics.org. 06:39.800 --> 06:44.180 So Chet does a lot of research and a lot around Python. 06:49.340 --> 06:54.800 So I think most of us here are familiar with open source intelligence, or OSINT. 06:54.800 --> 07:03.980 Again, what we're trying to do here is apply that wealth of information to meaningful intelligence that we can apply in a physical way. 07:03.980 --> 07:07.740 Whether that be concerns around business risk. 07:07.740 --> 07:14.880 Are we building a store in a foreign country, and are we concerned about someone targeting that store? 07:14.880 --> 07:23.100 Either while it's being built, or the first day it opens with some kind of flash mob that destroys the store, or things like that. 07:23.100 --> 07:31.200 Or maybe it applies to an organization that provides help for people around the world. 07:31.200 --> 07:36.140 Our thoughts and prayers go out to Maui and everything that's occurred there. 07:36.140 --> 07:44.580 But there's a wealth of data and pictures and information people are posting that can be leveraged to determine where to mobilize help next. 07:44.580 --> 07:47.680 And really provide that help to people that are in need. 07:47.680 --> 07:51.400 So this data can be applied in a variety of a lot of ways. 07:51.420 --> 07:57.460 Furthermore, it can be very cutting edge in terms of collecting it in a streaming format or live. 07:57.460 --> 07:59.520 So it can be very actionable. 07:59.520 --> 08:05.460 We'll talk about the overwhelming wealth of data out there and how to better curate that. 08:05.460 --> 08:12.960 So you don't spend 99% of your time trying to find those needles in the haystack that are most helpful for situational awareness. 08:12.960 --> 08:17.860 But actually flip that model upside down to pre-curate that data. 08:17.860 --> 08:21.840 And make it far more actionable in a very timely manner. 08:21.840 --> 08:33.840 As we mentioned here in the slide, we also want that to be a key decision making piece of information that we can use. 08:33.840 --> 08:39.800 But it doesn't displace information you may receive via private channels or other channels you have. 08:39.800 --> 08:43.500 And how you collect that intelligence and use it for situational awareness. 08:43.500 --> 08:46.700 Bottom line, it's meant to complement that data. 08:48.160 --> 08:54.400 The applied aspects of it could be related to threat assessments and business risk. 08:54.400 --> 08:58.760 Could also be related, as I mentioned, to disaster response. 08:58.920 --> 09:04.920 In this particular talk, we expose a lot of the research we've done around the Ukrainian war. 09:04.920 --> 09:13.400 To actually identify different attack patterns, paths that troops are moving, what things are being destroyed. 09:13.400 --> 09:20.820 How to profile tanks and other equipment out in the Ukraine and other areas. 09:21.080 --> 09:27.040 And it also can be used in more geopolitical type situations as well. 09:27.240 --> 09:29.640 So this data is becoming more and more helpful. 09:29.960 --> 09:34.880 But again, one of the big concerns may be around misinformation and disinformation. 09:34.880 --> 09:36.580 And we'll talk about that too. 09:39.060 --> 09:43.060 Lastly, before we get into how we actually perform this. 09:43.060 --> 09:48.040 And some of the Python scripts we've created for the collections in weeding through that data. 09:48.040 --> 09:51.520 Is, as I mentioned, can we trust this data? 09:51.860 --> 09:57.620 One of the things that we do is by collecting it in a live and streaming format. 09:57.620 --> 10:03.780 But furthermore, pinpointing a geo and collecting what we call ground truth. 10:03.780 --> 10:09.400 For a bounding box or a radius from which those posts are emanating from. 10:09.400 --> 10:11.940 We're not collecting data from around the world. 10:11.940 --> 10:14.140 We don't want data from around the world. 10:14.140 --> 10:17.220 It's littered with misinformation and disinformation. 10:17.500 --> 10:21.540 So you can leverage a technique that we'll talk about more here shortly. 10:21.540 --> 10:28.100 Where you can create a radius or a bounding box around a city such as Kiev within the Ukraine. 10:28.100 --> 10:33.700 You could do it around an area that you know has been impacted by an unfortunate disaster. 10:33.700 --> 10:40.460 This will give you much more legitimate posts that are, again, originating from that area. 10:40.460 --> 10:42.220 Giving you that ground truth. 10:42.220 --> 10:47.340 And we have found, bottom line, that that data is increasingly helpful. 10:47.340 --> 10:51.280 And weeds out more than 99% of the misinformation. 10:51.280 --> 10:57.500 So in getting to what we mentioned earlier about being proactive and pre-curating that data. 10:57.500 --> 11:00.520 So you don't have a huge data warehouse of useless data. 11:00.520 --> 11:04.960 This is one of the preliminary things you can do so you don't end up in that scenario. 11:07.530 --> 11:13.250 So let's talk about the tradecraft and then we'll talk about our research and a lot of the interesting findings. 11:17.350 --> 11:21.890 So conducting this involves a lot of data sources. 11:21.890 --> 11:25.410 It may be social media, it may be other sources as well. 11:25.410 --> 11:33.150 And understanding the chemistry of these different social networks and other mediums that provide this information. 11:33.190 --> 11:36.210 For example, if you're going to take a look at Discord. 11:36.410 --> 11:41.950 On Discord there are a variety of servers and channels that are focused on the Ukrainian war. 11:41.950 --> 11:48.630 That are focused specifically on nothing but tanks and other things that people are passionate about that they're posting. 11:48.630 --> 11:51.690 A lot of those posts are also non-biased. 11:51.690 --> 11:55.690 So there are pro-Ukrainian and pro-Russian posts going on. 11:55.690 --> 11:58.550 But again, we want to collect ground truth. 11:58.550 --> 12:04.790 So we want stuff originating from that area, not across lines, not from other portions of the world. 12:05.450 --> 12:13.370 In addition, that helps with countering all that counterintelligence, misinformation and disinformation. 12:13.370 --> 12:24.530 So when we perform these analysis techniques, we also are profiling and understanding which of these accounts over time are actually spreading misinformation. 12:24.610 --> 12:35.090 We'll talk about how we profile those so that we understand the top posters or people that may be posting legitimate information from that ground truth. 12:35.090 --> 12:40.370 And people who are basically muddying the waters with that misinformation. 12:40.370 --> 12:47.940 So in this case, we want less data, not more. 12:47.940 --> 12:53.940 Most people will get on social media, leverage a hashtag or a keyword and just start collecting a whole ton of data. 12:54.060 --> 13:05.340 If you talk to anyone that basically takes that and tries to automate that, there's just so much data collected, you end up spending too much time analyzing all that data. 13:05.340 --> 13:08.660 You can actually get ahead of the curve in a variety of ways. 13:10.800 --> 13:16.740 Also, you know, it was a long time ago when I started out as a UNIX system administrator. 13:16.760 --> 13:22.540 You know, my boss told me, listen, there's a lot of things to manage here on the network at the NASDAQ stock market. 13:22.640 --> 13:25.140 You know, laziness breeds efficiency. 13:25.220 --> 13:27.520 Focus on the most important things. 13:27.720 --> 13:32.340 You know, try to focus on, you know, less data, not more. 13:32.340 --> 13:34.200 And how we pre-curate that. 13:34.200 --> 13:35.700 So we'll talk about that, too. 13:35.900 --> 13:38.360 Also, we want to respect the terms of service. 13:38.360 --> 13:42.140 These vary based on the website, based on the social network. 13:42.140 --> 13:44.660 Make sure you're familiar with what those are. 13:44.660 --> 13:52.600 Some of them have very important privacy policies where you're not supposed to expose the accounts from which this data may be coming from. 13:52.600 --> 13:56.040 Also, scraping and other things sometimes are not allowed. 13:57.140 --> 14:02.600 But as you interact, for example, with these social networks, there are exposed APIs. 14:02.600 --> 14:06.760 For example, with Twitter, those have changed quite a bit recently. 14:06.760 --> 14:13.380 And now also there are tiers you have to pay for to get some of that data that otherwise with a free account you wouldn't get. 14:13.380 --> 14:19.760 So understanding that, too, is going to help enrich your data and also ensure that you're within the terms of service. 14:23.560 --> 14:34.920 So as we collect based on a different methodology rather than keyword or hashtag, we actually collect by geo. 14:34.920 --> 14:37.560 Again, we get that ground truth. 14:37.560 --> 14:42.800 Data that's emanating from within that particular radius or bounding box. 14:42.960 --> 14:46.460 In addition, it increases the authenticity of it. 14:46.460 --> 14:51.760 Because we're collecting not only posts, but we want those images, videos, and even emojis. 14:51.760 --> 14:59.760 And we pump those emojis actually through some ML that actually helps us in an automated way understand the sentiment, too. 14:59.760 --> 15:04.360 Which also helps us determine whether or not this information can be trusted. 15:08.780 --> 15:14.040 So we created a Python script that allows us to perform this collection. 15:14.560 --> 15:23.680 This allows us to either pick from a particular city or location and select the bounding box or radius for that location. 15:23.680 --> 15:31.280 Or you can actually put in the latitude and longitude and then define the radius or bounding box for what you want to collect. 15:31.480 --> 15:38.300 That fundamentally performs the collection by basically eliminating all the rest of the data from around the world. 15:38.440 --> 15:40.780 Are we going to miss a few important things? 15:40.780 --> 15:41.140 Yeah. 15:41.140 --> 15:48.300 But what I will say, that's a lot easier than collecting data from all around the world and trying to find those needles in the haystack. 15:48.300 --> 15:53.160 Instead, we're weeding out all those needles and we're getting roughly 90% of them. 15:53.160 --> 16:01.060 And that is far more actionable and timely in terms of leveraging this data for all of those applied aspects we mentioned earlier. 16:02.820 --> 16:08.560 We can additionally put in keywords and hashtags, even in other languages. 16:08.560 --> 16:11.480 And this will help further curate a lot of that data. 16:11.480 --> 16:16.840 There may be stuff within a radius around Kiev that have nothing to do with the war. 16:16.840 --> 16:22.960 So if we're looking to collect stuff related to the Ukrainian-Russian war, there may be stuff that's irrelevant. 16:28.280 --> 16:32.700 As we perform the collection, then, we're collecting each post. 16:32.700 --> 16:36.420 It could be a tweet, it could be a post, it depends on the social network. 16:36.420 --> 16:40.520 And collecting those images and collecting those videos. 16:40.700 --> 16:45.800 We'll talk next how that stuff is really important for cross-correlating that data. 16:46.200 --> 16:50.260 We also take a look at the accounts that are posting this information. 16:50.260 --> 17:02.620 So as we categorize them and determine who are the top people that are posting legitimate and very helpful information, we can actually focus on those and eliminate a lot of the other accounts. 17:02.620 --> 17:15.740 What we have found is, of the data that normally emanates from a bounding box, that there are about 10 to 20 people that dominate those posts with a wealth of positive, helpful information. 17:15.740 --> 17:21.200 But there are also about 10 to 20 that are littering it with misinformation. 17:21.560 --> 17:31.120 But again, knowing these from the get-go, as you're collecting this data, and weeding that out, again, helps curate the data in a very automated way. 17:37.910 --> 17:42.790 So we've gone, we've collected the data, we've done this in a very automated fashion. 17:42.790 --> 17:46.310 Again, leveraging geolocation information. 17:46.310 --> 17:56.630 And leveraging this in a chronological way, so that we have a timeline as well of the events, and we can use this for predictive analysis. 17:56.770 --> 17:59.370 So let's talk about how we leverage this data. 18:00.730 --> 18:08.150 Now, as we look through the data, we can see all the posts, all the tweets, and furthermore, any of the information and referring links. 18:08.150 --> 18:13.710 If it may point to Twitter, if it may point to Discord, if it may point to Telegram. 18:14.570 --> 18:21.330 And as we look through the data that we've collected, we start to find some really interesting information. 18:21.450 --> 18:30.190 If you're using some form of computer vision in your favorite cloud vendor, you can automate the analysis of these images, and we've done that. 18:30.450 --> 18:40.090 So if you're using optical character recognition, you may start to notice here on the screen that there are serial numbers on these military vehicles. 18:40.090 --> 18:50.490 And by knowing we collected that information for that vehicle, at that location, at that point in time, we know that that vehicle was located there. 18:50.570 --> 19:09.750 But if we're monitoring multiple locations around the Ukraine, for example, Kyiv, that they targeted first, then moved on to Kyiv, if we saw that military vehicle in Kyiv, and then later Kyiv, we're starting to see a motion on the map, right, of where this equipment is going, 19:09.750 --> 19:13.170 where the troops are going, and potentially where they may go next. 19:13.170 --> 19:20.530 Really interesting sliver of information I'll show you later, that was something that we found that we completely did not expect. 19:20.830 --> 19:25.030 So there's a lot of related data here across these devices. 19:25.030 --> 19:38.530 If you're using optical character recognition, whether it be an image or a video, it'll go ahead and scan all those images, and actually gave us chronologically, in time and location-wise, where that equipment was. 19:41.540 --> 19:47.700 And you may or may not have heard the terminology then, chrono-locationally. 19:47.700 --> 19:52.200 So we're taking a look at it chronologically, but also locationally as well. 19:52.200 --> 19:56.240 Kind of a new term that's been floating around, chrono-locationally. 19:59.510 --> 20:12.450 So to recap before we delve deeper into the data, is we're performing live streaming collection, secondly, by geo, to help pre-curate that data. 20:12.530 --> 20:17.990 We're not running a vacuum and collecting data from all around the world based on a keyword or hashtag. 20:18.230 --> 20:20.910 We're not doing that from the get-go. 20:20.910 --> 20:25.170 We're saying we want ground truth from this radius, from this location. 20:26.910 --> 20:32.490 In addition, since we're getting the location information, we can plop this on a map. 20:32.530 --> 20:44.210 One of the frequently asked questions we get is, how are you getting the location information then if that privacy is shut down or people don't have that enabled on their mobile devices when it's posted? 20:44.210 --> 20:48.950 It comes from the APIs and how these social networks are collecting the data. 20:48.950 --> 20:52.910 They know that it's been posted from within that bounding box. 20:52.910 --> 21:03.710 They won't say that that person is specifically on this street, on this corner in Kiev, or San Francisco, or Chicago, or wherever, but it's definitely within that region. 21:03.710 --> 21:08.010 And in this case, we don't care exactly where it's pinpointed on the map. 21:08.010 --> 21:13.150 We can build a one-mile radius around a region and get all that ground truth from there. 21:13.150 --> 21:20.950 And that's good enough for making these very important decisions in terms of mobilizing troops or mobilizing help in the event of a disaster. 21:21.950 --> 21:24.390 So again, we're not collecting this from the device. 21:24.390 --> 21:33.090 This comes directly from the social network, telling us that this person with this device posted from within that region or that bounding box. 21:37.870 --> 21:42.310 This takes us to another piece of really interesting information. 21:42.710 --> 21:45.590 A lot of activity going on in the war. 21:46.930 --> 21:55.590 Initially, the knee-jerk reaction is, let's collect everything we can around tanks and troops and what's being bombed and targeted. 21:55.590 --> 21:58.430 And that's great, and that's absolutely important. 21:59.010 --> 22:08.570 But when we started to kind of take a look at the data in a different way, leveraging transforms, we actually discovered a completely different narrative going on. 22:08.570 --> 22:11.370 And that narrative involved places of worship. 22:11.850 --> 22:20.930 So we found that churches, synagogues, mosques across the Ukraine were being destroyed as troops moved through those areas. 22:21.150 --> 22:35.550 So if we set aside all the data around what we knew about the troop movements and military equipment and things like that, we also started to see some very important trends of all of these places of worship being destroyed. 22:35.550 --> 22:55.370 We would not have seen that had it not been for some of the automation we'll talk about next, which is why now with the invent of AI, ChatGPT, Copilot, and those things, that you can now have a right-hand virtual person, if you will, to have you or have them look at it in a completely different way. 22:55.370 --> 22:58.070 And this is how we found this very important artifact. 22:58.490 --> 22:59.990 Why is that important? 23:01.070 --> 23:06.850 Well, as we started to map this out, you could use something as simple as Google Maps if you wanted to. 23:06.850 --> 23:09.670 You could go in here and plot these on a map. 23:09.870 --> 23:16.350 And in addition, with the automation, you can also include the initial post or tweet or whatever. 23:16.370 --> 23:18.550 And in addition, that image. 23:18.550 --> 23:29.930 So now as you start to bring this intelligence together, if you want to look at it on a map, you can start to see, in this case, all the places of worship that unfortunately have been destroyed during the war. 23:30.690 --> 23:42.930 And if you start to now look at that chronolocationally, or a chronological timeline by location, there's some very important predictions you can make here about where they might target next. 23:42.930 --> 23:46.230 We did that, and we found out we were 100% spot on. 23:46.230 --> 23:56.570 Not by tracking the tanks, not by tracking the troops, but by tracking the churches, synagogues, and mosques, all these places of worship that were actually being destroyed. 24:00.520 --> 24:04.200 So we take all that data and then plot that on a map. 24:04.360 --> 24:09.380 The captured image, the tweet, or the post, if it's from another social network. 24:10.740 --> 24:22.740 And what we found is, as we start to go through this, we then said, okay, we have another whole, you know, lump of data here around troop movement and tanks. 24:22.740 --> 24:25.300 We'll plot all that on a map, too. 24:25.300 --> 24:33.380 And you can do layers within Google Maps to help layer that or look at that data individually by type. 24:34.140 --> 24:36.840 So here, we mapped all the tanks. 24:36.840 --> 24:42.480 These were some of the tanks with the same serial numbers that we saw at different locations at different points in time. 24:42.480 --> 24:45.580 So what's going on with this particular one? 24:45.660 --> 24:50.240 The first one, you'll see the tank here, just outside Kiev. 24:50.440 --> 24:55.020 But if you then look at the next tank, it's heading towards the area of Donetsk. 24:55.020 --> 25:05.420 What we learned later is that attacks from the Russians then occurred, you know, from the Donetsk region, from the water, from the west side of the country. 25:05.420 --> 25:14.580 And basically what we're able to predict is that a lot of this military, once it had targeted Kiev, started to move in the direction of Donetsk. 25:14.580 --> 25:19.780 And we also started to see movement from the water hitting land targeting Donetsk. 25:19.780 --> 25:25.740 Bottom line, all of this military equipment and troops were converging on Donetsk. 25:27.500 --> 25:35.680 This tank that we show here is that piece of military equipment we showed earlier with that serial number B23. 25:35.720 --> 25:42.980 And we showed it first next to Kiev, then moving towards Donetsk, and then further, you know, closer to Donetsk. 25:48.450 --> 25:53.190 So you could leverage Twitter for this and some other social networks. 25:53.190 --> 25:59.690 If you go, for example, to Discord, there are some amazing servers out there with a ton of information around the war. 25:59.690 --> 26:06.250 As I mentioned, some people are just passionate about planes or tanks or you name it. 26:06.250 --> 26:15.130 So there is a wealth of data out there that we leverage with this to cross correlate the data we're getting from multiple social networks. 26:15.130 --> 26:19.850 Again, whether it be leveraging the APIs, we're doing this within the terms of service. 26:23.300 --> 26:41.920 What was really cool about that, then, is as we start to merge that data on a Google map, we can basically enable all of those layers to now look at this in the context of everything going on, whether it be military equipment, troops, churches and places of worship being destroyed, 26:41.920 --> 26:45.560 and many of the other activities by different categorizations. 26:46.160 --> 26:55.260 This definitely starts to build out a storyline chronologically to help understand what might happen next, as we showed with that piece of military equipment. 26:58.190 --> 27:01.710 There's a lot of different ways you can dashboard this. 27:01.770 --> 27:04.050 And this can be very helpful for an analyst. 27:04.050 --> 27:09.390 There are lots of either free tools or free trial tools you could use. 27:09.390 --> 27:12.310 In this particular case, I used Power BI. 27:12.310 --> 27:19.050 We took all the data, and with an Azure Logic app, fed this automatically into Power BI. 27:19.870 --> 27:24.170 Initially, I created all these dashboards for all this data. 27:24.170 --> 27:25.570 Why is this important? 27:25.570 --> 27:28.610 Well, over on the far right, you'll see a filter. 27:28.690 --> 27:31.530 This filter allows you to search against the data. 27:31.530 --> 27:46.690 You can either automate that, or if an analyst needs to investigate something, it gives you full sorting capability built right into the tool. 27:48.170 --> 27:53.570 In addition to this, you can also filter the tweets, filter things by emojis. 27:53.570 --> 27:55.990 There's a wealth of things you can do with this. 27:56.450 --> 28:03.730 Now, with things like Copilot or other things, is the ability to say, Hey, I want to incorporate some prompt engineering. 28:03.730 --> 28:05.650 I want to ask it some questions. 28:05.650 --> 28:12.270 And when I do, I want to say, Hey, build me a dashboard based on the intelligence you're seeing. 28:12.450 --> 28:22.070 And it'll build another tab with that additional intelligence from a completely different viewpoint, giving you other analytics and data points you didn't even think to look for. 28:22.610 --> 28:35.110 So, when you start to map it out in this way, and a lot of these tools even have a free trial, you can go in there and leverage these to get some really cool, actionable, and very short timeline type data. 28:39.130 --> 28:43.770 There's also a plethora of widgets built into a lot of these tools. 28:43.770 --> 28:56.870 What I had used within Power BI allowed me to not only do a lot of the mapping with ArcGIS, which is fed by Ursa, I believe, but also other forms of widgets like Word Cloud. 28:56.870 --> 29:04.410 This became really informative, too, because based on the data, there was a whole bunch of different keywords that popped up. 29:04.570 --> 29:06.830 So, I can see a variety of things there. 29:06.830 --> 29:13.970 The, obviously a popular used word in Ukrainian and Russian, and then you start to look at some of this other data. 29:16.610 --> 29:24.910 When I started to take a look at the top ten words, one of the ones that popped up was Oblast, which I believe is a region within Kiev. 29:25.090 --> 29:36.150 I had no idea that things were going on at that point in time within Kiev, but this actually told me that, through the Word Cloud, through the widget, through all this active data. 29:36.150 --> 29:49.450 So, again, another data point I would not have looked at by doing human analysis of all of this, but leveraging the widget and the automation in your favorite dashboarding, whether it be Grafana, whether it be Power BI, whether it be whatever, name your tool. 30:00.160 --> 30:04.080 So, then we start to get into extrapolation and prediction. 30:04.100 --> 30:18.300 We've already got some really interesting data points that we didn't even think to look for, but with the automation, with a lot of different perspectives, with the AI, with chat GBT, with prompt engineering, we got a lot of additional, very helpful data, 30:18.300 --> 30:20.320 and got that in a very timely manner. 30:20.320 --> 30:27.520 The data is automatically fading into it, and analysts can be right there doing searches, creating reports, and getting that data on demand. 30:29.480 --> 30:35.740 So, as I mentioned, we're able to track some of the Russian troop movements based on the data we collected. 30:36.600 --> 30:48.960 And, in addition, we also found certain areas, places of worship that were being destroyed, location, the time at which things were occurring, and other interesting data points. 30:50.640 --> 30:59.780 The question here, then, was, if I look specifically at these places of worship being destroyed, can I predict where the troops might be next? 30:59.780 --> 31:10.700 So, if I'm on the defense, or counter, and I need to figure out where to mobilize my teams, we were able to actually use this data to do a variety of prediction. 31:10.700 --> 31:23.160 Up at the top, you can see Chekerev, and, before that, we actually had Kikarev, where the troops were moving based on the chronological timeline, and we could see they were moving closer and closer to Kiev. 31:23.160 --> 31:27.740 It was pretty much expected they were going to target Kiev, but these data points proved that. 31:27.740 --> 31:34.820 And it gave us some really interesting data points to actually say, yep, they're on the move, they're all converging on Kiev. 31:35.400 --> 31:46.360 Then you'll see a splinter of, you know, some troops in a completely different direction, up one of the main highways, in the eastern direction. 31:46.360 --> 31:53.400 And we showed you earlier where some of the other troops splintered off and started going towards Donetsk, where they also converged. 31:53.540 --> 32:01.720 So a lot of this data definitely can be used for predictive extrapolation type of analysis, and we proved it. 32:01.980 --> 32:07.100 But we would not have known to look at just places of worship to get that data. 32:07.100 --> 32:16.400 So the automation, the prompt engineering, and a lot of these things are what actually got our laser focus on some of these other areas we weren't focused on. 32:27.730 --> 32:32.330 So a lot of us are starting to mess around with things like chat GPT. 32:32.650 --> 32:46.450 And what's kind of nice about it is, a lot of vendors have taken this and basically isolated it in a tenant for you, so that you're running it only against your data, and not exposing that data or that analysis to the external world. 32:47.210 --> 32:56.410 This prompt engineering, as I mentioned, can allow you to put in a variety of different prompts to look at just this data in a lot of unique ways. 32:56.670 --> 33:00.430 Tell me, what are the top trends we're seeing at this point in time? 33:00.430 --> 33:01.190 What are they? 33:01.190 --> 33:02.730 Are they places of worship? 33:02.730 --> 33:04.350 Are they troop movements? 33:04.350 --> 33:14.570 Are they posts from people that are suffering damage and need help on the ground within this geolocation, within this bounding box? 33:15.870 --> 33:27.110 Through this, we basically said, let me go ahead and put in a variety of questions, and then once I got this honed in on what I want, I basically can say, build me a dashboard. 33:27.210 --> 33:35.950 So whereas the previous dashboard I had was built manually by myself, I can now have the AI and the automation build it for me. 33:35.950 --> 33:38.770 And when it did, it looked completely different. 33:38.790 --> 33:39.910 And that was really cool. 33:39.910 --> 33:46.030 So we have that as a second tab, looking at all of this data in a different transform in a different way. 33:47.830 --> 33:52.870 Bottom line, it helped us identify a plethora of things we had missed. 33:57.710 --> 34:00.070 So how can we use this methodology? 34:01.030 --> 34:05.090 It can be applied to not only war, but disaster response. 34:05.090 --> 34:10.290 There are a lot of organizations that want to know, hey, we're having all of these California rains. 34:10.290 --> 34:12.390 There are dams building up with water. 34:12.390 --> 34:17.530 People are starting to post that the water is going over the dam and destroying some homes. 34:17.530 --> 34:24.270 Also help us find people in need within that geobounding box of that region that we weren't focused on. 34:24.270 --> 34:34.850 Someone just posted something about their home being destroyed and there's a bunch of flooding from a river that overflowed or the things that were going on in Sacramento. 34:35.010 --> 34:42.990 So this will obviously give you a variety of really interesting data points to help with the situational awareness and mobilization of help for these individuals. 34:44.230 --> 34:48.210 Another thing that we frequently do is the account profiling. 34:48.210 --> 35:02.470 As I mentioned, we basically whittle this down so that 50% of the data we're getting are from the top 10 to 20 people that are posting on that social network versus the whole gamut of that entire region. 35:02.470 --> 35:19.110 That provides us about 50% of the data and also helps us further weed out misinformation and disinformation because we also profile those accounts that are spreading misinformation or disinformation and profile those top 10 accounts as well so we can say exclude that data. 35:19.110 --> 35:21.490 I don't even want that included with it. 35:26.160 --> 35:28.760 So I hope you found this talk informative. 35:28.760 --> 35:32.000 I'll hang out for a little bit if you have any questions about this. 35:32.000 --> 35:33.660 Happy to chat about it. 35:33.660 --> 35:35.020 Hope you enjoyed it. 35:35.020 --> 35:36.420 We spent a few years. 35:36.420 --> 35:42.440 If you're interested in a copy of the research report, it's available at our site listed there. 35:42.780 --> 35:43.820 SilentSignals.com. 35:43.820 --> 35:45.320 You can download it for free. 35:45.420 --> 35:48.820 And see a deeper analysis around the research report. 35:48.820 --> 35:49.600 It's free. 35:49.820 --> 35:54.820 And we're also looking for people at any time that may want to get involved with the research. 35:54.920 --> 35:57.460 So if you are interested, we have an email there. 35:57.460 --> 36:02.880 And we also have our Twitter account where we post some of the latest information around some of our research. 36:02.880 --> 36:04.340 So thanks so much for your time today. 36:04.340 --> 36:05.220 I hope you enjoyed it. 36:07.760 --> 36:09.360 Mike Rago, everyone. 36:09.420 --> 36:10.160 Hi. 36:11.420 --> 36:15.620 So if you have a question for Mike, yeah, we can hang out here. 36:15.620 --> 36:16.700 We got time. 36:17.580 --> 36:19.520 I don't think you need to use a microphone. 36:19.520 --> 36:21.840 But yeah, thank you so much for being here. 36:21.980 --> 36:27.260 Tomorrow we will have two more talks presented by the Packet Hacking Village in the afternoon. 36:27.560 --> 36:29.920 I'll ask him a question. 36:29.920 --> 36:30.920 Thank you again. 36:31.820 --> 36:33.160 I've got a question. 36:33.160 --> 36:37.860 What level of Twitter API were you using to gather your data? 36:39.420 --> 36:40.520 Can you clarify? 36:40.520 --> 36:40.820 I'm sorry. 36:40.820 --> 36:42.300 I caught the part around Twitter API. 36:42.300 --> 36:43.460 A free version. 36:43.460 --> 36:44.540 It's limited. 36:44.540 --> 36:48.320 There's like a $5,000 a month Firehose version. 36:48.920 --> 36:49.540 Yeah. 36:49.700 --> 36:57.160 The last that we were, you know, doing our, you know, updates related to it, there were multiple tiers. 36:57.160 --> 37:04.160 So the free versus the other three tiers, there's like, don't quote me on this, and it may have changed again. 37:04.160 --> 37:08.740 But there was basic, I forget the middle one, and then like enterprise. 37:08.780 --> 37:12.580 The basic, I believe, the last I checked was $100 a month. 37:12.580 --> 37:12.880 Yeah. 37:12.880 --> 37:14.740 And that's what we've used. 37:14.740 --> 37:16.760 And we've gotten like a wealth of data. 37:16.760 --> 37:17.280 Yep. 37:17.280 --> 37:17.880 Yep. 37:17.940 --> 37:21.160 But we did have to move from free to a paid version. 37:21.160 --> 37:23.880 But the basic did give us a wealth of data. 37:23.880 --> 37:31.600 If you go on the Tweepy API website, it will list out the different tiers for you in different tabs. 37:31.600 --> 37:34.460 So you can see exactly what APIs are available. 37:34.940 --> 37:35.980 Good question. 37:38.640 --> 37:40.400 Well, I'll hang out for a few more minutes. 37:40.400 --> 37:43.660 If you have any questions, feel free to use the mic. 37:43.660 --> 37:46.960 Or if you want to come up and chat privately, that's fine, too. 37:47.080 --> 37:48.140 And thanks again. 37:48.140 --> 37:49.280 Hope you enjoy the show.