00:00:00.234-->00:00:05.239 >>Hi everybody uh, I'm Jason Hernandez. Uh our talk is titled tracking spies in the skies uh 00:00:07.374-->00:00:13.747 as you can see there, that's a photo of a uh domestic surveillance aircraft. Uh N496ww 00:00:13.747-->00:00:18.752 which uh, is an FBI aircraft. Uh that was flying over the Phoenix area over the last couple of 00:00:24.458-->00:00:29.463 years. Um, so again uh about the talk our main focus is on um, law enforcement and aerial 00:00:33.200-->00:00:38.205 surveillance mostly focus on domestic flights. Um so the F, FBI the DEA, um local uh 00:00:42.509-->00:00:47.514 sheriff's offices and police departments. Um they're all operating um fixed wing and, and 00:00:51.018-->00:00:56.023 helicopter aircraft. Um, with a variety of technology. Um, Sam Richards is gonna lead off uh 00:00:58.759-->00:01:02.629 with a little bit of discussion of the history of aerial surveillance and um the 00:01:02.629-->00:01:07.634 discovery of the FBI and DEAs um secret fleets. Um, and then Jerod MacDonald-Evoy is going to 00:01:11.238-->00:01:16.610 talk about some of the technology aboard the aircraft. Um and then I'll finish up and 00:01:16.610-->00:01:23.016 discuss uh, some algorithmic ways to, detect aerial surveillance um based on flight 00:01:23.016-->00:01:29.556 patterns. So, over to Sam. >>Alright, well thank you again everybody for coming out here. 00:01:29.556-->00:01:34.561 This is crazy. Um so I wanna say right away that it's, this talk it's a huge topic obviously with 00:01:38.065-->00:01:43.337 ya know tons of different agencies and municipalities and technologies, so this isn't 00:01:43.337-->00:01:47.975 gonna be all encompassing but we're going to try to do uh it's best justice we can to the 00:01:47.975-->00:01:54.781 topic. Uh so I guess for me it started off a long time ago just interested in all kind of 00:01:54.781-->00:02:01.021 conspiracies really. Uh the closest one that you can think of with this would be black 00:02:01.021-->00:02:05.792 helicopter conspiracy. So people were suspecting some sort of government activity with 00:02:05.792-->00:02:10.097 unmarked aircraft flying all over the place, um nobody really knew if there was actually 00:02:10.097-->00:02:16.203 something to it or not. And it turns out that there was interestingly enough. Um and so 00:02:16.203-->00:02:20.640 the first news report that kind of caught my eye and a lot of people's attention was from the 00:02:20.640-->00:02:25.645 Wall Street Journal. Uh just like on a lot of these initial reports it was just talking 00:02:25.645-->00:02:30.384 about 1 individual plane uh and it was pretty obvious that you could assume that it would be 00:02:30.384-->00:02:36.223 doing some sort of surveillance. And then when Baltimore happened with the Freddie B- uh Freddie 00:02:36.223-->00:02:42.129 Gray protest more people started noticing just a single engine Cessna uh circling at ya know a 00:02:42.129-->00:02:47.134 lot of times it was at night and it was pretty obvious to assume that that was monitoring the 00:02:47.134-->00:02:52.672 protest. Because the plane wasn't there before that. And again, with those articles it 00:02:52.672-->00:02:58.645 was just referencing 1 aircraft, not nothing really confirmed um ya know it wasn't ever 00:02:58.645-->00:03:04.685 acknowledges publicly by the agencies that were operating the aircraft obviously. And that's 00:03:04.685-->00:03:09.689 sort of when I started getting involved. Um, yea. So, I've put a story on medium.com uh 00:03:12.993-->00:03:18.765 initially I was just on my Twitter account. Um I got a tip from somebody I worked with at 00:03:18.765-->00:03:22.536 Minneapolis International Airport, uh he just sent me a screenshot from flightradar24 00:03:22.536-->00:03:27.908 and it showed just this plane circling all kinds of like, I think it was Southdale Mall in 00:03:27.908-->00:03:33.013 Minneapolis and other highly populated areas. So we immediately recognize that it 00:03:33.013-->00:03:37.584 was doing some sort of surveillance there uh and we just all open source 00:03:37.584-->00:03:44.124 information. We just pulled the aircraft's registration up on uh aircraft 1 dot com and from 00:03:44.124-->00:03:49.262 there, there was a list of 62 planes that had similar registration so that plane in 00:03:49.262-->00:03:55.735 particular was registered to FVX Research. Which doing just more open source investigating you 00:03:55.735-->00:04:00.340 could see that, that wasn't a real company. Um, they weren't on any Secretary of State 00:04:00.340-->00:04:05.212 websites, there was no legitimate business website or anything like that. So, at the 00:04:05.212-->00:04:10.784 bottom of that first list we found were 2 aircraft registered explicitly to the Department of 00:04:10.784-->00:04:16.990 Justice. Not behind a front name just explicitly DOJ. Uh those 2 planes were actually 737's. So 00:04:16.990-->00:04:22.295 not kind of aircraft that are used for spy planes but still it was, it was interesting to see 00:04:22.295-->00:04:28.401 that correlating right away. Uh and then from there using just the names of these companies we 00:04:28.401-->00:04:34.307 couldn't prove existed uh, I just started searching all those specific names like PSL Surveys, 00:04:34.307-->00:04:41.047 FVX Research, um Northwest Aircraft Leasing CO is one of my favorites. And putting those 00:04:41.047-->00:04:47.521 names into these little aircraft registration websites the list ballooned to over 100 aircraft 00:04:47.521-->00:04:53.193 and this was in May of 2015. So I put that list on Pastebin uh which I'm sure people are 00:04:53.193-->00:04:58.532 familiar with, it's just like a website you can dump information on and share it. Uh so at that 00:04:58.532-->00:05:04.905 same time I just put a little brief, ya know write up, like hashtag FBI sky spies, this is 00:05:04.905-->00:05:09.609 probably what's going on. Here's some links to images from flight radar where you can see these 00:05:09.609-->00:05:15.448 planes circling. Uh and it was all over the country, like, the, Seattle was really have at that 00:05:15.448-->00:05:20.554 time. But basically, anywhere there was a metropolitan area that's bigger, like bigger than 00:05:20.554-->00:05:27.127 a medium sized city. Um but also rural places too, like remote places. Greeley, Colorado 00:05:27.127-->00:05:33.200 outside of Denver was a really small town that was popular with the FBI at that time. And you 00:05:33.200-->00:05:39.673 can go next. So this was basically my main concern with these planes. Um, I knew about 00:05:39.673-->00:05:45.612 this program from a PBS documentary. Basically the, the drones in this case they, they 00:05:45.612-->00:05:50.350 patched together what are equivalent to just a ton of cellphone cameras and you make, 00:05:50.350-->00:05:55.021 you an image so large that you actually just, you keep these drones parked over for a long 00:05:55.021-->00:06:00.260 time and then you can see everything moving including people walking, cars and things 00:06:00.260-->00:06:06.933 like that. Uh and so in the in the war efforts they were using these to try and track down who 00:06:06.933-->00:06:10.971 placed an IED somewhere after the bomb exploded. And they would literally just review the 00:06:10.971-->00:06:16.176 footage and find where that person went. And then presumably the special forces or whatever 00:06:16.176-->00:06:23.016 would raid that house. And so that, that was publicly known uh developed by DARPA in the early 00:06:23.016-->00:06:27.420 2000s and that was, when I saw planes flying over the country I immediately thought of this 00:06:27.420-->00:06:34.127 technology. And so, that' that's a huge fear obviously if you have everybody in a city being 00:06:34.127-->00:06:38.565 surveilled with all their movements being tracked. Um so a little bit more on that 00:06:38.565-->00:06:44.738 technology after on probably, but back to the stories. So, after I put up my story on 00:06:44.738-->00:06:50.043 medium dot com with, it was just like less than 1-->000 words. Um the ai- the associated press 00:06:50.043-->00:06:55.548 broke that story into the mainstream about a week later. Uh and so those tweets with the, 00:06:55.548-->00:07:00.787 the flight tracking information and the medium blog post were going viral still. Uh so that 00:07:00.787-->00:07:06.559 was interesting. Uh they actually linked to it in the AP article initially. Uh they 00:07:06.559-->00:07:11.498 called it speculation, which didn't really hurt my feelings that much. But let's see here. 00:07:11.498-->00:07:16.503 So continuing on to the research. Um we basically found this office at the FAA's uh FOIA 00:07:19.439-->00:07:23.009 headquarters. And you can, you can submit a ton of tail numbers. They were actually r- 00:07:23.009-->00:07:27.881 incredibly responsive and helpful. This one came from a FOIA that I sent over 700 tail 00:07:27.881-->00:07:32.385 numbers to them. And they came back with just a massive spreadsheet and n- not much time 00:07:32.385-->00:07:38.425 at all. And uh you can kinda see the basic info. Like the tail numbers are highlighted. You see 00:07:38.425-->00:07:44.264 the dates. Um the airports, cities that they took off and landed at. But not, not a whole 00:07:44.264-->00:07:50.870 lot of information. Uh so y- you don't see the circles or if it was just a simple A to B flight. 00:07:50.870-->00:07:56.843 But you get a general idea. And so that, that was helpful but still sort of limited you know? 00:07:56.843-->00:08:01.915 If we're gonna go about trying to investigate these things just doing it manually on a flight 00:08:01.915-->00:08:07.120 radar website or doing it through public records requests it's very labor intensive. Um, 00:08:07.120-->00:08:13.560 and with 100s and 100s of these aircraft which are actually proliferating down from, you can 00:08:13.560-->00:08:19.399 hit the next one. Um, or maybe the next one, we'll go back to this one. SO you can see that 00:08:19.399-->00:08:24.437 these, these companies that are making this technology it's, it is an industry. It's 00:08:24.437-->00:08:30.243 surveillance industrial complex. Uh initially everything starts off with the war efforts it 00:08:30.243-->00:08:35.849 seems and then it eventually percolates down from there from federal to state to local. And, 00:08:35.849-->00:08:40.120 the bottom there it says, large city law enforcement, medium city law enforcement. That's a 00:08:40.120-->00:08:45.525 huge market share. So we're gonna clearly see this type of stuff continue to be, ya know 00:08:45.525-->00:08:52.065 even just some podunk sheriff will eventually have a Cessna. Um, with whatever capabilities 00:08:52.065-->00:08:58.071 so we need more people to look into it. Cause you looking at one plane, if you pull it up on 00:08:58.071-->00:09:03.410 flight radar, it you'll see just a bunch of flights right? And that's, that's obviously labor 00:09:03.410-->00:09:08.381 intensive so, I think people need to start taking it um taking it seriously. And kind of 00:09:08.381-->00:09:13.787 try to inquire more with whoever is operating the aircraft in their neck of the woods. Um so 00:09:13.787-->00:09:19.159 you want to go back to that open? This is really interesting to me. So this was actually made 00:09:19.159-->00:09:24.864 by homeland securities geospatial intelligence team. And you know you can see the red 00:09:24.864-->00:09:29.602 areas. I'm not sure how clear it is on the screen over there but that's where there was civil 00:09:29.602-->00:09:34.774 unrest. Uh the Freddie Gray protest response basically. And then they have all the other 00:09:34.774-->00:09:38.845 critical infrastructure that they're interested in protecting. But, so this this 00:09:38.845-->00:09:44.584 wouldn't' really be possible without some sort of spy plane over head. Uh you feed this down 00:09:44.584-->00:09:50.890 and obviously it's a huge benefit for any law enforcement or intelligence to have a pretty 00:09:50.890-->00:09:56.930 accurate and up-to-date map of an entire city and all of the activity at the time. Um, and 00:09:56.930-->00:10:01.935 then you wanna go and, yeah. >>So uh, I've been looking into um what the types of different 00:10:04.637-->00:10:08.274 aircrafts through public records. And through public records we've been able to and 00:10:08.274-->00:10:11.945 just watching ya know flight radar we've been able to determine what types of aircraft 00:10:11.945-->00:10:16.649 they use. The most common for airborne aerial surveillance are the small fixed wing Cessnas. 00:10:16.649-->00:10:20.854 It's like 1 engine prop plane you probably have seen them around. They're really popular 00:10:20.854-->00:10:26.526 with like flight schools or private owners. Um but they'll, they'll fly those around um all 00:10:26.526-->00:10:31.798 over. A lot of law enforcement has those both local and state and federal. Then the next type 00:10:31.798-->00:10:35.568 of aircraft, these large dual engine beechcraft. It kind of looks like this, this plane 00:10:35.568-->00:10:39.539 right here but instead of 1 engine up front it's got 2 on each wing and it can hold about 00:10:39.539-->00:10:44.143 15 to 20 people. Now when they got to the state legislator and ask for money to buy these they 00:10:44.143-->00:10:47.780 usually say, oh it's going to be for transport or things like that or you know moving 00:10:47.780-->00:10:53.253 investigators around the state. But they when you go through the records you find that they do 00:10:53.253-->00:10:56.956 have surveillance equipment installed on them as well. And they're using these large engine 00:10:56.956-->00:11:01.794 planes um specially customs and borders patrol has one that they fly up and down the border 00:11:01.794-->00:11:07.100 almost daily uh around Arizona and Texas. Just running surveillance on the border 00:11:07.100-->00:11:10.703 picking up radio signals and things like that. Then there's the military style plane the 00:11:10.703-->00:11:15.842 Pilatus. Uh very, very few law enforcement has these, have these. Uh Phoenix PD is one of 00:11:15.842-->00:11:20.547 the early adopters uh that we know of. And what's interesting about these, this specific plane 00:11:20.547-->00:11:26.719 it's the Specter model which was initially designed for the US Air Force to fly uh and do 00:11:26.719-->00:11:30.290 surveillance in Iraq and Afghanistan. Now the other interesting thing about this 00:11:30.290-->00:11:35.662 plane is first it's the largest 1 engine prop plane you can buy and it's also pressurized. Now 00:11:35.662-->00:11:39.933 if you have a pressurized cabin you can go up way higher and this plane is usually flying at 00:11:39.933-->00:11:46.306 around 17-->000 feet. Doing circles um like 10 mile orbits following some guy or, or 00:11:46.306-->00:11:51.778 looking at a residence. Um, these are, these are becoming more and more popular. This is 00:11:51.778-->00:11:56.015 about a 4.5 million dollar aircraft though. So that the, the larger cities are gonna be 00:11:56.015-->00:11:59.819 the ones that are gonna be more interested in the Pilatus. But the small wix- fixed wing 00:11:59.819-->00:12:04.424 cessnas, I mean there's not a huge barrier to entry for those for a lot of other local law 00:12:04.424-->00:12:09.862 enforcement because for example the Mary Copen county sheriff's office down in Arizona, they got 00:12:09.862-->00:12:13.900 one of their panes through Rico. Uh because there was a guy who was smuggling marijuana on a 00:12:13.900-->00:12:19.038 plane. They found him, took the plane decided, hey let's use this for our own uh benefit. So 00:12:19.038-->00:12:24.210 the other type of aircraft most people are familiar with uh with when it comes to police aircraft 00:12:24.210-->00:12:27.780 are helicopters, but they're loud they're noisy, they've gotta fly them low. They mainly 00:12:27.780-->00:12:33.519 use them for the guy who ran away on foot. Or to survey like a car accident. Now drones are 00:12:33.519-->00:12:38.291 still very new to law enforcement. Not very many are using them. Sheriff Joe out in 00:12:38.291-->00:12:43.930 Arpaio petitioned to try and get a predator drone but that never went through obviously. Thank 00:12:43.930-->00:12:48.501 goodness, um but uh eh, eh there're other law enforcement agencies that are getting like 00:12:48.501-->00:12:52.372 the quadcopters. They're saying they're trying to use them for search and rescue but we haven't 00:12:52.372-->00:12:56.242 really found any uh examples of them being used for surveillances yet and they're a 00:12:56.242-->00:12:59.812 little bit harder to keep track of. But law enforcement are getting a lot more interested in 00:12:59.812-->00:13:05.418 those especially with the ones that are backpack deployable. Um now equipment wise the most 00:13:05.418-->00:13:10.490 popular type of camera system they're on practically all of these air craft, is the Wescam 00:13:10.490-->00:13:16.496 by L3 Communications now um, the Wescam is interesting, they have 3 different varieties. Uh they 00:13:16.496-->00:13:19.232 have the MX10, MX15, and MX20. MX10 uh has like about a 70 time zoom ratio. It's usually on like 00:13:19.232-->00:13:21.901 the helicopters, because the helicopters don't have to go very high. Uh MX15 is kind of an 00:13:21.901-->00:13:26.906 in between for your law enforcement agency that can't ya know spend 1.5 million dollars 00:13:30.610-->00:13:35.615 on a camera system. Um and then the MX20 is though the most popular. The MX20 is the same 00:13:38.951-->00:13:44.090 camera system that is used on the predator drones. Um, they have a laser designator where 00:13:44.090-->00:13:50.063 they can uh well on predator drones the laser des- designator is used for missiles. But on a 00:13:50.063-->00:13:54.267 law enforcement aircraft they use that laser designator to, say they're following a, a guy 00:13:54.267-->00:13:58.604 who stole a car. They lock it on that car and then the camera doesn't ya know, the operator 00:13:58.604-->00:14:02.375 doesn't have to worry too much about moving it and they're just doing zoom or changing the lens. 00:14:02.375-->00:14:07.046 Cause you can see it's got like several different lens, lenses here. And one of those lenses 00:14:07.046-->00:14:13.086 usually is an infrared camera. FLIR or for looking infrared, um the company that makes those is 00:14:13.086-->00:14:17.790 called FLIR and they're, they're the most popular and FLIRs interesting uh because it's 00:14:17.790-->00:14:23.963 basically, imagine predator vision, that's what it is. It's seeing heat. Um and it can 00:14:23.963-->00:14:27.667 actually see usually, it can even see through walls and you can see people walking inside a 00:14:27.667-->00:14:33.506 home with the, with their heat signatures. But um there was a supreme court ruling Danny Lee 00:14:33.506-->00:14:39.579 Kyllo vs the United States which ruled that using uh infrared to look into someone's home is, it, 00:14:39.579-->00:14:45.084 it requires a warrant. So this guy he was, he was growing pot in his house and police u looked 00:14:45.084-->00:14:48.221 into his house and saw there was a lot of heat coming from the garage and then they said to the 00:14:48.221-->00:14:51.390 judge oh we think that means he's growing marijuana in there and got a warrant. Well Danny 00:14:51.390-->00:14:55.328 Lee Kyllo and his attorneys argued that the, the use of that infrared technology to see into 00:14:55.328-->00:15:00.967 his home constituted a search. Um, and so they won that and now they, police have to get 00:15:00.967-->00:15:05.905 warrants to use infrared in that, that type of way. But it's hard to track to see how police 00:15:05.905-->00:15:09.942 are using that because a lot of time cases that the aerial surveillance is used on, um 00:15:09.942-->00:15:15.148 they'll seal the dockets. Now cell site simulators are the other ar- uh type of equipment 00:15:15.148-->00:15:19.619 that we know has been used on these, these aircraft. The FBI's done it at least twice that we 00:15:19.619-->00:15:23.923 know of. Now cell site simulators, it's a little device if you don't' know coactively 00:15:23.923-->00:15:29.262 they're called Stingrays or IMSI catchers and they act like a cellphone tower. So if there's 00:15:29.262-->00:15:33.332 one in this room right now everyone's phone would go to that instead of the cell tower. 00:15:33.332-->00:15:37.403 And they could scoop up the data. Read metadata on the more advanced versions they could 00:15:37.403-->00:15:41.908 even turn it into a passive listening device. They get they can follow where you're going um 00:15:41.908-->00:15:46.546 and it's a really popular device. Um but if you can imagine flying one of those 00:15:46.546-->00:15:52.151 above a city of 1.5 million or so you're gonna be sweeping up a lot of data uh from that plane. 00:15:52.151-->00:15:57.757 Now the fourth thing that we have seen on these planes are what they call LETC devices. Now 00:15:57.757-->00:16:03.362 this is a stupid little acronym they like um that we found from uh, from records. From FFA 00:16:03.362-->00:16:07.333 records, on, on their airworthiness. They, airworthiness records y- you can 00:16:07.333-->00:16:12.038 get, they're public record and it shows everything that's been installed on an airplane. Um and 00:16:12.038-->00:16:17.076 we found it on several the aircraft it will say installed LETC antenna. LETC stands for 00:16:17.076-->00:16:21.280 law enforcement technical collection, um and when I reached out to customs and 00:16:21.280-->00:16:25.184 border patrol about the, the myriad of law enforcement technical collection antennas 00:16:25.184-->00:16:29.155 they had installed on their plane they said that they use them to collect signals across 00:16:29.155-->00:16:34.894 the electromagnetic spectrum. Well. [laughter] Electromagnetic spectrums a pretty big place 00:16:34.894-->00:16:38.798 right? So I asked them about that and they just said, no we collect across the entire 00:16:38.798-->00:16:43.803 electromagnetic spectrum which is ya know x-rays, gamma rays, cell phones, everything. So what 00:16:43.803-->00:16:48.007 we can assume they are using it, especially customs and border po- patrol probably listen to 00:16:48.007-->00:16:53.212 drug runners with radios or cell phones. Or even internet signals could be picked up with an LETC 00:16:53.212-->00:16:58.918 uh antenna. Uh but that's one we don't know a whole lot about yet. Now some examples of use 00:16:58.918-->00:17:05.157 are um, uh one of the most well known uh examples was when the FBI flew their airplane in 00:17:05.157-->00:17:10.196 support of the Baltimore police department during the uh Freddie Gray protest. Uh this is a 00:17:10.196-->00:17:14.467 screenshot from that, they were using IR uh and they were, they circled I think the protest I 00:17:14.467-->00:17:19.071 think for like 8 to 10 hours that day. Uh the released all the footage after myself and 00:17:19.071-->00:17:22.008 several other reporters kept bugging them about the footage because we knew it was their 00:17:22.008-->00:17:27.013 plane. Uh so that's 1 example of use even though the FBI publicly said that they don't surveil 00:17:29.081-->00:17:33.185 protests, um but obviously, they did here but they say that's because they were in support of 00:17:33.185-->00:17:36.188 Baltimore police department and they had asked for that. So that's the other thing is the 00:17:36.188-->00:17:39.825 FBI will often lend this aircraft out to the local law enforcement that might not have 00:17:39.825-->00:17:45.898 it. Now another example was the FBI also did some aerial surveillance of uh this suspect 00:17:45.898-->00:17:51.737 um in Arizona who they believe had been shooting a gun at cars on the interstate 10 out there. 00:17:51.737-->00:17:55.975 They flew up and down the, the freeway, followed him to his work, circled his work then 00:17:55.975-->00:17:59.845 circled his apartment uh then the next day they arrested him. So we can assume that they were 00:17:59.845-->00:18:04.350 probably getting intelligence on ya know how does he go into his house, what does he drive, where 00:18:04.350-->00:18:08.421 is he driving, who is he seeing? Before they went in with the SWAT team and got him. Now 00:18:08.421-->00:18:13.326 phoenix PD recently used their military style aircraft to follow a guy who had stolen a 00:18:13.326-->00:18:19.065 U-Haul and led police through uh the city. Um, and that's another example of it being used. So 00:18:19.065-->00:18:23.769 it's even been used in, in smaller crimes like that and not just major investigations. Now 00:18:23.769-->00:18:29.342 the planes they use front companies. Sam talked a lot about that. The FBI, FBP, DOJ 00:18:29.342-->00:18:31.978 they're the main ones that use front companies. Local law Enforcement doesn't really do 00:18:31.978-->00:18:35.815 that. But we still don't' know what's on those planes. But there's a 10 dollar records 00:18:35.815-->00:18:39.552 request that I encourage everyone here to do and we will put a link to how to do this if 00:18:39.552-->00:18:43.389 you're curious about an aircraft. It's 10 dollars, they send you a CD the next week and 00:18:43.389-->00:18:47.827 it comes with the airworthiness and registration history for the plane for its entire history and 00:18:47.827-->00:18:51.197 you can look through and see what's been installed you can see it changing hands. An 00:18:51.197-->00:18:57.536 example is, this plane here uh I'd saw it doing weird patterns near the border and it belonged 00:18:57.536-->00:19:01.574 to this Global GEO Mapping Inc who I couldn't find any information on anywhere. They 00:19:01.574-->00:19:06.545 just had a PO Box registration in Albuquerque. I get the registration uh CD in the mail 00:19:06.545-->00:19:10.716 look in it, look at it, literally the first page it says, you know registrated, 00:19:10.716-->00:19:15.721 registration access, or address PO Box in Albuquerque but physical address Kirtland 00:19:15.721-->00:19:20.993 Airforce Base, customs and border patrol. So you can, it's pretty easy to find these planes 00:19:20.993-->00:19:25.264 once you kind of start learning that um, but now they're getting they're catching on and we have 00:19:25.264-->00:19:31.604 this problem with Delaware where in Delaware you can register your aircraft to uh like ev- the 00:19:31.604-->00:19:36.609 Delaware Corporation Commissions Office uh and it just becomes a black hole and you can't get any 00:19:36.609-->00:19:41.447 information. Uh for example the Delaware Corporation Commissions Office has over 2-->000 planes 00:19:41.447-->00:19:47.186 registered to that building. Um including Google's Ash Floyd LLC which is their front company for 00:19:47.186-->00:19:51.957 their drone program. Uh that they're using to eh look at using drones to deliver things 00:19:51.957-->00:19:57.263 or uh update their maps. And they have about 10 hummingbird drones registered to the 00:19:57.263-->00:20:02.034 Delaware Corporation Commissions Office. Um so that's another new trick that they're using. Now 00:20:02.034-->00:20:07.706 this video real quick, this is of the Pilatus uh aircraft in Phoenix following a guy who had 00:20:07.706-->00:20:12.678 stolen the U-Haul. You can see there that the, they changed lenses because they have about 3 00:20:12.678-->00:20:17.550 different lenses. They were orbiting about 17-->000 feet 10 miles away. And as you can see 00:20:17.550-->00:20:22.621 um, it's pretty clear, you can get in really close with this camera and this is a very 00:20:22.621-->00:20:26.892 compressed version of the video they gave me. So I imagine even on, probably on their version 00:20:26.892-->00:20:30.863 the clarity is even better and you can probably read the license plate, see what state is 00:20:30.863-->00:20:36.202 on the side of the U-Haul and what mural they have. Um and right here you can tell that 00:20:36.202-->00:20:40.239 they're starting to designate the U-Haul with the laser designator there. In order to 00:20:40.239-->00:20:44.043 better track it. Um, now we're going to skip forward just a little bit here. See there they 00:20:44.043-->00:20:48.147 changed the lens again and in a second here it will go to IR, the infrared and so you'll get 00:20:48.147-->00:20:52.151 to see a little bit of what that does. Not sure why they went to IR on this, it was a clear day 00:20:52.151-->00:20:56.622 so, uh they might of just wanted to get some good footage but you can see that, you can see the 00:20:56.622-->00:21:02.995 heat from the, from the road from the car, um and it's, it's you can a lot of great 00:21:02.995-->00:21:08.434 information for them, or they get a lot of great information with that system. Um, now I'm 00:21:08.434-->00:21:15.107 gonna kick it over to Jason. >>Um, hi everybody again. Um, so we've discussed some ways to 00:21:15.107-->00:21:20.112 track down surveillance flights by uh following the paperwork uh documentation, registration, but 00:21:23.682-->00:21:30.623 I wanna look at how we more generally detect surveillance activity. Um, registrations can 00:21:30.623-->00:21:36.629 be changed and obscured. We're seeing government agencies get a little bit more clever about how 00:21:36.629-->00:21:41.634 they register aircraft and, and use shell companies. Um, eh, it's also um possible that 00:21:46.505-->00:21:51.544 contractors start to get more involved in surveillance. Uh some surveillance technologies 00:21:51.544-->00:21:58.083 are commercially available. Uh as the technology improves more and more will and you could see 00:21:58.083-->00:22:04.356 um corporations conducting more surveillance, um you could ever see wealthy private individuals 00:22:04.356-->00:22:10.362 funding and, and uh conducting surveillance. Um and then there's also kind of a, a big 00:22:10.362-->00:22:15.401 question we've, we've talked all primarily about the United States but um how much of this 00:22:15.401-->00:22:20.806 surveillance activity is happening in other parts of the world? Um, are agencies in the 00:22:20.806-->00:22:25.811 UK, in Europe, in other developed countries, or, or even uh the developing world 00:22:27.913-->00:22:32.918 conducting aerial surveillance of this type? And uh, if they aren't um when will then? Um, so 00:22:37.523-->00:22:44.296 I'm gonna discuss some, some techniques to, to filter out surveillance flights. Um based 00:22:44.296-->00:22:49.034 on taking advantage of knowledge about the, the technical and operational requirements that 00:22:49.034-->00:22:54.039 dictate relatively unique flight patterns. Um, and as you can see there, um that flight pattern 00:22:56.642-->00:23:02.581 doesn't look like somebody uh going, going on a commercial flight. That's um, kind of this 00:23:02.581-->00:23:08.387 odd cork screw patterns and then you can see there's a really heavily defined circle there and 00:23:08.387-->00:23:13.392 um whoever is at the center of that circle should probably get a lawyer. [off mic question] 00:23:17.763-->00:23:24.303 Possible. Yeah. [off mic comment] Okay, um so the question becomes how, how do we 00:23:24.303-->00:23:30.876 track aircraft. So as a hobbyist or an individual radar is not a practical way to, to track 00:23:30.876-->00:23:36.448 aircraft. But fortunately most aircraft are equipped with transponders that uh beacon a 00:23:36.448-->00:23:38.450 signal, uh with a unique identifier that's uh actually internat- internationally 00:23:38.450-->00:23:40.452 unique. Uh is- issued by the International Civil Aviation Organization. Um the protocol is 00:23:40.452-->00:23:42.454 called automatic dependent surveillance broadcast or ADS-B. Um and it's just how a beaconing 00:23:42.454-->00:23:47.459 signal. Uh and positions of aircraft can be calculated uh with the technique called 00:24:00.005-->00:24:04.943 multilateration. Uh that's where you compare the time difference of arrival for messages that um 00:24:10.649-->00:24:17.623 arrive at multiple receivers. Um you need 4 or more receivers to calculate positioning of a 00:24:17.623-->00:24:22.628 transmitter with multilateration. Uh obviously more receivers gives you a 00:24:22.628-->00:24:29.001 little bit better fix and better data. Uh and there are some websites and aggregator networks 00:24:29.001-->00:24:34.006 that collect feeds from ADS-B receivers and uh calculate out aircraft positions. Um some 00:24:36.675-->00:24:42.781 aircraft also transmit uh additional information, they'll transmit they're latitude and 00:24:42.781-->00:24:47.786 longitude as calculated by their GPS unit, um squat codes, and uh altitude. Um but we can't 00:24:51.724-->00:24:57.830 necessarily rely on the accuracy of the uh positions transmitted by the aircraft so uh 00:24:57.830-->00:25:04.403 multilateration is really helpful in confirming that from multiple sources. So, uh the 00:25:04.403-->00:25:09.408 question becomes how do we get a hold of ADS-B data at scale? Uh fortunately there's already a 00:25:13.078-->00:25:20.052 large and active community of radio and aviation and hacking enthusiast that collect ADS-B 00:25:20.052-->00:25:26.959 data and um send it to aggregator networks. Um, all that you need is a Raspberry Pi 00:25:26.959-->00:25:33.499 even your old model 1B that you might have in the closet will work. Um an RTL-SDR radio, which 00:25:33.499-->00:25:39.438 is a little software defined radio you can buy one for 25 dollars uh over in the vendor 00:25:39.438-->00:25:45.844 area if they're not out. Um and antenna of varying size and quality um and an internet 00:25:45.844-->00:25:50.849 connection. And um so for under 100 dollars you can pick up uh transmissions um from aircraft. 00:25:54.386-->00:25:58.924 And uh there are multiple aggregator networks that collect ADS-B data and, and calculate 00:25:58.924-->00:26:05.230 positions. Uh couple of the big ones, Flightradar24 dot come and Flightaware dot com. And I'll 00:26:05.230-->00:26:11.770 talk more about ADS-B exchange dot com uh which is a bit unique. Um and uh the FAA 00:26:11.770-->00:26:17.676 regulations require an increasing number of aircraft to transmit ADS-B, it's' just part 00:26:17.676-->00:26:24.450 of their next gen program to revamp uh air traffic control systems. So, by 20 20 most 00:26:24.450-->00:26:31.056 aircraft are uh supposed to be equipped with ADS-B out. Um and then there's similar regulations 00:26:31.056-->00:26:36.061 around the world um in the EU, India, Australia, etcetera. Okay but there are some challenges 00:26:40.666-->00:26:46.638 and limitations to the uh availability data. Um, the major commercial flight tracking sites 00:26:46.638-->00:26:51.643 augment their data with FAA radar data. Um the FFA gives you this data for free but you have 00:26:54.746-->00:27:00.652 to accept restrictions. And the, and the restriction is that uh tracking sites don't publish the 00:27:00.652-->00:27:06.825 positions of aircraft that are on their block list. Um and any aircraft owner can request that 00:27:06.825-->00:27:13.165 their aircraft be added to the block list for free. Um and even if the tracking site um is 00:27:13.165-->00:27:18.704 picking it up over ADS-B or some other method, they're, they're not allowed to publish it. Um so 00:27:18.704-->00:27:23.709 the commercial sites um, have some restrictions there. Um, also bulk access to data is uh, 00:27:26.678-->00:27:33.085 limited or expensive if you, it's not practical to get all of the data for all flights in real 00:27:33.085-->00:27:38.090 time from um these commercial sites. Um but AD ADS-B exchange is an exception. Uh they 00:27:41.093-->00:27:46.098 explicitly do not use FFA data and they don't censor flights. Um they also provide free access 00:27:48.200-->00:27:54.873 to live and historical data. Um, I encourage anybody who uses that data to, to donate to them. 00:27:54.873-->00:28:00.812 They don't endorse us but uh we are big fans of their work, so. So, um to pick surveillance 00:28:00.812-->00:28:05.817 flights out of that live streaming data um, there there's a bit of work that has to be 00:28:09.354-->00:28:16.128 done. Um, there are over 80-->000 flights a day. Uh the uncompressed data coming from 00:28:16.128-->00:28:21.133 ADS-B exchange works out to about 10 gigs a day. Um and at any given time there's 8-->000 to 00:28:23.569-->00:28:29.942 13-->000 aircraft being tracked in the air. Usually most of these are not surveillance flights. Um 00:28:29.942-->00:28:36.882 and so the question becomes how do we pick out the surveillance flights? So, um looking at 00:28:36.882-->00:28:41.887 surveillance flights versus other flights uh we know most non-surveillance traffic goes 00:28:45.557-->00:28:51.964 from point A to point B as quickly and efficiently as possible. Um and commercial 00:28:51.964-->00:28:57.469 traffic and, and most other traffic will try and minimize flying over populated airspace. 00:28:57.469-->00:29:03.742 Populated areas um so commercial airlines have noise restrictions, they are not 00:29:03.742-->00:29:08.747 allowed to fly too low over particular areas. Uh and then furthermore uh if you're in a 00:29:11.783-->00:29:16.555 metro area there are lot of different airports including generally a large commercial 00:29:16.555-->00:29:21.960 airport and each of these airports has their own ring that's controlled air space at 00:29:21.960-->00:29:27.966 different levels. So, this is a screenshot of um the airspace around Phoenix Sky Harbor 00:29:27.966-->00:29:32.971 airport. Um and you can see that bowtie pattern is the main, main area of controlled airspace, um 00:29:35.974-->00:29:40.712 around the approach for the runways of sky harbor. Uh and there are different altitude 00:29:40.712-->00:29:46.618 indicators that show that you can't fly below this range or above this range without getting 00:29:46.618-->00:29:53.325 clearance from the tower. So, that's um that's a hassle for, for a leisure pilot or a 00:29:53.325-->00:29:59.898 commercial travel. They want to get up and out of controlled airspace um as quickly as 00:29:59.898-->00:30:04.903 possible and not have to go in holding patterns, etcetera. So and then we also know from um, 00:30:08.874-->00:30:13.879 some of the, the technical information 'bout the equipment on board there are technical and 00:30:15.947-->00:30:20.986 operational constraints for running surveillance flights. There's an altitude sweet spot 00:30:20.986-->00:30:26.324 in that if you're flying too low you're going to make noise and you're going to be much more 00:30:26.324-->00:30:32.030 noticeable. Um you also have to deal with controlled airspace from smaller regional airports, 00:30:32.030-->00:30:37.035 um terrain, uh other challenges. On the other side of the coin um most of these aircraft are not 00:30:39.604-->00:30:44.976 pressurized and so they can't really get much higher than 10-->000 feet and they're not, not 00:30:44.976-->00:30:49.981 built for that. Um, they also would have to deal more so with uh airspace, controlled airspace 00:30:52.484-->00:30:57.489 from, from the major commercial airport in the area. Um and then just as 1 example, piece of 00:31:00.425-->00:31:07.032 equipment um cell site simulators, um have a range of approximately 2 miles in a metro 00:31:07.032-->00:31:12.671 area. Um according to some documents published by the intercept that may improve a bit 00:31:12.671-->00:31:18.076 but it gives you a good guideline in terms of um range. And if the aircraft is too high 00:31:18.076-->00:31:24.516 you're going to limit that range on the ground geometrically. Um also surveillance flights 00:31:24.516-->00:31:30.155 typically take off and land at the same airport. It just would get more costly and more 00:31:30.155-->00:31:35.127 impractical if you were trying to schedule surveillance flights between different airports and 00:31:35.127-->00:31:40.132 um so for example in Phoenix uh, the FBI uses the Deer Valley airport, uh which is 00:31:42.200-->00:31:48.039 conveniently about 5 to 10 minutes away from the field office there. Um, furthermore 00:31:48.039-->00:31:54.279 most surveillance is interested in people that live in densely populated metro areas. So, these 00:31:54.279-->00:32:01.019 flights are going to be corkscrewing around um over cities, not going straight from 00:32:01.019-->00:32:06.024 ya know Las Vegas to New York, etcetera. And then there's some other, other aims, so aircraft 00:32:08.360-->00:32:14.900 capabilities. Certain aircraft fly too fast it wouldn't' be practical to use a 737 to 00:32:14.900-->00:32:19.704 conduct surveillance over, over a city because it just would move too quickly you wouldn't 00:32:19.704-->00:32:26.478 have much dwell time over the city, aside from costs. Um, but then the aircraft can't be too 00:32:26.478-->00:32:31.983 small because they need to generate enough power or carry uh batteries to operate this 00:32:31.983-->00:32:37.422 relatively energy hungry surveillance equipment. Um they also need to be large enough to 00:32:37.422-->00:32:42.427 carry these relatively large surveillance pods. So, um, there's kind of a, a sweet spot 00:32:44.830-->00:32:49.835 in terms of size of aircraft as well. So, um with some of that information being considered um, 00:32:54.105-->00:33:00.979 I've worked on just a prototype of a, a surveillance score. So, where you look at the behavior 00:33:00.979-->00:33:05.984 of aircraft and you um try and thin it out as surveillance or not surveillance. Um, the main 00:33:08.186-->00:33:14.726 metric that I'm using is just calculating 90 degree turns um so that's where, where the 00:33:14.726-->00:33:20.498 heading of the aircraft changes more than 90 degrees and each time that happens you, you just 00:33:20.498-->00:33:27.305 increment the score. Um, most commercial traffic is going to only have a couple of turns, 00:33:27.305-->00:33:33.812 maybe to make final approach. Um, and that's all conditional based on altitude so again we, 00:33:33.812-->00:33:40.085 we know that there's generally a sweet spot for, for altitude so um, turns and activity that are 00:33:40.085-->00:33:46.524 in that, that range are scored higher. Um, lower flying flights might be somebody in a flight 00:33:46.524-->00:33:52.797 school that's practicing takeoffs and landings. Um, higher up it might be a, a 00:33:52.797-->00:33:59.638 commercial jet that's um in a holding pattern before coming in. So, and then there's some, 00:33:59.638-->00:34:04.576 some a lot of other uh angles to, to further improve uh detection. So considering the 00:34:07.379-->00:34:13.485 proximity to airports and controlled airspace might help filter out more, um more flight 00:34:13.485-->00:34:18.490 schools. Uh and then uh scoring based on the aircraft model, again we, we have a pretty good 00:34:21.126-->00:34:27.565 idea there, there's a good list of aircraft that are popular for surveillance and, and well, well 00:34:27.565-->00:34:34.339 built for that purpose. And so scoring those higher is, is another angle to, to highlight 00:34:34.339-->00:34:40.812 uh surveillance activity. Um, also I think that uh it's suspicious to request that your 00:34:40.812-->00:34:47.652 plane be on the block list and so I would highlight those as interesting, uh potential 00:34:47.652-->00:34:52.290 surveillance aircraft. Um a little bit more geometry might help to filter out survey 00:34:52.290-->00:34:57.295 activity which can sometime trick, trip this um basic turn counting. Um and then uh 00:35:01.733-->00:35:07.172 comparing flights to interesting geography will, will also highlight uh surveillance 00:35:07.172-->00:35:12.177 activity. So, for example Jerod found some flights that went, uh up and down along the Arizona uh 00:35:14.679-->00:35:19.684 US Mexico border. Um so highlighting the border as a interesting geographic area uh 00:35:23.488-->00:35:30.028 will help, help to uh pick out surveillance flights because there's not really any other 00:35:30.028-->00:35:36.067 reason that somebody is going to fly the length of the US Mexico border. Um also events like 00:35:36.067-->00:35:41.072 protests, uh the Freddie Gray protest, the protests at the shooting of Jamar Clark, Dakota 00:35:44.142-->00:35:49.147 access pipeline protest. We've all seen aerial surveillance and so some scoring of, of those 00:35:52.183-->00:35:57.188 interesting areas might be another angle to, to highlight and uh spot spy planes. So, just 00:35:59.958-->00:36:04.963 looking at, at that crude metric of counting the number of 90 degree turns. Um, most flights 00:36:09.667-->00:36:15.440 with 30 or more turns look like surveillance flights. In that when you start to dig into the 00:36:15.440-->00:36:21.212 paperwork you look where it's registered, you look at airworthiness documents um a 00:36:21.212-->00:36:26.217 large share of those aircraft are surveillance. Um so this is just an example screenshot of a 00:36:28.753-->00:36:33.758 uh, aircraft over Baltimore during the Freddie Gray protest. So, um the prototype code that 00:36:37.729-->00:36:42.734 I'm going to release after the talk is um, takes data um, in a real time live stream from ADS-B 00:36:49.240-->00:36:54.245 exchange and it um, holds it into virtual radar server. Um ADS-B exchange has a real-time 00:36:59.784-->00:37:04.723 API. Um that, that V- VRS works this. And virtual radar server then provides a JSON endpoint 00:37:08.593-->00:37:14.466 that you can grab bye snippets from. Um the reason why I'm using that angle is that um 00:37:14.466-->00:37:21.239 ADS-B exchange also offers historical data that's in that same JSON format. So in the 00:37:21.239-->00:37:27.579 future could take that historical data and um use pretty much the same methodology 00:37:27.579-->00:37:33.718 and work on refining the score and uh spotting historic surveillance events. Um, so 00:37:33.718-->00:37:38.723 that's just loaded into a queue um split across multiple threads that can be uh, distributed 00:37:40.859-->00:37:47.365 across multiple machines so that's helpful. Um, should 1 start to analyze um a large 00:37:47.365-->00:37:53.605 archive of historical flight data you spin out virtual machines and uh just kind of 00:37:53.605-->00:38:00.411 crank through it. And then that's all queued up. Um it's composed in redis which has some 00:38:00.411-->00:38:06.818 really nice geo hashing features. Um and uh the surveillance flights are dumped 00:38:06.818-->00:38:11.823 out as JSON files into uh, a web map so you can look at it on the web. And see these aircraft are, 00:38:15.560-->00:38:22.467 are highlighted as surveillance, or suspicious at least. Um and then it also um archives the 00:38:22.467-->00:38:27.472 files into mongoDB so you can do some geo queries and, and look at it um further. And do 00:38:30.775-->00:38:35.980 retrospective analysis. So maybe uh the surveillance score algorithm you want to improve it 00:38:35.980-->00:38:42.520 and tweak it and the reprocess all of your old data and see if anything else falls out. Um so 00:38:42.520-->00:38:47.525 that's the idea there. And this will all be open source. The um, the white boxes are, are code 00:38:52.230-->00:38:58.770 that I'm releasing. Um, so here's an example, screenshot of, of the web map uh this is 00:38:58.770-->00:39:04.776 actually a very new aircraft. It was purchased by the California highway patrol uh circling over 00:39:04.776-->00:39:09.781 the Sacramento area. You can see uh it kind of does some, some loops and corkscrews and then it 00:39:12.450-->00:39:18.089 goes around different areas. I think this might actually be um, looking for speeders. Uh I 00:39:18.089-->00:39:21.859 don't' know if anybody's driven through California and they have these signs, speed monitored by 00:39:21.859-->00:39:26.864 aircraft. I didn't really believe it until um, looking at the data a bit more. So, um. So, 00:39:29.500-->00:39:34.505 um if you're interested in this and wanna work on tracking spy planes there are a lot of things 00:39:38.109-->00:39:43.114 you can do. Um, it's pretty easy to set up an ADS-B receiver for less than 100 dollars. Um a good 00:39:46.284-->00:39:52.857 little weekend project and feed data to ADS-B exchange dot com. Um, you can donate to ADS-B 00:39:52.857-->00:39:57.862 exchange um they don't endorse us but uh you can also donate to us. Um, the code will be open as 00:40:00.698-->00:40:06.137 well so you can use fork and improve the application. It's still very much a prototype but 00:40:06.137-->00:40:11.142 um it does highlight interesting planes. Um, so if you have questions um, or wanna see more 00:40:13.645-->00:40:18.650 information you can go to our website uh we're gonna have a, um some links and article up uh 00:40:21.819-->00:40:26.824 hopefully later today once we get online. Um and links to Github soon. Thank you. Um. 00:40:33.498-->00:40:38.002 [applause] >>Thank you. [applause]