[00:04.300 --> 00:08.560]  Hi, thanks for coming to my talk. My name is Matt Smith. I'm a postdoc from the
[00:08.560 --> 00:11.120]  University of Oxford and today I'm going to talk to you about
[00:11.120 --> 00:14.720]  some research that we did recently where we tried to understand how pilots
[00:14.720 --> 00:18.180]  might react when faced with realistic attacks on
[00:18.180 --> 00:21.860]  their systems. So who are we? Well, we're a team of
[00:21.860 --> 00:23.520]  researchers from the University of Oxford
[00:23.520 --> 00:27.320]  and ArmaSwiss. ArmaSwiss is the research and development part
[00:27.320 --> 00:31.220]  of the Swiss Ministry of Defence and we've been involved in security
[00:31.220 --> 00:37.040]  research now for about eight years, covering topics such as ADS-B and ACARS,
[00:37.040 --> 00:40.200]  more recently broadening some other areas as well.
[00:40.200 --> 00:43.420]  We've also been involved in founding OpenSky, which is
[00:43.860 --> 00:46.820]  a really great project. I really recommend you check it out, especially if
[00:46.820 --> 00:51.180]  you're involved in aviation. It is a research-focused
[00:51.940 --> 00:56.760]  ADS-B MODIS collection network, so really would recommend checking that out.
[00:57.360 --> 01:04.420]  So the past 10 years has really seen a rise in research from
[01:05.660 --> 01:08.580]  a wide range of groups all around the world, publishing at
[01:08.580 --> 01:13.540]  top-tier conferences, in much more practical security venues like
[01:13.540 --> 01:18.480]  Black Hat and DEFCON, master's theses, all sorts of different bits of research.
[01:18.660 --> 01:22.820]  And often these papers have proposed really interesting attacks
[01:23.400 --> 01:27.060]  that really, you know, propose something new
[01:27.060 --> 01:33.060]  and focus on the part between the transmission of the signal and
[01:33.060 --> 01:35.500]  getting this onto the aircraft. And obviously you can kind of predict
[01:35.500 --> 01:39.980]  some effects that this might have on how the aircraft is flown,
[01:39.980 --> 01:43.920]  but it really does raise the question of how well do pilots handle
[01:44.600 --> 01:50.200]  these kind of attacks. So we think this is a really interesting
[01:50.200 --> 01:54.020]  question because pilots are often assessed on how they handle faults
[01:54.020 --> 01:57.320]  and they usually do this in a flight simulator,
[01:57.320 --> 02:00.020]  and this is a full motion one on the right-hand side here,
[02:00.020 --> 02:03.980]  and they're sort of a last line of defense against these faults
[02:03.980 --> 02:06.740]  because they have this really well-defined procedure, it's part of
[02:06.740 --> 02:09.760]  their training and, you know, some really key part of
[02:09.760 --> 02:14.780]  their job. And we started to think about how
[02:14.780 --> 02:18.800]  this fault handling skill might translate to mitigating attacks
[02:19.360 --> 02:23.420]  and also whether you can use flight simulation to actually understand the
[02:23.420 --> 02:27.180]  impact of attacks like you can use it to understand the impact of faults.
[02:28.300 --> 02:31.680]  So something we really want to focus on at this point and just highlight is
[02:31.680 --> 02:35.260]  the fact that this piece of work is actually interested
[02:35.260 --> 02:39.460]  in the effects of attacks rather than, say, the technical detail of the attacks
[02:39.460 --> 02:42.700]  themselves. So as we all know, normally the kind of
[02:42.700 --> 02:46.660]  chain of signal to action is there's some sort
[02:46.660 --> 02:50.020]  of legitimate signal from air traffic control or whatever it might be,
[02:50.580 --> 02:54.540]  gets to the aircraft and presents to the pilots somehow.
[02:54.740 --> 02:59.420]  The pilots act on that and then there's some sort of behavior
[02:59.420 --> 03:03.040]  that comes out of it and that's normal. As we know,
[03:03.040 --> 03:06.240]  many of these aviation communications are unencrypted, so
[03:06.880 --> 03:09.940]  you know, we have this part that I just talked about where there's been a lot of
[03:09.940 --> 03:13.640]  research, this creation of signals that are transmitted to the aircraft.
[03:13.640 --> 03:17.120]  But what we're interested in this piece of work is what happens when those
[03:17.120 --> 03:19.840]  signals arrive at the aircraft and are presented to the pilots. How do they
[03:19.840 --> 03:23.200]  perceive them? What do they do with them? Does it affect how the aircraft
[03:23.200 --> 03:30.780]  is flown? So to do this, we invited 30 tight-rated Airbus A320 pilots to
[03:30.780 --> 03:34.840]  come and fly in our little simulator that we built in our
[03:34.840 --> 03:40.700]  building. And within that simulator, we implemented some attacks on collision
[03:40.700 --> 03:44.300]  avoidance systems, ground proximity warning systems, and landing systems.
[03:44.300 --> 03:48.360]  And we're only going to cover two of those today just to keep it focused,
[03:48.360 --> 03:52.460]  but we do have some more info if people are interested in the other one.
[03:52.580 --> 03:59.380]  We used X-Plane 11, which is a high quality aircraft model, and we had a
[03:59.380 --> 04:03.680]  single pilot set up. So the experimenter provided some basic flying support in
[04:03.680 --> 04:06.500]  terms of pushing buttons and enabling modes, but nothing more than that. They
[04:06.500 --> 04:09.240]  didn't actually contribute to decisions made.
[04:09.800 --> 04:13.820]  In terms of how we actually carried this out, we had a familiarization flight
[04:13.820 --> 04:17.400]  first so the pilots could get used to the route and the controls, and then we
[04:17.400 --> 04:23.020]  had a simulator flight for each of the attacks that implemented the attack.
[04:23.120 --> 04:26.880]  And after they'd flown it, the pilots debriefed with us to tell us a little
[04:26.880 --> 04:30.780]  bit about what they observed and what they thought had gone on.
[04:31.400 --> 04:35.700]  We had quite a spread of experience actually, all the way from fairly newly
[04:35.700 --> 04:40.640]  qualified pilots through to people who'd spent pretty much their entire
[04:41.380 --> 04:43.280]  professional career flying.
[04:44.920 --> 04:49.240]  And so before we carry on, we need to think about what our attacker looks like.
[04:49.560 --> 04:52.300]  What might they want to achieve? And obviously the first thing you think
[04:52.300 --> 04:58.280]  about in this situation is, do they want to cause a crash? And in our kind of
[04:58.280 --> 05:02.300]  experiment, we weren't that interested in that because we had a simulator with
[05:02.300 --> 05:06.900]  some limits, we only had a single pilot in, and the situations in which an
[05:06.900 --> 05:11.140]  attacker would need to be involved in order to cause this kind of kinetic
[05:11.140 --> 05:16.060]  effect would be very extreme, right at the edge of the pilot's capabilities,
[05:16.060 --> 05:20.200]  and that wouldn't be a fair situation for the pilots in, but it also wouldn't
[05:20.200 --> 05:24.200]  really allow us to actually assess anything because they do have some limits
[05:24.200 --> 05:27.840]  in how they interface with the simulator. So instead, we're looking at softer
[05:27.840 --> 05:32.740]  effects, but ones that really do have an impact. So things like triggering
[05:32.740 --> 05:37.900]  go-arounds where the aircraft can't land and so they have to go and take
[05:39.240 --> 05:44.380]  another go at the landing. Forcing unexpected manoeuvres, this could be any
[05:44.380 --> 05:48.940]  sort of manoeuvre really outside of the intended profile of the aircraft.
[05:49.460 --> 05:53.960]  Or it could also be to push crew to switch the systems off that they rely on,
[05:53.960 --> 05:59.020]  which is particularly bad when some of these systems are really quite tightly
[05:59.020 --> 06:04.500]  related to safety. And all of these can have a range of sort of second order
[06:04.500 --> 06:08.800]  effects, if you will, things like causing delay or costing money. And obviously
[06:08.800 --> 06:13.720]  the worst of these is actually reducing safety, if that's what happens.
[06:14.620 --> 06:18.400]  The equipment needed for these attacks is fairly typical for this kind of
[06:18.960 --> 06:21.060]  this kind of experiment or this kind of
[06:22.760 --> 06:29.740]  attack, if you will. And basically here, we're looking at a scientific grade
[06:29.740 --> 06:34.460]  software-defined radio, a high gain amplifier and a directional antenna.
[06:34.540 --> 06:37.740]  Obviously, this will vary a little bit depending on the type of attack and
[06:37.740 --> 06:41.240]  exactly where the attacker is positioned, but fairly standard for this kind of
[06:41.240 --> 06:46.260]  research. So the first system that I'm going to talk about is TCAS. There will
[06:46.260 --> 06:50.020]  be other talks that go into much more detail on TCAS, so I'm not really going
[06:50.020 --> 06:56.020]  to get into the technical detail and specifics. But really what TCAS is trying
[06:56.020 --> 06:59.920]  to do is stop aircraft colliding into each other by stepping in before they
[06:59.920 --> 07:04.920]  get too close and moving them apart if need be. So let's say we have some sort
[07:04.920 --> 07:09.060]  of aircraft that's flying along. Now, TCAS uses Mode S, which you may know
[07:09.060 --> 07:13.100]  about. This is an air traffic control technology that allows air traffic
[07:13.100 --> 07:17.620]  control on the ground to interrogate the intruder, in this case, and the
[07:17.620 --> 07:22.560]  intruder will respond with some information. So this is continuously
[07:22.560 --> 07:27.520]  happening, and in the process of an aircraft responding, obviously other
[07:27.520 --> 07:33.220]  aircraft nearby can see that response. And in this case, the own ship can
[07:33.220 --> 07:37.540]  actually use that information to estimate, roughly speaking, where the
[07:37.540 --> 07:42.500]  intruders or the nearby aircraft might be. This is presented in the cockpit to
[07:42.500 --> 07:45.600]  the pilots. And you can see an example on the right-hand side here. This is
[07:45.600 --> 07:51.440]  what it might look like in an Airbus, with the red dot there being a much
[07:51.440 --> 07:55.880]  higher risk of collision, all the way through to the kind of no threat right
[07:55.880 --> 07:59.340]  at the top there. And obviously some other information is presented, such as
[07:59.340 --> 08:02.860]  altitude difference and whether that aircraft is climbing or descending.
[08:03.480 --> 08:08.480]  So let's say these aircraft continue to fly towards each other. At some point,
[08:08.480 --> 08:12.100]  just passively listening isn't enough, and we move into a phase called active
[08:12.100 --> 08:16.020]  surveillance. Now, at this point, the ownership in this case is directly
[08:16.020 --> 08:21.660]  interrogating the intruder, as if it's air traffic control. And if the system
[08:21.660 --> 08:25.140]  is thinking that these aircraft are starting to be too close, something
[08:25.140 --> 08:29.600]  called a traffic advisory will be issued. And this, in the cockpit, is basically
[08:29.600 --> 08:37.000]  just an annunciation of traffic traffic. And the pilots will then have to kind
[08:37.000 --> 08:41.120]  of get hands-on controls and be ready in case there are further instructions
[08:41.120 --> 08:46.220]  they need to follow. So if the aircraft continue to get closer, even after a
[08:46.220 --> 08:49.980]  traffic alert, it looks like at this point they're going to end up with
[08:49.980 --> 08:54.620]  something called a resolution advisory. And this involves the aircraft actually
[08:54.620 --> 08:59.060]  doing something to move them apart from each other. So they'll use a form of
[08:59.060 --> 09:02.960]  mode S, called coordination messages, to communicate what they're planning to
[09:02.960 --> 09:07.620]  do. And this helps maximize the separation. And once that's been decided,
[09:07.620 --> 09:10.500]  the RAs will be announced in the cockpit. And these are compulsory
[09:10.500 --> 09:14.180]  instructions and they must be followed usually within about five seconds. So
[09:14.180 --> 09:16.900]  let's say, in this case, the intruder has to climb and the own ship has to
[09:16.900 --> 09:21.140]  descend. Now, importantly, TCAS sort of depends on the fact that these
[09:21.140 --> 09:24.540]  aircraft are cooperative and actually don't want to crash into each other.
[09:24.940 --> 09:29.200]  And so that's something to bear in mind for the attack. And the attack is really
[09:29.200 --> 09:34.200]  quite simple, at least at a high level. So mode S has been shown to be insecure
[09:34.200 --> 09:40.080]  and vulnerable to injection by Costin and Schaefer and many others as well
[09:40.080 --> 09:43.700]  over the years. So if we have some aircraft that's flying along and is
[09:43.700 --> 09:48.520]  issuing a mode S interrogation, we have an attacker who is injecting mode S
[09:48.520 --> 09:52.480]  responses to create a sort of false intruder. And this false intruder will
[09:52.480 --> 09:57.420]  appear to be flying towards the target aircraft, which will eventually cause a
[09:57.420 --> 10:01.840]  TA and then an RA and require avoiding action. So in our simulator scenario, we
[10:01.840 --> 10:06.420]  saw this happen multiple times. And the aim was to see sort of how many
[10:06.420 --> 10:11.280]  resolution advisories pilots would fly. And we found, generally speaking, that
[10:11.860 --> 10:16.760]  pilots actually, after a handful of these resolution advisories, started to
[10:16.760 --> 10:20.120]  think something was up and something weird was going on. So we found that 26
[10:20.120 --> 10:23.660]  of our pilots ended up turning the sensitivity of TCAS down, so it only
[10:23.660 --> 10:29.480]  issued traffic advisories. And a further 11 of those actually turned the system
[10:29.480 --> 10:34.420]  to standby. So at that point, they received no TCAS announcements at all.
[10:34.420 --> 10:38.720]  And this typically happened, the first stage was after about four to five RAs,
[10:38.720 --> 10:42.220]  and the further switch down to standby was after another two to three. So it
[10:42.220 --> 10:46.240]  took a little while, but eventually, it got there. So the important thing to
[10:46.240 --> 10:52.300]  remember here is that switching to standby actually causes the loss of TCAS.
[10:52.300 --> 10:56.200]  And it means that you then shift the burden of keeping your aircraft away
[10:56.200 --> 10:59.780]  from others onto air traffic control. And this was often done because the
[10:59.780 --> 11:02.960]  pilots felt they had no choice but to do it, there was something going on with
[11:02.960 --> 11:07.140]  But this then means that air traffic control have to keep you further apart
[11:07.140 --> 11:11.480]  from other aircraft, they also have an extra bit of workload on their part in
[11:11.480 --> 11:16.920]  making sure that you're always further away. On top of that, there was also
[11:16.920 --> 11:20.340]  excess fuel burn by following RAs, you don't really have a choice in that, but
[11:20.340 --> 11:24.800]  you have to do these maneuvers. And that really does use some of your fuel.
[11:25.680 --> 11:29.860]  We found that most of the pilots continued on the route, but some of them
[11:29.860 --> 11:34.880]  felt that they needed to make extra maneuvers. So maybe to climb above clouds
[11:34.880 --> 11:38.540]  or to get out of cloud in some way or other. So they could see whether there
[11:38.540 --> 11:41.660]  was an intruder nearby, or in some cases actually go back to the departure
[11:41.660 --> 11:45.420]  airport. And this was just because they felt like they couldn't really
[11:46.780 --> 11:53.040]  fly with TCAS in an inoperative state. So generally speaking, this suggests
[11:53.040 --> 11:58.560]  that attackers can push pilots to fly these unnecessary RAs or reduce TCAS
[11:58.560 --> 12:03.780]  sensitivity. And if we look at some of the participant responses in a bit more
[12:03.780 --> 12:09.320]  detail, we found that many participants noted that RAs were actually really rare.
[12:09.320 --> 12:14.580]  And so this many RAs in close succession suggests something weird is going on.
[12:14.580 --> 12:20.740]  On top of that, one pilot who had had 17 years of flying experience had only
[12:21.280 --> 12:27.620]  had 10 RAs in his career, or less than 10 RAs even. And so it really gives you
[12:27.620 --> 12:32.340]  an idea of how rare these really are. Something that the participants told us
[12:32.340 --> 12:35.460]  was that weather would make attack identification much harder, because you
[12:35.460 --> 12:39.600]  can't just look out the window and see whether anything's flying towards you.
[12:40.300 --> 12:44.220]  And they also suggested that these sudden repeated RAs might actually
[12:44.220 --> 12:49.040]  affect other aircraft as well. So 28 of our participants felt that this
[12:49.040 --> 12:52.180]  attack lowered the safety of the aircraft because of this, but also because
[12:52.180 --> 12:56.980]  of things like it moving the passengers about on board.
[12:58.360 --> 13:04.540]  And finally for this attack, pilots had to reduce sensitivity in the key safety
[13:04.540 --> 13:09.220]  system because of this distraction. One participant put this really well.
[13:09.220 --> 13:13.520]  They said that it was a sort of crying wolf effect where, you know, they're
[13:13.520 --> 13:17.720]  trained to listen to this system, do what it tells them to do, and that it's a
[13:17.720 --> 13:21.240]  really important safety system, but they had to doubt that system and it might
[13:21.240 --> 13:23.920]  impact how they respond to TCAS in the future.
[13:24.680 --> 13:27.940]  So the next system I'll talk about is the instrument landing system. Again,
[13:27.940 --> 13:31.660]  there's going to be some much more detailed talks on how this works and some
[13:32.760 --> 13:37.820]  technical discussion of attacks on the system. But for this, we're just going to
[13:37.820 --> 13:41.880]  talk about the glide slope. So the idea here is that the instrument landing
[13:41.880 --> 13:47.460]  system provides a method to do precision approaches that can be done in a wide
[13:47.460 --> 13:52.100]  range of weather conditions. And it's really useful for busy air spaces where
[13:52.100 --> 13:55.080]  you need to land a lot of aircraft very accurately, for example.
[13:56.540 --> 14:02.420]  So generally speaking, the aircraft will follow a glide path down to the
[14:02.420 --> 14:05.900]  touchdown zone of the runway. The autopilot will help in following this
[14:05.900 --> 14:11.420]  glide path. And the way it does this is there are two lobes, two signal lobes.
[14:11.420 --> 14:15.820]  One of them is modulated at 150 hertz and one is modulated at 90 hertz.
[14:16.300 --> 14:21.060]  And the aircraft will measure the relative signal strength of these two lobes.
[14:21.060 --> 14:25.120]  And when the signal strength is equal, it is on the correct glide path.
[14:25.480 --> 14:29.360]  The glide path itself will depend a little bit on the airport and the approach,
[14:29.360 --> 14:31.420]  where usually it's about three degrees.
[14:32.060 --> 14:35.380]  So the attack is fairly simple because this system is fairly simple.
[14:35.380 --> 14:38.540]  There's no security in place. So we have this idea that an attacker
[14:38.540 --> 14:43.100]  transmits a false glide slope from the other side of the far end of the runway,
[14:43.100 --> 14:48.020]  maybe off to the side in some private land rather than actually on the airport.
[14:48.940 --> 14:53.400]  And they don't introduce a huge difference. It's 350 feet, 100 meters.
[14:53.440 --> 14:57.200]  But that's big enough to really make a huge difference in where the aircraft
[14:57.200 --> 15:02.840]  would touch down if it followed it. And the concept is that the aircraft will
[15:02.840 --> 15:09.640]  intercept either from above or the attacker might be able to overpower the real glide slope.
[15:09.640 --> 15:14.360]  And this will cause them to follow the false glide slope all the way to the ground.
[15:15.200 --> 15:22.240]  So as I mentioned, this is very similar to Harshad's attack on the localizer.
[15:22.240 --> 15:25.720]  So I really encourage you to check that talk out for much more technical detail.
[15:25.960 --> 15:32.260]  And the kind of overall idea of this attack is to get the aircraft to overshoot the runway
[15:32.260 --> 15:37.900]  and abort the approach or land deep. Generally speaking, we found that
[15:37.900 --> 15:42.480]  participants had to abort their approach. So they would arrive at the final stability check
[15:42.480 --> 15:47.980]  around the 1,000 feet in altitude mark. And they would realize that they were
[15:47.980 --> 15:52.040]  really on the wrong glide slope and had to have another go.
[15:52.040 --> 15:56.020]  However, in the following attempts, participants really avoided the glide slope.
[15:56.020 --> 15:58.720]  They used a wide range of different approach methods.
[15:58.760 --> 16:03.520]  Some just still using some ILS tools, but not glide slope like lock DME,
[16:03.520 --> 16:07.060]  some using RNAV, and some using a visual approach.
[16:07.060 --> 16:12.900]  Generally speaking, the decision to go around was made about, as I mentioned,
[16:12.900 --> 16:18.420]  the kind of 1,000 feet mark, about one mile to go. So just over a minute before landing.
[16:18.420 --> 16:22.480]  So this seems quite late, but it is fairly safe and kind of well within
[16:23.100 --> 16:29.400]  the normal kind of checks, if you will. But some pilots did choose to land anyway.
[16:29.400 --> 16:33.320]  When they did, they really had to make a very steep correction. It was often at the limit of
[16:33.320 --> 16:36.180]  the kind of correction they could make. So it wouldn't always be possible.
[16:36.180 --> 16:38.880]  It really would depend on the conditions on the day.
[16:40.020 --> 16:45.080]  So again, this suggests that, you know, attackers can push pilots to
[16:45.080 --> 16:50.140]  miss an approach in this case, and even abandon using the glide slope completely.
[16:51.100 --> 16:55.960]  And if we go back to the participant comments on this, they all identified an issue,
[16:55.960 --> 16:58.400]  but they very quickly lost confidence in the glide slope.
[16:58.400 --> 17:01.600]  So this attack really wouldn't work beyond one approach.
[17:01.600 --> 17:03.460]  And it might not work for multiple aircraft,
[17:03.460 --> 17:06.000]  because they would realize something weird was going on very quickly.
[17:06.780 --> 17:11.200]  They also commented that it was really a lot harder to manage this in low fuel situations.
[17:11.200 --> 17:15.580]  If you only have the fuel to do one more approach and a diversion,
[17:16.720 --> 17:19.760]  you've got a lot of stress and a lot of work to try and figure out
[17:19.760 --> 17:22.320]  what's going on and make sure it doesn't happen again.
[17:23.240 --> 17:26.540]  Something that we didn't really think about when we were devising this attack is
[17:26.540 --> 17:29.580]  how they might detect something was going on.
[17:29.660 --> 17:31.860]  And one of the key factors in this was actually
[17:31.860 --> 17:34.400]  the precision path approach indicators, or the PAPIs.
[17:34.400 --> 17:36.560]  So these are some lights to the side of the runway.
[17:36.560 --> 17:39.880]  And if you have two red and two white lights, you're on the correct glide slope.
[17:40.420 --> 17:43.640]  And pilots used this, they said that it was important,
[17:43.640 --> 17:46.220]  they could see very quickly that they were off the glide slope.
[17:46.240 --> 17:49.480]  But they didn't know that in poor weather, this would be a bit harder to spot,
[17:49.480 --> 17:53.620]  because you obviously have the PAPIs, but you might not be able to see the ground very well.
[17:53.620 --> 17:55.880]  And so you would lose that visual reference.
[17:56.840 --> 18:01.180]  Also, experience with glide slope behaving weirdly helps and hinders.
[18:01.180 --> 18:05.260]  Pilots commented that glide slopes can bend a little bit sometimes,
[18:05.260 --> 18:08.820]  they can behave in weird ways, but that also kind of held them back,
[18:08.820 --> 18:15.340]  because it meant that they sort of allowed the deviation to run for a little while.
[18:15.340 --> 18:18.060]  They weren't sure if it was just a quirk of the glide slope.
[18:18.720 --> 18:22.960]  Also, the wide range of secondary approach methods really suggested uncertainty.
[18:22.960 --> 18:27.860]  There wasn't one clear way of how to actually handle this attack,
[18:27.860 --> 18:32.160]  which is a problem in trying to advise pilots on how to manage this kind of thing.
[18:32.480 --> 18:35.700]  There was also some concern about this short glide slope,
[18:35.700 --> 18:38.180]  so the diehard approach, if you will,
[18:38.180 --> 18:43.200]  where the glide slope would actually be shifted short of the threshold,
[18:43.200 --> 18:48.440]  and if followed, would lead the aircraft to land not on the runway at all.
[18:49.920 --> 18:52.140]  That's not something we chose to look at in this case,
[18:52.140 --> 18:56.740]  but for reasons we discussed earlier, but it is a really interesting variant of this attack.
[18:57.960 --> 19:00.000]  So what did we find overall?
[19:00.000 --> 19:03.800]  Well, we saw that if attacks can cause spurious alarms,
[19:03.800 --> 19:06.740]  then the system often will just be turned off or ignored.
[19:07.440 --> 19:11.240]  And this sort of means that attackers are effectively forcing pilots away
[19:11.240 --> 19:13.420]  from systems by attacking them.
[19:13.620 --> 19:17.020]  On top of that, attacks have real potential for disruption,
[19:17.020 --> 19:22.880]  though it's not always easy to see how that disruption will pan out.
[19:22.880 --> 19:24.660]  It can be quite hard to predict.
[19:24.660 --> 19:27.400]  So this also means, in a way,
[19:27.400 --> 19:30.680]  that it can have a wider, more unpredictable system impact.
[19:33.180 --> 19:36.080]  Participants were very quick to identify unusual behavior,
[19:36.080 --> 19:37.820]  and this was really encouraging to see.
[19:37.820 --> 19:40.920]  That procedure really helped carry them through
[19:40.920 --> 19:42.540]  and get them to a point where they could figure out
[19:42.540 --> 19:46.300]  something wasn't quite working properly, and they could act on it.
[19:46.600 --> 19:51.460]  And on top of that, the attack success partly depends on wider system effects.
[19:51.460 --> 19:53.940]  So we touched on this a little bit, but things like traffic,
[19:53.940 --> 19:59.340]  how busy air traffic control are, how many legs the pilot has flown that day,
[19:59.340 --> 20:03.460]  really has a big impact on whether some of the more obvious attacks
[20:03.460 --> 20:05.740]  might fly under the radar or not.
[20:06.680 --> 20:08.260]  So where do we go from here?
[20:08.260 --> 20:11.960]  Well, we've shown that ILS and TCAS both have attacks
[20:11.960 --> 20:16.160]  that can have an impact on how the aircraft is flown.
[20:16.580 --> 20:21.120]  And these really do have a disruptive effect.
[20:21.120 --> 20:23.440]  They aren't just a bit of an annoyance.
[20:23.440 --> 20:27.540]  They cause the aircraft to do something that it really wouldn't normally do.
[20:28.560 --> 20:32.440]  Larger impacts, though, it really remains to be seen how likely they are,
[20:32.440 --> 20:34.540]  whether it's possible to cause them,
[20:34.540 --> 20:38.940]  whether under some sort of very specific circumstances, this could work.
[20:39.140 --> 20:41.700]  Our view at the moment is it's very hard to predict
[20:41.700 --> 20:44.140]  what the larger impact might be,
[20:44.140 --> 20:48.400]  and it would be very hard to cause any sort of collision risk
[20:48.400 --> 20:49.700]  or multi-aircraft effect
[20:49.700 --> 20:54.240]  just because of the numerous kind of circumstantial bits
[20:54.240 --> 20:56.440]  that would need to be pieced together for that to happen.
[20:57.580 --> 21:00.140]  The encouraging thing from this, though, is the existing procedure
[21:00.140 --> 21:03.260]  really is an ideal starting point for how to handle these attacks.
[21:03.260 --> 21:05.140]  So we're not starting from scratch.
[21:05.140 --> 21:09.580]  Regulators are already talking about how to incorporate awareness training,
[21:09.580 --> 21:11.980]  and the pilots did really well already.
[21:12.100 --> 21:15.380]  So there's something really encouraging to build on that.
[21:15.880 --> 21:18.960]  If you found this talk interesting and you'd like to learn more,
[21:18.960 --> 21:22.820]  we had a full paper on this published earlier this year at NDSS 2020.
[21:23.080 --> 21:26.120]  We also have another attack in there on the ground proximity warning system
[21:26.120 --> 21:29.620]  and a lot more detail on all sorts of different responses.
[21:29.660 --> 21:32.800]  So do check that out. The link is just there on the slide.
[21:33.920 --> 21:35.300]  And thanks for listening.
[21:35.300 --> 21:37.500]  I'd be really happy to answer any questions in the chat.