Rogue Drone Pilots Face A New Foe: Deep Learning AI

In the wake of airspace-invading drones causing the precautionary shutdowns of three major airports—London's Gatwick in December and Heathrow in early January, along with Newark Liberty International, NJ last week—aviation authorities the world over are now considering the acquisition of commercial drone detection, tracking, and capture systems, in a bid to keep airliners and passengers safe.

However, they could be jumping the gun.

The reason? Experts are warning that today's counter-drone systems are no cure-all, as they cannot defend against some types of criminal drone threat—especially the most determined airspace saboteurs using home-built, customized drones that may not respond to conventional countermeasures.

However, a number of ingenious ideas are now being investigated to shore up some of the gaps in functionality, and some of them are even based on the technology of the moment: deep learning.

Aviation safety authorities worry about drones because they can penetrate cockpit windscreens, injuring the pilots, or risk starting inflight fires by breaching the fuel tanks in aircraft wings, or causing sudden engine failure during a critical part of flight like takeoff or landing, perhaps leading to a crash.

"The chance of colliding with the equivalent of a flying house brick as I make an approach, or a takeoff, concerns me a lot. I know just how much damage it could do to an aircraft," says Leo Nugent, an airline training captain based in London and a flight safety advocate with the British Airline Pilots Association, a trade union.

This is why legislation exists in most nations to keep consumer and commercially used drones well out of the way of civilian airplanes, and location-sensing geofences on many drones prevent their use near airports. Yet laws and geofences are little impediment to determined scofflaw airspace hackers who think they have the technical chops to get away with endangering airplanes and the lives they carry.

Detect, Track, Disrupt

Enter counter-drone technologies. These generally sense a drone's Wi-Fi-like radio control signal, then use cameras and radar to track it and, at an opportune time, they swamp the control signal with a more powerful one, allowing law enforcement to take control of the drone and capture it. Alternatively, they may jam the drone's GPS receiver, which can force the drone to return to its launch spot, giving officers a chance to track down the rogue pilot. This demonstration video from Sydney, Australia-based counter-drone systems maker DroneShield shows how such systems work.

Capturing both the drone and its controller device (typically a smartphone, tablet, or a dedicated remote control unit) can provide valuable digital and physical forensics, such as the rogue pilot's DNA from skin cells caught on the drone's sharp-edged rotor blades, or fingerprints on SIM and SD cards. Even if the pilot is not found red-handed, there can still be major clues to his or her identity.

Despite the fact that no one has been arrested or charged for either the Gatwick or Heathrow drone incursions, use of counter-drone technology appears to have frightened the perpetrators off (for now, at least), and this has encouraged the British government as to its overall efficacy.

"The Gatwick issue was solved only by the smart and innovative use of new technology," Transport Minister Chris Grayling told the British Parliament in early January, lauding the deployment of a "new kind of response" by the U.K. Ministry of Defence (MoD), thought to be based on the Drone Dome Counter Unmanned Aircraft System made by Rafael Advanced Defense Systems of Haifa, Israel.

"We have now been approached by airports around the world for our advice on how to handle something similar," Grayling added.

All of Britain's major airports are being encouraged by the MoD to get such "military-grade anti-drone equipment," which does not want the Royal Air Force called in every time a drone appears near an airport.

Painted Into a Corner?

Experts worry that it's too early to be committing to any particular countermeasure technologies, as they may fall short when it really matters, like when a drone is bearing down on a packed holiday jet.

"I am deeply concerned that the counter-drone technology being purchased will not future-proof British airports from drone attack," says Noel Sharkey, a roboticist and autonomous systems analyst at the University of Sheffield in the U.K. "They will not be able to deal with systems where the operator is not in communication with the drone, for instance."

Sharkey adds, "These systems also face problems coping with multiple drones. So we need to stop playing catch up and get ahead of the game."

The threat stems from the fact that, although drones are now a widely available consumer product, it is simple for someone determined to endanger aircraft to buy all the components for a drone (airframes, rotors, motors, lithium batteries, autopilots) online. They can then build their own drone and fly it with freely available, open-source controller software, which would have no geofence restrictions built in.

In addition, as Sharkey suggests, such as customized drone could be pre-programmed to follow a flight path that causes heavy disruption at airports, so there would be no Wi-Fi link to the user's controller, and so nothing for a counter-drone system to hack or track. The same could apply to a hacked consumer drone.

While he think airports ultimately will have to invest in counter-drone technology, Darrell Swanson, a U.K.-based aviation consultant specializing in airport economics, says users of counter-drone systems should not expect them to provide a complete defense. "The biggest threat could come from home-built drones flying pre-programmed tracks to get around any built-in geofencing," he says, adding that attackers might also avoid GPS jamming by triangulating their drone's position from nearby mobile transmitters. 

Deep Droning

That counter-drone systems are indeed a work in progress is evident from recent technical conferences.

At the Pattern Recognition and AI conference in Union, NJ, last August, for instance, John Murray and a team of autonomous systems engineers from the University of Hull in the U.K. revealed how deep learning could be used to locate a rogue drone pilot. Murray's team worked with Metis Aerospace, a counter-drone systems maker in Lincoln, U.K., to test their idea. Using two radio receivers a short distance apart in field trials, they received radio signals from different makes of drone controllers while they were in use and trained a deep neural network to infer the position of the user.

What they ended up with was a system that can automatically locate the position of a rogue drone user within 40 meters, at a range of 500 meters. They are now going on to train the system to locate the drone itself, for situations where radio signals are sensed but the device is not visible.

In yet another innovation, Tam Vu and colleagues at the University of Colorado in Denver have worked out how a counter-drone system can sniff out an approaching drone's 2.4-GHz radio control signal amid the noise of other Wi-Fi signals from smartphones, tablets, cars, airplanes, and even airport coffee shops.

To do this, the Colorado team noticed that the frame structure of a multirotor drone vibrates in such a way that it superimposes a telltale 50-220-Hz signal on its Wi-Fi channel, one that identifies the make of drone, too, which aids in hacking it. In that context, all a counter-drone system would need to do is seek that component by filtering out non-drone signals. This, too, could be computed quickly by a deep learning artificial intelligence system.

It's clever stuff, but for Sheffield's Sharkey, it's all a little too late. The threat to airspace safety presented by consumer drones, highlighted by roboticists over the last decade, has been ignored, he says, and the incidents at Gatwick, Heathrow, and Newark show how civilians are set to reap the whirlwind.

"​The under-regulated rise of the consumer drone threat shows a dereliction of duty by governments worldwide," Sharkey says. "They have been blinded by economy-boosting predictions of billions of dollars in income from drone services, despite the warnings from many of us since 2007."

Paul Marks is a technology journalist, writer, and editor based in London, U.K.