Autonomous Drones Challenge Human Champions in First “Fair” Race

Watching robots operate with speed and precision is always impressive, if not, at this point, always surprising. Sophisticated sensors and fast computing means that a powerful and agile robot, like a drone, that knows exactly where it is and exactly where it's going can reliably move in highly dynamic ways. This is not to say that it's easy for the drone, but if you've got a nice external localization system, a powerful off-board computer, and a talented team of roboticists, you can perform some amazingly agile high-speed maneuvers that most humans could never hope to match.

I say most" humans, because there are some exceptionally talented humans who are, in fact, able to achieve a level of performance similar to that of even the fastest and most agile drones. The sport of FPV (first-person view) drone racing tests what's possible with absurdly powerful drones in the hands of humans who must navigate complex courses with speed and precision that seems like it shouldn't be possible, all while relying solely on a video feed sent from a camera on the front of the drone to the pilot's VR headset. It's honestly astonishing to watch.

A year ago, autonomous racing quadrotors from Davide Scaramuzza's Robotics and Perception Group at the University of Zurich (UZH) proved that they could beat the world's fastest humans in a drone race. However, the drones relied on a motion-capture system to provide very high resolution position information in real time, along with a computer sending control information from the safety and comfort of a nearby desk, which doesn't really seem like a fair competition.

Earlier this month, a trio of champion drone racers traveled to Zurich for a rematch, but this time, the race would be fair: no motion-capture system. Nothing off-board. Just drones and humans using their own vision systems and their own computers (or brains) to fly around a drone racing track as fast as possible.

To understand what kind of a challenge this is, it's important to have some context for the level of speed and agility. So here's a video of one of UZH's racing drones completing three laps of a track using the motion-capture system and off-board computation. This particular demo isn't fair," but it does give an indication of what peak performance of a racing drone looks like, with a reaction from one of the professional human pilots (Thomas Bitmatta) at the end:

As Thomas says at the end of the video, the autonomous drone made it through one lap of the course in 5.3 seconds. With a peak speed of 110 kilometers per hour, this was a staggering 1.8 seconds per lap faster than Thomas, who has twice won FPV drone racing's MultiGP International World Cup.

The autonomous drone has several advantages in this particular race. First, it has near-perfect state estimation, thanks to a motion-capture system that covers the entire course. In other words, the drone always knows exactly where it is, as well as its precise speed and orientation. Experienced human pilots develop an intuition for estimating the state of their system, but they can't even watch their own drone while racing since they're immersed in the first-person view the entire time. The second advantage the autonomous drone has is that it's able to compute a trajectory that traverses the course in a time-optimal way, considering the course layout and the constraints imposed by the drone itself. Human pilots have to practice on a course for hours (or even days) to discover what they think is an optimal trajectory, but they have no way of knowing for sure whether their racing lines can be improved or not.

Elia Kaufmann prepares one of UZH's vision-based racing drones on its launch platform.Evan Ackerman/IEEE Spectrum

So what, then, would make for a drone race in which humans and robots can compete fairly but doesn't ask the robots to be less robotic or the humans to be less human-y?

• No external help. No motion-capture system or off-board compute. Arguably, the humans have something of an advantage here, since they are off-board by definition, but the broader point of this research is to endow drones with the ability to fly themselves in aggressive and agile ways, so it's a necessary compromise.
• Complete knowledge of the course. Nothing on the course is secret, and humans can walk through it and develop a mental model. The robotic system, meanwhile, gets an actual CAD model. Both humans and robots also get practice time-humans on the physical course with real drones, and the system practices in simulation. Both humans and robots can use this practice time to find an optimal trajectory in advance.
• Vision only. The autonomous drones use Intel RealSense stereo-vision sensors, while the humans use a monocular camera streaming video from the drone. The humans may not get a stereo feed, but they do get better resolution and higher frames per second than the RealSense gives the autonomous drone.

Three world-class human pilots were invited to Zurich for this race. Along with Thomas Bitmatta, UZH hosted Alex Vanover (2019 Drone Racing League champion) and Marvin Schapper (2021 Swiss Drone League champion). Each pilot had as much time as they wanted on the course in advance, flying more than 700 practice laps in total. And on a Friday night in a military aircraft hangar outside of Zurich, the races began. Here are some preliminary clips from one of the vision-based autonomous drones flying computer-to-head with a human; the human-piloted drone is red, while the autonomous drone is blue:

With a top speed of 80 km/h, the vision-based autonomous drone outraced the fastest human by 0.5 second during a three-lap race, where just one or two-tenths of a second is frequently the difference between a win and a loss. This victory for the vision-based autonomous drone is a big deal, as Davide Scaramuzza explains:

This demonstrates that AI-vs.-human drone racing has the potential to revolutionize drone racing as a sport. What's clear is that superhuman performance with AI drones can be achieved, but there is still a lot of work to be done to robustify these AI systems to bring them from a controlled environment to real-world applications. More details will be given in follow-up scientific publications.

I was at this event in Zurich, and I'd love to tell you more about it. I will tell you more about it, but as Davide says, the UZH researchers are working on publishing their results, meaning that all the fascinating details about exactly what happened and why will have to wait a bit until they've got everything properly written up. So stay tuned-we'll have lots more for you on this.

The University of Zurich provided travel support to assist us with covering this event in person.