Robot Vehicles Make Contactless Deliveries Amid Coronavirus Quarantine

For the past two months, the vegetables have arrived on the back of a robot. That's how 16 communities in Zibo, in eastern China, have received fresh produce during the coronavirus pandemic. The robot is an autonomous van that uses lidars, cameras, and deep-learning algorithms to drive itself, carrying up to 1,000 kilograms on its cargo compartment.

The unmanned vehicle provides a "contactless" alternative to regular deliveries, helping reduce the risk of person-to-person infection, says Professor Ming Liu, a computer scientist at the Hong Kong University of Science and Technology (HKUST) and cofounder of Unity Drive Innovation, or UDI, the Shenzhen-based startup that developed the self-driving van.

Since February, UDI has been operating a small fleet of vehicles in Zibo and two other cities, Suzhou and Shenzhen, where they deliver meal boxes to checkpoint workers and spray disinfectant near hospitals. Combined, the vans have made more than 2,500 autonomous trips, often encountering busy traffic conditions despite the lockdown.

"It's like Uber for packages-you use your phone to call a robot to pick up and deliver your boxes," Professor Liu told IEEE Spectrum in an interview via Zoom.

Even before the pandemic, package shipments had been skyrocketing in China and elsewhere. Alibaba founder Jack Ma has said that his company is preparing to handle 1 billion packages per day. With the logistics sector facing major labor shortages, a 2016 McKinsey report predicted that autonomous vehicles will deliver 80 percent of parcels within 10 years.

That's the future UDI is betting on. Unlike robocars developed by Waymo, Cruise, Zoox, and others, UDI's vehicles are designed to transport goods, not people. They are similar to those of Nuro, a Silicon Valley startup, and Neolix, based in Beijing, which has deployed 50 robot vans in 10 Chinese cities to do mobile delivery and disinfection service.

Photo: UDI A self-driving vehicle delivers lunch boxes to workers in Pingshan District in Shenzhen. Since February, UDI's autonomous fleet has made more than 800 meal deliveries.

Professor Liu, an IEEE Senior Member and director of the Intelligent Autonomous Driving Center at HKUST, is unfazed by the competition. He says UDI is ready to operate its vehicles on public roads thanks to the real-world experience it has gained from a string of recent projects. These involve large companies testing the robot vans inside their industrial parks.

One of them is Taiwanese electronics giant Foxconn. Since late 2018, it has used UDI vans to transport electronic parts and other items within its vast Shenzhen campus where some 200,000 workers reside. The robots have to navigate labyrinthine routes while avoiding an unpredictable mass of pedestrians, bicycles, and trucks.

Autonomous driving powered by deep learning

UDI's vehicle, called Hercules, uses an industrial-grade PC running the Robot Operating System, or ROS. It's also equipped with a drive-by-wire chassis with electric motors powered by a 8.4-kWh lithium-ion battery. Sensors include a main lidar, three auxiliary lidars, a stereo camera, four fisheye cameras, 16 sonars, redundant satellite navigation systems, an inertial measurement unit (IMU), and two wheel encoders.

The PC receives the lidar point-clouds and feeds them into the main perception algorithm, which consists of a convolutional neural network trained to detect and classify objects. The neural net outputs a set of 3D bounding boxes representing vehicles and other obstacles on the road. This process repeats 100 times per second.

Image: UDI UDI's vehicle is equipped with a main lidar and three auxiliary lidars, a stereo camera, and various other sensors [top]. The cargo compartment can be modified based on the items to be transported and is not shown. The chassis [bottom] includes an electric motor, removable lithium-ion battery, vehicle control unit (VCU), motor control unit (MCU), electric power steering (EPS), electro-hydraulic brake (EHB), electronic parking brake (EPB), on-board charger (OBC), and direct-current-to-direct-current (DCDC) converter.

Another algorithm processes images from forward-facing cameras to identify road signs and traffic lights, and a third matches the point-clouds and IMU data to a global map, allowing the vehicle to self-localize. To accelerate, brake, and steer, the PC sends commands to two secondary computers running real-time operating systems and connected to the drive-by-wire modules.

Professor Liu says UDI faces more challenging driving conditions than competitors like Waymo and Nuro that conduct their tests in suburban areas in the United States. In Shenzhen, for example, the UDI vans have to navigate through narrow streets with double parked cars and aggressive motorcycles that whiz by narrowly missing the robot.

Click here for additional coronavirus coverage

Over the past couple of months, UDI has monitored its fleet from its headquarters. Using 5G, a remote operator can receive data from a vehicle with just 10 milliseconds of delay. In Shenzhen, human intervention was required about two dozen times when the robots encountered situations they didn't know how to handle-too many vehicles on the road, false detections of traffic lights at night, or in one case, a worker coming out of a manhole.

Photo: UDI One of UDI's autonomous vehicles equipped with a device that sprays disinfectant operates near a hospital in Shenzhen.

For safety, UDI programmed the vans to drive at low speeds of up to 30 kilometers per hour, though they can go faster. On a few occasions, remote operators took control because the vehicles were driving too slowly, becoming a road hazard and annoying nearby drivers. Professor Liu says it's a challenge to balance cautiousness and aggressiveness in self-driving vehicles that will operate in the real world.

He notes that UDI vehicles have been collecting huge amounts of video and sensor data during their autonomous runs. This information will be useful to improve computer simulations of the vehicles and, later, the real vehicles themselves. UDI says it plans to open source part of the data.

Mass produced robot vans

Professor Liu has been working on advanced vehicles for nearly two decades. His projects include robotic cars, buses, and boats, with a focus on applying deep reinforcement learning to enable autonomous behaviors. He says UDI's vehicles are not cars, and they aren't unmanned ground robots, either-they are something in between. He likes to call them "running robots."

Liu's cofounders are Professor Xiaorui Zhu at Harbin Institute of Technology, in Shenzhen, and Professor Lujia Wang at the Shenzhen Institutes of Advanced Technology, part of the Chinese Academy of Sciences. "We want to be the first company in the world to achieve mass production of autonomous logistics vehicles," says Wang, who is the CTO of UDI.

To do that, the startup has hired 100 employees and is preparing to put its assembly line into high gear in the next several months. "I'm not saying we solved all the problems," Professor Liu says, citing system integration and cost as the biggest challenges. "Can we do better? Yes, it can always be better."