Watch This Giant Chopstick Robot Handle Boxes With Ease

Although robots are already in warehouses, shuffling small items between bins for shipping or storage, they have yet to take over the job of lugging big, heavy things. And that's just where they could be of the most use, because lugging is hard for people to do.

Several companies are working on the problem, and there's likely to be plenty of room for all of them, because the opportunity is enormous. There are a lot of trailers out there that need to be unloaded. Arguably the most interesting approach comes from Dextrous Robotics, which has a robot that moves boxes around with a giant pair of chopsticks.

We first wrote about Dextrous Robotics in 2021, when they were working on a proof of concept using Franka Panda robotic arms. Since then, the concept has been proved successfully, and Dextrous has scaled up to a much larger robot that can handle hundreds of heavy boxes per hour with its chopstick manipulators.

The chopstick type of approach is very robust," Dextrous CEO Evan Drumwright tells us. We can carry heavy payloads and small items with very precise manipulation. Independently posable chopsticks permit grasping a nearly limitless variety of objects with a straightforward mechanical design. It's a real simplification of the grasping problem."

The video above shows the robot moving about 150 boxes per hour in a scenario that simulates unloading a packed trailer, but the system is capable of operating much faster. The demonstration was done without any path optimization. In an uncluttered environment, Dextrous has been able to operate the system at 900 boxes per hour, about twice as fast as the 300 to 500 boxes per hour that a person can handle.

Of course, the heavier the box, the harder it is for a person to maintain that pace. And once a box gets heavier than about 20 kilograms, it takes two people to move it. At that point, labor becomes far less efficient. On paper, the hardware of Dextrous's robot is capable of handling 40 kg boxes at an acceleration of up to 3 gs, and up to 65 kg at a lower acceleration. That would equate to 2,000 boxes per hour. True, this is just a theoretical maximum, but it's what Dextrous is working toward.

If the only problem was to move heavy boxes quickly, robots would have solved it long ago. However, before you can move the box you first have to pick it up, and that complicates matters. Other robotics companies use suction to pick things up. Dextrous alone favors giant chopsticks.

Suction does have the advantage of being somewhat easier to handle on the perception and planning side: Find a flat surface, stick to it, and there you go. That approach assumes you can find a flat surface, but the well-ordered stacks of boxes seen in most demo videos aren't necessarily what you'll get in a warehouse. Suction has other problems: It typically has a payload limit of 20 kg or so, it doesn't work very well with odd-size boxes, and it has trouble operating in temperatures below 10 C. Suction systems also pull in a lot of dirt, which can cause mechanical problems.

A suction system typically attaches to just one surface, and that limits how fast it can move without losing its grip or tearing open a box. The Dextrous chopsticks can support a box on two sides. But making full use of this capability adds difficulty to the perception and planning side.

Just getting to this point has been hardcore," Drumwright says. We've had to get to a level of precision in the perception system and the manipulation to be able to understand what we're picking with high confidence. Our initial engineering hurdle has been very, very high."

Manipulating rigid objects with rigid manipulators like chopsticks has taken Dextrous several years to perfect. Figuring out how to get a robot to perceive and understand its environment, figure out the best item to pick, and then manipulating that item and doing all that in a reasonable length of time-that is really, really hard," Drumwright tells us. I'm not going to say we've solved that 100 percent, but it's working very well. We still have plenty of stuff left to do, but the proof of concept of actually getting a robot that does contact-based manipulation to pick variably sized objects out of an unconstrained environment in a reasonable time period? We've solved that."

Here's another video showing a sustained box-handling sequence; if you watch carefully, you'll notice all kinds of precise little motions as the robot uses its manipulators to slightly reposition boxes to give it the best grasp:

All of those motions makes the robot look almost like it's being teleoperated, but Drumwright assures me that it's completely autonomous. It turns out that teleoperation doesn't work very well in this context. We looked at doing teleop, and we actually could not do it. We found that our controllers are so precise that we could not actually make the system behave better through teleop than it did autonomously." As to how the robot decides what to do what it does, I can't tell you exactly where these behaviors came from," Drumwright says, Let's just call it AI. But these are all autonomous manipulation behaviors, and the robot is able to utilize this diverse set of skills to figure out how to pick every single box."

You may have noticed that the boxes in the videos are pretty beat up. That's because the robot has been practicing with those boxes for months, but Dextrous is mindful of the fact that care is necessary, says Drumwright. One of the things that we were worried about from the very beginning was, how do we do this in a gentle way? But our newest version of the robot has the sensitivity to be very gentle with the boxes."

I asked Drumwright what would be the most difficult object for his robot to pick up. I suggested a bowling ball (heavy, slippery, spherical). Challenging, but by no means impossible," was his response, citing research from Siddhartha Srinivasa at the University of Washington showing that a robot with chopsticks can learn to do dynamic fine manipulation of spherical objects. Dextrous isn't above cheating slightly, though, by adding a thin coating of hard rubber to the chopsticks' end effectors to add just a tiny bit of compliance-not enough to mess with planning or control, but enough to make grasping some tricky objects a little easier.

By a year ago, Dextrous had shown that it could move boxes at high speeds under limited scenarios. For the past year, it has been making sure that the system can handle the full range of scenarios that it's likely to encounter in warehouses. Up next is combining those two things-cranking the speed back up while still working reliably and autonomously.

On the manipulation side, the system is fully autonomous," Drumwright says. We currently have humans involved in driving the robot into the container and then joysticking it forward once it's picked all that it can reach, but we're making that fully autonomous, too." And the robot has so far been quite reliable, requiring little more than lubrication.

According to Drumwright, the biggest challenge on the business side at this point is simply manufacturing enough robots, since the company builds the hardware in-house. The remaining question is how long it will take to make the transition from experiment to product. The company is starting a few commercial pilots, and Drumwright says the thing that's slowing them down the most is building enough robots to keep up with demand.

We've solved all of the hardest technical problems," he says. And now, it's the business part."