Food Service Robots Just Need the Right Ingredients

Food prep is one of those problems that seems like it should be solvable by robots. It's a predictable, repetitive, basic manipulation task in a semi-structured environment-seems ideal, right? And obviously there's a huge need, because human labor is expensive and getting harder and harder to find in these contexts. There are currently over a million unfilled jobs in the food industry in the United States, and even with jobs that are filled, the annual turnover rate is 150 percent (meaning a lot of workers don't even last a year).

Food prep seems like a great opportunity for robots, which is why Chef Robotics and a handful of other robotics companies tackled it a couple years ago by bringing robots to fast casual restaurants like Chipotle or Sweetgreen, where you get served a custom-ish meal from a selection of ingredients at a counter.

But this didn't really work out, for a couple of reasons. First, doing things that are mostly effortless for humans are inevitably extremely difficult for robots. And second, humans actually do a lot of useful things in a restaurant context besides just putting food onto plates, and the robots weren't up for all of those things.

Still, Chef Robotics founder and CEO Rajat Bhageria wasn't ready to let this opportunity go. The food market is arguably the biggest market that's tractable for AI today," he told IEEE Spectrum. And with a bit of a pivot away from the complicated mess of fast casual restaurants, Chef Robotics has still managed to prepare over 20 million meals thanks to autonomous robot arms deployed all over North America. Without knowing it, you may even have eaten such a meal.

The hard thing is, can you pick fast? Can you pick consistently? Can you pick the right portion size without spilling? And can you pick without making it look like the food was picked by a machine?" -Rajat Bhageria, Chef Robotics

When we spoke with Bhageria, he explained that there are three basic tasks involved in prepared food production: prep (tasks like chopping ingredients), the actual cooking process, and then assembly (or plating). Of these tasks, prep scales pretty well with industrial automation in that you can usually order pre-chopped or mixed ingredients, and cooking also scales well since you can cook more with only a minimal increase in effort just by using a bigger pot or pan or oven. What doesn't scale well is the assembly, especially when any kind of flexibility or variety is required. You can clearly see this in action at any fast casual restaurant, where a couple of people are in the kitchen cooking up massive amounts of food while each customer gets served one at a time.

So with that bottleneck identified, let's throw some robots at the problem, right? And that's exactly what Chef Robotics did, explains Bhageria: we went to our customers, who said that their biggest pain point was labor, and the most labor is in assembly, so we said, we can help you solve this."

Chef Robotics started with fast casual restaurants. They weren't the first to try this-many other robotics companies had attempted this before, with decidedly mixed results. We actually had some good success in the early days selling to fast casual chains," Bhageria says, but then we had some technical obstacles. Essentially, if we want to have a human-equivalent system so that we can charge a human-equivalent service fee for our robot, we need to be able to do every ingredient. You're either a full human equivalent, or our customers told us it wouldn't be useful."

Part of the challenge is that training robots do perform all of the different manipulations required for different assembly tasks requires different kinds of real world data. That data simply doesn't exist-or, if it does, any company that has it knows what it's worth and isn't sharing. You can't easily simulate this kind of data, because food can be gross and difficult to handle, whether it's gloopy or gloppy or squishy or slimy or unpredictably deformable in some other way, and you really need physical experience to train a useful manipulation model.

Setting fast casual restaurants aside for a moment, what about food prep situations where things are as predictable as possible, like mass-produced meals? We're talking about food like frozen dinners, that have a handful of discrete ingredients packed into trays at factory scale. Frozen meal production relies on automation rather than robotics because the scale is such that the cost of dedicated equipment can be justified.

There's a middle ground, though, where robots have found (some) opportunity: When you need to produce a high volume of the same meal, but that meal changes regularly. For example, think of any kind of pre-packaged meal that's made in bulk, just not at frozen-food scale. It's an opportunity for automation in a structured environment-but with enough variety that actual automation isn't cost effective. Suddenly, robots and their tiny bit of flexible automation have a chance to be a practical solution.

We saw these long assembly lines, where humans were scooping food out of big tubs and onto individual trays," Bhageria says. They do a lot of different meals on these lines; it's going to change over and they're going to do different meals throughout the week. But at any given moment, each person is doing one ingredient, and maybe on a weekly basis, that person would do six ingredients. This was really compelling for us because six ingredients is something we can bootstrap in a lab. We can get something good enough and if we can get something good enough, then we can ship a robot, and if we can ship a robot to production, then we will get real world training data."

Chef Robotics has been deploying robot modules that they can slot into existing food assembly lines in place of humans without any retrofitting necessary. The modules consist of six degree of freedom arms wearing swanky IP67 washable suits. To handle different kinds of food, the robots can be equipped with a variety of different utensils (and their accompanying manipulation software strategies). Sensing includes a few depth cameras, as well as a weight-sensing platform for the food tray to ensure consistent amounts of food are picked. And while arms with six degrees of freedom may be overkill for now, eventually the hope is that they'll be able to handle more complex food like asparagus, where you need to do a little bit more than just scoop.

While Chef Robotics seems to have a viable business here, Bhageria tells us that he keeps coming back to that vision of robots being useful in fast casual restaurants, and eventually, robots making us food in our homes. Making that happen will require time, experience, technical expertise, and an astonishing amount of real-world training data, which is the real value behind those 20 million robot-prepared meals (and counting). The more robots the company deploys, the more data they collect, which will allow them to train their food manipulation models to handle a wider variety of ingredients to open up even more deployments. Their robots, Chef's website says, essentially act as data ingestion engines to improve our AI models."

The next step is likely ghost kitchens where the environment is still somewhat controlled and human interaction isn't necessary, followed by deployments in commercial kitchens more broadly. But even that won't be enough for Bhageria, who wants robots that can take over from all of the drudgery in food service: I'm really excited about this vision," he says. How do we deploy hundreds of millions of robots all over the world that allow humans to do what humans do best?"