Robot Learns to Cook Your Perfect Omelet

Cooking robots have come a long way in a relatively short amount of time. We're not yet at the point where we've got robot arms dangling from the ceiling that do all the work for us, but there are a bunch of robots out there with reasonable cookie-making skills. However, we've mostly seen cooking robots that are programmed to follow a specific recipe, rather than cooking robots that are programmed to cook you exactly what you want. Sometimes these are the same thing, but often cooking is (I'm told) much more about adapting a recipe to your individual taste.

For personal cooking robots to make us food that we love, they're going to need to be able to listen to our feedback, understand what that feedback means, and then take actions to adapt their recipe or technique to achieve the desired outcome. This is more complicated than, say, adding less salt next time because it was too salty this time-the robots will have to work with less easily definable things like appearance and texture, and the totally subjective nature of human taste.

At ICRA 2020 this week, roboticists from the University of Cambridge, in England, presented a paper on OmeletteBot, a fully autonomous end-to-end omelet-cooking robot. What's new here is that the robot is able to optimize its omelet making for different people based on how they react to a few sample omelets. Your perfect omelet is now within reach.

Here's the omelet making process, which is neat to watch, although it's not really the focus of the paper:

The goal here is for the quality of the omelet to be optimized for an individual human across a few different metrics, all of which are subjective. In this case, the omelet is being evaluated on three metrics: flavor, appearance, and texture, each on a 1 to 10 scale. Meanwhile, the robot has a variety of different parameters it can adjust, including shakes of salt, shakes of pepper, whisking time, in-pan mixing time, and time on heat.

The ideal way to approach this problem would be to have a human sample every possible combination of parameters to find their ideal omelet, but this is obviously impractical. And making everything more difficult is that humans get full quickly, our memory is short, and we change our minds often. Maybe the first omelet you taste is pretty great, so you give it an 8/10 on flavor, but your next omelet tastes twice as good-then what do you do?

The researchers employed a solution to this problem called batch Bayesian optimization. The more traditional sequential Bayesian optimization would take a human's score of each omelet and use it to modify the cooking process for the next omelet, but this approach doesn't work well because the human feedback is, as the researchers tell us, often ambiguous and relative." By running the optimization process only after all scores have been collected, the robot is able to explore many more combinations of variables, yielding a substantially better end result. Instead of the omelets gradually getting better as you go, you'll instead be tasting them randomly, but you'll end up with a much tastier omelet.

For more details, we spoke with authors Kai Junge and Fumiya Iida via email. Remember, since this is an ICRA paper, if you're registered for the conference you can view their video presentation and ask your own questions in the paper Slack channel at this link.

IEEE Spectrum: Why did you decide to work with omelets instead of (say) pancakes or cookies?

Omelet is one of the most popular dishes, while it has quite a variety in terms of individual preferences about how it should be cooked. Another consideration was the complexity of cooking itself. We decided to cook omelets because they are not too difficult for robots to cook, unlike many other choices.

Is it frustrating to work on such a subjective problem, when the best" answer (best omelet) is different for every person?

There are a full of real-world problems in which ground truth" is ambiguous or fluctuating, just like the human preferences to omelets. We found it rather interesting and exciting, than frustrating, to tackle such problems.

Can you describe how this could be an important tool for home-cooking robots?

Home cooking robot is a real challenge because of its complex nature of operations. However, in our opinion, there should be a number of implications in the learning process proposed by our paper. In particular, we found it very fascinated by the fact that human taste can be somewhat characterized by the rather simple machine-learning technique, and we are hoping that such techniques can be applicable to many kitchen appliances in the near future.

If I had a new omelet-cooking robot in my home, what process would I go through to use the method in your paper to find my optimal omelet?

You eat as many omelets as possible, and give feedback to the robot every time. The robot should be able to adjust its recipe automatically based on our feedback, and hopefully it converges to your preference.

What are you working on next?

There are many remaining challenges around robotic cooking, and robotic automation in kitchen operations, more generally. We are working on the issues such as more dexterous robotic manipulation of food and cooking tools, better programming/learning methods of robots to cook more complicated cooking tasks. There also exist many pain points" in regular household kitchens, such as food preparation, dishwasher loading/unloading, cleaning of worktops, and assistance for elderly and handicapped people, which are our next challenges.

Improving Robotic Cooking Using Batch Bayesian Optimization," by Kai Junge, Josie Hughes, Thomas George Thuruthel, and Fumiya Iida from the University of Cambridge, is being presented at ICRA 2020. < Back to IEEE Journal Watch