Cornell Researchers Taught a Robot To Take Airbnb Photos

A team of researchers from Cornell University used a computational aesthetic system to teach an AI robot "to not only determine the most pleasing picture in a given dataset, but capture new, original -- and most importantly, good -- shots on its own," writes Engadget's A. Tarantola. The project is called AutoPhoto and was presented last fall at the International Conference on Intelligent Robots and Systems. From the report: This robo-photographer consists of three parts: the image evaluation algorithm, which evaluates a presented image and issues an aesthetic score; a Clearpath Jackal wheeled robot upon which the camera is affixed; and the AutoPhoto algorithm itself, which serves as a sort of firmware, translating the results from the image grading process into drive commands for the physical robot and effectively automating the optimized image capture process. For its image evaluation algorithm, the Cornell team led by second year Masters student Hadi AlZayer, leveraged an existing learned aesthetic estimation model, which had been trained on a dataset of more than a million human-ranked photographs. AutoPhoto itself was virtually trained on dozens of 3D images of interior room scenes to spot the optimally composed angle before the team attached it to the Jackal. When let loose in a building on campus, as you can see in the video above, the robot starts off with a slew of bad takes, but as the AutoPhoto algorithm gains its bearings, its shot selection steadily improves until the images rival those of local Zillow listings. On average it took about a dozen iterations to optimize each shot and the whole process takes just a few minutes to complete. "You can essentially take incremental improvements to the current commands," AlZayer told Engadget. "You can do it one step at a time, meaning you can formulate it as a reinforcement learning problem." This way, the algorithm doesn't have to conform to traditional heuristics like the rule of thirds because it already knows what people will like as it was taught to match the look and feel of the shots it takes with the highest-ranked pictures from its training data, AlZayer explained. "The most challenging part was the fact there was no existing baseline number we were trying to improve," AlZayer noted to the Cornell Press. "We had to define the entire process and the problem." Looking ahead, AlZayer hopes to adapt the AutoPhoto system for outdoor use, potentially swapping out the terrestrial Jackal for a UAV. "Simulating high quality realistic outdoor scenes is very hard," AlZayer said, "just because it's harder to perform reconstruction of a controlled scene." To get around that issue, he and his team are currently investigating whether the AutoPhoto model can be trained on video or still images rather than 3D scenes.

Read more of this story at Slashdot.