'GrokNet', the AI behind Facebook Shops, looks for body type, skin tone, location, socioeconomic class in photos
Yay, Clearview AI but for shopping!
Facebook Mark Zuckerberg today announced the launch of Facebook Shops, an e-commerce feature to allows business users to list and sell products on Facebook and Instagram.
Here's the Facebook blog post launching the new shop feature.
Today we're introducing Facebook Shops, which will make it easy for businesses to set up a single online store on both @facebookapp and @instagram, and soon @messenger and @WhatsApp too pic.twitter.com/bBloRGBFzP
- Facebook (@Facebook) May 19, 2020
Here's more at CNBC about Facebook Shops.
Separately, Kyle Wiggers at VentureBeat reports on some of the very creepy personal details examined for the shopping experience by Facebook's artificial intelligence:
Facebook says its AI-powered shopping systems segment, detect, and classify images to know where products appear and deliver shopping suggestions. One of those systems - GrokNet - was trained on seven data sets containing images of products that millions of users post, buy, and sell in dozens of categories, ranging from SUVs to stiletto heels to side tables. Another creates 3D views from 2D videos of products, even those obscured by dim or overly bright lighting, while a third spotlights apparel like scarfs, ties, and more that might be partially obscured by their surroundings.
Facebook says that GrokNet, which can detect exact, similar (via related attributes), and co-occurring products across billions of photos, performs searches and filtering on Marketplace at least twice as accurately than the algorithm it replaced. For instance, it's able to identify 90% of home and garden listings compared with Facebook's text-based attribution systems, which can only identify 33%. In addition to generating tags for colors and materials from images before Marketplace sellers list an item, as part of a limited test, it's tagging products on Facebook Pages when Page admins upload a photo.
In the course of training GrokNet, Facebook says it used real-world seller photos with challenging" angles along with catalog-style spreads. To make it as inclusive as possible for all countries, languages, ages, sizes, and cultures, it sampled examples of different body types, skin tones, locations, socioeconomic classes, ages, and poses.
Rather than manually annotate each image with product identifiers, which would have taken ages - there are 3 million possible identifiers - Facebook developed a technique to automatically generate additional identifiers using GrokNet as a feedback loop. Leveraging an object detector, the approach identifies boxes in images surrounding likely products, after which it matches the boxes against a list of known products to keep matches within a similarity threshold. The resulting matches are added to the training set.
Well, that's not creepy at all.
Read more at venturebeat.com:
Facebook details the AI behind its shopping experiences
[Kyle Wiggers, May 19, 2020, via techmeme.com]
FB is working on some very powerful stuff here -> Facebook details the AI behind its shopping experiences, accounting for consumer traits like body type, skin tone, location, socioeconomic class, age, and poses https://t.co/OwOE7cvGJU pic.twitter.com/FyyNJi3rXi
- Glenn Gabe (@glenngabe) May 20, 2020
"Our long-term vision is to build an all-in-one AI lifestyle assistant that can accurately search and rank billions of products, while personalizing to individual tastes." https://t.co/3vLFi5l3xn
- Emil Protalinski (@EPro) May 19, 2020
it also may give average people the heebie jeebies in a world in which people are growing uncomfortable about tracking. https://t.co/LUHzPQN44n
- Tony Romm (@TonyRomm) May 19, 2020
This must give @Pinterest the heebie-jeebies:
The new products are powered by a family of new AI and ML systems that can search and rank products, while personalizing its recommendations to individual tastes."#GrokNethttps://t.co/jx2yVOj5cV
- Chris Messina (@chrismessina) May 19, 2020