"Edge AI": encapsulating machine learning classifiers in lightweight, energy-efficient, airgapped chips

Writing in Wired, Boing Boing contributor Clive Thompson discusses the rise and rise of "Edge AI" startups that sell lightweight machine-learning classifiers that run on low-powered chips and don't talk to the cloud, meaning that they are privacy respecting and energy efficient.

Thompson focuses on Picovoice, which makes a speech-recognition system that has a limited vocabulary and whose per-unit hardware costs are "a few bucks apiece" -- cheap enough that you can embed one in a coffee-maker or a music system so you can voice-control them. They won't banter with you like Alexa, but they can reliably, quickly and cheaply recognize the phrases needed to brew a cup of coffee or cue up a song.

Other classifiers also operate on this model -- Xnor.ai, acquired by Apple, makes an image classifier that's so low-powered it can run on the minute voltages given off by houseplants.

There are free, open source versions of this technology if you want to get started.

But before you get too excited about the energy savings of running AI at the edge instead of in the cloud, remember that the process of generating those models is incredibly energy-intensive, equivalent to the lifetime carbon emissions of five automobiles.

What's more, edge AI is speedy. There are no pauses in performance, no milliseconds lost while the device sends your voice request to play Smash Mouth's "All Star" halfway across the continent to Amazon's servers, or to the NSA's sucking maw of thoughtcrime data, or wherever the hell it winds up. Edge processing is "ripping fast," says Todd Mozer, CEO of Sensory, a firm that makes visual- and audio-recognition software for edge devices. When I interviewed Mozer on Skype, he demo'd some neural-net code he'd created for a microwave, and whatever command he uttered-"Heat up my popcorn for two minutes and 36 seconds"-was recognized instantly.

Worried About Privacy at Home? There's an AI for That [Clive Thompson/Wired]

(Image: Bill Ward, CC BY, methodshop .com, CC BY-SA, modified)