Brain-like Computers Tackle the Extreme Edge

Neuromorphic computing draws inspiration from the brain, and Steven Brightfield, chief marketing officer for Sydney-based startup BrainChip, says that makes it perfect for use in battery-powered devices doing AI processing.

The reason for that is evolution," Brightfield says. Our brain had a power budget." Similarly, the market BrainChip is targeting is power constrained. "You have a battery and there's only so much energy coming out of the battery that can power the AI that you're using."

Today, BrainChip announced their chip design, the Akida Pico, is now available. Akida Pico, which was developed for use in power-constrained devices, is a stripped-down, miniaturized version of BrainChip's Akida design, introduced last year. Akida Pico consumes 1 milliwatt of power, or even less depending on the application. The chip design targets the extreme edge, which is comprised of small user devices such as mobile phones, wearables, and smart appliances that typically have severe limitations on power and wireless communications capacities. Akida Pico joins similar neuromorphic devices on the market designed for the edge, such as Innatera's T1 chip, announced earlier this year, and SynSense's Xylo, announced in July 2023.

Neuron Spikes Save Energy

Neuromorphic computing devices mimic the spiking nature of the brain. Instead of traditional logic gates, computational units-referred to as neurons'-send out electrical pulses, called spikes, to communicate with each other. If a spike reaches a certain threshold when it hits another neuron, that one is activated in turn. Different neurons can create spikes independent of a global clock, resulting in highly parallel operation.

A particular strength of this approach is that power is only consumed when there are spikes. In a regular deep learning model, each artificial neuron simply performs an operation on its inputs: It has no internal state. In a spiking neural network architecture, in addition to processing inputs, a neuron has an internal state. This means the output can depend not only on the current inputs, but on the history of past inputs, says Mike Davies, director of the neuromorphic computing lab at Intel. These neurons can choose not to output anything if, for example, the input hasn't changed sufficiently from previous inputs, thus saving energy.

Where neuromorphic really excels is in processing signal streams when you can't afford to wait to collect the whole stream of data and then process it in a delayed, batched manner. It's suited for a streaming, real-time mode of operation," Davies says. Davies' team recently published a result showing their Loihi chip's energy use was one-thousandth of a GPU's use for streaming use cases.

Akida Pico includes its neural processing engine, along with event processing and model weight storage SRAM units, direct memory units for spike conversion and configuration, and optional peripherals. Brightfield says in some devices, such as simple detectors, the chip can be used as a stand-alone device, without a microcontroller or any other external processing. For other use cases that require further on-device processing, it can be combined with a microcontroller, CPU, or any other processing unit.

BrainChip's Akida Pico design includes a miniaturized version of their neuromorphic processing engine, suitable for small, battery-operated devices.BrainChip

BrainChip has also worked to develop AI model architectures that are optimized for minimal power use in their device. They showed off their techniques with an application that detects keywords in speech. This is useful for voice assistance like Amazon's Alexa, which waits for the Hello, Alexa' keywords to activate.

The BrainChip team used their recently developed model architecture to reduce power use to one-fifth of the power consumed by traditional models running on a conventional microprocessor, as demonstrated in their simulator. I think Amazon spends $200 million a year in cloud computing services to wake up Alexa," Brightfield says. They do that using a microcontroller and a neural processing unit (NPU), and it still consumes hundreds of milliwatts of power." If BrainChip's solution indeed provides the claimed power savings for each device, the effect would be significant.

In a second demonstration, they used a similar machine learning model to demonstrate audio de-noising, for use in hearing aids or noise canceling headphones.

To date, neuromorphic computers have not found widespread commercial uses, and it remains to be seen if these miniature edge devices will take off, in part because of the diminished capabilities of such low-power AI applications. If you're at the very tiny neural network level, there's just a limited amount of magic you can bring to a problem," Intel's Davies says.

BrainChip's Brightfield, however, is hopeful that the application space is there. It could be speech wake up. It could just be noise reduction in your earbuds or your AR glasses or your hearing aids. Those are all the kind of use cases that we think are targeted. We also think there's use cases that we don't know that somebody's going to invent."