The NPU in your phone keeps improving—why isn’t that making AI better?
Almost every technological innovation of the past several years has been laser-focused on one thing: generative AI. Many of these supposedly revolutionary systems run on big, expensive servers in a data center somewhere, but at the same time, chipmakers are crowing about the power of the neural processing units (NPU) they have brought to consumer devices. Every few months, it's the same thing: This new NPU is 30 or 40 percent faster than the last one. That's supposed to let you do something important, but no one really gets around to explaining what that is.
Experts envision a future of secure, personal AI tools with on-device intelligence, but does that match the reality of the AI boom? AI on the edge" sounds great, but almost every AI tool of consequence is running in the cloud. So what's that chip in your phone even doing?
What is an NPU?Companies launching a new product often get bogged down in superlatives and vague marketing speak, so they do a poor job of explaining technical details. It's not clear to most people buying a phone why they need the hardware to run AI workloads, and the supposed benefits are largely theoretical.