Knowing What You Don’t Know
Why the next real breakthrough in AI isn't a bigger brain - it's a machine that can admit ignorance.
A reader caught me out.
Last column I argued that the great AI buildout - the hundreds of billions pouring into data centers and the GPUs that fill them - is aimed at the wrong layer. We are spending as if the bottleneck were the size of the model's brain, when the real bottleneck is getting the right information in front of it. Cheap retrieval, I said, not expensive cognition.
A reader replied, pointing out the name Jevons.
In 1865, a young English economist named William Stanley Jevons noticed something strange about coal. As steam engines got more efficient - as they wrung more work out of every lump - Britain did not burn less coal. It burned more. Efficiency made steam power cheaper, cheaper made it worth using everywhere, and everywhere" swamped the savings many times over. The better we got at not wasting the stuff, the more of it we wanted.
The reader's point was simple and, annoyingly, correct. Even if I'm right that retrieval is cheap and the brains are overbuilt, that won't shrink the GPU bill. Make AI cheaper to run and we will simply run more of it. Demand eats the savings. The buildout survives. Jevons always wins.
He's right. I concede the whole thing.
And I want to thank him, because in correcting me he handed me a better column.
Here is what I should have said the first time. The case for what comes next in AI was never really about cost. Cost is a weak argument; cost gets competed away, and Jevons makes sure of it. The argument that does not get competed away - the one still standing after the dust settles - is honesty.
There is exactly one problem in artificial intelligence that no amount of cheaper compute, and no amount of bigger compute, has ever solved or can solve by getting cheaper or bigger: the machine does not know what it does not know.
Ask today's best models a question they cannot answer, and they will not pause. They will not hedge. They will hand you a fluent, confident, beautifully formatted answer that happens to be wrong, and they will deliver it with precisely the same swagger they bring to the answers that are right. We have taught them to sound certain. We have not taught them to be calibrated. And you cannot Jevons your way out of that. Make a confident liar a thousand times cheaper and you have fixed nothing - you have a thousand times more confident lying.
In a consumer toy, this is a parlor trick gone wrong. The chatbot invents a court case, the lawyer who trusted it gets sanctioned, everyone has a good laugh, life goes on. In an enterprise, it is the whole reason the technology keeps stalling at the door.
I have watched this movie up close. A bank, a hospital, an insurer, a law firm - they do not deploy a system that is confidently wrong five percent of the time. They can't. Five percent confidently wrong, in a contract or a diagnosis or a compliance filing, is not a rounding error. It is a lawsuit, a recall, a fine, a firing. So the pilot dazzles everyone in the demo and then dies quietly in procurement, and the executives go back to muttering that AI isn't ready" - and they are right, but not for the reason they think.
The thing standing between AI and the enterprise was never speed and was never price. It is trust. And trust is not a mood; it is a property. It requires the machine to know the boundary of its own knowledge and to tell you, out loud, when you have walked past it.
Twenty-four hundred years ago the smartest man in Athens built a whole philosophy on four words: I know that I know nothing. Socrates' entire edge was that he knew the edge - he could feel where his competence ran out. That, not raw recall, is what we actually mean when we call someone an expert. The junior analyst answers every question. The senior one says, I'd have to check." We trust the second one more, and we are right to.
We have built, at staggering expense, the AI junior analyst. Confident everywhere. Calibrated nowhere. The breakthrough that matters - the one I would put real money on - is not a model that knows more. It is a model that knows when it doesn't, and has the nerve to say so.
And here is why this argument, unlike my last one, is bulletproof. Efficiency is a commodity; it falls in price until it is nearly free, and Jevons drags the demand along behind it. But knowing what you don't know is not an efficiency. It is a capability. It either lives in the system or it doesn't. You cannot out-cheap your way to it, which means no one can Jevons their way past it. The moment a buyer can choose between an AI that fabricates and one that flags its own ignorance, there is no contest - and no price war that changes the outcome. Honesty does not get absorbed by demand. It gets demanded.
Can such a thing actually be built - a system that checks itself against what it genuinely knows and raises its hand when it has wandered outside that - or is I don't know" forever beyond a machine that is, at bottom, an engine for guessing the next plausible word? I think it can be built. I think the architecture for it looks nothing like the brain-in-a-bigger-jar we have been financing. (Full disclosure: I have co-founded a company, 2Brains, built around exactly this problem, so discount my optimism by whatever margin you judge fair.) But the how is a column for next time.
For now I will leave you with the reader who set me straight. He read my argument, found the spot where it didn't hold, and said so plainly. He knew the edge of what I had proven, and he had the nerve to name it.
That is the whole trick.
The machines should be so lucky.
The post Knowing What You Don't Know first appeared on I, Cringely.
Digital Branding
Web DesignMarketing