Can AI models reason: Just a stochastic parrot?

OpenAI has just released its full o1 model-a new kind of model that is more capable of multi-step reasoning than previous models. Anthropic, Google and others are no doubt working on similar products. At the same time, it's hotly debated in many quarters whether AI models actually reason" in a way similar to humans.

Emily Bender and her colleagues famously described large language models as nothing more than stochastic parrots-systems that simply repeat their training data blindly, based on a statistical model, with no real understanding (reminiscent of the Chinese Room experiment). Others have made similar comments, describing LLMs as n-gram models on steroids" or a fancy extrapolation algorithm."

There is of course some truth to this. AI models sometimes generate remarkable results and yet lack certain basic aspects of understanding that might inhibit their sometimes generation of nonsensical results. More to the point of parroting" the training data, recent work from Yejin Choi's group has shown how LLMs at times will cut and paste snippets from its various training documents, almost verbatim, to formulate its outputs.

Are LLMs (just) glorified information retrieval tools?

The implication of these concerns is that an LLM can only" repeat back what it was taught (albeit with errors). However this view does not align with the evidence. LLM training is a compression process in which new connections between pieces of information are formed that were not present in the original data. This is evidenced both mathematically and anecdotally. In my own experience, I've gotten valid answers to such obscure and detailed technical question that it is hard for me to believe would exist in any training data in exactly that form. Whether you would call this reasoning" or not might be open to debate, but regardless of what you call it, it is something more than just unadorned information retrieval like a stochastic parrot."

What is your experience? Let us know in the comments.

The post Can AI models reason: Just a stochastic parrot? first appeared on John D. Cook.