LLM's Don't Reason: People Do
LLMs produce beautiful prose, not reliable reasoning. But they're great at augmenting people who do.
Two recent papers addressing LLM’s and reasoning are worth attention.
The first, from researchers at Caltech and Stanford, is bluntly titled “Large Language Model Reasoning Failures.” The second, from Wharton, is called “Thinking: Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender.” There are many more like them but the conclusion is the same: large language models don’t reason. They produce a convincing semblance of reasoning, but it is not the same thing.
However, LLMs create authoritative sounding output. Like a snake oil salesman, the pitch is polished and the product is empty. Even purposefully wrong. People believe it anyway to their detriment.
This matters enormously for strategy.
At VSTRAT we’ve been building AI-augmented strategy tools since 2003, over a decade before generative AI existed. And from the beginning our approach has been deliberate: we use structured, proven frameworks to help humans reason better. We do not build systems that churn out a finished recommended strategy. It’s not that we can’t. It’s that we purposefully don’t.
Here’s why.
The field of behavioral economics, pioneered by Tversky and Kahneman and extended by Shiller, Thaler, and others, draws a sharp line between homo economicus (the perfectly rational decision maker that exists in textbooks) and homo sapiens (the actual human being). Real people think with their heads and their guts. They have insight and intuition that no model can replicate.
And in strategy, rejecting the pattern is often the whole point.
Was it predictable, following patterns of successful businesses, for FedEx to launch an overnight delivery service? No. How about for Netflix to pivot to streaming when broadband penetration was young? For a distant third-place Nintendo to bet the company on the Wii when analysts said they were wasting money? For a recently bankrupt Marvel to risk everything on building their own movie studio?
Every one of those moves was against the patterns LLM’s love. And every one of them redefined an industry. A purely pattern matching machine would not have made any of them.
This is the danger of what the Wharton researchers call “cognitive surrender”: letting an LLM do your strategic thinking because its output sounds confident and polished. The words are beautiful. The decks shine. And the reasoning isn’t there. If the goal is to rubber-stamp a decision already made, the insta-deck works fine; the robot will parrot what you want it to. But that was never strategy.
We love AI. We’ve been working at the intersection of AI and strategy for decades, long before ChatGPT, long before generative AI itself. But we draw a hard line between augmentation and replacement. The role of AI in strategy is to help people think, not to think for them.
The market is starting to figure this out. The infamous MIT study showing that 95% of AI projects fail? Those are the projects built on hype and self-proclaimed expertise. Meanwhile, the Bureau of Labor Statistics reports a 4.9% productivity gain compared to the long-running 2.2% average. That’s applied AI doing what it does best: augmenting human work, not replacing human judgment.
One last thought. We’ve started noticing something in our Google Analytics: referral traffic from ChatGPT. When we investigated, the pattern was clear. If people ask ChatGPT for a strategy, it points them somewhere else. If they ask for help thinking through strategy, it points them to us.
Even ChatGPT knows it’s not a strategy tool.