A paper at ICLR 2026 found that making language models better at reasoning also makes them more likely to invent tool calls. The paper is titled "The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination", and its result runs against a common assumption that smarter models are more trustworthy.

What the researchers found
Chenlong Yin, Zeyang Sha, Shiwen Cui, Changhua Meng and Zechao Li used reinforcement learning to train large language models to reason more effectively. As the models improved on reasoning benchmarks, their tool hallucination rates rose in step.
The effect was not caused by overfitting or domain leakage. The team trained models on mathematics tasks and tested them on unrelated tool-use scenarios, and the increase in hallucination still appeared. It also showed up regardless of method: in fine-tuned models and when chain-of-thought prompting was used at inference time.
In practice, an agent trained to be better at multi-step reasoning becomes more likely to call tools that do not exist, invent API parameters, or claim to have carried out actions it never performed.
The authors also looked inside the models. They report that reasoning optimisation "disproportionately collapses tool-reliability-related representations", and that this happens in late-layer residual streams. Put simply, the part of the model that holds back from acting when it should not act is what gets weakened during reasoning training. The model gets better at hard problems and, at the same time, more confident about answers it has made up.
This matters because much of the industry has sold reasoning as a fix for hallucination. Marketing from major AI labs has leaned on the line "our latest model hallucinates less because it reasons better." For tool use, this paper suggests the opposite.
How they measured it
The authors built a diagnostic benchmark called SimpleToolHalluBench. It tests one behaviour: when given an impossible task, does the agent refuse, or does it fabricate a tool call?
The benchmark removes tools or replaces them with misleading alternatives. A well-calibrated agent should see that the action cannot be done and decline. A poorly calibrated one invents a tool and says it used it. Across models trained with progressively more reasoning reinforcement learning, reasoning scores and hallucinated tool calls rose together.
The paper also tested two common fixes, prompt engineering and Direct Preference Optimization (DPO). Both reduced hallucination somewhat, and neither closed the gap. Both also reduced the model's overall usefulness: it became more cautious and refused more legitimate tasks. With current training methods, there appears to be a real trade-off between capability and reliability.
Where this hits deployed agents
Agents are moving from demos into production, including in government services, financial operations and enterprise automation. In those settings an agent that fabricates a database query, invents an API endpoint or reports sending an email it never sent can do real damage. These findings suggest the training that makes agents more capable may be raising that risk.
Four practical responses follow from the paper.
Validate tool calls outside the model. At runtime, check that every requested tool exists and that its parameters are valid before anything executes. Do not rely on the agent's own account of what it did.
Track fabricated tool calls in production. Task completion rates can look healthy while the agent reaches them through hallucinated shortcuts. Monitor for invented tools and unverified claims of action specifically.
Accept some lost capability where stakes are high. DPO and similar methods only partly work, but partial improvement is better than none in high-risk work, even if the agent becomes more cautious.
Look past reasoning benchmarks. A higher reasoning score says nothing reassuring about tool reliability, and this paper suggests the two can move in opposite directions.
For teams buying an agent platform, that last point turns into a set of direct questions for the vendor. How does your model score on a test like SimpleToolHalluBench, where the correct answer is to refuse? What happens in your system when the agent requests a tool that does not exist? And can you show us logs that separate actions the agent actually executed from actions it only said it executed?