💡LLMs Encode 95-98% of Facts, But Recall 26-34% Without Thinking
LLMs Know More Than They Can Remember
TL;DR
New research shows large language models encode 95-98% of facts but struggle to recall them without inference-time thinking, shifting the focus from model size to recall mechanisms.
A new study reveals that large language models (LLMs) like GPT-5 and Gemini-3 encode 95-98% of tested facts but fail to recall 26-34% of those facts without thinking. This means the bottleneck for factual accuracy is recall, not storage. Engineering teams can build more reliable applications by focusing on inference-time computation rather than larger models or external databases. The study categorizes knowledge into five profiles, including direct recall, encoding failure, and recall with thinking, with inference-time thinking successfully retrieving 40-65% of encoded facts initially missed.
Key Points
New research shows LLMs encode 95-98% of tested facts, but recall only 26-34% without thinking.
Inference-time thinking retrieves 40-65% of encoded facts initially missed, acting as a recovery mechanism.
Scaling model size does not automatically resolve the gap between encoding and recall, suggesting storage isn't the issue.
Rare facts are encoded at rates similar to popular facts, but there's a 25% recall gap for long-tail facts.
LLMs struggle with reverse questions but recognize correct answers in multiple-choice format.
Why It Matters
If you're building applications with LLMs, understanding recall mechanisms is key. The study shows that while models encode 95-98% of facts, they recall only 26-34% without thinking. This means focusing on inference-time computation can improve reliability without needing larger models or external databases.
Frequently Asked Questions
Why does this matter?
If you're building applications with LLMs, understanding recall mechanisms is key. The study shows that while models encode 95-98% of facts, they recall only 26-34% without thinking. This means focusing on inference-time computation can improve reliability without needing larger models or external databases.
What happened?
New research shows large language models encode 95-98% of facts but struggle to recall them without inference-time thinking, shifting the focus from model size to recall mechanisms.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,436 builders reading daily.