🔬OpenDS-World: Best Deep-Search Agent Hits Just 39%
TL;DR
OpenDS-World is a live-web benchmark of 200 human-verified deep search tasks. Twelve frontier models were tested in one tool environment, and the best reached 39.0% accuracy.
OpenDS-World is a live-web benchmark of 200 human-verified deep search tasks. Twelve frontier models were tested in one tool environment, and the best reached 39.0% accuracy. Humans scored 21.1% across 1,000 attempts.
Key Points
200 tasks, each independently verified by humans
12 frontier models evaluated under a unified tool environment
Top model accuracy was 39.0% versus 21.1% for human attempts
Failures cluster in tool coordination and long-horizon search management
Paper compares search cost, termination behavior and failure recovery across models
Why It Matters
Agents already beat people at hard web research, yet still fail most tasks. Treat deep-search agents as drafting aids that need verification.
Quick Facts
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,567 builders reading daily.