🔬BizBench: Top Agents Pass Only 64% of Business Tasks
TL;DR
BizBench tests long-horizon business agents on 50 tasks drawn from a major payments company, with 601 tools and 789 documents. Claude Opus 5 leads at 64.0% strict success and GPT-6 Sol follows at 61.3%.
BizBench tests long-horizon business agents on 50 tasks drawn from a major payments company, with 601 tools and 789 documents. Claude Opus 5 leads at 64.0% strict success and GPT-6 Sol follows at 61.3%. Outcome-only scoring hides invalid steps.
Key Points
Environment: 789 knowledge documents, 601 business tools, 50 multi-turn tasks
Strict success: Claude Opus 5 at 64.0%, GPT-6 Sol at 61.3%
Qwen3.8-27B reached 42.0% goal attainment but only 32.7% strict success
DeepSeek-V4.1-Flash averaged 191.3 tool calls per run, about 70% more than reference runs
Tasks include resource limits, timing dependencies, prior commitments and outdated guidance
Why It Matters
Agents that reach the goal while breaking constraints are a liability in payments. Score your own agent on process compliance, not just final outcomes.
Quick Facts
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.