Skip to content
Source: MIT Technology Review

📈METR Finds AI Task Horizons Doubling Every Four Months, Judgment Tasks Lag Behind

Your AI evals are about to get a lot harder

TL;DR

METR's new report says AI task completion length still doubles roughly every four months on coding and math-style work. Judgment-heavy tasks lag badly, and the team says trends there will be far harder to pin down. If you're building evals or planning on agents doing open-ended research, that gap matters more than the headline curve.

The exponential is still running, but only on the tasks we can grade automatically. METR's updated time horizon plot shows frontier models completing tasks that take humans longer and longer, doubling roughly every four months. The catch is that the new suite adds open-ended work, and models are noticeably weaker there than at coding. If you're shipping agents, this is the split you need to plan around. Coding and math improve fast because reinforcement learning works when a solution either passes or fails. Choosing a research direction or making a judgment call doesn't have a clean reward signal, so benchmarks for those tasks are sparse and progress is harder to measure. Don't assume the coding curve transfers to the work your agents do after the tests pass. The numbers that matter: the original plot tracked doubling every four months, and the updated version extends that to much longer human-equivalent tasks. Newer models like Opus 5.5 land at "borderline above the bar" on METR's evaluation design tasks, which a year ago every model failed to reach. That's promising, but not yet usable for METR's own research. Worth watching: whether the judgment-task scores keep climbing at the same rate. METR itself says it doesn't expect trends there to be as clean or robust, so treat any single forecast with suspicion.

METR Finds AI Task Horizons Doubling Every Four Months, Judgment Tasks Lag Behind — MIT Technology Review

Key Points

1

METR's original plot showed the length of tasks frontier models could complete doubling roughly every four months, measured by how long human testers needed for the same tasks.

Keep reading — it's free

Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.

Also get
or

Already a member? Sign in

METRtime horizonAI evaluationreinforcement learningagents

Comments

Subscribe to join the conversation...

Be the first to comment