Skip to content
Apple Machine Learning Research·

🧠SSR Cuts Multimodal Agent Reasoning Latency by Over 90 Percent

Your agents might stop overthinking every single step

TL;DR

Researchers propose Selection-based Structured Reasoning, which replaces open-ended agent thinking with picking from reusable reasoning candidates. Per-turn reasoning latency drops over 90 percent, and success rates stay competitive with leading search agents of the same size.

Multimodal agents have a dumb habit: they write out long freeform reasoning before every action, even when most of it goes nowhere. A new framework called Selection-based Structured Reasoning (SSR) says stop generating that text and start choosing it instead. Bottom line: the model picks from pre-written, reusable reasoning candidates scored by their likelihood in the current context. If you're building agents that browse, search, or act across images and text, this hits your latency budget directly. Every token of reasoning you cut is inference cost you stop paying, and per-question time is what your users feel. SSR's trick is that the candidates are pre-specified, so scoring them can happen in parallel instead of one token at a time. That's a structural change to how agent loops run, not a prompt tweak. The numbers are the reason to read this. Per-turn reasoning latency drops by more than 90 percent, and total per-question inference latency falls 28 to 54 percent. Accuracy is the catch people will ask about, and the paper says SSR keeps average success rate competitive with leading search agents at the same scale. It was tested on seven multimodal search benchmarks using 2B and 4B models, so don't assume the same gains at 70B without evidence.

SSR Cuts Multimodal Agent Reasoning Latency by Over 90 Percent — Apple Machine Learning Research

Key Points

1

SSR reduces per-turn reasoning latency by over 90 percent by scoring pre-specified reasoning candidates in parallel instead of generating open-ended text each turn.

Keep reading — it's free

Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.

Also get
or

Already a member? Sign in

multimodal-agentsreasoninginference-latencyllm-agentsefficiency

Comments

Subscribe to join the conversation...

Be the first to comment