🤖LLM Router Deprecation: End of an Era for Complexity Classification
Complexity classification is dead, long live simplicity
TL;DR
The LLM router, launched March 2023 and shut down Sept. 1, 2023, has been deprecated due to its unpredictability in automated workflows. Engineers now face trade-offs between model consistency and cost savings.
Google just killed the LLM router with its shutdown on September 1st, signaling a shift towards simpler models for most use cases. The router's complexity classification was problematic as it couldn't predict behavior consistently across sessions. For developers, this means understanding the subtleties of different models is crucial to avoid hidden costs in automated workflows. Cache reads are between 75% and 90% cheaper than uncached inputs, but the unpredictability of jumping from one model to another can negate these savings.
Key Points
LLM router was launched March 2023 and deprecated June 2023
Router classified requests into four tiers: simple, standard, complex, reasoning
Cache reads are up to 90% cheaper than uncached inputs
System prompts and conversation history often represent many tokens
LLM routers can break behavior consistency during working sessions
Why It Matters
If you're managing automated agentic workflows or autonomous agents, the unpredictability of LLM routers means higher costs. Isolating different requests with specific models is now superior for most cases. The shift to simpler models will likely reduce hidden costs and improve consistency.
Frequently Asked Questions
Why does this matter?
If you're managing automated agentic workflows or autonomous agents, the unpredictability of LLM routers means higher costs. Isolating different requests with specific models is now superior for most cases. The shift to simpler models will likely reduce hidden costs and improve consistency.
What happened?
The LLM router, launched March 2023 and shut down Sept. 1, 2023, has been deprecated due to its unpredictability in automated workflows. Engineers now face trade-offs between model consistency and cost savings.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.