🤖Cloudflare Auto Router Beta: AI Gateway's New Routing Model
Auto Router beats Cloudflare's own benchmark
TL;DR
Cloudflare's Auto Router, now in public beta, uses AI to route requests more efficiently, outperforming Cloudflare's existing router in benchmarks. It's free during beta but pricing details are pending.
Cloudflare just dropped Auto Router in public beta through AI Gateway, and it's already outperforming the company's own router in benchmarks. The model uses a two-stage process: AI Gateway builds a candidate pool, then a multi-head classification model runs on Workers AI to pick the best model for each request. This new system allows teams to balance reliability and cost more effectively, with a scoring matrix that considers task complexity and context length. The catch? Without a session ID, requests may land on different models, complicating prompt caching and increasing costs.
Key Points
Auto Router uses a two-stage process: AI Gateway builds a candidate pool, then a multi-head classification model runs on Workers AI.
Keep reading — it's free
Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.
Already a member? Sign in
Comments
Be the first to comment