⚡Ai2 Cuts GPU Debug Waits From 2 Hours to 30 Seconds
TL;DR
Ai2 replaced its priority-based GPU scheduler with time budgets, fair-share scheduling and a preemption contract. Teams now get 98% of the GPU hours they were owed while the cluster stays at 98% occupancy.
Ai2 replaced its priority-based GPU scheduler with time budgets, fair-share scheduling and a preemption contract. Teams now get 98% of the GPU hours they were owed while the cluster stays at 98% occupancy.
Key Points
Debug-workload p90 wait fell from 2 hours to 30 seconds; the simulator had predicted 5 minutes
Largest H100 cluster: median queue wait dropped from 5 minutes to 24 seconds, p90 from 2.8 hours to 1.8 hours
Workloads declare a minimum runtime of up to 8 hours, then can be preempted and requeued
Fair share is tracked over a 7-day sliding window; 13 of 15 team allocations got at least 95% of their hours
Repairs needing human intervention dropped 74%
Why It Matters
Demand sat at 2-3x capacity before and after, so the win came from policy rather than new hardware. Any team juggling a shared GPU pool can copy the budget-plus-contract design.
Quick Facts
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,564 builders reading daily.