⚡Google Ditches Python for XSE, Serving 6 Million Queries Per Second

Your Python inference servers just got a faster rival

Google just pulled Python out of its AI serving path. The XLA Serving Engine, or XSE, now handles six million queries per second across Search, Ads, and YouTube Shorts, and it's not a research demo. If you're serving ranking, recommendation, embedding, or vision models with tight latency budgets, this matters. Your model code still lives in Python and JAX, but the hot path no longer runs through a Python interpreter. Google also plans TorchTPU integration so PyTorch users can export models through StableHLO into an XLA SavedModel and serve them from a C++ runtime on TPUs. The numbers: in Google's tests, XSE delivered two to five times the throughput of its best optimized Python server while using a quarter of the memory. XLA SavedModels run on GPUs and CPUs too, and Google has partnered with Arm and Intel to build out the ecosystem. XSE is not aimed at large-scale LLM inference, so don't expect it to replace your vLLM setup anytime soon.