🤖Google and Meta Open Source TorchTPU for PyTorch on TPUs
PyTorch now runs natively on TPUs, changing the game for model deployment
TL;DR
Google and Meta have open-sourced TorchTPU, a native PyTorch backend for TPUs, simplifying model deployment and making PyTorch more versatile. This move could significantly impact the cost and efficiency of AI model inference and training.
Google and Meta just open-sourced TorchTPU, a native PyTorch backend for TPUs, making it easier to deploy models on Google's specialized hardware. This is a big deal for anyone using PyTorch, as it simplifies the process of moving models from GPUs to TPUs with just a model.to("tpu") command. The new backend also supports Hugging Face models out of the box, and it's part of a broader push for open-source software that runs on any chip in any cloud. With inference now the biggest data center workload, this could cut costs and improve performance for teams running large models. The vLLM and SGLang inference engines run natively, and TorchTitan supports various model types for training at scale.
Key Points
TorchTPU replaces PyTorch/XLA, a lazy-tensor bridge from 7 years ago, simplifying model deployment.
TPUs join GPUs and CPUs as a native option for PyTorch, making it easier to switch between hardware.
Inference has overtaken training as the largest data center workload, driving the need for efficient solutions.
DeepSeek's Flash model became 60% cheaper in 37 days, highlighting potential cost savings with TorchTPU.
Agent token consumption has grown 14-fold since February 6, 2026, underscoring the importance of efficient inference.
Why It Matters
If you're running PyTorch models on TPUs, TorchTPU simplifies the process with a single command. For instance, moving a model from GPU to TPU is as simple as changing model.to("cuda") to model.to("tpu"). This change can significantly reduce the overhead of deploying models on specialized hardware, making it more accessible and cost-effective for teams.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,567 builders reading daily.