For years, running PyTorch on Google's TPUs meant either porting code to PyTorch/XLA or learning JAX. TorchTPU, a native PyTorch backend, removes that detour: a TPU becomes just another device, like a GPU. Hugging Face says its Transformers, Diffusers and Kernels libraries now run on it directly, so loading a Hub model onto a TPU takes a single setting.
The old route had real costs. PyTorch/XLA used lazy tensors, so code did not always run when you expected, and small changes could trigger fresh graph compilations that were hard to debug. Hugging Face built Optimum TPU to smooth this over; with TorchTPU it is no longer needed for Transformers or Diffusers. The new backend runs in eager mode by default, supports torch.compile when you want speed, handles distributed training, and ships with kernels written in Pallas, Google's low-level kernel language. Models too big for one chip can be split across chips as they load. In one example, a mixture-of-experts kernel from the Tokamax library made decoding 21.5 times faster on Qwen3-30B-A3B and 12.8 times faster on Gemma 4 26B-A4B at a batch size of 64.
This matters most for cost. TPUs are Google's own chips, built for the matrix maths that neural networks spend their time on, and available on Google Cloud from a single chip to pods of thousands. Teams that stuck with Nvidia GPUs partly because of PyTorch can now price TPU capacity as a real alternative without rewriting their code. It arrives alongside a similar native backend for AWS Trainium, TorchNeuron, and a deal bringing OpenAI's models to Google Cloud, which together point to a less Nvidia-centred market for the tools we review, including PyTorch, Transformers and Diffusers.
It is not finished. Hugging Face says 96.5% of the Transformers tests that run on TPU pass, it has reported 34 issues to the TorchTPU project so far, and nightly testing is still being set up. The speed-ups quoted are for specific kernels at a large batch size, not end-to-end comparisons with mature CUDA setups. The open question for anyone budgeting a move is how TPU performance and price compare with GPUs on their own workloads, and that will need measuring rather than assuming.