Google and Meta Open-Source TorchTPU to Bring Native PyTorch Support to Google TPUs

Google and Meta have released TorchTPU as an open-source project, providing a native PyTorch backend for Google's tensor processing units. The announcement, reported by InfoQ's Sergio De Simone, positions the work as a way for existing PyTorch code to run on TPU hardware with minor edits, covering eager and compiled execution modes, distributed computing and hardware-specific optimizations.
The move follows Google's earlier route to PyTorch-on-TPU support: since 2020 it has used PyTorch/XLA, which relied on JAX, XLA and lazy tensors. According to Google, that design carried real costs. PyTorch/XLA demanded intrusive source-code changes to mark out regions that could be compiled, had severe compile-time latency, did not support PyTorch's native eager mode, and trailed the PyTorch ecosystem in supporting newer distributed-computing primitives.
TorchTPU is built around a different priority: eager execution first. In practice, porting a workload from GPUs means swapping the cuda device for tpu and the nccl distributed backend for tpu_dist. Developers can instead use torch.accelerator.current_accelerator to detect whichever accelerator is present and drop the explicit backend argument from the process-group initialization. The example Google shows takes a standard torch.distributed script, changes the device line and lets init_process_group pick its backend, leaving the rest of the model and data-parallel setup intact.
Eager mode comes in three variants: strict, debug, and DeferAndFuse. The last is a hybrid eager/compiled mode that Google says can run 50 percent or more faster than Strict Eager while reducing peak HBM memory. HBM here means the high-bandwidth memory attached to the accelerator, the resource that most often caps model size and batch size in training and inference. A hybrid mode that speeds up execution while lowering that peak is the kind of combination practitioners usually have to trade off, so the claim is worth watching as independent users reproduce it.
To check whether existing models run untouched, Google built an integration pipeline that evaluates roughly 5,000 Hugging Face models spanning more than 500 architectures, in both eager and compiled modes, on TPU generations from v5e onward. Google describes the methodology as corpus stratification with balanced architectural selection: a two-tier process intended to weight real-world adoption while keeping architectural diversity, cut redundant testing of near-duplicate variants, and still exercise production weights. This suggests the goal is to measure coverage across the long tail of community models rather than a handful of flagship networks.
Performance figures came from a 30-billion-parameter Muse Glimmer model written in standard PyTorch. On a single TPUv7 chip, inference reached 25.7k tokens per second at a sequence length of 5,120 and 22.4k tokens per second at 10,240. For training on four TPUv7 chips, throughput was 5.6k tokens per second at 5,120 and 4.6k tokens per second at 10,240. Longer sequences lowering throughput is expected, since attention cost grows with context length, but the numbers give developers a rough sense of what a single chip handles at these settings.
Google also ran training experiments with Qwen3 MoE 235B, a large mixture-of-experts model, scaling from 1,024 to 8,192 TPUv7 chips. For inference on large-scale MoE workloads using vLLM, it reported competitive performance per dollar. Mixture-of-experts models activate only part of their parameters per token, so cost efficiency and memory capacity matter as much as raw throughput, which makes performance-per-dollar the metric enterprises tend to track when comparing accelerators.
Reaction on Hacker News was broadly positive. One commenter, tingkuanpei, said Google should have done this much earlier, noting how rich the PyTorch ecosystem is. Others pointed out that beyond offering TPUs through Google Cloud, Google sells them for on-premises deployments to enterprise clients, naming Anthropic as one and raising the possibility of Meta following. That framing highlights a strategic point: a native PyTorch path lowers the switching cost for teams that would otherwise stay on GPUs simply because their code and tooling assume CUDA.
TorchTPU is early and still evolving. The roadmap lists dynamic shapes, compilation performance, operator coverage, reinforcement learning and quantization. Of these, operator coverage and dynamic shapes are the classic blockers for porting real models, since a missing kernel or a rigid shape assumption can quietly force a fallback or a code rewrite. Quantization matters for serving costs, and reinforcement learning tooling has become a major PyTorch workload. A likely trade-off for early adopters is uneven coverage outside well-trodden model families, with the Hugging Face sweep offering evidence of breadth but not a guarantee for every custom kernel.
For developers, the appeal is straightforward: a backend that keeps eager semantics and needs only device and backend substitutions means less rewriting than PyTorch/XLA required, and the accelerator-detection API makes code portable across GPU and TPU fleets. For product teams, the practical question is whether performance-per-dollar on TPUs beats their current GPU baseline for the specific models they serve. Why it matters: native PyTorch support widens the realistic choice of hardware for teams already invested in PyTorch, and the involvement of Meta signals that cross-vendor portability is being treated as a shared concern; my inference is that the competitive pressure will fall on tooling completeness and operator coverage more than on raw chip specifications. Early-stage gaps in dynamic shapes and operators are likely to determine how quickly teams can move production workloads rather than experiments.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
TorchTPU is an open-source project developed by Google in collaboration with Meta that provides a native PyTorch backend for Google TPUs. It enables PyTorch workloads to execute on TPU hardware with minimal code modifications, supporting eager and compiled execution, distributed computing, and hardware-specific optimizations. By Sergio De Simone