AivexaNewsSearch
AI news for builders and product teamsChecked every hour

PyTorch Conference 2026: Google and Meta Open-Source TorchTPU for Native PyTorch on TPUs

Collected Oct 25, 2026

At PyTorch Conference North America 2026 in San Jose, Google and Meta open-sourced TorchTPU, a native PyTorch backend for Google's Tensor Processing Units. The launch was detailed in October 21 keynotes by Meta software engineer Zongwei Zhou and Bill Jia, Google Cloud's VP of engineering for Core ML/AI. Meta, which created PyTorch in 2016, said it intends to shift larger-scale LLM workloads onto TPUs using the new backend.

TorchTPU replaces PyTorch/XLA, the lazy-tensor bridge dating from 2020 that the announcement said required intrusive source-code modifications. The pitch is portability by a single line: swapping model.to("cuda") for model.to("tpu"). Google Cloud's announcement, co-authored by Jia, framed the change as TPUs joining GPUs and CPUs as a native option for PyTorch.

The practical reach is broad because TorchTPU hooks into PyTorch's low-level dispatcher. According to the speakers, Hugging Face models worked out of the box, the vLLM and SGLang inference engines run natively, and TorchTitan supports dense, mixture-of-experts and diffusion models for large-scale training. Ray and llm-d handle scaling across hosts on Kubernetes. Four repositories were released: TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan.

On performance, Jia showed near-linear scaling beyond 8,000 TPU chips, with per-chip throughput holding roughly steady as clusters grew. Google's published figures include a Qwen3 235B mixture-of-experts run on 8,192 TPU v7x chips at 28% model FLOPs utilization. The announcement itself is candid that the one-line change is a starting point rather than a finish line: a model's shapes, sharding and ops may be skewed toward GPU architecture, so it should run out of the box but will not hit peak TPU performance without tuning. Google's answer to that gap is its TPU agent suite, now in preview on GitHub, which plugs into coding harnesses to automate onboarding, diagnostics and performance tuning.

Zhou said Meta's PyTorch team co-developed the backend with Google to make the TPU a first-class native PyTorch target, citing production requirements around performance, reliability and maintainability. Jana van Greunen, Meta's director of PyTorch engineering, said in Google Cloud's announcement that advancing AI at Meta requires immense compute scale and deep hardware diversity and that the company cannot be constrained by a single hardware architecture.

AWS moved in parallel, putting TorchNeuron, a native PyTorch backend for its Trainium chips, into public beta on October 20 with .to('neuron') as the sole code change. Both backends register through PrivateUse1, PyTorch's extension point for custom hardware. TorchNeuron's first release has documented gaps: The Next Web reported it does not yet support dynamic input shapes and skips roughly 22,000 PyTorch tests. In an analysis for Forkast, Blair Hayes argued the lock-in has relocated rather than vanished, with optimization and supply layers staying under vendor control.

Conference framing put the demand behind all this in perspective. Mark Collier, executive director of the PyTorch Foundation, said inference has overtaken training as the largest data center workload, driving what he called an incredible hunger for tokens, and described a community goal of open source software that runs on any chip in any cloud and connects to any agent. Foundation slides showed how quickly the market shifts: DeepSeek's Flash model became 60% cheaper in 37 days, and Kimi K3 ran 68% faster on the same hardware in 33 days. Mazin Gilbert of the Agentic AI Foundation said agent token consumption passed human consumption on February 6, 2026, and has since grown 14-fold against 2.8-fold for humans. Those figures draw on public data from a single platform, OpenRouter, so their scope deserves scrutiny before reuse.

Other sessions filled in the edges. In a talk on deploying robot policies with ExecuTorch, Meta's Anthony Shoumikhin and Jacob Szwejbka argued policies can stay in PyTorch from training to device without a conversion stack, with the same quantization and operator graphs running from TensorRT on NVIDIA Jetson to OpenVINO on Intel. On cost, RadixArk's Yueming Yuan said fusing two log-probability computations into one kernel made that step more than 10 times faster in the Miles reinforcement learning framework, and delta updates cut data moved per weight sync from 62 GB to under 1 GB in one large run. Anyscale's SkyRL separates training, inference and environments and orchestrates them with Ray; Hugging Face's OpenEnv, which joined the Agentic AI Foundation on October 22, gives every RL environment a shared interface, and Hugging Face's Sergio Paniego Blanco used TRL and OpenEnv to train a small Liquid AI model through four coding agents, lifting its score on held-out tasks from 42% to 54%.

This suggests the significance is less about a single flag flipping and more about PyTorch becoming a genuine multi-vendor substrate. PrivateUse1 gives vendors a standard registration path, so developers can keep one framework while choosing accelerators. The trade-off is familiar: a common front end does not guarantee common performance, since kernels, sharding strategies and compiler work remain vendor-specific and often vendor-owned. The Hacker News thread, which reached 379 points, captured that mix of relief and doubt, with commenters welcoming first-party support after years of fighting XLA while others asked where a TPU can be bought at all; one whose company evaluated on-premises TPUs said it would stick with Nvidia for at least one more generation.

Why it matters: teams weighing a move get a low-friction path to TPUs, but the one-line change gets a model running, not tuned, so budget for sharding and kernel work and confirm capacity before committing. Inference-heavy products, where token costs dominate, are the likeliest early beneficiaries, though that is inference on my part rather than a stated result. Vendor lock-in arguably shifts from framework APIs toward optimization and supply, meaning portability claims should be tested against real workloads. Session recordings are expected on the PyTorch Foundation's YouTube channel within two weeks of the event.

Read at InfoQ · AI, ML & Data Engineering

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Google and Meta open-sourced TorchTPU, a native PyTorch backend for Google's TPUs, at PyTorch Conference North America 2026, and Meta said it plans to move larger LLM workloads onto TPUs with it. AWS released TorchNeuron for Trainium the same week, while developers questioned TPU availability and where vendor lock-in now sits. By Andrew Hoblitzell