AivexaNewsSearch
AI news for builders and product teamsChecked every hour

ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser

Collected Nov 10, 2026

PyTorch has introduced a WebGPU backend for ExecuTorch that lets PyTorch models execute on the GPU inside a web browser. A model is exported once with torch.export into an ExecuTorch .pte file, and that same file runs in any browser with WebGPU support through ExecuTorch's runtime for WebAssembly. The release avoids the conventional step of first converting a model into another ecosystem's format, such as GGUF, ONNX or TFLite. The company says the backend is still in early development and that it plans further work on speed, stability, and operator, data-type and model coverage. It also published a browser-based playground and per-model Hugging Face repositories containing .pte files, model cards and export recipes.

The claimed breadth of model support goes beyond text generation. The announcement lists Llama 3.2 1B, Qwen2.5-0.5B and Qwen3-0.6B as 4-bit quantized LLM examples, plus Gemma 3 1B. Vision-language work is shown with SmolVLM for image question answering and captioning. Pure vision coverage spans Depth Anything V2 for monocular depth, SAM 2.1 Hiera Tiny for promptable segmentation, D-FINE for object detection, Real-ESRGAN for super-resolution and MODNet for background removal. Audio examples include Whisper base for speech-to-text and Phonon-31M-EN for text-to-speech. The post also lists on-device training: LoRA and DiReFT fine-tuning for Llama 3.2 1B, Qwen2.5 0.5B and Qwen3 0.6B, aimed at personalized assistants and domain adaptation. That range matters because it suggests the operator coverage generalizes past the narrow matrix-multiply patterns of autoregressive text models.

Performance is measured against ExecuTorch's own CPU path: multi-threaded WebAssembly with XNNPACK. The test used quantized Llama 3.2 1B and Qwen3-0.6B in Google Chrome on an Apple M1 Pro, with each data point a median of five runs after three warmups, across 128 to 8K tokens of context, using identical token IDs for both backends. WebGPU is reported as 4–7× faster on prefill and 2–10× faster on decode. The two backends use different numerics: WebGPU runs 4-bit groupwise weights with group size 64 and fp32 activations, while the CPU side uses XNNPACK's 4-bit weights with 8-bit dynamic activations. Timing was wall-clock for both, because GPU timestamps aren't available for CPU execution. This suggests the comparison is fair on wall-clock terms but should be read as a backend-level comparison rather than an apples-to-apples precision study; results on other hardware or browsers may differ.

Architecturally, models go through torch.export() and a WebGPUPartitioner, then execute as WGSL compute shaders. At runtime the backend allocates GPU buffers and dispatches shaders into command buffers. Dynamic shapes are handled with allocate-at-max buffers and per-dispatch workgroup recalculation, which the team says enables batched prefill, and shapes are routed to specialized WGSL kernels at runtime. Because one artifact is meant to serve many targets, WGSL is translated to Metal Shading Language on macOS, SPIR-V for Vulkan on Linux and Android, and HLSL for D3D12 on Windows. For the browser, the C++ runtime compiles to WebAssembly via Emscripten, and emdawnwebgpu bridges its WebGPU calls to the browser's JavaScript API. WebGPU is the successor to WebGL and adds native compute shaders, shared memory, storage buffers and barriers, which are the features that make GPU inference in a tab practical.

Deployment is deliberately minimal: a .pte file served over a CDN and loaded by a web page. Pipeline compilation, memory management and dispatch scheduling are handled by the backend, and data stays in GPU device memory within the tab. The stated advantages are privacy (prompts, images, camera frames and audio never leave the device), lower network latency, offline use and reduced server-side inference cost. The playground runs many of the listed models on the same WebAssembly plus WebGPU runtime.

In practice, the trade-offs are the usual ones for browser inference. Model weights must be downloaded to the client, so file size and cold-start time shift to the user, and quality is constrained by what fits in GPU memory and a reasonable download. Quantized weights keep size down but can cost accuracy. WebGPU support and driver quality vary by browser and device, and the backend's early status implies gaps in operators and data types will surface on less common architectures. A single cross-platform artifact is convenient, but developers should expect per-platform kernel behavior to differ.

Why it matters: Teams that want an on-device option without a native app or a server round-trip can now keep the PyTorch toolchain end to end, exporting once and shipping a .pte file like any other web asset. Privacy-sensitive or offline features in LLM, vision and speech products become feasible directly in the tab, though the early-stage status and large model downloads mean production use likely calls for careful fallbacks and size budgeting. It also positions ExecuTorch as a competitor to GGUF, ONNX and TFLite paths for browser deployment, which in my reading is the most significant structural shift here.

Read at PyTorch

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

TL;DR PyTorch models now run in the web browser on the GPU through ExecuTorch, with no conversion to another framework’s format. You export with torch.export to an ExecuTorch .pte file,... The post ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser appeared first on PyTorch .