AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Fast by Default: Compile-Time Heuristics for Helion Kernels

Collected Nov 12, 2026

PyTorch's Helion, a domain-specific language for performance-portable machine learning kernels, now ships compile-time heuristics for kernel configuration. The work addresses autotuning as a bottleneck: one previously tuned paged-attention kernel across 72 shapes used 10 hours in Helion's quick mode and 25 hours at full effort. The new heuristics read a kernel's intermediate representation (IR) and dataflow at compile time and emit a configuration directly, with no benchmarking or search. They are merged into Helion and enabled by default on H100 and B200 with the Triton backend.

Kernels are classified into three families depending on the operations they contain, and a family-specific heuristic then sets the tuning knobs. Any kernel containing a matmul operation, such as hl.dot or torch.baddbmm, goes to the matmul heuristic regardless of what else it includes. A reduction without matmul goes to the reduction heuristic, and everything else receives the pointwise heuristic. If a kernel's extents are not all known at compile time, none of the heuristics fire and Helion falls back to its previous default, so fully dynamic loops remain in a degraded state.

The shared foundation is a resource estimate. A liveness analysis walks the compiler IR to find where each value is last used, then estimates where Triton will place it: registers, shared memory, or, on Blackwell hardware, tensor memory. From that it derives occupancy, meaning how many thread blocks can live on a streaming multiprocessor at once, and register pressure, meaning whether the assembler will spill. These predictions are approximate because the placement decisions belong to Triton, not Helion, which the team flags as a central limitation.

The matmul heuristic illustrates how the knobs are chosen. If the launch produces fewer thread blocks than the GPU has SMs, block size shrinks on a grid axis to create more work. The number of pipeline stages rises as far as shared memory allows, but the heuristic accounts for stages crowding out occupancy. Warps are chosen to steer Triton toward the larger matrix-multiply instruction, wgmma on Hopper or tcgen05 on Blackwell, which requires at least four warps and an M block of 64 or more. When there is little dynamic matmul work, many small-multiply blocks that overlap their loads can be better. Kernels with several matrix multiplies rank each by importance, prioritizing any with a loop-carried accumulator, and the top-ranked operation settles disagreements over shared knobs such as num_warps.

The reduction heuristic balances three pressures on warp count: enough parallelism for wide reductions, enough blocks per SM to stay resident, and avoiding spills. Block sizes shrink to buy residency, but the unit-stride axis is kept large enough to preserve coalesced loads. A persistence decision lets a reduction stay resident in one pass when the kernel provably reads a row twice, as RMSNorm does to compute a scale and then apply it, accepting some spilling to avoid a second memory trip.

Benchmarks on H100 are grouped by workload. Across 96 kernel-and-shape cells covering nine linear-attention variants, the computed config reaches 1.18x the hand-written Triton in Flash Linear Attention (FLA) for the forward pass and 1.12x with backward included. That compares with 0.52x for Helion's previous no-autotune default and 1.26x for full autotuning. On 153 cells spanning six production reduction kernels from vLLM, the computed config reaches 1.27x vLLM's own CUDA/C++ operators, against 0.58x for the old default and 1.31x for full search, so tuning bought about 3% in that case.

On everyday kernels compared with torch.compile in max-autotune mode, the heuristics show 1.09x over 80 reduction cells, where Helion's old default was 0.27x. Across 70 matmul shapes the figure is 1.88x, versus 0.51x for the default, with two caveats: some comparisons are not apples-to-apples because the PyTorch version materializes intermediates that the Helion kernel fuses, and jagged HSTU attention regresses from 3.72x to 3.27x because the old default happened to land on a good configuration.

A second set of helper heuristics generates roughly 5 to 20 alternative configs to seed the autotuner when it is enabled. Across 38 matmul and multi-matmul kernels, seeding cut the configs needed to reach a decent result by 35x and those needed to get within 25% of the best by 41x, but only 1.5x to get within 5% and 1.2x to finish. The final config was within a fraction of a percent either way, so the benefit is search time, not quality.

In practice, this is a tradeoff between guaranteed quality and instant results. Autotuning can still win by benchmarking thousands of candidates, which remains the right call when shipping a fixed set of shapes and paying once, according to the team. The heuristics mainly change the floor: an untuned Helion kernel no longer starts far behind handwritten libraries. For teams that write per-shape kernels or ship many shapes, the per-shape tuning cost is the multiplier the static approach avoids. A likely inference is that the default path becomes much more attractive for exploratory work and for shapes not seen during tuning, though dynamic-extent kernels and jagged workloads keep the old behavior.

The team lists future directions: compiling once and reading the assembler's actual register and shared-memory usage to correct estimates, firing on kernels whose extents are unknown at compile time, covering knobs such as tensor descriptors and launch style, specializing heuristics per family, and extending beyond NVIDIA, since the constants were tuned against H100 and B200 even though the analysis is parameterized by hardware limits.

Why it matters: Developers using Helion on H100 or B200 get a better default without spending GPU-hours on tuning, which matters for workloads with many kernels or shapes. The gap between no tuning and full tuning is now small on several production-like kernels, so skipping the search becomes a reasonable default, an inference from the reported numbers rather than a guarantee. Specialists can still run autotuning and use the heuristic configs as seeds to speed it up.

Read at PyTorch

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

TL;DR Helion now uses compile-time heuristics that analyze a kernel’s IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and... The post Fast by Default: Compile-Time Heuristics for Helion Kernels appeared first on PyTorch .