AivexaNewsSearch
AI news for builders and product teamsChecked every hour

From KTIR to SpyreCode: An MLIR Dataflow Backend for IBM Spyre

Collected Oct 20, 2026

The PyTorch project has published the second of three posts describing how software for IBM's Spyre accelerator is compiled, focusing on the boundary between torch-spyre's Inductor frontend and a new backend compiler. The central artifact is KTIR (Kernel Tile IR), an MLIR representation documented in RFC 0682 and implemented in the torch-spyre/ktir-mlir-frontend repository. A new Dataflow Scheduler, open source under Apache-2.0 together with its dialects, consumes KTIR and emits DataflowIR, which Spyre's device-specific stages then use to generate SpyreCode — a symbolic SpyreBinary plus a job plan that the runtime prepares once and launches repeatedly, filling in tensor addresses and shapes at launch.

Three design ideas frame the stack. The frontend, meaning Inductor, decides the parallel decomposition, while the backend's job is dataflow scheduling within that mapping rather than rediscovering it. Scheduling is treated as pipeline construction: Spyre decouples access from execution, with loads, stores and compute running on separate engines that communicate through FIFOs, so the scheduler starts from the simplest legal pipeline and refines it one readable MLIR rewrite at a time. And the boundary between the halves is an IR rather than a library call. KTIR, the scheduler's ScheduleIR and DataflowIR are all ordinary MLIR built on upstream dialects such as linalg, scf, arith, memref and tensor.

KTIR is written against a deliberately simple hardware picture: cores, each with a compute engine and its own on-chip scratchpad, joined by an interconnect that also reaches off-chip memory. Every compute tile can see every memory — its own scratchpad, other cores' scratchpads and device memory. That mirrors Spyre's shape: 32 cores on a ring, 2 MB of scratchpad each, backed by off-chip device memory, with data moving in sticks of 128 bytes (64 fp16 values). The abstraction deliberately omits FIFOs, load/store units and decoupled execution engines, because KTIR states what each tile computes and where its data lives, not how it gets there.

The IR is compositional: arithmetic, reductions and control flow come from upstream dialects, while a dedicated ktdp dialect adds explicit memories and explicit accesses. Its operations include get_compute_tile_id, construct_memory_view (interpreting a region at a possibly symbolic address as a strided tensor, with no allocation or data movement), construct_distributed_memory_view (composing per-core views into one logical tensor), construct_access_tile (an integer domain plus an affine traversal order, so tiles can be rectangular, strided, skewed or triangular), construct_indirect_access_tile for gathers such as paged KV-cache lookups, load and store as the only data movement, and inter_tile_produce/inter_tile_reduce for cross-core communication like all-reduces of partial sums. A worked example in the post splits an elementwise add over 96×64 fp16 tensors across 32 tiles of three rows each, with grid = [32] and arithmetic on the tile ID fixing the partition that the backend must honor.

A second, smaller dialect called spyreop exposes Spyre intrinsics as ordinary MLIR operations: activations and elementwise functions (exp, gelu, sigmoid, silu, softplus, sqrt, rsqrt, reciprocal, realdiv), normalization building blocks (a combined mean and mean of squares, and LayerNorm's normalizing and scaling halves, some fused), integer and address arithmetic including idx32toaddr for gathers, plus compare and select. It also defines df16, IBM's 16-bit DLFloat format with a 1-6-9 sign/exponent/mantissa split rather than IEEE half precision's 1-5-10, and fused types carrying a pair of floats as one value. Intrinsics sit inside linalg.generic bodies so they fuse with surrounding elementwise work; how each expands into hardware instructions lives in the architecture description, not the scheduler.

KTIR maps Inductor's decisions one-to-one: work division becomes the grid attribute and affine arithmetic on the tile ID; stick-granularity layout becomes view strides and access-tile order; coarse tiling and working-set reduction become scf.for loops; scratchpad residency chosen by the allocator becomes a view in a ct_local memory space at the assigned offset; core-to-core relayouts become access tiles over distributed views; reductions split across cores become inter_tile_produce/reduce; and PyTorch-owned addresses or runtime shapes stay symbolic, including through a runtime_arg type with granularity and upper-bound parameters.

What most distinguishes KTIR from GPU-oriented tile IRs, in the authors' framing, is its treatment of on-chip memory. Triton-style models let each program instance load blocks from global memory and leave staging through shared memory, registers and caches to the compiler and hardware, with on-chip storage private and short-lived. Spyre has no hardware-managed cache, so keeping tensors on-chip between operations is a major performance lever, and Inductor's scratchpad allocator decides which buffers stay resident, where, and on which core. KTIR records that decision directly. A distributed memory view lets a producer store into per-core ct_local views and a consumer load from them without touching device memory, turns core-division mismatches between producer and consumer into ordinary loads that cross scratchpad boundaries, and keeps placement binding because the memory space and coordinate set form the placement contract.

The backend is a single MLIR pipeline in two halves meeting only at DataflowIR. The scheduler is an MLIR pass pipeline driven by an architecture description written in the ktdf_arch dialect: a graph whose nodes are memories and execution units with SIMD lane counts per element type, word sizes and access granularities, and FIFO depths, and whose edges are datapaths with transfer granularities, with groups scoping resources per core. Schedules use three constructs — nested scf.for loops, ktdf.pipeline for stages overlapping across iterations, and ktdf.parallel for identical instances splitting an iteration space. Passes fall into three groups with fixed IR contracts at their boundaries, and at four checkpoints (pre_mapping, pre_scheduling, post_scheduling, post_lowering) the scheduler applies rewrite patterns written in PDL that live in the architecture description, so a device can supply its own lowerings without forking the scheduler.

Why it matters: teams targeting Spyre, or building compilers for accelerators without hardware-managed caches, get a concrete open blueprint for splitting parallel decomposition from dataflow scheduling across a stable IR boundary. The open-source scheduler and dialects under Apache-2.0 mean device support can be added through architecture descriptions and PDL patterns rather than code changes, which in practice lowers the cost of retargeting — an inference, since the post does not discuss porting effort. The symbolic-address and runtime-shape handling is also directly relevant to anyone compiling PyTorch models whose tensor shapes or allocations are only known at launch.

Read at PyTorch

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

TL;DR Torch-spyre’s Inductor decides how each kernel’s work is divided across Spyre’s cores and manages tensor residency between on-chip scratchpad and off-chip memory, and the backend compiler turns those decisions... The post From KTIR to SpyreCode: An MLIR Dataflow Backend for IBM Spyre appeared first on PyTorch .