AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Bringing ExecuTorch to Arm Cortex-M CPUs

Collected Oct 15, 2026

PyTorch has added a Cortex-M backend to ExecuTorch, its on-device inference runtime, so trained PyTorch models can be exported, quantized and run directly on Arm Cortex-M microcontroller CPUs with Arm CMSIS-NN acceleration. The backend is available now in the ExecuTorch repository. It covers the full Cortex-M range — Scalar cores (M0, M0+, M3, M23), DSP cores with SIMD multiply-accumulate (M4, M7, M33, M35P), and MVE/Helium cores (M55, M85) with 128-bit vector operations — from one workflow, with the target chosen at compile time.

The gap it fills is the conversion step. Deploying to microcontrollers previously meant translating models into another framework's format, keeping two toolchains, and chasing numerical differences between them. With the new backend the model stays inside the PyTorch stack that already serves mobile, desktop and server. It also complements existing Arm MCU support through the Ethos-U NPU, so parts with or without an accelerator use the same path.

The mechanism is ahead-of-time operator replacement. Compiler passes swap standard PyTorch operators for CMSIS-NN-backed custom operators; at inference those call into CMSIS-NN, Arm's open-source library of optimized neural network kernels for Cortex-M. A purpose-built quantizer, built on PyTorch's PT2E framework, produces INT8 graphs laid out the way CMSIS-NN kernels expect.

Several decisions move work from the device to the compiler. Redundant dtype conversions, quantize/dequantize pairs and memory-format transposes are stripped during lowering. Quantization scale and zero-point arithmetic is folded into integer multiplier/shift form, so kernel inference stays in fixed point. Convolution weights are transposed from PyTorch's native OIHW layout to CMSIS-NN's OHWI at export time, so the permute cost is paid once. Tensor and scratch buffers are laid out statically, including across multiple memory regions such as TCM and SRAM, with placement customizable so latency-critical buffers land in fast local memory. There is no heap allocator and no runtime memory management, which means no fragmentation risk under sustained load. Kernel variants matching the core's ISA tier are selected at compile time rather than through runtime dispatch.

The workflow in the published example is: export with torch.export; prepare and calibrate with CortexMQuantizer; convert; re-export the quantized model; lower with CortexMPassManager given a CortexMTargetConfig; then serialize to a .pte file. That binary is linked into firmware alongside the runtime and CMSIS-NN, then flashed.

Footprint is the headline constraint, and the numbers are small. The ExecuTorch core runtime is under 20 KiB of flash. Operator selective build links only the operators a model uses, so code size scales with the model rather than the framework.

The benchmarks cover four of the five MLPerf Tiny models, measured on a single Cortex-M55 core at 400 MHz on an Alif Ensemble E8, with that board's Ethos-U55 and U85 NPUs unused. DS-CNN keyword spotting ran in 5.825 ms (172 inferences per second) at 26.00 KiB peak RAM and 37.67 KiB model flash, 73.70 KiB total. MobileNet v1 for visual wake words took 21.450 ms (47 per second), 85.15 KiB RAM, 253.07 KiB model flash, 289.10 KiB total. ResNet-8 image classification ran in 15.290 ms (65 per second), 55.14 KiB RAM, 89.77 KiB flash, 119.59 KiB total. The autoencoder for anomaly detection was fastest at 1.208 ms (828 per second), 5.20 KiB RAM, 272.03 KiB flash, 287.00 KiB total. All four use under 90 KiB of RAM, three under 60 KiB; model code is 15 to 37 KiB. Figures were compiled with GCC 15.2.1 using -Oz for the runtime and -O3 for kernels, with logging off. The authors note these are their own measurements, not an official MLPerf Tiny submission, and are not directly comparable to MLCommons leaderboard results.

For chips that pair a Cortex-M CPU with an Ethos-U NPU, the EthosUPartitioner hands supported operations to the NPU via the Vela compiler; what stays outside runs on the CPU in the same graph. Cortex-M passes run after Ethos-U partitioning and reuse the graph's existing quantization parameters, so a model quantized with EthosUQuantizer can use CMSIS-NN for CPU operations that meet its constraints, falling back to ExecuTorch's portable (unaccelerated) kernels otherwise. In practice this means incomplete NPU operator coverage is not a blocker: a model can ship on mixed CPU/NPU execution now and migrate more of the graph as coverage grows. That is a meaningful change for teams shipping on parts like the Alif E8, whose NPUs were idle in these measurements.

A person-detection demo shows the practical ceiling: a µYOLO model (pruned depthwise-separable CNN backbone, linear predictor head) on a Renesas EK-RA8D1 board with a Cortex-M85 at 480 MHz. The INT8 .pte is 452 KiB, total flash 640 KiB, peak RAM 254 KiB, and inference takes 24 ms per 128x128 RGB frame, or 12.7 frames per second end to end. Placing the largest operators' weights in DTCM and enabling the instruction cache cut latency from 191 ms to 24 ms, nearly 8x. The demo application in the repository targets the Corstone FVP so it can be brought up without hardware.

Tooling supports iterative debugging: intermediate representations stay executable, so outputs can be compared at each stage, graphs can be inspected with Model Explorer, and ETDump reports per-operator execution costs on the target. The team's stated next steps are mixed-precision support (A8W4, A16W8) across the full operator set, exposing CMSIS-NN 8.0.0's fp32 and fp16 MVE kernels, broader coverage (attention, layer normalization, CMSIS-DSP operators such as FFT), more end-to-end examples including streaming wake word, and validation on more hardware.

Why it matters: embedded and TinyML developers who train in PyTorch can now keep one toolchain from training through firmware, removing the format conversion and cross-framework numerical debugging that used to sit between the two. This suggests smaller teams can target MCUs without maintaining a separate embedded ML pipeline. The trade-offs are quantization-bound: INT8 is the path shown, floating-point kernels are still to come, and attention-style architectures are not yet covered — so teams with unusual operators should check the supported-operator list before committing.

Read at PyTorch

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

TL;DR Deploying PyTorch models on microcontrollers has historically required converting to other framework formats, maintaining separate toolchains, and debugging cross-framework numerical mismatches. ExecuTorch eliminates this conversion step entirely. The Cortex-M... The post Bringing ExecuTorch to Arm Cortex-M CPUs appeared first on PyTorch .