TinyTorch: Don’t Just Import PyTorch. Build It.

TinyTorch is a free, open-source curriculum in which learners build a working machine learning framework from scratch in pure Python, from tensors through transformers, using PyTorch's own API. It runs on a laptop with 4 GB of RAM and no GPU. The project is at tinytorch.ai and lives in the Machine Learning Systems repository, with code under MIT and curriculum under CC BY-SA 4.0.

The curriculum consists of twenty modules in four tiers, driven by a CLI called tito and delivered as Jupyter notebooks with portions left for learners to fill in. Prerequisites are Python and comfort with NumPy; no GPU, cloud account, or prior ML systems background is required. Each tier depends on the one below it. Foundation fits a half-semester module, all twenty fit a four-credit course, and self-paced learners take from a few intense weeks to several months.

Three design decisions are described. Systems from day one: Module 01 ships a memory_footprint() method before matrix multiplication. Progressive disclosure: the Tensor class stays free of gradient machinery through Module 05, and Module 06 adds enable_autograd() via runtime monkey-patching rather than inheritance, a choice the authors say mirrors PyTorch 0.4 merging Variable into Tensor. Build to validate: six historical milestones, from Rosenblatt's Perceptron in 1958 to the 2017 transformer and MLPerf-style benchmarking, require a working implementation, including clearing 75% on CIFAR-10 in the 1998 milestone.

The authors state the resemblance to PyTorch stops at the API surface: there is no dispatcher, no C++ or CUDA layer, no JIT, and nothing distributed. Pure Python runs between 100 and 10,000 times slower than PyTorch.

TinyTorch began as CS 249r, a Harvard TinyML graduate seminar launched in 2020, and grew from course notes. Andrea now maintains it from ETH Zurich. The post reports 682 community members across 92 institutions since a December 2025 launch, more than 27,000 repository stars, 95 or more contributors, and courses at 50 or more universities.

The post says TinyTorch is in preview with classroom readiness targeted for Fall 2026. It states learning outcomes have not been measured and that there is no controlled data showing TinyTorch students debug production systems better than students in conventional courses. The scope is single-node and CPU-only, and the authors say it teaches nothing about GPU kernels, distributed training, or gradient synchronization, with parallel data loading and GPU memory management deliberately left out to preserve the 4 GB floor.