AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Building Reasoning Models from the Ground Up: A Full Video Course

Collected Oct 24, 2026

Sebastian Raschka has published an eight-part video course that walks through building a reasoning model from scratch in Python and PyTorch, published on his Ahead of AI newsletter. The series runs about 11 hours and accompanies his book Build a Reasoning Model (From Scratch), available from Manning and Amazon, with a print edition in color. The author describes it as a sequel to last year's Coding LLMs from the Ground Up course, though he says it can be watched without that prerequisite. He also notes he is taking a short break and plans to return with LLM research roundups next month.

The course starts from a small pre-trained Qwen3 model rather than training a base model itself, so the work focuses on post-training and inference methods: evaluation, inference-time scaling, reinforcement learning and distillation. The author recommends comfort with Python and PyTorch. Early text-generation and evaluation examples can run on a CPU; later experiments, especially training, are much more practical on a GPU. The first video covers environment setup, and the companion repository includes hardware and environment recommendations, including remote-GPU use through VS Code.

Video 1 frames the relationship between conventional LLMs, reasoning models and agents, then sets up the environment with Python and PyTorch dependencies installed via uv, plus hardware choices and troubleshooting. Video 2 loads Qwen3 0.6B weights, implements tokenization and decoding, greedy decoding and a streaming text generator, handles end-of-sequence tokens, measures generation speed, adds a KV cache for reusing attention keys and values, and combines that with torch.compile to compare hardware performance. That gives the baseline the rest of the series evaluates and improves.

Video 3 builds a math-answer verifier: extracting final answers, normalizing them and checking mathematical equivalence, then using it to score the base model on the MATH-500 dataset. The same verifier returns later as the reward signal for reinforcement learning with verifiable rewards. Video 4 covers inference-time scaling with fixed weights: temperature scaling, multinomial sampling and top-p filtering to produce diverse answers, then self-consistency through majority voting over final answers, with an examination of how accuracy trades off against the extra compute spent on more candidates.

Video 5 adds scoring functions to compare multiple answers and break ties in self-consistency, including a simple heuristic scorer and token probabilities and sequence likelihoods computed in log space for numerical stability. It then implements a self-refinement loop in which the model critiques and revises an answer, evaluated on MATH-500.

Video 6 moves to training with a simplified version of Group Relative Policy Optimization for RLVR: generating groups of responses, scoring them with the verifier, and turning relative rewards into weight updates, including token probabilities, a simplified GRPO loss, a training loop with logging and checkpoints, and comparisons between RLHF and RLVR. Video 7 revisits those runs with more diagnostics such as entropy and advantage statistics, clipped policy ratios, KL loss terms and format rewards conditioned on answer correctness, and discusses reward hacking and limits of the simplified KL formulation.

Video 8 closes with distillation from a stronger teacher: comparing soft and hard and on-policy and off-policy variants, collecting teacher-generated answers for the MATH dataset (using DeepSeek R1), preparing tokenized splits, computing answer-token log probabilities and cross-entropy loss, running supervised training, and comparing distilled checkpoints on MATH-500. For a shorter conceptual pass, the author also points to a roughly 90-minute AI Reasoning Models course on LinkedIn Learning.

A useful framing in the post is an engine analogy: the model is the engine, and an agent harness is the tooling and context-management layer around it. In that view, most current-generation LLMs are reasoning models or offer a reasoning mode, so understanding how the engine is built matters for anyone assembling agents. This suggests the course is best read as implementation-level grounding rather than a survey of the research literature; a likely trade-off is that a 0.6B model keeps compute modest but will not reproduce frontier-scale results, so the lessons are about method, and progress on MATH-500 across checkpoints is the main yardstick rather than absolute capability.

Why it matters: developers who want hands-on experience with verifiers, self-consistency, GRPO-style RLVR and distillation now have a structured, code-first path built on an openly available small model, which lowers the barrier to experimenting with post-training. For product teams, the engine-versus-harness framing is a reminder that reasoning capability and agent tooling are separate investments, and that evaluation harnesses such as the MATH-500 verifier are reusable infrastructure rather than throwaway course code. As an inference, expect tutorials like this to become a common reference point for teams prototyping their own post-training pipelines, even though results at this model scale should not be read as predictions for larger systems.

Read at Ahead of AI

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Over 11 hours of coding videos on evaluation, inference-time scaling, reinforcement learning, and distillation, with Python and PyTorch.