AI news for builders and product teamsUpdated Nov 18, 2026, 11:00 UTC
AI news, straight from the source.
Releases, research, and ideas for developers and product teams. Updated every hour.
Across the feeds
Newest first
Lilian Weng published a post explaining prompt engineering, also called in-context prompting, as methods to steer LLM behavior without updating model weights. The post covers zero-shot and few-shot prompting, example selection and ordering, instruction prompting, self-consistency sampling, chain-of-thought prompting, and automatic prompt design.

Lilian Weng published version 2.0 of her "The Transformer Family" post, a refactored and expanded update to her 2020 article. The new version restructures the section hierarchy, adds more recent papers, and is described as a superset of the old version at about twice its length.

A technical blog post by Lilian Weng provides an overview of methods for optimizing inference of large transformer models, covering distillation, quantization, pruning, sparsity, mixture-of-experts, and architecture-specific improvements. It explains challenges such as high memory usage, including KV cache memory, and reviews techniques to reduce memory footprint, computation, and latency.

Lilian Weng published a technical blog post titled "Some Math behind Neural Tangent Kernel," providing a deep dive into the motivation, definition, and proofs behind neural tangent kernel (NTK) theory. The post covers NTK's connection to Gaussian processes and the proof of deterministic convergence for infinite-width networks.

Lilian Weng's post surveys one approach to vision language models: extending pre-trained language models to consume visual signals. It groups methods into four buckets and describes techniques such as jointly training image and text, learned image embeddings as frozen LM prefixes, cross-attention fusion, and training-free decoding guided by vision-based scores.

Lilian Weng published Part 3 of her "Learning with not Enough Data" series, covering synthetic data generation for training. The post details data augmentation methods for images, text, audio, and architecture, plus data synthesis using pretrained language models as noisy annotators or data generators.

This article, part two of a series on learning with limited labeled data, covers active learning methods that select samples for human labeling under a budget. Topics include acquisition functions, uncertainty and diversity sampling, deep acquisition functions, and representativeness measures.

Lilian Weng published Part 1 of a series on learning with insufficient data, covering semi-supervised learning, which combines labeled and unlabeled data. The post surveys four general approaches and specific consistency regularization, pseudo labeling and self-training techniques, drawing mostly on vision research.

Lilian Weng's article surveys techniques for training very large neural networks across many GPUs, covering data, model, pipeline, and tensor parallelism, mixture-of-experts layers, and memory-saving designs. It notes updates adding expert choice routing and a related OpenAI Blog post.

Lilian Weng's blog post "What are Diffusion Models?" explains diffusion-based generative models, covering forward and reverse diffusion processes, connections to score networks and Langevin dynamics, guidance methods, and later updates adding latent diffusion, progressive distillation, and consistency models.

Lilian Weng published a technical overview of contrastive representation learning, covering training objectives such as contrastive loss, triplet loss, N-pair loss, NCE and InfoNCE, plus common setups and key ingredients including heavy data augmentation, large batch sizes and hard negative mining.

Lilian Weng published a post examining methods for reducing toxic content in pretrained language models. It covers toxicity definitions and categorizations, data collection and annotation, detection approaches, and detoxification techniques.

Lilian Weng's blog post surveys methods for controlling neural text generation with pretrained language models, covering decoding strategies, guided and trainable decoding, prompt design, and fine-tuning approaches. It was updated through September 2021 to add P-tuning, Prompt Tuning, and unlikelihood training.

Lilian Weng published a technical overview of approaches for building open-domain question answering systems, covering open-book retriever-reader and retriever-generator frameworks, closed-book generative QA, and related techniques.

A survey-style report describes Neural Architecture Search (NAS), the automatic design of neural network architectures. It outlines the three components defined by Elsken et al. 2019—search space, search algorithm, and evaluation strategy—and reviews search spaces, search algorithms, and evaluation strategies from prior work.

An updated overview of exploration strategies in deep reinforcement learning, covering classic methods, key problems like hard exploration and noisy-TV, and intrinsic reward techniques including count-based and prediction-based approaches.

Lilian Weng updated her post on the Transformer family on 2023-01-27, refactoring it to incorporate new Transformer models since 2020. The revised version, titled The Transformer Family Version 2.0, is the recommended reference on the topic.

A technical overview of curriculum learning for reinforcement learning, covering task-specific curricula, teacher-guided methods, self-play, automatic goal generation, skill-based curricula, and curriculum through distillation. The article notes updates made on 2020-02-03 and 2020-02-04 and cites work including Elman (1993) and Bengio et al. (2009).

Lilian Weng's self-supervised representation learning post covers pretext tasks for unlabeled data, including image distortion, patch prediction, colorization, generative modeling, and contrastive learning, with update notes from 2020 through 2021.

Lilian Weng published a technical post on Evolution Strategies, covering Gaussian ES, CMA-ES parameter updates, Natural Evolution Strategies, and applications in deep reinforcement learning including OpenAI ES and exploration methods.