The latest Research stories across our sources, prepared from the publishers’ own reporting.
An article by Jérémy Cohen in The Gradient examines whether large language models could advance autonomous driving, surveying research areas including perception, planning and generation, and citing models such as Talk2BEV, DriveGPT and Wayve's GAIA-1. The author says it is too early to tell whether LLMs can be trusted in self-driving cars.
Researchers present vec2text, a method that recovers text from its embedding vectors, achieving 92% exact match on 32-token sequences and a BLEU score of 97. The work, published at EMNLP 2023, raises questions about the privacy of embeddings stored in vector databases.
A Gradient article by an author with about 20 years of machine learning experience describes common ways models appear to perform well while failing on real-world data, including misleading data, data leakage and inappropriate metrics. It cites a review by Roberts et al. of Covid prediction models and a Toronto water quality system, and points to the REFORMS checklist.
Lilian Weng published an article surveying methods for collecting and evaluating high-quality human-annotated data for machine learning, covering rater agreement, disagreement, and annotation aggregation techniques.
This article reviews how deep learning is applied to single-cell sequencing data analysis, covering autoencoder architectures such as denoising and variational autoencoders and their use in imputation and denoising tasks.
Lilian Weng published an overview post on adversarial attacks against large language models, covering threat models, attack classification, and specific techniques such as token manipulation, gradient-based attacks, and jailbreak prompting.
A Gradient article surveys neural algorithmic reasoning, the effort to capture classical computation such as shortest path-finding in deep neural networks. It covers algorithmic alignment theory, graph neural networks for algorithm execution, and applications including mathematics and computer networking.
Chip Huyen published a long technical post explaining multimodality and large multimodal models (LMMs), covering why multimodal data matters, the different data modalities and multimodal tasks, and the fundamentals of systems like CLIP and Flamingo. It also surveys active LMM research areas including multimodal outputs and efficient training adapters.
Chip Huyen outlines ten open research directions in large language model work, based on conversations with people in industry and academia. She highlights hallucinations and context learning as the most discussed areas, and names multimodality, new architectures and GPU alternatives as the ones she is most excited about.
Lilian Weng published an overview of LLM-powered autonomous agent systems, describing planning, memory, and tool use as the key components, with proof-of-concept demos including AutoGPT, GPT-Engineer, and BabyAGI as examples.
Lilian Weng published a post explaining prompt engineering, also called in-context prompting, as methods to steer LLM behavior without updating model weights. The post covers zero-shot and few-shot prompting, example selection and ordering, instruction prompting, self-consistency sampling, chain-of-thought prompting, and automatic prompt design.
Lilian Weng published version 2.0 of her "The Transformer Family" post, a refactored and expanded update to her 2020 article. The new version restructures the section hierarchy, adds more recent papers, and is described as a superset of the old version at about twice its length.
A technical blog post by Lilian Weng provides an overview of methods for optimizing inference of large transformer models, covering distillation, quantization, pruning, sparsity, mixture-of-experts, and architecture-specific improvements. It explains challenges such as high memory usage, including KV cache memory, and reviews techniques to reduce memory footprint, computation, and latency.
Lilian Weng published a technical blog post titled "Some Math behind Neural Tangent Kernel," providing a deep dive into the motivation, definition, and proofs behind neural tangent kernel (NTK) theory. The post covers NTK's connection to Gaussian processes and the proof of deterministic convergence for infinite-width networks.
Lilian Weng's post surveys one approach to vision language models: extending pre-trained language models to consume visual signals. It groups methods into four buckets and describes techniques such as jointly training image and text, learned image embeddings as frozen LM prefixes, cross-attention fusion, and training-free decoding guided by vision-based scores.
Lilian Weng published Part 3 of her "Learning with not Enough Data" series, covering synthetic data generation for training. The post details data augmentation methods for images, text, audio, and architecture, plus data synthesis using pretrained language models as noisy annotators or data generators.
This article, part two of a series on learning with limited labeled data, covers active learning methods that select samples for human labeling under a budget. Topics include acquisition functions, uncertainty and diversity sampling, deep acquisition functions, and representativeness measures.
Lilian Weng published Part 1 of a series on learning with insufficient data, covering semi-supervised learning, which combines labeled and unlabeled data. The post surveys four general approaches and specific consistency regularization, pseudo labeling and self-training techniques, drawing mostly on vision research.
Lilian Weng's article surveys techniques for training very large neural networks across many GPUs, covering data, model, pipeline, and tensor parallelism, mixture-of-experts layers, and memory-saving designs. It notes updates adding expert choice routing and a related OpenAI Blog post.
Lilian Weng's blog post "What are Diffusion Models?" explains diffusion-based generative models, covering forward and reverse diffusion processes, connections to score networks and Langevin dynamics, guidance methods, and later updates adding latent diffusion, progressive distillation, and consistency models.