Build specialized multimodal models with LFM3

Liquid AI has released LFM3, the next generation of its Liquid Foundation Models, published on the Hugging Face blog with open weights available on Hugging Face. The family puts text, vision, audio, omni and decision models on one shared backbone, all in the 2B parameter range, and every member branches from a single base checkpoint, LFM3-1.7B-Base. The stated goal is "specialized intelligence": pick the model that matches your input modality, fine-tune it on your own data on a single GPU, and deploy it wherever the application runs.
Four changes separate LFM3 from the earlier LFM2 family. First, more capability per parameter: a new tri-hybrid architecture and training recipe let LFM3-1.7B outperform LFM2.5-2.6B while using 35% fewer parameters. Second, every modality shares one backbone, yielding six 2B-class models including a first omni model and video input for the vision model. Third, an edge focus: the text model fits in about 1 GB at 4-bit precision, new in-house vision and audio encoders speed up processing, and a 256K context window is meant to let agents run on-device. Fourth, specialization, since each model fine-tunes on a single GPU.
The architecture work leaned on Liquid AI's hardware-in-the-loop search, which now covers 14 device and backend combinations spanning phones, a Raspberry Pi 5, laptops, cloud CPUs, GPUs and the Qualcomm Hexagon NPU. LFM3-1.7B-Base has 24 layers built from three layer types: 13 gated short convolutions with kernel size 7 (up from 3 in LFM2), 5 sliding-window attention layers with a 256-token window, and 6 global attention layers. Three further techniques matter to anyone who cares about inference cost and quantization. Head-wise gated attention applies a sigmoid gate to each head's output, so a head can switch off when it contributes nothing, which the company says can suppress activation outliers that hurt quantization. Partial RoPE rotates only the highest-frequency half of each head in global attention layers, leaving the other half to match on content regardless of distance, while sliding-window layers keep full RoPE. Sandwich norm normalizes each block's input and output to keep activations well-scaled.
The payoff is claimed at long context. Per parameter, LFM3-1.7B matches LFM2-1.2B on short prompts across edge devices and is about 25% faster with 8K tokens in context. Pre-training covered 22T tokens with the same 128K vocabulary as LFM2.5-8B-A1B; the final 3.5T tokens formed a mid-training stage on curated data with more agentic traces and tool calls, which also extended the context window to 256K tokens.
The vision and audio models add two new in-house encoders. The vision encoder combines convolution and attention and uses 3x less compute than the SigLIP2 encoder in previous LFM vision-language models; the 164M-parameter encoder matches Qwen3.5-2B's vision encoder in accuracy at 2.2x fewer FLOPs, both measured with the same LFM2.5-230M language backbone. For video, it embeds each pair of sampled frames into one temporal slice, halving the temporal sequence length. The audio encoder mixes sliding-window attention and short convolutions and is fully causal, so the model can prefill while the user is still speaking, with time to first token as low as 50 ms. A new audio tokenizer, LiquidCodec, converts generated tokens to speech at roughly 87x real time on a single CPU thread on an Apple M4 Max and supports streaming decoding.
Model by model, LFM3-1.7B is the text entry, aimed at local agents, tool calling, structured outputs and domain fine-tuning; it is post-trained for harnesses such as Hermes Agent, Pi and OpenClaw. On WebArena-Verified it scores 38.42 against 3.02 for LFM2.5-2.6B, and the company says it beats Qwen3.5-9B on long tasks in browsers and business apps. LFM3-VL-2B is the first LFM vision model accepting video, reasons step by step before answering, and handles object grounding and spatial reasoning with robotics in mind; at 1.9B parameters versus 3.1B for LFM2.5-VL-3B, it raises the average across 62 vision benchmarks from 57.6 to 71.4, described as on par with Gemma 4 12B. LFM3-Audio-2B covers voice chat, speech recognition, synthesis and translation across 18 languages, takes text or audio in and returns text, speech or both, and can turn spoken requests into tool calls; speech quality rises to UTMOSv2 3.83 from 3.22, and it adds audio understanding and tool calling. LFM3-Omni-2B is the first omni model, with text, vision and audio input and text or speech output, and it beats models up to 10.7B parameters such as Qwen2.5-Omni-7B on Daily-Omni, WorldSense and AV-SpeakerBench for audio-visual video understanding. The d1-2B and d1-omni-2B decision models return structured answers to a batch of questions over one input in a single forward pass, each with a calibrated probability, and can point at objects, read document values, estimate numeric ranges and select multiple options; d1-2B replaces d1-3B with a third fewer parameters and scores 75.5 against 64.0 on ImajevBench.
In practice, the shared backbone is the interesting part for product teams. Choosing among six small checkpoints instead of one large generalist trades breadth for footprint and latency, and the claim that a short fine-tune turns a 2B model into a task expert is the bet worth testing against your own data. A likely trade-off is that a 1.7B model will lag frontier models on open-ended reasoning even where it wins narrow agentic and decision benchmarks, so the benchmarks to trust are the ones closest to your workload.
Usage is through transformers, with a minimum version depending on model (5.19 for d1, 5.20.2 for audio and omni, 5.21.0 for vision), generally with trust_remote_code=True. The release includes fine-tuning notebooks for text, vision, audio and omni covering data preparation through evaluation. As a worked example, Liquid AI adapted LFM3-VL-2B into three robotics models for orchestration, navigation and manipulation; the manipulation VLA, trained jointly across 15 embodiments, reaches 96.2% average success across four LIBERO suites and 45.7% on RoboCasa-GR1 tabletop tasks, generating action chunks in under 50 ms on a Jetson AGX Orin with TensorRT FP16.
Why it matters: teams building on-device or privacy-sensitive applications now have a matched set of roughly 2B open-weight models covering text, image, video, audio, combined audio-visual input and structured decisions, all fine-tunable on one GPU. That lowers the barrier to domain-specific deployment on phones, laptops and embedded hardware, though whether a 2B specialist beats a larger hosted model depends on the task, and my inference is that the agentic and decision benchmarks here are the most relevant signals for buyers. The claimed competitive comparisons should be validated on your own evaluation sets before committing.
Based on reporting from the original publisher. Visit the source for full context and later updates.