AivexaNewsSearch
AI news for builders and product teamsChecked every hour

NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt

Collected Oct 28, 2026

NVIDIA has published Nemotron 3.5 Transcribe, an open-weight speech model, on Hugging Face and ModelScope, together with Granary v2, the open data-curation pipeline and manifests behind its training, and NeMo Speech fine-tuning recipes. The release combines several speech tasks that are usually shipped as separate models into one checkpoint: transcription across 40 languages, translation to or from English for 31 of them, speaker attribution in 13 languages, and audio question answering.

The model accepts recordings up to an hour long and can label up to eight speakers. Its distinguishing design choice is promptability: developers steer the output with natural-language instructions rather than fixed configuration, asking for a verbatim transcript, a speaker-labelled transcript, a translation, or an answer to a question about the meeting. The model card ships recommended presets, a full prompt guide, and a matrix showing which tasks are supported per language; audio question answering is trained in English only. Word-level timestamps are supported.

Architecturally, Transcribe pairs a speech encoder with a hybrid Mamba–Transformer mixture-of-experts language model, so only about 4B of its roughly 30B parameters are active per token. Speaker cues from a Sortformer component reach the language model alongside acoustic features, which lets the model tag each turn during generation instead of attaching speaker labels to words in a separate pass afterward. That is the mechanism behind its diarization claims.

The benchmark numbers NVIDIA reports are strong. On the Hugging Face Open ASR Leaderboard, it ranks first among open-weight models for English at 5.33% average word error rate, with 4.62% mean WER across seven languages and 10.98% mean WER on long-form audio in the model card's comparison. On Artificial Analysis's speech-to-text ranking it ties for first among open-weight models at 2.8% AA-WER with about 4B active parameters, against 24B for the model it ties; on the held-out voice-agent set AA-AgentTalk it scores 1.8% WER versus 2.3% for the next-best open-weight model. NVIDIA says it ranks ahead of several proprietary speech APIs on that all-model chart.

On speaker attribution, Transcribe posts the lowest diarization error rate on David AI's 21-language leaderboard at 18.26%, narrowly ahead of NVIDIA's dedicated Nemotron 3 Diarization model at 18.38%, and ranks second overall on that benchmark's combined transcription-and-attribution track. It also leads VoiceArena's 14-system English full-session benchmark at 32.4% cpWER, ahead of a Nemotron 3 Diarization pipeline at 34.2%, with deletions at 8.32% of reference words against a median near 31%.

Deployment is single-GPU, spanning Jetson AGX Thor and DGX Spark at the edge up to Blackwell data-center GPUs. On one RTX PRO 6000 Blackwell Server Edition GPU at NVFP4 precision, NVIDIA reports 100 hours of audio processed in under three minutes on 30-second clips at high concurrency. Serving options include vLLM or local execution with NeMo-Speech.cpp, and hosted endpoints are available from Baseten, DeepInfra and Simplismart. One boundary worth noting for product planning: the model is built for recorded audio, not live streaming, so real-time captioning would need a different stack.

Granary v2 is the data side of the story. Its manifest covers 463,062 hours across 48 languages and 16 source corpora, up from 25 languages in v1; NVIDIA is explicit that these figures describe the dataset release, not the full training mixture or sampling weights. The pipeline, built with NeMo Curator and runnable on Ray and Dynamo backends, produces three transcript forms per recording (raw, inverse-normalized raw, and clean), plus audio captions, code-switching restoration, speech-grounded Q&A pairs, contextual biasing terms and scored translations. A high-fidelity filtering stage removed noisy, clipped and multi-speaker segments, cutting VoxPopuli from 354k to 111k hours across 22 European languages. Source audio must be downloaded separately under each corpus's original license.

For domain adaptation, NVIDIA walks through a medical case study on Eka Care's open Indian medical speech benchmark of 3,619 clips. A prompt naming English and the Latin alphabet cut full-benchmark WER from 30.8% to 23.8%. Fine-tuning on roughly 153 hours of synthetic speech, 29 hours of accented Common Voice English and 63 hours of MultiMed recordings, training the encoder and projection with LoRA on the language model, brought it to 11.8%. On 2,764 held-out clips, WER fell from 22.44% to 12.17%, a 45.8% relative reduction, with keyword WER down from 9.2% to 6.5% and semantic WER from 10.0% to 7.6%; that 12.17% beats an open medical-specialist model at 13.79% and a general-purpose ASR model at 14.92%. The trade-off is general English: LibriSpeech test-clean WER rose slightly from 1.54% to 1.69%. The full adaptation took about 14 GPU-hours on one GB300. NVIDIA recommends keeping at least 100 real labelled clips for checkpoint selection.

My reading is that the interesting shift here is control at two layers. At inference, prompts replace per-task models and per-task pipelines; at training, explicit written targets replace whatever transcript convention a corpus happened to contain. That matters because most production ASR failures are formatting and vocabulary failures, not acoustic ones. The caveats are real: benchmark wins on average WER do not guarantee behaviour on your accents or jargon, the streaming limitation rules out live use, and the fine-tuning recipe needs representative labelled audio to pay off.

Why it matters: Teams building meeting tools, call analytics, captioning or clinical dictation get a single open checkpoint covering transcription, diarization and translation, with hosted endpoints for those who do not want to self-host. The published manifests and LoRA workflow lower the cost of adapting to a niche vocabulary, with the medical case suggesting a few days of GPU time rather than a research project. Product teams should treat the leaderboard figures as a starting point and budget a small labelled evaluation set for their own domain.

Read at Hugging Face Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt