AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Scale Agentic Coding Workflows with Meta’s Muse Spark Open on NVIDIA GB300 NVL72

Collected Oct 15, 2026

Meta has released Muse Spark Open, a coding-focused large language model built for agentic workflows, with NVIDIA detailing how it runs on the GB300 NVL72 rack-scale system. The model is a sparse mixture-of-experts (MoE) design with 816 billion total parameters, of which 42 billion are active per token. It carries a 1 million token context window and was trained on NVIDIA's GB300 NVL72, a platform built around Blackwell Ultra GPUs.

The defining numbers are worth unpacking. In a sparse MoE model, each token is routed only to a subset of the network's experts rather than through all parameters. Here, routing selects the top 8 of 256 experts per token, which is where the 42B active figure comes from. That arrangement is meant to deliver capability associated with a very large model while keeping per-token serving costs closer to a much smaller one, because far fewer weights are touched on each forward pass. This is the same broad pattern used by several recent frontier open models, and it is the reason a model of this total size can be served economically at all.

The 1M token context window is positioned as a way to hold an entire project in a single session, including dependencies, legacy code, and thousands of files, without splitting a task into smaller pieces. For agentic coding, where a model plans a multifile refactor, navigates a repository, recovers from failing tests, and coordinates subagents across many turns, context is often the binding constraint. Meta describes the model as planning, acting, observing, and self-correcting, with tool use, output evaluation, and iterative refinement as core behaviors. It also supports goal conditioning to maintain direction and context compaction to preserve relevant knowledge across long-running sessions.

Beyond code, the model is described as handling web search including multimodal browsing, spawning and coordinating subagents for decomposable tasks, and computer use across desktop, mobile, and browser. It generates separate commentary before each action so a developer can see what the agent intends to do and why. Underneath, a 1.8 billion parameter perception encoder processes interleaved text, images, video frames, and audio in a unified pass rather than through separate per-modality pipelines. That matters for workflows where a session mixes screenshots, diagrams, error logs, and video walkthroughs; one example given is the model diagnosing a bug from a screen recording with narrated audio by reading the codebase to find the root cause.

On performance, the GB300 NVL72 integrates 72 Blackwell Ultra GPUs into a single platform with a 72-GPU NVLink domain running at 130 TB/s. NVIDIA argues this bandwidth is especially relevant for sparse MoE serving: when each token is routed to eight of 256 experts, expert traffic that would otherwise cross conventional networks can become the bottleneck, so a large coherent NVLink domain directly affects how much of the theoretical throughput a deployment actually realizes. NVIDIA says Muse Spark Open running agentic workloads on GB300 NVL72 delivers up to 50K tokens per second per GPU and up to 600 tokens per second per user out of the box. A figure caption notes that on four Blackwell GPUs with speculative decode, the model exceeded 50K tokens per second per GPU and 600 tokens per user, a useful hint that these results are tied to speculative decoding rather than a single naive decode path.

Memory and precision are part of the story as well. NVFP4 checkpoints generated by Meta and tuned for Blackwell Ultra compress the footprint of all 816B parameters, with native FP4 Tensor Core acceleration. Lower precision means fewer bytes moved per token, which NVIDIA says raises tokens per second per user, and the freed memory can hold a larger KV cache for concurrent long-context sessions. This suggests the throughput and cost-per-token gains are not purely about raw compute but about the interaction between quantization, memory capacity, and how many long sessions can be served at once. A likely trade-off is that aggressive FP4 quantization requires careful validation, since it is generally the point at which accuracy-sensitive workloads can drift; NVIDIA does not state accuracy loss figures here.

Deployment is intended to be flexible across inference stacks. SGLang, vLLM, and TensorRT LLM provide direct control over serving configuration, parallelism, and batching, with TensorRT LLM adding compiled engines tuned for NVIDIA GPUs. NVIDIA Dynamo publishes deployment recipes for all three and separates prefill from decode across dedicated GPU pools so prompt processing does not stall token generation; its KV cache-aware routing sends each request to the worker that already holds the relevant prefix, avoiding recomputation as context accumulates over a long agentic session. For teams that need to adapt the model, post-training is supported through NeMo AutoModel, a PyTorch-native training library with native Hugging Face checkpoint support and no model conversion, offering full supervised fine-tuning and LoRA out of the box. When a workload outgrows one node, Megatron Bridge takes the same checkpoints and adds Megatron Core tensor, pipeline, and expert parallelism, so teams can move from experimentation to production training without reconverting the model.

Getting started paths are straightforward: NVFP4 weights are downloadable from Hugging Face, and production deployments can use a downloadable NVIDIA NIM, which reduces setup to a single command and automatically selects a serving backend and precision for the detected hardware.

Why it matters: teams building coding agents now have an open, very large MoE model explicitly tuned for long-horizon, tool-using work rather than single-shot completions, which is where many current coding models fall short. The combination of a 1M token context window and multimodal perception means a single agent session can ingest a full repository alongside screenshots, logs, and video, reducing the amount of orchestration scaffolding developers must write. The practical cost question is whether the sparse design plus FP4 serving on Blackwell Ultra delivers the stated throughput in real deployments; that depends on the serving stack and hardware mix, so teams should benchmark against their own workloads. My inference is that this pushes competitive pressure toward context length, tool-calling reliability, and serving economics rather than raw parameter count alone.

Read at NVIDIA Developer Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Meta launches Muse Spark Open, a coding-optimized 816B mixture-of-experts (MoE) model with 42B active parameters and a 1M token context window. Trained on...