AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Running Frontier-Class AI Models Across 8 NVIDIA DGX Spark Nodes

Collected Oct 22, 2026

At GTC Berlin, NVIDIA announced official support for clusters of eight DGX Spark nodes, the desktop AI systems it positions for local model development. The headline number is memory: a single Spark carries 128 GB of unified memory, four nodes reach 512 GB, and eight nodes reach 1 TB. Support for eight-node clusters is being added to NVIDIA Sync, the tool whose Cluster Assistant handles setup and device management, and is due by the end of the year.

The memory ceiling is what changes which models developers can run. NVIDIA says the eight-node configuration can load models that would not start on a four-node system, citing the full GLM 5.3 (753B) checkpoint from Z.ai as an example. That model takes roughly 470 GB in NVFP4 form, so it fits within a 1 TB cluster with room to spare. NVIDIA also ran the lighter GLM 5.3 Flash across eight nodes and measured about 80 tokens per second in generation, which it describes as suitable for interactive local work against a frontier-class mixture-of-experts model. Benchmark notes say measurements used MTP3, an inference configuration, at concurrency one, four and eight, with prefill measured at an 8K input sequence length per request.

A second strand of the announcement is a Spark-tuned collective communication backend called RoCEnante, aimed at tensor-parallel inference. Tensor parallelism splits one model across nodes, and each decode step then requires collective operations such as all-reduce to combine partial results. RoCEnante replaces the usual ring all-reduce, which forwards data between neighboring ranks over several rounds, with an all-to-all exchange followed by local reduction: every rank sends its input to every peer and reduces locally, coordinated by a single GPU kernel. Because data reaches all peers directly rather than through a forwarding chain, latency drops for small messages at low concurrency and modest rank counts, which NVIDIA says is the pattern typical of decode on local hardware. The cost is more traffic, since each rank broadcasts its full input; as messages or node counts grow, that overhead makes ring all-reduce's bandwidth efficiency more attractive again. NVIDIA is explicit that this is not a replacement for NCCL, which remains the supported default and the better choice at high concurrency, on larger clusters or in ring topologies. The backend has been merged into FlashInfer and falls back transparently to NCCL if a startup test fails; it only applies to switch-connected topologies because it relies on direct point-to-point exchange between every node. NVIDIA credits the original one-shot all-reduce approach to a community contributor, Jason Cook (el8).

Reported gains: at four nodes with NVFP4, NVIDIA saw roughly 5% higher prefill and 21% higher decode throughput; at eight nodes on the non-quantized model, decode throughput was about 18% higher. On a decode scaling chart using Qwen3.8-27B-NVFP4 with speculative decoding disabled on a switch topology, eight nodes gave a 4.43x speedup with RoCEnante against 3.05x without it, which NVIDIA frames as a 45% improvement in scaling efficiency.

NVIDIA also published scaling figures for a dense Qwen3.8-27B (NVFP4) model with no speculative decoding or MTP, run from one to eight nodes with llama-benchy at 8192-token input and 256-token output at concurrency one. Decode throughput rose from about 12 tokens per second on one node to 52 on eight. The topology finding is the practical takeaway: a physical ring does well for a single model spread over all nodes, while a switch is needed to split the cluster into independent groups or to run workloads requiring any-to-any connectivity. Four nodes wired directly in a ring work well because NCCL defaults to ring algorithms at that scale for tensor-parallel traffic, giving a cheaper, simpler four-node inference box. At eight nodes, or when splitting into two groups of four, a ring cannot express the topology; all-to-all and traditional MPI-style clustered workloads also want a switch rather than paying a hop penalty.

For multi-model serving, NVIDIA points to NeMo Switchyard, which puts a lightweight classifier in front of requests and routes each one, or each session, to an appropriate model pool, with a load balancer spreading traffic across replicas. In a prototype, OpenBMB's MiniCPM5 2B handled simple tasks and Qwen3.8 Flash Next handled complex ones. On a 92-case tool-use benchmark including hard cases, the routed agent scored 91/100, matching a Qwen-only configuration. Since MiniCPM serves roughly an order of magnitude more requests per second per replica, shifting traffic raises the cluster's ceiling: allocating three Qwen replicas of two Sparks each and two MiniCPM replicas of one Spark each, NVIDIA reports about 1.5x aggregate throughput over a Qwen-only baseline when half of requests go to Qwen, and about 3.0x when only a quarter do. Routing policy is a YAML file, with a sample provided.

Training scales differently, and better. When each node holds a full model copy, the only cross-node communication is a gradient synchronization once per step, so adding nodes scales close to linearly. NVIDIA pre-trained a nanochat model from scratch, about 896.5M parameters with 20 transformer layers, hidden dimension 1,280 and 2,048-token sequences, using a global batch of 524,288 tokens per step and 32 sequences per GPU microbatch. Eight nodes produced a 7.79x speedup at 97.4% efficiency. The code is on GitHub.

My reading is that the announcement is less about raw performance than about the memory tier. A 1 TB local cluster with roughly 80 tokens per second on a 470 GB model puts serious open-weight experiments on a desk rather than in a cloud queue, though the ring-versus-switch split means the cheap four-node setup and the full eight-node one are different purchases with different trade-offs. The NCCL caveats around RoCEnante also suggest teams should expect the default path to remain the safe choice at higher concurrency.

Why it matters: Developers and product teams working with large open models gain a documented path to serving frontier-scale checkpoints and orchestrating several models locally, with published numbers for inference, routing and small-model pre-training they can use for capacity planning. The end-of-year Sync support is the gating item; until then, clustering leans on the existing networking guide and vLLM recipes.

Read at NVIDIA Developer Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

AI developers aren’t just running a single model anymore—they’re orchestrating multiple models in parallel, chaining inference calls, and experimenting...