AivexaNewsSearch
AI news for builders and product teamsChecked every hour

NVIDIA Nemotron 3.5 Super VL is now available in Amazon SageMaker JumpStart

Collected Nov 9, 2026

AWS announced that NVIDIA Nemotron 3.5 Super VL, the vision-language member of the Nemotron 3.5 family, is available in Amazon SageMaker JumpStart. It accepts images, video and text and returns text, and it can be deployed either through SageMaker Studio or with the SageMaker Python SDK, or from its Hugging Face model page via a Deploy menu. JumpStart ships two variants: an NVFP4 build quantized from BF16 using quantization-aware distillation and intended for optimized inference, and the BF16 full-precision reference weights intended for customization.

The architecture is unusual and worth understanding. The language backbone is a 120B-parameter hybrid with 12B active parameters per token: 88 layers that interleave Mamba-2 state-space layers and Mixture-of-Experts layers with 8 attention layers, plus a shared Multi-Token Prediction head. The MoE portion has 512 routed experts with 22 active per token, and the backbone was initialized from Nemotron 3 Super Base and distilled from Nemotron 3 Ultra Base. Vision comes from a C-RADIO encoder, which processes images and sampled video frames, and a connector that maps those representations into the backbone's embedding space; encoder and connector stay in BF16 in both variants. The model claims context up to 1 million tokens, supports per-request reasoning control, tool calling and JSON schema output through response_format, and is licensed under the OpenMDW License Agreement, version 1.1, covering English, French, German, Italian, Japanese, Spanish and Chinese. Training ran through continued pre-training, long-context and software-engineering mid-training, vision-language training, two-stage multimodal supervised fine-tuning, reinforcement learning with verifiable rewards and human feedback in NeMo RL and NeMo Gym, multi-teacher on-policy distillation, and a final MTP-head stage. The NVFP4 checkpoint runs 4-bit weights and activations on Blackwell GPUs, keeping the vision encoder, connector, Mamba convolutions and embeddings in BF16 and the MoE router in FP32.

On NVIDIA's published model-card benchmarks with reasoning on, the two checkpoints are close: MMLongBench-Doc is 62.82 for both, OCRBench v2 English is 68.12 BF16 versus 68.34 NVFP4, CharXiv reasoning is 80.82 versus 79.88, Video-MME v2 is 32.85 versus 32.54, and OSWorld-Verified is 51.14 versus 52.30. In practice, this suggests quantization costs little on these tasks, which is the usual argument for picking the smaller footprint when latency and cost dominate; a developer should still validate on their own data, since quantization effects are workload-specific.

SageMaker's measured throughput, using a generate profile of 250 input tokens, 1,500 output tokens and five concurrent streaming requests, shows NVFP4 on ml.g7.24xlarge at 63.07 output tokens per second mean per request with 0.45 seconds to first token, ml.g7.48xlarge at 64.01 tokens per second and 0.51 seconds, and BF16 on ml.p5en.48xlarge at 140.85 tokens per second and 0.53 seconds. Because the variants run on different GPU systems, these are not a clean precision comparison. Long prompts shift the picture: with a 10,000-token summarize profile, first-token latency on ml.g7.24xlarge reaches 12.62 seconds.

Supported instances and footprints: NVFP4 runs on G7 instances with NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs, four of them (128 GB, tensor parallel 4) on the default ml.g7.24xlarge and eight (256 GB, tensor parallel 8) on ml.g7.48xlarge; BF16 runs on ml.p5en.48xlarge with eight H200 GPUs and 1,128 GB. Checkpoint size is roughly 75 GB for NVFP4 and 249 GB for BF16. New accounts start with a zero quota for ml.g7 instances, so a Service Quotas increase must be requested before deploying; the SDK path needs SageMaker Python SDK v3.25 or later.

Serving configurations matter more than the headline 1M-token figure suggests. JumpStart configurations actually serve contexts of 520 tokens (interact), 2,275 (generate default), 13,650 (summarize) and 15,600 (modify), and prompt plus completion must fit inside that budget. A 1,536 x 1,024 page image consumes about 1,536 tokens, so a single document page already eats most of a generate budget. The endpoint serves one image and one video per request, requiring media inline as base64 data URLs because it runs with network isolation; PDFs must be rendered to images first. Frame sampling for video is fixed at endpoint launch and per-request media_io_kwargs are rejected. The MTP head supports speculative decoding in principle, but the JumpStart configuration does not enable it, and LoRA adapters are not loaded, so fine-tuned weights require the BF16 reference path. Deployment takes about 10 minutes on ml.g7.24xlarge, and NVIDIA recommends temperature 1.0 and top_p 0.95 for text and 0.6 with top_p 0.95 for image and video.

Why it matters: teams building document-heavy or computer-use agents now get a large open-licensed VLM through a managed endpoint, with a quantized default that fits on four Blackwell GPUs rather than requiring an eight-way H200 node. The practical constraints are the serving context windows, single-image-per-request handling and inline-only media, which likely push multi-page pipelines toward per-page requests or stitched images. Choosing BF16 is mainly a decision about customization rather than raw quality, since the reported benchmark gap is small.

Read at AWS Machine Learning Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

NVIDIA Nemotron 3.5 Super VL, a 120B-parameter multimodal vision-language model, is now available in Amazon SageMaker JumpStart. Deploy the NVFP4 or BF16 variant in a few steps, then run text, image, and video inference for document intelligence, computer-use agents, and multimodal agentic workloads.