NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4

Instructions to use EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForMultimodalLM
model = AutoModelForMultimodalLM.from_pretrained("EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4

SGLang

How to use EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4
```

NVIDIA Nemotron 3.5 Super VL 120B-A12B — NVFP4 with embedded MTP

A byte-identical copy of NVIDIA's NVFP4 checkpoint nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 at revision 7d4bc747b6bdc2cff64c58d65da0d08dd9caa25c, served on Apple silicon by the Darkbloom native nemotron_h_omni runtime. The language trunk, the C-RADIO vision tower and projector, and the multi-token prediction (MTP) head are all in one indexed checkpoint: one download. The 4,120 MTP tensors ship in model-00088-of-00088.safetensors, so no separate draft model or MTP file is needed.

Status: staged for the Darkbloom network (catalog id nvidia-nemotron-3.5-super), not yet live. Darkbloom serves it on Apple M5 Max (128 GB) and M5 Ultra; so far it is measured on hardware only on a 256 GB M3 Ultra (provider 6f9e0e64, SDK 53b11b32): cold prefill about 480–520 tok/s from 2K to 65K-token prompts, a repeated 16K-token prompt answers 7× sooner from the prefix cache, and MTP output is byte-identical to target-only decoding.

This is NVIDIA's own quantization, not an EigenLabs requantization or an MLX conversion: every weight, config, tokenizer and template file is NVIDIA's bytes. NVIDIA's original model card is kept as UPSTREAM_README.md.

What is in it

Part	Storage	Source
Routed experts	ModelOpt NVFP4: packed E2M1 values, FP8-E4M3 block scale per 16, FP32 per-tensor scale	NVIDIA (quantization-aware distillation from the BF16 model)
Attention / Mamba-2 / shared projections	ModelOpt mixed precision (FP8 and BF16 where NVIDIA kept them)	NVIDIA
Norms, router, Mamba state parameters, C-RADIO vision tower, projector	as stored (BF16/F32)	NVIDIA
MTP head (4,120 tensors, shard 88)	as stored	NVIDIA

Field	Value
Architecture	`nemotron_h_omni`: Nemotron-H hybrid Mamba-2 / attention latent MoE (120B total, 12B active), C-RADIO vision encoder and projector, 1 MTP head
Safetensor files	75,010,188,240 bytes in 88 shards (tensor payload 74,986,817,096)
Indexed tensors	169,505, including 4,120 MTP tensors in shard 88
Index SHA-256	`c368acdbcfe882236af1ec79df4a1fb6127e6ed9f742d59c01c21fe96ddc2741`
`config.json` SHA-256	`35189a8038d1b1f934e58cba50eb7e97b42a4da569424e35b9fd1048058bc7ea`
Shard 88 SHA-256	`6e576283e02f46a5a547e08a892515cd4e77a0fa4e6955325cfbbfab24e37479`
Served context (Darkbloom)	262,144 tokens, prompt plus output, up to 32,768 output tokens (the configuration allows 1,048,576; longer windows are not qualified here)
Input	text, PNG and JPEG images
Darkbloom hardware	Apple M5 Max (128 GB) and M5 Ultra
License	OpenMDW-1.1 — see License and attribution

The checkpoint is not uniform 4-bit and is not affine quantization: loaders must apply ModelOpt's W = e2m1(code) × e4m3(scale) × weight_scale_2 per 16-element block and keep the FP8 and BF16 components as stored.

Multi-token prediction (MTP)

Embedded: the trained MTP head is in this checkpoint (shard 88) and reuses the target model's embedding and output head. There is no separate draft model to download.
Darkbloom: the native runtime serves text with MTP on by default, with exact (greedy-equivalent) verification of every proposed token; the draft depth follows the number of decoding requests (3, 2, 2, 2, then 1). Image requests run target-only on the same model automatically. No flags, separate files or environment variables are needed.
vLLM / SGLang / TensorRT-LLM (NVIDIA GPUs): use NVIDIA's original repository and its documented MTP settings.

On the full checkpoint, greedy text with MTP on is byte-identical to target-only decoding (9 of 9 prompts, paged KV). Net speedup versus target-only on matched workloads is being measured; accepted draft tokens alone do not prove one.

Reasoning controls

The pinned chat template's own controls, exposed through the OpenAI-compatible reasoning_effort (or reasoning.effort):

Value	Behavior
omitted, `high`	Thinking on (the template default)
`medium`	Thinking on, with the template's efficient-reasoning instruction
`none`, `off`, `0`	Thinking off
`low`, `xhigh`, contradictory controls	Refused with HTTP 400

These select template behavior; they are not output-token budgets. Other reasoning fields (max_tokens, exclude) are not supported and are currently ignored.

Serving defaults on Darkbloom

Feature	Default	Notes
MTP speculation	on (text)	images target-only
Paged attention KV	on	segmented native paged storage
Exact prefix cache	on	recurrent (Mamba) state plus attention KV and MTP prompt history at fixed 2,048-token chunk boundaries; partitioned by tenant and by image identity
Activation overflow	saturating (FP8 448, block scale 448, E2M1 6), as deployment engines convert	in-range values are bit-identical to ModelOpt's eager rule; inputs that rule turned into NaN now answer

Chat template

chat_template.jinja is NVIDIA's template, unchanged (SHA-256 1369e90b19976f99ce2070eb21ade8fcb4af640f008e14e82af5c5e89a9cc816). Darkbloom renders it byte-for-byte against the reference tokenizer on a 30-case golden corpus (tools, tool-call history, images, reasoning on/off).

Download

from huggingface_hub import snapshot_download

model_path = snapshot_download(
    repo_id="EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4",
    revision="<full commit hash>",
)

Pin a full commit hash for reproducible deployments. Darkbloom providers download the same 96 runtime files plus LICENSE.txt and UPSTREAM_README.md from the Darkbloom model CDN as one manifest (version 2026-10-20-nvfp4-r2, aggregate SHA-256 56a1f05e980a602ebc09a64203689a656198069e1c9dd4c6698af833f2dceca6).

Validation and limitations

Measured on an M3 Ultra Mac Studio (256 GB) with the Darkbloom native runtime:

Check	Result
Integrity	All 88 shards verified against pinned SHA-256 on every load; 98-file Darkbloom manifest, aggregate `56a1f05e…ceca6`
Managed serving (7 tests)	Text, PNG/JPEG image matrix, images beside speculating text, three reloads with memory returned, a 33,233-token needle retrieval
Prefix cache on paged KV	3 restores, 12,288 prompt tokens saved, bounded eviction, cold and warm answers identical, no reuse across tenants or changed images
Long context	A 258,036-token needle returns the exact passphrase through the managed endpoint (paged KV, prefix cache, MTP); a 65,520-token prompt stays inside its reserved prefill memory (7.71 of 9.13 GB)
Reasoning controls	44 of 44 checks: every value above, streaming and not, text and image; unsupported values refused
Tools and JSON	Forced tool call and JSON object output under `none`, `medium` and `high`
Earlier build soak	10 minutes, 372 mixed text/image requests, zero failures

Prompt (cold)	Time to first token	Prefill
2,033 tokens	3.9 s	517 tok/s
16,369 tokens	33.1 s	495 tok/s
32,768 tokens	63.7 s	514 tok/s
65,520 tokens	136.8 s	479 tok/s
258,036 tokens (needle, end to end)	888.8 s	290 tok/s
16,369 tokens, repeated (prefix cache)	4.5 s	—

The 2K and 16K rows were measured on the preceding build (contiguous KV); the 32K and 65K rows on the build above.

Not established here: a physical M5 Max 128 GB machine (the default load fits its memory cap with about 14.6 GiB to spare; a 16-image request does not), audio and video input, contexts above 262,144 tokens, and hosted usage parity. These checks do not establish BF16 equivalence or quality on any particular workload.

License and attribution

Model, training and NVFP4 quantization: NVIDIA, released under the OpenMDW License 1.1 ("ready for commercial or non-commercial use"). LICENSE.txt is the full license text; keep it with any copy of these weights.
NVIDIA's original model card, including its evaluation, bias, safety and privacy notes, is UPSTREAM_README.md.
Mirror and Darkbloom runtime qualification: EigenLabs. This is not an official NVIDIA release.

Downloads last month: 223

Safetensors

Model size

66B params

Tensor type

I64

F32

BF16

F8_E4M3

Model tree for EigenLabs/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4

Base model

nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16

Quantized

nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4

Quantized

(5)

this model