Status (2026-07-07): no weights published yet. This repository currently contains only the model card β€” it marks a planned variant that has not been released. Follow the repo to be notified when files land.

KV-cache quantization without any fork (recommended, 2026): upstream llama.cpp/Ollama now cover this natively β€” use -ctk q8_0 -ctv q8_0 (half KV memory, negligible quality loss: perplexity +0.002–0.05) or -ctk q4_0 -ctv q4_0 (quarter memory, β‰ˆ7.6% perplexity increase). In Ollama: OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1. Keep K and V types symmetric to stay on the fast fused Flash-Attention path. Since April 2026, mainline llama.cpp also applies Hadamard rotation to KV activations (PR #21038), which greatly improves low-bit KV quality (opt-out: LLAMA_ATTN_ROT_DISABLE=1).

The RotorQuant/TurboQuant fork flow below is experimental/legacy: the TurboQuant llama.cpp PR was closed without merging (June 2026) and the fork is unmaintained relative to mainline. It is NOT required to use this model.

Nemotron-3-Nano-Omni-30B-A3B-Reasoning - mmproj-F16

Multimodal projector split file for Nemotron-3-Nano-Omni-30B-A3B-Reasoning. Required by llama-mtmd-cli when running multimodal inference against any of the GGUF weight variants in this family.

This card is a reference; the actual mmproj-F16.gguf binary is published by the upstream community. For canonical NVIDIA BF16 weights see nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16.

Quickstart

# This card is a reference for the multimodal projector binary used by
# llama-mtmd-cli. Pair it with any GGUF weight variant in this family:

huggingface-cli download majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-mmproj-F16 mmproj-F16.gguf --local-dir ./mmproj

llama-mtmd-cli \
  -m ./model/Q4_K_M.gguf \
  --mmproj ./mmproj/mmproj-F16.gguf \
  --image example.jpg \
  -p "Describe this image"

Modality matrix

Modality Encoder Quantization in this variant
Text LLM backbone (Mamba-2 + Transformer hybrid Sparse MoE) per the variant suffix
Image CRADIO v4-H BF16 (kept full-precision in every non-GGUF variant; GGUF uses mmproj-F16 split file)
Audio Parakeet-TDT-0.6B-v2 BF16 (same rationale)
Video Parakeet-TDT-0.6B-v2 + frame sampler BF16 (≀ 2 min, 256 frames @ 2 FPS)

NVIDIA's official FP8 / NVFP4 recipe keeps both encoders + the cross-modal MLP projectors in BF16 to preserve multimodal accuracy. We follow that convention in every quantized variant we ship.

Runtime quirks

llama.cpp

Use llama-mtmd-cli for multimodal inference; pass --mmproj mmproj-F16.gguf (see majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-mmproj-F16).

Do NOT use CUDA 13.2 β€” produces gibberish. Pin CUDA 12.x or use the Metal/CPU paths.

Ollama

Text-only; multimodal is blocked because Ollama doesn't yet support the mmproj split-file pattern.

Reasoning mode

enable_thinking defaults to True. To disable extended reasoning (e.g., for latency-sensitive cases), pass enable_thinking=False to the chat template / generate call. No separate "no-think" variant card exists β€” this is a runtime flag, not a model variant.

Variants in this family

(Showing 56 sibling variants under majentik/nemotron3-nano-omni-30b-*. The current variant β€” mmproj-F16 β€” is bolded.)

Variant Runtime Approx size Use case
mmproj-F16 llama-mtmd-cli ~1-2 GB Multimodal projector (pair with any GGUF)

About the RotorQuant / TurboQuant labels

RotorQuant and TurboQuant are this project's release labels, not distinct quantization algorithms β€” for any given tier, both brand repos carry byte-identical weights produced with the standard MLX / llama.cpp quantizers. No brand-specific speedup is claimed or measured.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-mmproj-F16

Finetuned
(21)
this model

Dataset used to train majentik/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-mmproj-F16