Nemotron 3 Super
Open-weights 120B-total / 12B-active hybrid Mamba-Attention Mixture-of-Experts text model from NVIDIA's Nemotron 3 family, with long-context support up to 1M tokens and NVFP4/BF16 deployment variants.
Nemotron 3 Super is NVIDIA's open-weights large language model (LLM) release that separates total model capacity from per-token compute. It combines hybrid Mamba-attention sequence layers with a Mixture-of-Experts (MoE) feed-forward path so a 120-billion-parameter model activates about 12 billion parameters on each token.
At a glance
- Family
- nemotron
- Source type
- Open weights
- Released
- March 2026
- Authors
- NVIDIA
- Modalities
- Text
- Parameter count
- 120 billion total parameters
- Context length
- 1,048,576 tokens
- Precision
- bf16, nvfp4
What It Is
Nemotron 3 Super is an open-weights text large language model (LLM) in NVIDIA's Nemotron 3 family. NVIDIA publishes it as a sparse hybrid design: roughly 120 billion total parameters with about 12 billion active parameters per token. The public release includes Brain floating point 16 (BF16) and NVIDIA FP4 (NVFP4) checkpoint variants for different deployment budgets.Inputs And Outputs
The model takes text tokens and returns text tokens. It is not multimodal in the published release. Checkpoint names such as Nemotron-3-Super-120B-A12B-BF16 and the NVIDIA Inference Microservices (NIM) identifier nvidia/nemotron-3-super-120b-a12b refer to the same family member with different packaging for local weights versus managed serving.Architecture
NVIDIA describes Nemotron 3 Super as a hybrid Mamba-Attention Mixture-of-Experts (MoE) stack rather than a plain dense transformer decoder. Sequence mixing alternates between Mamba-style state-space blocks and attention blocks, while the feed-forward path routes each token through a small active expert set inside a much larger expert pool (branded LatentMoE in NVIDIA materials). This pattern is meant to hold large total capacity without paying the full dense cost on every token.Output
ProbabilitiesSoftmaxOutput
LinearTransformer architectureHybrid sequence sublayer containerSparse expert sublayer containerN×RMSNormHybrid
Mamba +
AttentionRMSNormLatentMoE
ExpertsEmbeddingRoPEInput
EmbeddingInput
TokensSoftmax to Output
ProbabilitiesOutput
Linear to SoftmaxRMSNorm to Output
LinearLatentMoE
Experts to RMSNormRMSNorm to LatentMoE
ExpertsRMSNorm to RMSNormHybrid
Mamba +
Attention to RMSNormEmbedding to Hybrid
Mamba +
AttentionEmbedding to RMSNormInput
Embedding to EmbeddingRoPE to EmbeddingInput
Tokens to Input
Embedding
Important Modules
The modules below anchor how to read the architecture graph and related registry pages. The selective state-space module explains the Mamba-style sequence blocks, the Mixture-of-Experts routing story is the main efficiency lever, and attention modules cover the alternating attention path in the hybrid stack.Training
NVIDIA's technical report frames post-training for agentic and reasoning workloads, but the public materials emphasize architecture and deployment formats over a step-by-step training recipe. Treat training details here as high-level context from the release sources rather than a reproducible lab checklist.Training regimes
No training regimes listed yet.
Linked papers
- Nemotron 3 Super
Practical Notes
The release advertises up to 1 million tokens of context and positions NVFP4 variants for inference-efficient serving alongside BF16 weights. Those choices matter because long-context models often trade memory and routing overhead for reach; Nemotron 3 Super is presented as a design that keeps active compute closer to a smaller dense model while retaining a larger total parameter budget.References
- NVIDIA. "Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning." 2026.
- NVIDIA. "NVIDIA Nemotron 3 Super." NVIDIA Research, 2026.
- NVIDIA. "NVIDIA-Nemotron-3-Super-120B-A12B-BF16 Model Card." Hugging Face, 2026.
- NVIDIA. "NVIDIA NIM Nemotron 3 Super 120B-A12B." NVIDIA NIM Documentation, 2026.