Cosmos 3
Open omnimodal world model family from NVIDIA for physical AI, combining autoregressive reasoning with diffusion-based generation across text, image, video, audio, and action outputs through Nano and Super checkpoints.
Cosmos 3 is NVIDIA's open-weights omnimodal world model family for physical AI. It couples autoregressive reasoning with diffusion-based generation so one model family can ground language in images and video, synthesize worlds with sight and sound, and plan robot actions from shared multimodal context.
At a glance
- Family
- cosmos
- Source type
- Open weights
- Released
- May 2026
- Authors
- NVIDIA
- Modalities
- Text, Image, Video, Audio, Multimodal
- Parameter count
- 16B Nano and 64B Super family variants
What It Is
Cosmos 3 is an open-weights omnimodal world model family from NVIDIA, released on 2026-05-31 with Nano and Super checkpoints described as 16-billion-parameter and 64-billion-parameter variants. NVIDIA frames it as a physical AI foundation model that connects understanding, generation, simulation, and action through one shared backbone rather than separate single-task models. Public materials position Cosmos 3 as a model family researchers can inspect, adapt, and deploy, not only a technical report title or a closed serving platform.Inputs And Outputs
Cosmos 3 works across text, image, video, audio, and action-oriented tasks described in NVIDIA's release materials. Readers can condition the family on language prompts, still images, video clips, and audio references, then obtain text answers, generated images, video with synchronized audio, and robot trajectories or policy outputs depending on the selected capability. That multimodal span is the core idea: the same world-model backbone is meant to move between perception, generation, forward simulation, inverse dynamics, and manipulation planning instead of treating each modality as an isolated product surface.Architecture
NVIDIA describes Cosmos 3 as pairing an autoregressive reasoning path with a diffusion-based generation path inside one omnimodal world model. The reasoner side grounds language in spatial layout, temporal cues, object state, and action evidence from images and video. The generator side denoises or synthesizes visual and audio outputs, while conditioning inputs steer both paths toward a requested scene, clip, or trajectory. The architecture graph below shows that split at a high level; the technical report and research page remain the authoritative sources for checkpoint-specific wiring.Video
OutputText
OutputImage
OutputAudio
OutputAction
OutputDiffusion-Based
GeneratorAutoregressive
ReasonerMultimodal
ConditioningText
PromptImage
InputVideo
InputAudio
InputAutoregressive
Reasoner to Text
OutputAutoregressive
Reasoner to Action
OutputDiffusion-Based
Generator to Image
OutputDiffusion-Based
Generator to Video
OutputDiffusion-Based
Generator to Audio
OutputAutoregressive
Reasoner to Diffusion-Based
GeneratorMultimodal
Conditioning to Diffusion-Based
GeneratorMultimodal
Conditioning to Autoregressive
ReasonerText
Prompt to Multimodal
ConditioningImage
Input to Multimodal
ConditioningVideo
Input to Multimodal
ConditioningAudio
Input to Multimodal
Conditioning
Important Modules
Public Cosmos 3 materials emphasize architecture-level ideas—world modeling, multimodal conditioning, autoregressive reasoning, and diffusion generation—rather than a short list of named reusable modules on this site. The related concept pages below are the best anchors for those mechanisms until a future registry slice names Cosmos-specific modules explicitly.No modules listed yet.
Training
NVIDIA's technical report frames Cosmos 3 as an omnimodal world-model release trained to couple understanding and generation across modalities for physical AI workloads. The public sources summarize training at the family level and focus on what the checkpoints can do after training rather than publishing a step-by-step reproducible recipe on this page.Training regimes
No training regimes listed yet.
Linked papers
- Cosmos 3
Practical Notes
Cosmos 3 is useful to read as an open world-model family that unifies reasoning and generation for robotics, driving, smart-space, and audio-visual synthesis scenarios described in NVIDIA's release pages. The Nano and Super variants give builders a smaller and a larger checkpoint choice, while the GitHub repository and Hugging Face collection provide the practical entry points for weights and examples. Treat deployment, licensing, and availability details as release-channel facts to confirm in the linked primary sources rather than as fixed promises on this overview page.References
- NVIDIA. "Cosmos 3: Omnimodal World Models for Physical AI." arXiv:2606.02800, 2026.
- NVIDIA. "Cosmos 3 — Cosmos Lab." NVIDIA Research, 2026.
- NVIDIA. "nvidia/Cosmos3-Super Model Card." Hugging Face, 2026.
- NVIDIA. "nvidia/cosmos." GitHub, 2026, https://github.com/nvidia/cosmos.
- NVIDIA. "Cosmos3 Collection." Hugging Face, 2026.