What is NVIDIA Cosmos 3?
Cosmos 3 is NVIDIA's family of open world foundation models for physical AI, released on May 31, 2026. One Mixture-of-Transformers network pairs an autoregressive reasoner, which reads text, images and video and answers in text, with a diffusion generator that produces images, video with 48 kHz stereo audio, and robot or vehicle actions. It comes as Nano (16B parameters, 8B reasoner plus 8B generator), Super (64B, 32B plus 32B) and, since July, a 4B Edge model for Jetson-class devices.
On November 5, 2026 NVIDIA added four checkpoints post-trained from Nano for interactive simulation. Transfer-Auto renders synchronised driving footage across up to 11 cameras, with optional LiDAR, under varied weather and lighting; Sim-Auto is its streaming version, returning camera and LiDAR frames chunk by chunk so a driving policy can act and see the result; Sim-Transfer turns a physics simulator's depth, segmentation and edge maps into realistic video; Sim-Bimanual predicts what two-arm YAM and AgiBot robots see as a policy or a joystick drives them.
All weights are under the OpenMDW 1.1 licence, which allows commercial and non-commercial use. The generator works at 256p, 480p or 720p and makes 5 to 400 frames per clip, 189 by default. NVIDIA tests it on Linux in BF16 only.
Is NVIDIA Cosmos 3 worth it?
Cosmos 3 matters less as a video generator than as one open model that can watch, predict and act. The same weights describe a scene, render what happens next given an action, infer the action that caused a change, and output a policy's next move with its predicted consequence. NVIDIA reports that it leads open models on PAI-Bench, R-Bench, Physics-IQ, RoboLab and VANTAGE-Bench, and that it topped Artificial Analysis's open-weight text-to-image and image-to-video boards without audio. Those are NVIDIA's own claims; on November 9 the with-audio image-to-video board listed no Cosmos model.
The November checkpoints turn that into closed-loop simulation. Sim-Auto and Sim-Bimanual were distilled to four denoising steps and then trained on their own outputs so small errors do not compound over a rollout, and each latent step covers four RGB frames. NVIDIA is candid about the gap: Sim-Auto needs your own simulation runtime and scene-map renderer to close the loop, and the post says it is still unclear where these models can reliably support policy learning and evaluation.
Hardware is the main cost. NVIDIA recommends vLLM-Omni on eight H200, H100 or A100 GPUs for Super, where a 50-step video takes about 55 seconds, or about 3 minutes on two H200s. Nano runs on one H200 and is aimed at an RTX PRO 6000 workstation; Sim-Bimanual was tested on one GB200. Only the Reasoner has a NIM container so far, with FP8 and NVFP4 checkpoints that NVIDIA says run up to twice as fast.
Its licence is the clearest advantage over closed world models such as Genie 4: OpenMDW 1.1 permits commercial use, and NVIDIA publishes post-training recipes, coding-agent skills and synthetic datasets for robotics, driving and warehouse scenes. The model card warns that outputs approximate physics without a simulator and should not be treated as physically accurate, so a team still validates any policy on real hardware.
Where it earns its place
A simulation engineer at an autonomous-driving start-up with an 8-camera rig
Multiply rare scenes (night rain, glare) from logged drives for perception training
Post-train Transfer-Auto on the rig's camera layout, then render each logged scene under new weather and lighting from scene-map controls and a text prompt. The weights are free; the cost is the GPU time for post-training and generation.
A robot-learning researcher with a two-arm AgiBot or YAM setup
Score policy checkpoints without booking hours on the physical robot
Run Sim-Bimanual as a learned simulator: feed the policy's actions, get predicted camera views back, and rank checkpoints before testing the best few on hardware. NVIDIA warns this is not yet proven for policy evaluation.
A product lead at a warehouse-robotics vendor
Build on a world model without a usage-based licence or a closed API
Ship a fine-tuned Nano inside the product under OpenMDW 1.1 and run it on customer-site RTX PRO 6000 workstations, or the 4B Edge model on Jetson, rather than paying per call to a hosted model.
Before you commit
- Setup
- Download from Hugging Face and serve with the vllm/vllm-omni:cosmos3 Docker image, or load Cosmos3OmniPipeline in Diffusers; Linux and an Ampere, Hopper or Blackwell GPU required. The simulation checkpoints also need Docker and the NVIDIA Container Toolkit.
- Integrations
- Diffusers, vLLM 0.21 (CUDA 13) or 0.19.1 (CUDA 12.8), vLLM-Omni and SGLang; Reasoner NIM on NGC; build.nvidia.com demos for Nano; training code and agent skills in the NVIDIA cosmos-framework repo.
- Data and privacy
- OpenMDW 1.1 allows commercial and non-commercial use, and self-hosted runs keep data local. NVIDIA says training used 1.3 billion data points across 393 datasets.
- Cost at scale
- Super: eight H200, H100 or A100 GPUs recommended, about 55 seconds per 50-step video on H200s; distilled four-step Super is up to 25 times faster, by NVIDIA's figures. Nano fits one H200. VRAM needs are not published; layer-wise offload helps on smaller GPUs.
- Limits
- 256p to 720p output, 5 to 400 frames; video input capped at 5 frames and audio at 0.5 seconds; actions only for listed embodiments such as Franka, AgiBot, UR and WidowX. SGLang lacks video-to-video, sound and action generation. Artifacts include temporal inconsistency and imprecise physical interactions.
What’s good
- One open model for scene reasoning, video prediction and robot actions
- OpenMDW 1.1 licence permits commercial use
- Streaming driving and two-arm robot checkpoints for closed-loop rollouts
- Post-training recipes, agent skills and synthetic datasets published
What’s not
- Super wants eight data-centre GPUs; about 55 seconds per 50-step video on H200s
- Benchmark leads are NVIDIA's own; independent boards do not yet confirm them
- Closed-loop driving needs your own simulation runtime and scene-map renderer
- No generator NIM or hosted inference provider yet
Best for
Robotics and autonomous-driving teams on NVIDIA hardware who want an open, commercially licensed world model to post-train on their own sensors and robots.
NVIDIA Cosmos 3 alternatives
NVIDIA Cosmos 3: common questions
- Is NVIDIA Cosmos 3 worth it?
- Cosmos 3 is the open world model a robotics or driving team builds on, but the new simulation checkpoints are kits for teams with a simulator and a GPU cluster, not finished simulators. We rate it 4.2 out of 5.
- How much does NVIDIA Cosmos 3 cost?
- Free under the OpenMDW 1.1 licence, commercial use allowed. Reasoner NIM container needs an NGC API key; hosted pricing not published, and no Hugging Face inference provider serves it.
- What are the downsides of NVIDIA Cosmos 3?
- Super wants eight data-centre GPUs; about 55 seconds per 50-step video on H200s; benchmark leads are NVIDIA's own; independent boards do not yet confirm them; closed-loop driving needs your own simulation runtime and scene-map renderer; no generator NIM or hosted inference provider yet.
- Who is NVIDIA Cosmos 3 best for?
- Robotics and autonomous-driving teams on NVIDIA hardware who want an open, commercially licensed world model to post-train on their own sensors and robots.
- How much does NVIDIA Cosmos 3 cost for a team or in production?
- Super: eight H200, H100 or A100 GPUs recommended, about 55 seconds per 50-step video on H200s; distilled four-step Super is up to 25 times faster, by NVIDIA's figures. Nano fits one H200. VRAM needs are not published; layer-wise offload helps on smaller GPUs.
- Is NVIDIA Cosmos 3 safe for company data?
- OpenMDW 1.1 allows commercial and non-commercial use, and self-hosted runs keep data local. NVIDIA says training used 1.3 billion data points across 393 datasets.
- What are the best NVIDIA Cosmos 3 alternatives?
- The closest alternatives we have reviewed are Genie 4, LingBot-World 2.0 (Infinity), LTX-2.5 and HunyuanVideo 1.5.
Answers checked Nov 9, 2026.
Make NVIDIA Cosmos 3? Show this rating on your site
