Nvidia Releases Cosmos 3 Family, Four Open-Weight Nano Checkpoints for Interactive World Simulation

Nvidia has published four open-weight checkpoints under its new Cosmos 3 platform, targeting researchers building interactive world simulators for robotics and autonomous driving. The models—Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual—are post-trained variants of the 16-billion-parameter Cosmos 3 Nano foundation model, released under the Linux Foundation's OpenMDW-1.1 license [7], which permits commercial and non-commercial use [1]. Nvidia describes Cosmos 3 as the first fully open omni-model for physical AI [5], a claim that rests on the model's unified handling of language, image, video, audio, and action within a single Mixture-of-Transformers (MoT) architecture [7].

Four Checkpoints, Four Simulation Tasks

Each checkpoint addresses a different simulation workflow:

• Transfer-Auto generates synchronized multi-view camera video (up to 11 views) and LiDAR from structured driving scenarios and text prompts, intended for offline synthetic data generation [1].
• Sim-Auto is a causally distilled, few-step streaming variant of Transfer-Auto that produces successive chunks of camera and LiDAR observations for closed-loop driving simulation [1].
• Sim-Transfer converts simulator-produced spatial controls (depth, segmentation, blur) plus a text prompt into photorealistic RGB video, leaving physics and ground-truth state to an external simulator [1].
• Sim-Bimanual predicts the next visual observation from a bimanual robot's actions, supporting YAM and AgiBot embodiments controlled by policy or human-operated joysticks [1].

All four share a four-stage post-training recipe: bidirectional adaptation on full clips, teacher-forcing causal training, consistency distillation, and self-forcing distribution-matching distillation on the student's own rollouts [3]. The consistency distillation stage trains a student that generates each chunk in a few denoising steps; the optimized Bimanual setup predicts two latent time steps per chunk [1]. The technical report notes that policy-driven closed-loop validation remains separate from the reported benchmarks [3].

Architecture: One Model, Multiple Modes

Cosmos 3 uses a MoT backbone with separate reasoner and generator towers that share a unified representation space [7]. Language tokens are processed autoregressively by the reasoner; vision, audio, and action tokens are diffused by the generator, with joint attention allowing the generator to condition on the reasoner's output [7]. The same weights can operate as a vision-language model, text-to-image/video generator, video-transfer model, forward/inverse dynamics model, or joint video-action policy by changing which tokens are clean or noisy at inference [7].

Three scales are defined: Edge (4B total, based on a 2B transformer), Nano (16B total, based on an 8B transformer), and Super (64B total, based on a 32B transformer) [7]. Only Nano and Super checkpoints are released so far; Edge is slated for a later release [7]. The four new checkpoints are all Nano-scale, 16B-parameter models [1].

Licensing and Developer Tooling

Code, model checkpoints, curated synthetic datasets, and evaluation benchmarks are released under OpenMDW-1.1 [7]. Nvidia also provides Cosmos Framework—an open, modular codebase for training, post-training (including LoRA, SFT, GRPO, PPO, OPD, DiffusionNFT), optimization (distillation, quantization), and inference across Hugging Face Diffusers, PyTorch, SGLang, vLLM-Omni, vLLM, TRT-LLM, and TRT-Edge-LLM [2]. For production deployment, Cosmos NIM microservices offer optimized containers with speculative decoding (measured 2–3× speedup on H200), quantization, and KV-cache-aware routing [2]. The NIM path requires an NGC API key and Docker; the open Framework path runs on Linux with an Nvidia GPU, Docker, and the Nvidia Container Toolkit [1][2].

Benchmark Claims and Independent Verification

The Cosmos 3 technical report states: "Our post-trained Cosmos 3 models were ranked as the best open-source Text-to-Image and Image-to-Video models by Artificial Analysis, and the best policy model by RoboArena at the time the technical report was written" [7]. These rankings reflect the state of those leaderboards at the time of writing. Table 1 in that report shows Cosmos3-Super leads on Text-to-Image (91.36 vs. 90.85 for Gemini 3 Pro Image), while both Super and Nano outperform baselines on Text-to-Video (80.0/79.4 vs. 79.1 for Veo-3.1) and robot forward dynamics (26.0/25.5 vs. 23.0 for Ctrl-World) [7]. These results come from Nvidia's own evaluations in the technical report.

What Remains Unknown

• The "200+ releases" figure cited in Nvidia's broader open-data catalog spans Physical AI, robotics, autonomous vehicles, biology, drug discovery, retrieval and evaluation benchmarks, and sovereign AI—not exclusively Cosmos 3 checkpoints [5].
• Hardware minimums for the open Framework are not specified; the published vLLM-Omni setup was tested on a single GB200, explicitly noted as "not a minimum-memory specification" [1].
• Cosmos3-Edge weights are not yet available, and no timeline is given.
• Closed-loop policy evaluation results for the four new checkpoints are not included in the technical report [3].

Nvidia is hosting a livestream on November 5 to demonstrate the Framework [2]. The checkpoints, technical report, and Framework code are accessible via Hugging Face and GitHub [1][3][6].