NVIDIA launches Cosmos 3 open-source model for robotics

NVIDIA has launched Cosmos 3, a new open-source artificial intelligence model designed to help robots, autonomous vehicles, and other systems understand and interact with the physical world. The company announced the model this week, positioning it as a foundational tool for the development of physical AI.

The model integrates vision reasoning, world generation, and action prediction into a single system, which NVIDIA calls an "omnimodel." It is built on a Mixture-of-Transformers architecture that pairs a reasoning component to analyze scenes with a generation component to produce videos and action sequences. This allows the system to interpret object interactions and motion before predicting future states and generating corresponding actions. The model was trained on a vast multimodal dataset that includes text, images, video, ambient audio, and action data collected from both humans and robots.

NVIDIA released two versions of the model immediately. Cosmos 3 Super is aimed at applications requiring high physics accuracy, such as advanced robotics and autonomous vehicle training. Cosmos 3 Nano is optimized for faster inference, capable of generating results in fractions of a...