Google DeepMind Unveils SIMA 2 AI Agent for Virtual Worlds

Google DeepMind unveiled SIMA 2 on Thursday, an advanced AI agent designed to navigate and interact within diverse virtual 3D worlds, marking a significant evolution in artificial intelligence for gaming and beyond.

The new agent, short for Scalable Instructable Multiworld Agent, builds on the original SIMA released in March 2024. Powered by Google's Gemini large language model, SIMA 2 demonstrates enhanced reasoning capabilities, allowing it to interpret high-level instructions, explain its actions, and adapt to unfamiliar environments. Researchers at DeepMind trained the agent on footage from eight commercial video games, including Goat Simulator 3, No Man's Sky, and Valheim, as well as proprietary virtual worlds. It learns by observing screen pixels and simulating keyboard and mouse inputs, enabling it to perform tasks like turning right and entering a barn in Goat Simulator 3 or identifying and interacting with a distress beacon on a rocky planet in No Man's Sky.

In demonstrations, SIMA 2 responded fluidly to varied prompts, such as disobeying an order to pick flowers instead of working at a workbench or navigating to a "house the color of a ripe tomato" by reasoning that...