Mistral unveils 8B robot navigation model with single camera

French artificial intelligence company Mistral has unveiled its first model for robotic navigation, a system that guides robots using only a single color camera and plain language instructions. The model, named Robostral Navigate, represents a significant technical departure from existing approaches that typically rely on more expensive and complex sensor arrays like LiDAR, depth sensors, or multiple cameras.

The 8-billion-parameter model is designed to enable robots to autonomously navigate complex environments including offices, residential and commercial buildings, and outdoor settings. According to the company, it achieves a 76.6 percent success rate on the Room-to-Room in Continuous Environments (R2RCE) benchmark, a standard test for evaluating a robot's ability to follow instructions in environments it has not encountered during training. This performance reportedly beats the best single-camera approach by 9.7 percentage points and outperforms systems using depth sensors or multiple cameras by 4.5 points.

A key innovation of Robostral Navigate is its "pointing" method for navigation. Given a task and a history of observations, the model predicts the robot's next move by...