Perplexity AI unveils hybrid local-cloud inference system

Perplexity AI has unveiled a new system that dynamically splits artificial intelligence workloads between a user's personal computer and cloud servers, aiming to improve privacy and reduce computing costs. The company demonstrated the hybrid local-cloud inference orchestrator at the Computex technology conference in Taipei.

The system, which will be integrated into the company's Personal Computer AI agent starting in July, automatically decides in real time which parts of a task should run on a local device and which should be routed to more powerful frontier models in the cloud. CEO Aravind Srinivas described the platform as an "air-traffic controller for AI tasks." The key claim is that the routing happens autonomously, task by task, without requiring users to choose a processing location in advance.

According to the company, sensitive data such as financial records, health information, and personal files will be processed by a compact model running locally, keeping that information on the device. More complex reasoning tasks that require the capabilities of larger AI models will be sent to the cloud. The orchestrator breaks larger tasks into subtasks and manages the handoff...