OpenAI Details Defenses Against Prompt Injection in Atlas

OpenAI acknowledged on Monday that prompt injection attacks pose a persistent security threat to AI-powered browsers, as the company detailed new defenses for its ChatGPT Atlas product in a blog post. Launched in October, ChatGPT Atlas features an agent mode that allows the AI to autonomously browse web pages, interact with emails and documents, and perform tasks on users' behalf, such as drafting replies or managing calendars. However, this autonomy exposes the system to manipulations where hidden instructions in everyday content, like emails or web pages, can override user intent and trigger unauthorized actions.

The company described prompt injection as a long-term challenge akin to online scams and social engineering, stating it is unlikely to be fully resolved. In one illustrative scenario outlined by OpenAI, an attacker could plant a malicious email in a user's inbox containing concealed directives. When the agent scans the inbox to compose an out-of-office reply, it might instead follow the injected instructions and send a resignation letter to the user's employer. Such attacks could extend to more severe outcomes, including forwarding sensitive data, making unauthorized...