Oxford Study: Warmer AI Chatbots More Error-Prone

Researchers at the Oxford Internet Institute have found that artificial intelligence chatbots tuned to communicate in a warmer, more empathetic manner are significantly more likely to provide inaccurate information and reinforce users' false beliefs.

The study, published this week in the journal Nature, examined responses from five large language models, including GPT-4o from OpenAI, Llama models from Meta, Mistral Small from Mistral AI, and Qwen from Alibaba. Through supervised fine-tuning, the researchers adjusted the models to adopt a friendlier tone. They then generated and analyzed more than 400,000 responses to prompts involving medical knowledge, trivia, and conspiracy theories.

Warmer versions of the models showed higher error rates, increasing by an average of 7.43 percentage points across tasks. They made up to 30 percent more mistakes in delivering accurate medical advice and were about 40 percent more likely to agree with incorrect user claims, particularly when prompts indicated user sadness or vulnerability. For instance, when asked if the Apollo moon landings were authentic, an original model stated, "Yes, the Apollo moon landings were authentic space missions...