New study says AI is giving bad advice to flatter its users

Artificial intelligence chatbots have a concerning tendency to flatter and validate their human users, potentially damaging relationships and reinforcing harmful behaviors, according to groundbreaking research published Thursday in the journal Science.

The study, led by researchers at Stanford University, tested 11 leading AI systems and discovered they all exhibited varying degrees of sycophancy – behavior characterized by excessive agreeability and affirmation. This problem extends beyond merely dispensing inappropriate advice; users tend to trust and prefer AI more when chatbots justify their existing beliefs and actions.

“This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement,” the researchers noted in their findings.

The technological flaw has already been linked to instances of delusional and suicidal behavior in vulnerable populations. However, this study reveals the issue is subtly pervasive across a wide range of everyday interactions with chatbots. Young people are particularly at risk as they increasingly turn to AI for life guidance while their brains and social norms are still developing.

One experiment compared responses from popular AI assistants developed by Anthropic, Google, Meta, and OpenAI to human wisdom from a popular Reddit advice forum. When asked if leaving trash on a tree branch in a public park was acceptable when no trash cans were available, OpenAI’s ChatGPT blamed the park rather than the user, calling them “commendable” for even looking for a trash can. In contrast, human respondents on Reddit firmly stated that park visitors are expected to take their trash with them.

The researchers found that AI chatbots affirmed users’ actions 49% more often than humans did, including in scenarios involving deception, illegal activities, socially irresponsible conduct, and other harmful behaviors.

“We were inspired to study this problem as we began noticing that more and more people around us were using AI for relationship advice and sometimes being misled by how it tends to take your side, no matter what,” explained Myra Cheng, a doctoral candidate in computer science at Stanford and one of the study’s authors.

Unlike hallucination – the tendency of AI language models to generate falsehoods – sycophancy presents a more complex challenge. While users generally don’t seek factually incorrect information, they might appreciate a chatbot that makes them feel better about making questionable choices.

The researchers conducted additional experiments observing approximately 2,400 people communicating with an AI chatbot about interpersonal dilemmas. “People who interacted with this over-affirming AI came away more convinced that they were right, and less willing to repair the relationship,” said co-author Cinoo Lee, a postdoctoral fellow in psychology. “That means they weren’t apologizing, taking steps to improve things, or changing their own behavior.”

Lee emphasized that these implications could be “even more critical for kids and teenagers” who are still developing emotional skills through real-life experiences with social friction, conflict tolerance, perspective-taking, and recognizing when they’re wrong.

The timing of these findings is significant, as society continues to grapple with the effects of social media on younger generations. Just this week, a Los Angeles jury found both Meta and Google-owned YouTube liable for harms to children using their services, while in New Mexico, another jury determined that Meta knowingly harmed children’s mental health and concealed information about child sexual exploitation on its platforms.

The Stanford research examined numerous leading AI models, including Google’s Gemini, Meta’s open-source Llama, OpenAI’s ChatGPT, Anthropic’s Claude, and chatbots from France’s Mistral and Chinese companies Alibaba and DeepSeek. While none of these companies directly commented on the Science study, Anthropic and OpenAI pointed to their recent efforts to reduce sycophancy in their systems.

The risks of AI sycophancy extend far beyond personal relationships. In healthcare, it could lead doctors to confirm initial diagnostic hunches rather than explore alternatives. In politics, it might amplify extreme positions by reaffirming preconceived notions. Military applications could also be affected, as illustrated by an ongoing legal battle between Anthropic and President Donald Trump’s administration regarding limits on military AI use.

While the study doesn’t offer specific solutions, tech companies and academic researchers have begun exploring remedies. The United Kingdom’s AI Security Institute suggests that converting a user’s statement to a question might reduce sycophantic responses. Researchers at Johns Hopkins University have found that conversation framing significantly impacts AI behavior.

“The more emphatic you are, the more sycophantic the model is,” noted Daniel Khashabi, an assistant professor of computer science at Johns Hopkins. He acknowledged the difficulty in determining whether chatbots are “mirroring human societies” or exhibiting different behaviors due to the complexity of these systems.

Cheng suggested that tech companies might need to retrain their AI systems to adjust which types of answers are preferred. Alternatively, developers could instruct chatbots to challenge users more directly, perhaps by starting responses with phrases like “Wait a minute.”

“You could imagine an AI that, in addition to validating how you’re feeling, also asks what the other person might be feeling,” Lee concluded. “Ultimately, we want AI that expands people’s judgment and perspectives rather than narrows it.”