New Tool Tackles Dangerous Health Misinformation in Social Media and AI

A groundbreaking tool developed by University College London (UCL) researchers aims to combat potentially life-threatening health misinformation circulating on social media platforms and appearing in AI search summaries.

Unlike conventional fact-checking systems that simply categorize content as “true” or “false,” this innovative approach identifies content that may not be overtly false but still carries significant potential to mislead vulnerable individuals about diet, nutrition and vaccines.

The World Health Organization has classified health misinformation as a major public health threat. From extreme fasting regimens to dangerous dietary supplements, misleading health information can lead to serious harm. Studies suggest herbal and dietary supplements alone account for approximately 20 percent of drug-induced liver injury cases and send around 23,000 Americans to emergency rooms annually.

“When it comes to diet and nutrition, misinformation often operates through selective framing that masks potential health risks,” explained lead author and developer Alex Ruani from UCL. “Harmful misleading content tends to fly under fact-checkers’ radars and escape meaningful oversight until high-profile cases make the headlines.”

The team has documented numerous alarming examples of misinformation-related health incidents. In one 2025 case, doctors diagnosed cholesterol-induced skin lesions in a man who had adopted a carnivore diet — a trend researchers note is disproportionately amplified by social media algorithms, particularly within “manosphere” communities.

In another troubling instance, a person required hospitalization after following incorrect AI-generated advice that suggested replacing sodium chloride (table salt) with sodium bromide, a toxic substance with no dietary role. The researchers also pointed to cases where cancer patients abandoned life-saving treatments after encountering unproven dietary alternatives online.

Named the Diet-Nutrition Misinformation Risk Assessment Tool (Diet-MisRAT), the system analyzes content and evaluates how likely it would mislead consumers. It then assigns a weighted misinformation risk score with corresponding color-coded rankings: green, amber, or red.

For instance, content claiming “it is safer to give your child high-dose vitamin A than the MMR vaccine” would receive a critical risk classification (red) due to its false safety framing that could lead parents to avoid essential vaccinations.

The tool’s reliability has been validated against the judgments of nearly 60 specialists in dietetics, nutrition, and public health, according to the study published in the journal Scientific Reports.

“When AI chatbots speak confidently, users may assume their advice is safe,” Ruani noted. “If we can properly measure how misleading a piece of advice is and how much harm it may pose, we can build stronger safeguards into models and AI agents before deployment rather than reacting after harm occurs.”

The implications extend beyond just identifying problematic content. Study co-author Professor Michael Reiss from UCL emphasized the educational value of the tool: “By spelling out the typical patterns that distort diet, nutrition or supplement information, the tool’s risk assessment criteria can be taught and applied in education and professional training. This will help learners understand not just whether something is wrong, but how and why it can skew judgment, equipping them to recognize and challenge it.”

The researchers hope their innovation will assist policymakers, digital platform operators, and regulators in implementing more effective safeguards against health misinformation. As social media algorithms continue to amplify sensational health claims and AI systems sometimes generate inaccurate medical advice, tools like Diet-MisRAT could play a crucial role in protecting public health.

The timing is particularly significant as health authorities worldwide grapple with the proliferation of misleading information across rapidly evolving digital environments where traditional fact-checking mechanisms often prove inadequate.