New Tool Offers Systematic Approach to Assessing Diet and Nutrition Misinformation Risk

A novel assessment framework designed to evaluate misinformation risk in diet and nutrition content has been developed through a rigorous multi-phase process, according to researchers who created the tool. The Misinformation Risk Assessment Model (MisRAM) represents a significant step forward in systematically identifying potentially misleading health information before it reaches the public.

The development followed a structured approach modeled after the World Health Organization’s methodology for environmental hazard assessment. Researchers first created a five-step framework that guided the extraction and formalization of misinformation risk factors into a functional scoring instrument specifically focused on diet and nutrition content.

“We established this structured roadmap to characterize and assess risk factors in the informational environment,” explained the research team. “This allowed us to develop tools capable of estimating the risk of misinforming recipients based on the presence and nature of misinformation traits within any content of interest.”

The five-step framework begins with defining the assessment’s scope and objectives, then moves through identifying misinformation risk factors, characterizing and stratifying those factors, quantifying their presence, and finally evaluating the overall misinformation risk level.

In implementing the framework, researchers first defined their focus on the public health threat posed by diet-health misinformation in medium to long-form lay content. They then conducted a methodical scan of historical and contemporary information environments to extract recurring content traits consistently associated with misleading information.

These risk factors were classified into interconnected dimensions—inaccuracy, incompleteness, deceptiveness, and health harm—and sorted into a three-tiered risk stratification (low, moderate, and high-risk). This allowed the team to create a scoring system that accounts for both the density and severity of misinformation traits.

The final tool, called Diet-MisRAT (Diet Misinformation Risk Assessment Tool), produces a total score that translates into one of five risk categories: very low, low, moderate, high, or very high risk. These classifications can help inform individual decision-making, institutional responses, and systemic applications such as algorithmic risk flagging.

Following its development, the tool underwent extensive validation through five testing rounds with various groups, beginning with an expert panel review. Two experienced professors with backgrounds in nutrition science, dietetics, science education, and pedagogy provided critical feedback that informed refinements to the tool’s structure, content, and guidance.

The researchers then established an a priori benchmark dataset as a reference standard for appropriate response options when applying the tool. This involved independent application by the developer and expert validators, followed by agreement analysis and benchmark refinement.

Subsequent rounds involved testing with postgraduate dietitians in training, postgraduate students in nutrition science, and highly experienced nutrition professionals with extensive career experience ranging from a decade to over 30 years. In total, 33 postgraduate nutrition students and 15 senior nutrition professionals participated in these phases.

The final testing phase explored the tool’s potential for automation by engaging ChatGPT models to apply the Diet-MisRAT under zero-shot prompt-based conditions. The AI models completed the assessment in an average of just 25 seconds, compared to the 37.5 minutes averaged by human nutrition professionals.

“With appropriate automated AI deployment, such as parallelized API solutions or batch-processing, the tool could feasibly be scaled for simultaneous multi-piece analysis in future,” the researchers noted.

Statistical analyses assessed how closely participants aligned with the benchmark answers, with Pearson’s correlation coefficients calculated between each participant’s item-level scores and the corresponding benchmark values. The internal consistency of the tool was evaluated using Cronbach’s alpha.

For the AI testing round, standard performance metrics commonly used in natural language processing were applied, including accuracy, precision, sensitivity, and F1 scores, to evaluate how well the models aligned with expert benchmark responses.

The Diet-MisRAT represents a promising development in the fight against health misinformation, offering a systematic way for professionals, educators, researchers, and policymakers to identify potentially harmful content. As misinformation continues to proliferate across digital platforms, such tools may become increasingly valuable for protecting public health.