Evaluating Evidence: A Statistician’s Perspective on Distinguishing Truth from Misinformation

In a striking critique of today’s health discourse, the recent dietary guidelines unveiled by U.S. Health Secretary Robert F. Kennedy Jr. have ignited a familiar pattern of polarized responses. The guidelines, branded under the slogan “Make America Healthy Again,” drew both support and criticism from various health organizations, highlighting a growing problem in how we evaluate scientific evidence.

The American Heart Association praised the emphasis on vegetables, fruits, and whole grains. However, other organizations expressed concern about recommendations for red meat and full-fat dairy products, with some critics going so far as to label them “blatant misinformation.”

This labeling reflects a troubling trend in public discourse. The term “misinformation” has become ubiquitous in media and policy discussions, often deployed against viewpoints someone simply disagrees with rather than being reserved for actual falsehoods. While genuine misinformation can indeed undermine democracy, harm public health, and even fuel violence, the increasing tendency to weaponize this label threatens productive scientific dialogue.

The challenge stems partly from the inherent complexity of evaluating scientific evidence, particularly in nutrition science. Isolating the health effects of specific foods is notoriously difficult, as countless genetic and lifestyle factors can influence outcomes. This is why research studies often identify only associations or correlations between food consumption and health effects, rather than clear causation.

Even in more straightforward scenarios, assessing evidence remains surprisingly complex. Consider a simple example: If a die is rolled seven times and shows an odd number six times, does this suggest the die is loaded? The answer depends entirely on which statistical approach is applied.

Using the p-value approach—currently the most common statistical measure in science—one might conclude there’s insufficient evidence the die is loaded, since there remains a reasonable probability that a fair die could produce such results. Conversely, using an e-value approach, one might conclude the die is likely loaded, as the observed pattern would be much more likely with a loaded die than a fair one.

Neither conclusion is inherently wrong. They simply reflect different thresholds for what constitutes meaningful evidence. The p-value asks, “How unexpected is this result if the die is fair?” while the e-value asks, “How much more consistent is this result with a loaded die than a fair one?” These approaches can be calibrated to reach the same conclusion, but they frame evidence differently.

This statistical nuance creates significant problems in how the public interprets risk. Consider how differently people might react to these two statements about the same risk: “Those who eat this food regularly are 25 times more likely to develop cancer” versus “Eating this food increases your cancer risk from 0.01% to 0.25%.” Some might avoid the food based on the dramatic relative increase, while others might continue consuming it, focusing on the still-small absolute risk.

Neither reaction is definitively correct, as risk tolerance varies among individuals. Yet in today’s climate, expressing a personal risk calculation that differs from mainstream opinion can lead to accusations of spreading “misinformation.”

This dynamic undermines productive scientific discourse. The conventional p-value threshold has already contributed to numerous irreproducible findings in science. Using this shaky standard to label conflicting interpretations as “misinformation” only further impedes scientific progress.

The controversy surrounding Kennedy’s dietary guidelines exemplifies this problem. Nutrition science involves complex trade-offs and interpretations of imperfect evidence. Reasonable experts can reach different conclusions when evaluating the same data on red meat or dairy consumption.

As we navigate an increasingly complex information landscape, especially with concerns about AI potentially worsening misinformation spread, we should reserve the term “misinformation” for demonstrable falsehoods—not for conclusions based on different interpretations of evidence or varying risk thresholds.

The path forward requires greater nuance in how we discuss scientific evidence, particularly in fields like nutrition where definitive answers remain elusive. Only by acknowledging the inherent uncertainty in scientific inquiry can we foster the kind of productive dialogue needed to advance public health.