Listen to the article
In a significant evaluation of artificial intelligence’s capabilities in healthcare, researchers have found that many mainstream chatbots still fall short when providing medical information. Nearly half of AI-generated responses to health questions were deemed “problematic,” often containing misinformation, missing crucial context, or presenting incorrect information with unwarranted confidence.
The study, published in a medical journal, tested several major AI systems including Google’s Gemini, High-Flyer’s DeepSeek, Meta AI, OpenAI’s ChatGPT, and XAI’s Grok. Each platform was asked ten questions across various health domains including cancer, vaccines, stem cells, nutrition, and athletic performance.
The results revealed concerning gaps: 49.6% of responses were problematic, with 19.6% classified as highly problematic and potentially harmful if followed literally. While no single chatbot clearly outperformed the others, researchers noted that Grok produced more highly problematic answers than expected. Responses about vaccines and cancer showed better accuracy, while information about stem cells, athletic performance, and nutrition demonstrated significant weaknesses.
Another issue emerged in readability—many answers were written at an advanced level, making them difficult for the average patient to understand. Citation quality was also subpar across all platforms, with many references being incomplete or inadequate.
Against this backdrop, specialized medical AI platforms like SurvivorNet’s “My Health Questions” demonstrate how purpose-built technology can avoid these pitfalls. Unlike general-purpose chatbots, this platform was designed specifically for cancer care, grounded in clinical guidelines and backed by medically reviewed research from leading oncologists.
“My Health Questions allows users to create personalized health profiles by entering details like age, gender, and location, enabling the platform to refine its responses over time. Breast cancer survivor Christine Santasiero and her sister Lauren described the tool as “the perfect second opinion” during Christine’s diagnosis.
Dr. Maurice Franklin, a public health educator who recognized warning signs of prostate cancer after finding elevated PSA levels, turned to the platform for guidance between medical appointments. The AI recognized his anxiety and provided evidence-based reassurance while maintaining a safety-first approach that physicians prioritize.
Another patient, Gabby Cooper, a Penn State graduate undergoing treatment for stage 2 Hodgkin lymphoma, used the platform to prepare for her oncologist appointments. When she developed complications like colitis that raised concerns about her chemotherapy regimen, the tool helped her formulate specific, informed questions to discuss with her doctor.
The American Society of Clinical Oncology (ASCO) recently released six guiding principles for AI in oncology, emphasizing transparency, stakeholder awareness, equity, accountability, privacy, and human-centered application. These principles underscore that while AI has tremendous potential in cancer care, human clinical judgment remains essential.
Dr. Basak Dogan, Director of Breast Imaging Research at UT Southwestern’s Simmons Cancer Center, highlights one of AI’s promising advantages: “Traditional models often perform poorly for patients from diverse backgrounds who may not know their full family history. AI provides an objective assessment based on the individual’s biology, which can democratize access to high-risk screening.”
Dr. Beth Mittendorf, Chief of Multidisciplinary Oncology at Dana-Farber Cancer Institute, emphasizes the balanced approach needed: “AI should help us identify risk earlier, tailor prevention more intelligently, and use specialist resources more effectively. The goal is not to replace clinical judgment, but to augment it.”
As artificial intelligence continues to expand its role in healthcare, the contrast between general AI chatbots and specialized medical platforms highlights the importance of purpose-built solutions that combine technological efficiency with medical expertise. While general AI systems still struggle with medical accuracy, specialized platforms show promising results in providing patients with reliable information, reducing healthcare disparities, and supporting the doctor-patient relationship.
The future of AI in oncology lies not in replacing human expertise, but in creating tools that make complex medical information more accessible, personalized, and actionable for patients navigating their cancer journey.
Fact Checker
Verify the accuracy of this article using The Disinformation Commission analysis and real-time sources.
12 Comments
The healthcare sector is a critical area where we need to be extremely cautious about relying on AI systems. Providing inaccurate medical advice can have serious consequences. More safeguards and oversight are clearly required.
I agree completely. Healthcare is an area where we simply can’t afford to have unreliable information. Rigorous testing and validation of these systems is an absolute necessity.
Interesting findings on the accuracy of AI chatbots in healthcare. It’s concerning that nearly half the responses contained problematic information. More work is needed to improve the reliability of these systems.
You’re right, these findings are quite worrying. Healthcare information needs to be accurate and trustworthy. Chatbots still have room for improvement.
This is an important study, but I wonder how the results would change if the chatbots were asked more direct, factual questions instead of open-ended ones. The flaws may lie in the conversational interface as much as the underlying knowledge.
That’s a fair observation. The study design could be refined to better isolate the capabilities of the AI models themselves versus the limitations of conversational UIs. More research is needed.
This study is a wake-up call that we still have a long way to go before AI chatbots can be trusted to provide high-quality, reliable information on complex health topics. Continued research and development is needed.
I’m not surprised to see that chatbots struggled with topics like stem cells, nutrition, and athletic performance. Those are complex areas where even human experts can disagree. More research is needed.
Good point. The more nuanced and subjective the topic, the harder it is for an AI to provide reliable, comprehensive information. Chatbots have limitations that need to be better understood.
This study highlights the importance of rigorous testing and validation for AI systems, especially in sensitive domains like healthcare. Inaccurate or misleading information can have serious consequences.
Absolutely. AI systems should be held to high standards when it comes to providing critical information that can impact people’s health and wellbeing.
While the findings are concerning, I’m glad to see this kind of rigorous evaluation being done. It’s crucial that we understand the limitations of AI-powered chatbots, especially in high-stakes domains. Transparency and accountability are key.