AI’s Fact-Checking Problem: When ChatGPT Fabricates Reality

ChatGPT has emerged as an indispensable tool for millions of users worldwide, but it’s increasingly being recognized as the Wikipedia of our generation—helpful but factually unreliable. The AI chatbot’s tendency to “hallucinate” or fabricate information has become a significant concern for users seeking accurate information.

These hallucinations occur when the AI confidently presents false information as fact rather than acknowledging knowledge gaps. In a recent series of tests, ChatGPT consistently fabricated details across various topics, from automotive history to legal cases, while maintaining an authoritative tone that makes its inaccuracies difficult to detect.

When asked about electric cars from the 1940s, ChatGPT confidently described the Henney Kilowatt and Morrison Electric trucks as examples from that era. In reality, the Henney Kilowatt wasn’t produced until 1959, and the company name is Morrison-Electricar, not Morrison Trucks. This fabrication is particularly problematic given that the first mass-produced electric vehicle for American consumers was GM’s EV1, which didn’t appear until around 1990.

The problem extends beyond technical subjects. When prompted about song lyrics for “Chase the Kangaroo” by the 1970s band Love Song, ChatGPT not only provided detailed lyrics but also described the song’s folk-rock sound and gentle guitar work. The reality? Love Song never recorded such a track—the AI had completely fabricated both the connection and the analysis.

Legal information poses perhaps the most concerning area for AI hallucinations. Despite well-publicized incidents of lawyers submitting ChatGPT-fabricated legal cases in court documents—resulting in dismissed cases and professional embarrassment—the problem persists. When asked about legal cases involving fathers suing sons over car sales, ChatGPT cited specific cases like “Matter of Szabo’s Estate (1979)” and “Anderson v. Anderson (1994)” with altered facts to fit the narrative. The former actually concerned stocks and bonds, while the latter involved divorce proceedings—neither addressed car sales between family members.

Academic research suffers similar distortions. In response to a request for academic quotes about social media’s psychological impact, ChatGPT fabricated author names for real studies and misattributed quotes from well-known researchers. For scholarly work, such inaccuracies could prove disastrous, potentially leading to failed assignments or compromised research integrity.

Google’s competing AI, Gemini, appears somewhat more reliable with factual information. When presented with ChatGPT’s responses for fact-checking, Gemini often responded with dismissive or even sarcastic corrections, at one point describing ChatGPT’s academic citations as “a corrupted, recycled, and partially fabricated mess.”

Industry observers attribute Gemini’s relative accuracy to Google’s reputation as a search engine company, where factual reliability directly impacts brand trust. However, Gemini isn’t immune to hallucinations either. In one test, it incorrectly claimed the article’s author had written for The Onion, highlighting that all current AI systems remain vulnerable to factual errors.

The issue of AI hallucinations has significant implications beyond mere inconvenience. Legal professionals have already faced consequences after relying on fabricated cases, and students using these tools for research risk academic penalties. For journalists, researchers, and business professionals, the risk of propagating false information poses serious ethical and professional concerns.

While OpenAI has worked to improve ChatGPT’s factual accuracy—fixing some previously notorious errors like mixing up Porsche car models—the fundamental problem persists across all current AI chatbots. Users are increasingly advised to verify any factual claims from AI sources before incorporating them into important work.

As these AI tools become more deeply integrated into daily workflows, the challenge of distinguishing between reliable information and convincing fabrication will likely remain a critical skill for users across all sectors.