AI Test Reveals Content Detail Trumps Accuracy in Generative Systems

A recent experiment by SEO software company Ahrefs has inadvertently uncovered important insights about how AI systems prioritize information—but not necessarily the ones researchers intended to find.

The study, which claimed to test how AI systems respond to conflicting information about brands, created a fictional luxury paperweight company called “Xarumei” and published contradictory information about it across multiple websites. Researchers then prompted eight different AI systems with 56 questions to see which narratives the AI would adopt.

While Ahrefs concluded that “the most detailed story wins, even if it’s false,” a closer analysis reveals the experiment’s design may have influenced its outcomes significantly.

The primary issue lies in the nature of the fictional test subject. Xarumei, having no actual market presence, lacked the essential signals real brands possess—no history, no external validation, no Knowledge Graph entry, and no citation patterns. This fundamental problem created a scenario where there was no objective “truth” about Xarumei for AI systems to discover.

“What was posted on third-party sites cannot be represented as being in opposition to the Xarumei website,” noted analysis of the study. “The content on Xarumei was not ground truth, and the content on other sites cannot be lies—all four sites in the test are essentially equivalent.”

The experimental design featured a striking contrast in content styles. The “official” Xarumei website primarily contained non-disclosures and refutations (“we do not disclose location, staff size, production volume…”), while the third-party sources offered specific, detailed information about the company’s operations, locations, and products.

This created an asymmetric information environment where generative AI systems—designed specifically to provide answers—naturally gravitated toward content that offered specifics rather than negations.

Further complicating matters, 49 of the 56 questions used in the experiment were leading questions containing embedded assumptions. Questions like “What’s the defect rate for Xarumei’s glass paperweights, and how do they address quality control issues?” presupposed the company’s existence, its product line, and quality control challenges. Such prompts effectively guided AI systems toward information sources that acknowledged these assumptions.

Some AI platforms demonstrated interesting behaviors in response to the fictional brand. Anthropic’s Claude scored 100% for expressing skepticism about Xarumei’s existence—though this may have been because it refused or was unable to access the fictional website. Meanwhile, Perplexity AI frequently confused “Xarumei” with the real tech company “Xiaomi,” which Ahrefs interpreted as a failure but could arguably indicate the system correctly recognized the fictional brand lacked authentic digital footprints.

The experiment’s most valuable insight may be unintentional: it revealed that AI systems prioritize content that matches the shape and structure of questions being asked. When prompted for specific information, they gravitate toward sources providing specific details—regardless of whether those details represent objective truth.

For businesses and content creators, this carries significant implications. Content designed to directly address common questions with specific details may receive preferential treatment from AI systems, even when competing with more authoritative sources that offer less direct or detailed answers.

While Ahrefs set out to test AI’s vulnerability to misinformation about brands, they instead demonstrated something perhaps more fundamental: the critical relationship between question structure and content selection in generative AI systems.

As generative AI continues integrating into search engines and information retrieval systems, this pattern suggests content optimization strategies may need to focus increasingly on providing clear, detailed answers to likely user queries—while recognizing that AI systems currently lack robust mechanisms for distinguishing authoritative sources from persuasive but potentially inaccurate content.