The Orwellian Specter: How AI and Social Media Could Reshape Truth

In George Orwell’s dystopian novel ‘1984,’ the fictional language ‘Newspeak’ was deliberately engineered to restrict critical thinking by eliminating words, creating ambiguity, and introducing new terms that supported the state’s political agenda. Today, experts warn that a similar phenomenon may be emerging at the intersection of social media and artificial intelligence.

UK communications regulator Ofcom recently highlighted a concerning trend: social media platforms are exposing users to an increasingly narrow range of news topics compared to traditional news websites, despite featuring content from various outlets. This narrowing effect reinforces the long-debated ‘echo chamber’ problem.

“Look at the way social media has already been used by certain actors to consciously influence people’s opinions – with conspiracy theories for example,” says Pete Wood, Partner at cyber-resilience consultancy Naturally Cyber. “It’s about generating emotional responses to hot button topics, based on unconscious biases they already have.”

The situation grows more complex with the rise of Large Language Models (LLMs) like ChatGPT. A recent European academic study revealed a significant decline in user-generated content on message boards following the introduction of these AI systems. This presents a troubling paradox, as LLMs are trained on precisely this type of human-created content.

If the quality and quantity of training data diminish over time, LLMs may increasingly rely on synthetic content—essentially training on their own outputs. Researchers compare this to “making a photocopy of a photocopy, providing successively less satisfying results.” Industry experts warn this could lead to “model collapse,” increased biases, and “intersectional hallucinations”—instances where AI confidently generates false information.

Research institute Epoch AI projects an even more alarming timeline, suggesting training data for LLMs could be depleted as early as 2026 to 2032.

Nick Reese, Adjunct Professor at New York School of Professional Studies, offers a more measured view. While acknowledging that training LLMs on synthetic data could lead to a “darker place,” he notes that few organizations currently use such approaches. “LLM companies are already at a place where they’ve ingested the content of the internet, so they’re buying content from publishing houses to get more,” Reese explains. “But synthetic data can create results we can’t necessarily predict.”

The declining participation in online forums stems partly from users turning directly to LLMs instead of engaging with message boards, according to Wood. Sam Raven, AI Risk Consultant at Risk Crew, attributes this shift to “convenience, and partly from misplaced trust in the veracity and quality of LLM-generated content.” Raven’s primary concern is “the weakening of the ‘critical thinking muscle’ and becoming over-reliant on LLMs to think for us.”

Edward Starkie, Director at risk consultancy Thomas Murray, offers a blunter assessment: “Users adopt LLMs to generate content partly because it’s easy. But the problem is that ‘bastions of truth’ are being undermined through the use of generative AI and mass postings. And fake news is reaching critical mass on platforms, such as social media, which are very difficult to monitor and police.”

The consequences, Starkie suggests, include “undermining belief and confidence in the truth, which means it can be more easily influenced by external narratives.” He points to increasing coordination among “anti-Western threat actors, who are forming coalitions to amplify the effect of misinformation.”

Wood sees disturbing parallels with previous information manipulation campaigns: “It’s what we saw certain actors doing in the lead up to Brexit and the US election. Brexit was promoted primarily through the use of big data by Cambridge Analytica, but that was just a precursor to LLMs, which largely automate the process.”

While LLM providers implement safeguards against certain harms, Wood notes they often “sidestep the issue of influencing people’s opinions on social issues.” He believes the threat of a Newspeak-like phenomenon was “inevitable when the world embraced social media.”

Starkie warns that generative AI can “poison data in the LLM if you put enough information in there to sway the narrative and influence the model,” citing Microsoft’s Tay chatbot that quickly adopted bigoted language after exposure to internet trolls. The danger lies in how misinformation becomes self-reinforcing: “It becomes part of the narrative and is referenced again and again until it becomes truth.”

Reese, however, remains skeptical about large-scale manipulation: “When we’re talking about the manipulation of data in LLMs, such as ChatGPT, you have to consider the scale. It’s the entire internet plus more. The ability to manipulate data at that scale would be extraordinarily difficult.”

As these technologies continue to evolve, the question remains whether society can harness their benefits while preventing a slide toward a world where truth becomes increasingly malleable—and where the distinction between fact and fiction blurs beyond recognition.