Listen to the article
The Digital Battlefield: How Extremist Rhetoric Thrives Despite Moderation
Two groundbreaking academic works have shed new light on the persistent challenge of extremist rhetoric in our digital landscape. Jennifer Mather Saul’s “Dogwhistles and Figleaves” and Tamar Mitts’ “Safe Havens for Hate” together provide a comprehensive examination of how harmful political speech continues to flourish both online and offline despite efforts to curb it.
When QAnon co-opted the legitimate #SavetheChildren hashtag in 2020, transforming an anti-trafficking campaign into a vehicle for conspiracy theories about liberal elites, it exemplified the sophisticated strategies extremist groups employ to evade detection while expanding their reach. This incident serves as a perfect case study for both authors’ central arguments.
Saul, a philosopher of language, examines how manipulative speech techniques have weakened social norms against expressing racist views and falsehoods. Meanwhile, Mitts, a political scientist specializing in online hate speech, documents the remarkable digital resilience of extremist organizations that strategically navigate content moderation policies.
“The urgency stems not solely from the sharp upswing in harmful political speech, but from the degree to which that rhetoric has become normalised,” writes Saul. This normalization has particular consequences beyond rhetoric – as she grimly notes, “increasingly hateful language is often a precursor to violence and even genocide.”
Saul’s analysis centers on two key rhetorical devices: dogwhistles and figleaves. Dogwhistles contain dual meanings – a surface message for general audiences and a coded secondary message targeting those who share certain biases. The term “inner city,” for instance, might simply reference urban areas to most people while conveying racial stereotypes to others.
Figleaves, meanwhile, provide plausible deniability for otherwise offensive statements – dismissing a slur as “just a joke” or attaching phrases like “that’s what people are saying” to extremist claims. Together, these techniques gradually shift the boundaries of acceptable public discourse.
While social media platforms have increased efforts to remove harmful content, Mitts demonstrates how extremist groups maintain their digital presence through sophisticated adaptation. Groups like the Islamic State and the Proud Boys employ three key tactics: migrating between platforms, mobilizing supporters, and modifying their messaging.
When banned from mainstream platforms, these organizations make calculated moves to alternative spaces that offer less moderation but more “authenticity.” As Mitts explains, they carefully weigh the trade-off between reach and content freedom, often accepting smaller audiences on platforms like Gab or Parler if it allows them to maintain their core messaging.
Once established on these alternative platforms, extremist groups focus on attracting individuals receptive to their ideology – often those harboring grievances about perceived political bias or cultural displacement. Alternatively, to maintain presence on heavily moderated mainstream platforms, they adapt their messaging by softening content or employing coded language.
“The Islamic State has even hidden propaganda in appropriated content or added digital noise to text and images to throw off AI moderation algorithms deployed by large platforms,” Mitts reveals in her research.
Content moderation faces significant challenges despite increased efforts. Mitts finds that both government interventions like the EU Digital Services Act and industry self-regulation have shown limited effectiveness. Her research suggests that while cross-platform cooperation (convergence) might be a powerful countermeasure, it brings risks of collateral damage, including disproportionate impacts on marginalized communities and potential misuse by authoritarian governments to suppress legitimate dissent.
Both scholars highlight how those feeling aggrieved or marginalized are most susceptible to propaganda – a particularly concerning insight as resentments increasingly manifest as real-world violence. This vulnerability, coupled with today’s emotion-fueled attention economy that rewards inflammatory content with engagement, creates a perfect environment for harmful speech to thrive.
“Political lies are nothing new,” Saul reminds us, but what has changed are the incentive structures that make inflammatory rhetoric politically advantageous. In our current media landscape, extreme content spreads rapidly, making moderation not just a technical challenge but increasingly a partisan issue.
As societies grapple with these challenges, fundamental questions arise: Should we focus on understanding the root causes of grievances or risk validating them? Should we strengthen norms against harmful speech, pursue pre-emptive “inoculation” strategies against misinformation, or place greater responsibility on listeners to reject harmful content?
These debates, as fraught as they are necessary, might ultimately be crucial to preserving a functional public sphere despite its deepening fractures.
Fact Checker
Verify the accuracy of this article using The Disinformation Commission analysis and real-time sources.
18 Comments
This is an important and timely analysis of a critical issue. The rise of propaganda and extremism in the digital age is a complex challenge that requires a multifaceted response. I’m particularly interested in the case studies and potential solutions outlined in the research.
Absolutely. The sophisticated tactics employed by these groups, such as co-opting legitimate campaigns, underscore the urgent need for more effective and adaptive policies and tools to combat the spread of disinformation and harmful ideologies online.
This is a concerning trend, especially given the potential downstream impacts on vulnerable communities and democratic discourse. I’m curious to learn more about potential policy interventions discussed in the research.
Yes, the case study of the #SavetheChildren hijacking is a prime example of the sophisticated tactics used. Tackling this will require a multi-pronged approach.
Interesting to see how the ‘attention economy’ has fueled the spread of propaganda and extremism. It’s a complex challenge without easy solutions, but this research provides important insights.
Absolutely. The resilience of these extremist groups in navigating content moderation is worrying. More robust and adaptive policies are clearly needed.
The rise of propaganda and extremism in the attention economy is a complex and multifaceted challenge. This research provides a valuable framework for understanding the dynamics at play and the urgent need for more effective interventions.
Absolutely. The case studies highlighted in the article underscore the sophisticated tactics employed by these groups. Addressing this issue will require a comprehensive and adaptive approach.
Fascinating analysis of how extremist rhetoric persists online despite moderation efforts. It highlights the need for more sophisticated tools and policies to counter the manipulative tactics of these groups.
Yes, the co-opting of legitimate hashtags is a particularly concerning tactic. Platforms need to be proactive in identifying and shutting down these efforts early on.
Fascinating and troubling insights. The persistence of extremist rhetoric despite moderation efforts is a worrying trend that deserves serious attention. I’m curious to learn more about the policy recommendations or technological solutions proposed in the research.
The proliferation of extremist rhetoric despite moderation efforts is deeply troubling. This research underscores the urgent need for more effective strategies to combat the spread of disinformation and harmful ideologies online.
The insights from this research are both fascinating and deeply troubling. The ability of extremist groups to exploit the attention economy and evade detection is a serious threat to democratic discourse and social cohesion. I’m eager to learn more about the policy recommendations and technological interventions discussed.
This is a sobering and important analysis. The ability of extremist groups to co-opt legitimate campaigns and leverage content moderation policies is deeply concerning. I look forward to learning more about potential solutions discussed in the research.
It’s alarming to see how extremist groups are able to exploit the attention economy and evade detection. This research highlights the complex challenges involved in curbing the rise of propaganda and extremism in the digital age.
Agreed. The insights from these studies could inform important policy discussions and technological solutions to address this critical issue.
This research sheds important light on the insidious ways in which extremist rhetoric can spread and thrive in the digital landscape. The co-opting of legitimate campaigns is a particularly concerning tactic that requires a multifaceted response.
Agreed. The resilience of these groups in navigating content moderation policies is a significant challenge that calls for more robust and adaptive solutions.