Online Library

Extremist exploitation of social media platforms is an important regulatory question for civil society, government, and the private sector. Extremists exploit social media for a range of reasons—from spreading hateful narratives and propaganda to financing, recruitment, and sharing operational information. Policy responses to this question fit under two headings, strategic communication and content moderation. At the center of both of these policy responses is a calculation about how best to limit audience exposure to extremist narratives and maintain the marginality of extremist views, while being conscious of rights to free expression and the appropriateness of restrictions on speech. This special issue on “Countering Extremists on Social Media: Challenges for Strategic Communication and Content Moderation” focuses on one form of strategic communication, countering violent extremism. In this editorial we discuss the background and effectiveness of this approach, and introduce five articles which develop multiple strands of research into responses and solutions to extremist exploitation of social media. We conclude by suggesting an agenda for future research on how multistakeholder initiatives to challenge extremist exploitation of social media are conceived, designed, and implemented, and the challenges these initiatives need to surmount.

The alt-right is a growing radical right-wing network that is particularly effective at mobilizing emotion through digital communications. Introducing ‘white thymos’ as a framework to theorize the role of rage, anger, and indignation in alt-right communications, this study argues that emotive communication connects alt-right users and mobilizes white thymos to the benefit of populist radical right politics. By combining linguistic, computational, and interpretive techniques on data collected from Twitter, this study demonstrates that the alt-right weaponizes white thymos in three ways: visual documentation of white victimization, processes of legitimization of racialized pride, and reinforcement of the rectitude of rage and indignation. The weaponization of white thymos is then shown to be central to the culture of the alt-right and its connectivity with populist radical right politics.

Social media and the Internet play an important role in the proliferation of hateful and extreme speech. Looking to contemporary networks of digitally mediated extreme right-wing communication, this essay explores the form, dynamics, and potential governance of digital hate culture. It focuses on the cultural practices and imagination present in the networks of digital hate culture to illuminate how two frames, the Red Pill and white genocide, unify the different groups that take part in these networks. After providing a high-level overview of these networks, this essay explains three formal features of digital hate culture that make it ungovernable: its swarm structure, its exploitation of inconsistencies in web governance between different actors, and its use of coded language to avoid moderation by government or private sector actors. By outlining its cultural style and ungovernable features, this essay provides policy professionals and researchers with an understanding of contemporary digital hate culture and provides suggestions for future approaches to consider when attempting to counter and disrupt the networks on which it depends.

VOX-Pol Blog post.

As terrorist, extremist, and hateful content has become widespread on social media, platforms have responded with content moderation – the flagging, review, and enforcement of rules and standards on user-generated content online. This chapter provides an introduction to contemporary content moderation practices, technologies, and contexts and outlines key debates in the field. The chapter explores the changing regulatory context around hate speech and terrorist content online in the EU. This changing context, such as Germany’s NetzDG law and the EU’s 2021 regulation on terrorist content online, is setting new standards and statutory requirements for platforms and websites. While content moderation is a form of self-regulation run by companies themselves, the regulatory context is shifting towards multistakeholder governance. This chapter then maps key stakeholders and technologies in content moderation as they pertain to radicalisation, looking at trusted flaggers, Internet Referral Units, and automated content moderation. After providing an overview of the regulatory context, stakeholders, and technologies in content moderation, the chapter explores normative questions about deplatforming, how transparency hinders research and accountability in content moderation, and explores inconsistencies in the enforcement of content moderation rules, particularly in the case of the far right. By providing an outline of the regulatory context, mapping multistakeholder arrangements in content moderation, and exploring key debates, this chapter provides readers with an introduction to the growing literature on content moderation and countering radicalisation.

ISIS will likely maintain the capacity to align its military and information operations (IO) in the coming years. Continuing conflicts and the plodding effort to address the underlying conditions where it has taken root will likely help ISIS retain physical sanctuary and command and control capability in Iraq, Syria, and North Africa, even if it loses control of major cities. ISIS’s IO campaign has supported multiple objectives, including control over territory, coercion of populations, and recruitment. This campaign has enabled ISIS’s survival and execution of international terror attacks. It may ultimately usher in a “Virtual Caliphate” – a radicalized community organized online – that empowers the global Salafi-jihadi movement and that could operate independently of ISIS. This “Virtual Caliphate,” the emergence of which becomes more likely the longer ISIS’s physical caliphate exists, would represent a unique challenge to American national security. Other hostile actors, beyond ISIS and the global Salafi-jihadi movement, are also adopting elements of a broader IO campaign, highlighting the requirement for
the U.S. to formulate a determined response. The U.S. possesses inherent advantages, including material resources, military strength and convening power, with which to confront this evolving threat. It also has challenges to overcome, including the lack of a government-wide strategy – supported by the necessary resources and proper bureaucratic organization – to counter enemy IO. The U.S. should continue to counter ISIS and other enemies in this arena by focusing on rolling them back on the ground, degrading their technical capabilities and other means they employ to reach their intended audiences, and helping facilitate the emergence of compelling counter-narratives amenable to American interests.