Online Library

Extreme digital speech (EDS) is an emerging challenge that requires co-ordination between governments, civil society and the private sector. In this report, a range of experts on countering extremism consider the challenges that EDS presents to these stakeholders, the impact that EDS has and the responses taken by these actors to counter it. By focusing on EDS, consideration of the topic is limited to the forms of extreme speech that take place online, often on social media platforms and multimedia messaging applications such as WhatsApp and Telegram. Furthermore, by focusing on EDS rather than explicitly violent forms of extreme speech online, the report departs from a focus on violence and incorporates a broader range of issues such as hateful and dehumanising speech and the complex cultures and politics that have formed around EDS.

Extremist exploitation of social media platforms is an important regulatory question for civil society, government, and the private sector. Extremists exploit social media for a range of reasons—from spreading hateful narratives and propaganda to financing, recruitment, and sharing operational information. Policy responses to this question fit under two headings, strategic communication and content moderation. At the center of both of these policy responses is a calculation about how best to limit audience exposure to extremist narratives and maintain the marginality of extremist views, while being conscious of rights to free expression and the appropriateness of restrictions on speech. This special issue on “Countering Extremists on Social Media: Challenges for Strategic Communication and Content Moderation” focuses on one form of strategic communication, countering violent extremism. In this editorial we discuss the background and effectiveness of this approach, and introduce five articles which develop multiple strands of research into responses and solutions to extremist exploitation of social media. We conclude by suggesting an agenda for future research on how multistakeholder initiatives to challenge extremist exploitation of social media are conceived, designed, and implemented, and the challenges these initiatives need to surmount.

The alt-right is a growing radical right-wing network that is particularly effective at mobilizing emotion through digital communications. Introducing ‘white thymos’ as a framework to theorize the role of rage, anger, and indignation in alt-right communications, this study argues that emotive communication connects alt-right users and mobilizes white thymos to the benefit of populist radical right politics. By combining linguistic, computational, and interpretive techniques on data collected from Twitter, this study demonstrates that the alt-right weaponizes white thymos in three ways: visual documentation of white victimization, processes of legitimization of racialized pride, and reinforcement of the rectitude of rage and indignation. The weaponization of white thymos is then shown to be central to the culture of the alt-right and its connectivity with populist radical right politics.

Social media and the Internet play an important role in the proliferation of hateful and extreme speech. Looking to contemporary networks of digitally mediated extreme right-wing communication, this essay explores the form, dynamics, and potential governance of digital hate culture. It focuses on the cultural practices and imagination present in the networks of digital hate culture to illuminate how two frames, the Red Pill and white genocide, unify the different groups that take part in these networks. After providing a high-level overview of these networks, this essay explains three formal features of digital hate culture that make it ungovernable: its swarm structure, its exploitation of inconsistencies in web governance between different actors, and its use of coded language to avoid moderation by government or private sector actors. By outlining its cultural style and ungovernable features, this essay provides policy professionals and researchers with an understanding of contemporary digital hate culture and provides suggestions for future approaches to consider when attempting to counter and disrupt the networks on which it depends.

VOX-Pol Blog post.

As terrorist, extremist, and hateful content has become widespread on social media, platforms have responded with content moderation – the flagging, review, and enforcement of rules and standards on user-generated content online. This chapter provides an introduction to contemporary content moderation practices, technologies, and contexts and outlines key debates in the field. The chapter explores the changing regulatory context around hate speech and terrorist content online in the EU. This changing context, such as Germany’s NetzDG law and the EU’s 2021 regulation on terrorist content online, is setting new standards and statutory requirements for platforms and websites. While content moderation is a form of self-regulation run by companies themselves, the regulatory context is shifting towards multistakeholder governance. This chapter then maps key stakeholders and technologies in content moderation as they pertain to radicalisation, looking at trusted flaggers, Internet Referral Units, and automated content moderation. After providing an overview of the regulatory context, stakeholders, and technologies in content moderation, the chapter explores normative questions about deplatforming, how transparency hinders research and accountability in content moderation, and explores inconsistencies in the enforcement of content moderation rules, particularly in the case of the far right. By providing an outline of the regulatory context, mapping multistakeholder arrangements in content moderation, and exploring key debates, this chapter provides readers with an introduction to the growing literature on content moderation and countering radicalisation.