Online Library

Social media companies have long struggled with what to do about extremist content on their platforms. While most companies include provisions about “extremist” content in their community standards, until recently, such content was often vaguely defined, providing policymakers and content moderators a wide berth in determining what to remove, and what to allow. Unfortunately, companies have responded with overbroad and vague policies and practices that have led to mistakes at scale that are decimating human rights content.

Recognizing that militant, non-state groups utilize social media and online platforms to reach members, sympathizers, and potential recruits, state agencies and social media corporations now increasingly regulate access to accounts affiliated with such groups. Scholars examining deplatforming efforts have, to date, focused on the extent of audience loss after account restrictions and the identification of strategies for regrouping online followers on the same or different platforms over time. Left unexplored is if and how militant non-state groups adapt their official messaging strategies in response to platform restrictions despite continuing online access to them. To begin to fill that gap, this study compares ISIS’s 550 images displayed in the group’s official newsletter al-Naba 6 months before and after Europol’s November 2019 take-down of terrorist affiliated accounts, groups, channels, and bots on Telegram. It conducts a content analysis of images related to militaries and their outcomes, non-military activities and their outcomes, and presentational forms. The findings demonstrate that ISIS visually emphasizes its standard priming approach but shifts its agenda-setting strategy. While retaining some of its standard visual framing practices, the group also alters frames, particularly those related to images showing opposing militaries and military outcome.

Online abusive content detection is an inherently difficult task. It has received considerable attention from academia, particularly within the computational linguistics community, and performance appears to have improved as the field has matured. However, considerable challenges and unaddressed frontiers remain, spanning technical, social and ethical dimensions. These issues constrain the performance, efficiency and generalizability of abusive content detection systems. In this article we delineate and clarify the main challenges and frontiers in the field, critically evaluate their implications and discuss potential solutions. We also highlight ways in which social scientific insights can advance research. We discuss the lack of support given to researchers working with abusive content and provide guidelines for ethical research.

Online extremist movements are increasingly using social media communities to share content, spread their ideologies, recruit members, and mobilize offline activities. In recent years, mainstream platforms, including Twitter and Facebook, have adopted policies to remove or deplatform some of these movements. Yet online extremists are well-known for their abilities to adapt, self-censor, and migrate across online platforms. How successful have these extremist movement deplatformings been? To answer this question, we begin by training a classifier to identify content generated by four prominent extremist movements: white supremacists, patriot/militia groups, QAnon, and Boogaloos. After doing so, we use this classifier to analyze approximately 12 million posts generated by about 1500 online hate communities across 8 social media platforms, including both mainstream and alternative platforms. We find that the deplatformings of Boogaloos and QAnon by mainstream platforms were initially highly successful, but that both movements were able to find ways to re-introduce their content on these platforms. These findings highlight the challenges of movement-based deplatforming, and they point toward important implications for content moderation.

This book is a timely and significant examination of the role of counter-messaging via social media as a potential means of preventing or countering radicalization to violent extremism. In recent years, extremist groups have developed increasingly sophisticated online communication strategies to spread their propaganda and promote their cause, enabling messages to be spread more rapidly and effectively. Countermessaging has been promoted as one of the most important measures to neutralize online radicalizing influences and is intended to undermine the appeal of messages disseminated by violent extremist groups. While many such initiatives have been launched by Western governments, civil society actors, and private companies, there are many questions regarding their efficacy. Focusing predominantly on efforts countering Salafi-Jihadi extremism, this book examines how feasible it is to prevent or counter radicalization and violent extremism with counter-messaging efforts. It investigates important principles to consider when devising such a program. The authors provide both a comprehensive theoretical overview and a review of the available literature, as well as policy recommendations for governments and the role they can play in counter-narrative efforts. As this is the first book to critically examine the possibilities and pitfalls of using counter-messaging to prevent radicalization or stimulate de-radicalization, it is essential reading for policymakers and professionals dealing with this issue, as well as researchers in the field.