Online Library

Debates around the effectiveness of high-profile Twitter account suspensions and similar bans on abusive users across social media platforms abound. Yet we know little about the effectiveness of warning a user about the possibility of suspending their account as opposed to outright suspensions in reducing hate speech. With a pre-registered experiment, we provide causal evidence that a warning message can reduce the use of hateful language on Twitter, at least in the short term. We design our messages based on the literature on deterrence, and test versions that emphasize the legitimacy of the sender, the credibility of the message, and the costliness of being suspended. We find that the act of warning a user of the potential consequences of their behavior can significantly reduce their hateful language for one week. We also find that warning messages that aim to appear legitimate in the eyes of the target user seem to be the most effective. In light of these findings, we consider the policy implications of platforms adopting a more aggressive approach to warning users that their accounts may be suspended as a tool for reducing hateful speech online.

The internet clearly matters to terrorists, but online content by itself rarely causes people to carry out terrorist attacks. Responses should therefore not be limited to the mass removal of terrorist content from online platforms.

Extreme militant groups use their media campaigns to share information, recruit and radicalize followers, share worldviews, and seek public diplomacy ends. While previous research documents that various on-the-ground events correspond to changes in the groups’ messaging strategies, studies of how competing militant groups influence one another’s media campaigns are nascent. This study helps fill that gap by examining how successful attacks by one militant group correspond to changes in both the perpetrating and competing groups’ visual media messaging strategies. It examines attack success through the lens of violent acts that result in direct impact (measured through death counts) and indirect impact (measured through traditional media coverage levels). The study utilizes a content analysis of 1882 authority-related images in AQAP’s al-Masra newsletter and ISIS’s al-Naba’ newsletter appearing three issues before and after each attack, and a chi-square analysis comparing four ISIS attack conditions (high death/high media, high death/low media, low death/high media, and low death/low media). The findings show that a high number of resulting deaths, rather than a high level of media coverage, correspond to changes in the media campaigns of both the perpetrators and the competing groups, with key differences in visual content based on group identity.

This working paper explores trends in extremist Facebook data from July 2023 to June 2024. We examined engagement, sentiment, and topics within Facebook groups categorized as anti-Israel/Semitic, anti-Palestine/Muslim, and anti-both, mapping these trends against five major events related to the recent Israel-Hamas conflict. Our findings support the hypothesis that shifts in trends correspond with these key events, showing varying patterns across different group categories. We observed decreased activity proportion in anti-both groups and increased activity proportion in the two one-sided hate groups at the conflict’s onset. This pattern reversed after the Israeli troop withdrawal from Khan Yunis, Gaza. During the conflict, negative content proportion surged, and neutral content proportion fell in all the three group categories. Anti-Palestine/Muslim groups’ discourses shifted from religious to social media activism and political/protest around the time the war began, while anti-Israel/Semitic groups moved from political/protest to religious topics a couple of weeks before the war.

Report from the “Information Operations and Winning the Peace” workshop, held at the U.S. Army War College (USAWC), Carlisle Barracks, Pennsylvania

Social media and tech companies face the challenge of identifying and removing terrorist and extremist content from their platforms. This paper presents the findings of a series of interviews with Global Internet Forum to Counter Terrorism (GIFCT) partner companies and law enforcement Internet Referral Units (IRUs). It offers a unique view on current practices and challenges regarding content removal, focusing particularly on human-based and automated approaches and the integration of the two.