Online Library

Debates around the effectiveness of high-profile Twitter account suspensions and similar bans on abusive users across social media platforms abound. Yet we know little about the effectiveness of warning a user about the possibility of suspending their account as opposed to outright suspensions in reducing hate speech. With a pre-registered experiment, we provide causal evidence that a warning message can reduce the use of hateful language on Twitter, at least in the short term. We design our messages based on the literature on deterrence, and test versions that emphasize the legitimacy of the sender, the credibility of the message, and the costliness of being suspended. We find that the act of warning a user of the potential consequences of their behavior can significantly reduce their hateful language for one week. We also find that warning messages that aim to appear legitimate in the eyes of the target user seem to be the most effective. In light of these findings, we consider the policy implications of platforms adopting a more aggressive approach to warning users that their accounts may be suspended as a tool for reducing hateful speech online.

VOX-Pol Blog post.

The Islamic State (ISIS) was uniquely effective among extremist groups in the Middle East at recruiting Westerners. A major way ISIS accomplished this was by adopting Hollywood-style narrative structures for their propaganda videos. In particular, ISIS utilized a heroic martyr narrative, which focuses on an individual’s personal glory and empowerment, in addition to traditional social martyr narratives, which emphasize duty to kindred and religion. The current work presented adult participants (n = 238) video clips from ISIS propaganda which utilized either heroic or social martyr narratives and collected behavioral measures of appeal, narrative transportation, and psychological dispositions (egoism and empathy) associated with attraction to terrorism. Narrative transportation and the interaction between egoism and empathy predicted video recruitment appeal. A subset of adults (n = 80) underwent electroencephalographic (EEG) measurements while watching a subset of the video-clips. Complementary univariate and multivariate techniques characterized spectral power density differences when perceiving the different types of narratives. Heroic videos show increased beta power over frontal sites, and globally increased alpha. In contrast, social narratives showed greater frontal theta, an index of negative feedback and emotion regulation. The results provide strong evidence that ISIS heroic narratives are specifically processed, and appeal to psychological predispositions distinctly from other recruitment narratives.

Much research has focused on the role of the alt-right in pushing far-right narratives into mainstream discourse. In this work, we focus on the alt-right’s effects on extremist narratives themselves. From 2012 to 2017, we find a rise in alt-right, 4chan-like discourse styles across multiple communication platforms known for white supremacist extremism, such as Stormfront. This discourse style incorporates inflammatory insults, irreverent comments, and talk about memes and online “chan” culture itself. A network analysis of one far-right extremist platform suggests that central users adopt and spread this alt-right style. This analysis has implications for understanding influence and change in online white supremacist extremism, as well as the role of style in white supremacist communications. Warning: This paper contains examples of hateful and offensive language.

VOX-Pol Blog Post.

Is it possible to guarantee users online safety while also guaranteeing the right to freedom of expression? If there are limits to be placed on speech, who decides? In her keynote lecture of the VOX-Pol summer school on topics in violent online political extremism hosted by the Center for Media, Data and Society at the CEU School of Public Policy, Jillian C. York examines these questions, and looks at the current state of freedom of expression on the Internet, as well as the grave threats it faces. Jillian C. York is a writer and activist focused on the intersection of technology and policy. Based in Berlin, she serves as the Director for International Freedom of Expression at the Electronic Frontier Foundation, where she works on issues of free expression, privacy, and digital security. Jillian is also a fellow at the Center for Internet & Human Rights at the European University Viadrina, and a member of the Deep Lab collective.