Online Library

This paper, part of the Legal Perspectives on Tech Series, was commissioned in conjunction with the Congressional Counterterrorism Caucus.

This article contributes to public and policy debates on the value of social media disruption activity with respect to terrorist material. In particular, it explores aggressive account and content takedown, with the aim of accurately measuring this activity and its impacts. The major emphasis of the analysis is the so-called Islamic State (IS) and disruption of their online activity, but a catchall “Other Jihadi” category is also utilized for comparison purposes. Our findings challenge the notion that Twitter remains a conducive space for pro-IS accounts and communities to flourish. However, not all jihadists on Twitter are subject to the same high levels of disruption as IS, and we show that there is differential disruption taking place. IS’s and other jihadists’ online activity was never solely restricted to Twitter; it is just one node in a wider jihadist social media ecology. This is described and some preliminary analysis of disruption trends in this area supplied too.

This report seeks to contribute to public and policy debates on the value of social media disruption activity with respect to terrorist material. We look in particular at aggressive account and content takedown, with the aim of accurately measuring this activity and its impacts. Our findings challenge the notion that Twitter remains a conducive space for Islamic State (IS) accounts and communities to flourish, although IS continues to distribute propaganda through this channel. However, not all jihadists on Twitter are subject to the same high levels of disruption as IS, and we show that there is differential disruption taking place. IS’s and other jihadists’ online activity was never solely restricted to Twitter. Twitter is just one node in a wider jihadist social media ecology. We describe and discuss this, and supply some preliminary analysis of disruption trends in this area.

How does removing the leadership of online hate organizations from online platforms change behavior in their target audience? We study the effects of six network disruptions of designated and banned hate-based organizations on Facebook, in which known members of the organizations were removed from the platform, by examining the online engagements of the audience of the organization. Using a differences-in-differences approach, we show that on average the network disruptions reduced the consumption and production of hateful content, along with engagement within the network among periphery members. Members of the audience closest to the core members exhibit signs of backlash in the short term, but reduce their engagement within the network and with hateful content over time. The results suggest that strategies of targeted removals, such as leadership removal and network degradation efforts, can reduce the ability of hate organizations to successfully operate online.

In its offline aspect the broader right-wing movement is comprised of a range of groups and idealogical variances that have traditionally had difficulty coalescing into a coherent movement with broad appeal. In its online aspect, right-wing extremist practice is focused on spreading ideaology, recruiting and radicalization, and building transnational communities.

Exposure to radicalizing information has been associated with support for violent extremism. It is, however, unclear whether specific information use behavior, namely, a distinct pattern of incidental exposure (IE) to and active selection (AS) of radicalizing content, indicates stronger violent extremist attitudes and radical action intentions. Drawing on a representative general population sample (N = 1509) and applying latent class analysis, we addressed this gap in the literature. Results highlighted six types of information use behavior. The largest group of participants reported a near to zero probability of both IE to and AS of radicalizing material. Two groups of participants were characterized by high or moderate probabilities of incidental exposure as well as a low probability of active selection of radicalizing content. The remaining groups displayed either low, moderate, or high probabilities of both IE and AS. Importantly, we showed between-group differences regarding violent extremist attitudes and radical behavioral intentions. Individuals reporting near zero or high probabilities for both IE to and AS of radicalizing information expressed the lowest and strongest violent extremist attitudes and willingness to use violence respectively. Groups defined by even moderate probabilities of AS endorsed violent extremism more strongly than those for which the probability for incidental exposure was moderate or high but AS of radicalizing content was unlikely.