Online Library

A revealing and gripping investigation into how social media platforms police what we post online—and the large societal impact of these decisions. Most users want their Twitter feed, Facebook page, and YouTube comments to be free of harassment and porn. Whether faced with “fake news” or livestreamed violence, “content moderators”—who censor or promote user-posted content—have never been more important. This is especially true when the tools that social media platforms use to curb trolling, ban hate speech, and censor pornography can also silence the speech you need to hear.

In this revealing and nuanced exploration, award-winning sociologist and cultural observer Tarleton Gillespie provides an overview of current social media practices and explains the underlying rationales for how, when, and why these policies are enforced. In doing so, Gillespie highlights that content moderation receives too little public scrutiny even as it is shapes social norms and creates consequences for public discourse, cultural production, and the fabric of society. Based on interviews with content moderators, creators, and consumers, this accessible, timely book is a must-read for anyone who’s ever clicked “like” or “retweet.”

The increased threat of right-wing extremist violence necessitates a better understanding of online extremism. Radical message boards, small-scale social media platforms, and other internet fringes have been reported to fuel hatred. The current paper examines data from the right-wing forum Stormfront between 2001 and 2015. We specifically aim to understand the development of user activity and the use of extremist language. Various time-series models depict posting frequency and the prevalence and intensity of extremist language. Individual user analyses examine whether some super users dominate the forum. The results suggest that structural break models capture the forum evolution better than stationary or linear change models. We observed an increase of forum engagement followed by a decrease towards the end of the time range. However, the proportion of extremist language on the forum increased in a step-wise matter until the early summer of 2011, followed by a decrease. This temporal development suggests that forum rhetoric did not necessarily become more extreme over time. Individual user analysis revealed that super forum users accounted for the vast majority of posts and of extremist language. These users differed from normal users in their evolution of forum engagement.

A popular stereotype is that women will play more minor roles than men as environments become more dangerous and aggressive. Our analysis of new longitudinal data sets from offline and online operational networks [for example, ISIS (Islamic State)] shows that although men dominate numerically, women emerge with superior network connectivity that can benefit the underlying system’s robustness and survival. Our observations suggest new female-centric approaches that could be used to affect such networks. They also raise questions about how individual contributions in high-pressure systems are evaluated.

Exposure to radicalizing information has been associated with support for violent extremism. It is, however, unclear whether specific information use behavior, namely, a distinct pattern of incidental exposure (IE) to and active selection (AS) of radicalizing content, indicates stronger violent extremist attitudes and radical action intentions. Drawing on a representative general population sample (N = 1509) and applying latent class analysis, we addressed this gap in the literature. Results highlighted six types of information use behavior. The largest group of participants reported a near to zero probability of both IE to and AS of radicalizing material. Two groups of participants were characterized by high or moderate probabilities of incidental exposure as well as a low probability of active selection of radicalizing content. The remaining groups displayed either low, moderate, or high probabilities of both IE and AS. Importantly, we showed between-group differences regarding violent extremist attitudes and radical behavioral intentions. Individuals reporting near zero or high probabilities for both IE to and AS of radicalizing information expressed the lowest and strongest violent extremist attitudes and willingness to use violence respectively. Groups defined by even moderate probabilities of AS endorsed violent extremism more strongly than those for which the probability for incidental exposure was moderate or high but AS of radicalizing content was unlikely.

Many researchers, practitioners, and policy-makers continue to raise questions about the role of the Internet in facilitating violent extremism and terrorism. A surge in research on this issue notwithstanding, relatively few empirically grounded analyses are yet available. This chapter provides researchers with five key suggestions for progressing knowledge on the role of the Internet in facilitating violent extremism and terrorism so that we may be better placed to determine the significance of online content and activity in the latter going forward. These five suggestions relate to (1) collecting primary data across multiple types of populations; (2) making archives of violent extremist online content accessible for use by researchers and on user-friendly platforms; (3) outreaching beyond terrorism studies to become acquainted with, for example, the Internet studies literature and engaging in interdisciplinary research with, for example, computer scientists; (4) including former extremists in research projects, either as study participants or project collaborators; and (5) drawing connections between the on- and offline worlds of violent extremists.

The media frequently describes the 2017 Charlottesville ‘Unite the Right’ rally as a turning point for the alt-right and white supremacist movements. Social movement theory suggests that the media attention and public discourse concerning the rally may have engendered changes in social identity performance and visibility of the alt-right, but this has yet to be empirically tested. The presence of the movement on YouTube is of particular interest, as this platform has been referred to as a breeding ground for the alt-right. The current study investigates whether there are differences in language use between 7142 alt-right and progressive YouTube channels, in addition to measuring possible changes as a result of the rally. To do so, we create structural topic models and measure bigram proportions in video transcripts, spanning approximately 2 months before and after the rally. We observe differences in topics between the two groups, with the ‘alternative influencers’, for example, discussing topics related to race and free speech to a larger extent than progressive channels. We also observe structural breakpoints in the use of bigrams at the time of the rally, suggesting there are changes in language use within the two groups as a result of the rally. While most changes relate to mentions of the rally itself, the alternative group also shows an increase in promotion of their YouTube channels. In light of social movement theory, we argue that language use on YouTube shows that the Charlottesville rally indeed triggered changes in social identity performance and visibility of the alt-right.