Online Library

VOX-Pol Guest Lecture Series, Autumn 2022

Speaker: J.M. Berger

Title: The Conceptualisation of Extremism Online

The Counter-Terrorism Committee unanimously adopted the Delhi Declaration at the closing of its special meeting on countering the use of new and emerging technologies for terrorist purposes, held in India on 28-29 October 2022.

The special meeting focused on three main areas: unmanned aerial systems, terrorist financing and information and communications technologies. The meeting also examined overarching issues relating to the use of technologies for terrorist purposes, to include human rights, gender considerations, and future programming by the international community. The Delhi Declaration provides non-binding guidance to Member States and stakeholders in countering terrorist use of new and emerging technologies.

This article examines algorithmic content moderation, using the moderation of violent extremist content as a specific case. In recent years, algorithms have increasingly been mobilized to perform essential moderation functions for online social media platforms such as Facebook, YouTube, and Twitter, including limiting the proliferation of extremist speech. Drawing on Katherine Hayles’ concept of “cognitive assemblages” and the Critical Security Studies literature, we show how algorithmic regulation operates within larger assemblages of humans and non-humans to influence the surveillance and regulation of information flows. We argue that the dynamics of algorithmic regulation are more liquid, cobbled together and distributed than it appears. It is characterized by a set of shifting human and machine entities, which mix traditional surveillance methods with more sophisticated tools, and whose linkages and interactions are transient. The processes that enable the consolidation of knowledge about risky profiles and contents are, therefore, collective and distributed among humans and machines. This allows us to argue that the cognitive assemblages involved in content moderation become a cobbled space of preemptive calculation.

Cyberhate: The Far Right in the Digital Age explores how right-wing extremists operate in cyberspace by examining their propaganda, funding, subcultures, movements, offline violence, and the ideologies that drive it. Scholars and practitioners from a wide range of disciplines and professions including criminal justice, psychology, cybersecurity, religion, law, education, and terrorism studies contribute to provide an extensive analysis of the far-right online political landscape. Specific topics include laws surrounding cyberhate, propaganda, bitcoin funding, online subcultures such as the manosphere, theories that explain why some take the path of violence, and specific movements including the alt-right and the terroristic Atomwaffen Division. Relying on manifestos and other correspondence posted online by recent perpetrators of mass murder, this book focuses on specific groups, individuals, and acts of violence to explain how concepts like “white genocide” and incel ideology have motivated recent deadly violence.

The rise of the Internet has dramatically increased the degree to which youth may be exposed to online hate content, and simplified the process of sharing this content with others. Viewing messages that contain hate speech or language vilifying others can increase an individual’s risk of radicalization to extremist views and the acceptance of violent ideologies. Researchers have begun to explore the risk factors for exposure to such content, with prior studies demonstrating a relationship between low self-control and online activities being important correlates. Few studies have utilized youth samples to assess these relationships, or explored the voluntary consumption and sharing of content. This study attempts to address this gap in the literature using self-report responses provided by a sample of 1,193 youths in South Australia. A series of quantitative models are estimated assessing the relationships between self-control, opportunities to view content using both on and off-line measures, and four dependent variables related to exposure to or sharing of hate content. The implications of this analysis for our understanding of the utility of criminological theory to radicalization and countering violent extremism are discussed in detail.

In this paper we examine how offline protests attended by members of the Australian far-right shape online interactions. Tweets about the 2019 St Kilda beach rally were collected. Users were manually classified as supporters (n = 104) or opponents of the rally (n = 872). Network analysis demonstrated that interactions between the two groups increased at the time of the rally. Natural language processing showed that both groups became angrier and used more “othering” language during the rally. However, there were stark differences in the moral worldviews, highlighting the very different moral positions that underpin engagement with, and opposition to, the far-right agenda.