Online Library

This project by GNET looks at the user journeys of individuals who enter and participate in the online spaces of extremist communities. A user journey here refers to the distinct path a user may follow to reach their goals when accessing and using an online space belonging to extremist communities.

User journeys are particularly important in offering insights into the rationale and motivations of users on the one hand, and to the inner workings of extremist online communities on the other. This is vital for understanding their goals and objectives.

In selecting the ideologies for this project, we drew upon extremist communities – rather than extremist and terrorist organisations or groups – including those actors that participate in the extremist milieu and share ideas but do not necessarily operate in concert. These ideologies include those of formal and well‐defined extremist organisations of White supremacist and anti‐government extremist groups in the United States, supporter networks of Islamic State (IS), and looser communities of extremist actors, including accelerationists, incels and chan site members who operate on shared platforms, congregating around common beliefs without the connection of formal membership.

This project is a response to the growing interest in understanding how individuals enter and participate in online spaces of extremist communities. A core goal of the project was to understand the role of algorithms in leading users to extremist communities, including the changes in algorithmic recommendations that lead users to more extreme content online. However, examining these changes proved impossible due to the precautions taken by the expert contributors to the project, such as the use of separate technology and VPNs throughout their research.

The project also highlights the distinct posting behaviour and operational security protocols of different groups, usually along ideological lines.

Several alternative social media platforms have emerged in response to perceptions that mainstream platforms are censoring traditional conservative ideologies. However, many of these alternative social media platforms have evolved to be outlets for hate speech and violent extremism. This study examines hate-based channels on Telegram from a US perspective. While Telegram has often been studied in relation to ISIS, less is known about its usage by US extremist users and movements. The authors used OSINT and observational methods on a sample of 125 Telegram channels containing hate speech and violent extremist content from far-right and far-left perspectives. The authors hypothesized that there would be a greater and growing presence of far-right activity compared to farleft activity due to current migration trends away from mainstream social media by the far-right. The authors also sought to observe the presence of disinformation campaigns, conspiracy theories, and accelerationism on Telegram. This study had four major findings: (1) the findings supported the hypothesis that more channels were host to farright dialogues, yet there were several far-left channels present, (2) 64.8% of the channels grew in size over a oneweek period, (3) 47 of the 125 channels were connected to well-known violent extremist movements or hate groups, and (4) QAnon and the COVID-19 pandemic were the most prominent sources of disinformation and conspiracy theories on Telegram. The findings of this study highlight that alternative social media platforms are a growing environment for a range of hateful ideologies and are aiding the spread of disinformation campaigns. This study concludes with a discussion on future strategies to combat the influence of the Internet on radicalization outcomes.

US Department of Defense Anti-Terrorism Handbook 2004

Hateful and antagonistic content published and propagated via the World Wide Web has the potential to cause harm and suffering on an individual basis, and lead to social tension and disorder beyond cyber space. Despite new legislation aimed at prosecuting those who misuse new forms of communication to post threatening, harassing, or grossly offensive language – or cyber hate – and the fact large social media companies have committed to protecting their users from harm, it goes largely unpunished due to difficulties in policing online public spaces. To support the automatic detection of cyber hate online, specifically on Twitter, we build multiple individual models to classify cyber hate for a range of protected characteristics including race, disability and sexual orientation. We use text parsing to extract typed dependencies, which represent syntactic and grammatical relationships between words, and are shown to capture ‘othering’ language – consistently improving machine classification for different types of cyber hate beyond the use of a Bag of Words and known hateful terms. Furthermore, we build a data-driven blended model of cyber hate to improve classification where more than one protected characteristic may be attacked (e.g. race and sexual orientation), contributing to the nascent study of intersectionality in hate crime.

Hateful and antagonistic content published and propagated via the World Wide Web has the potential to cause harm and suffering on an individual basis, and lead to social tension and disorder beyond cyber space. Despite new legislation aimed at prosecuting those who misuse new forms of communication to post threatening, harassing, or grossly offensive language – or cyber hate – and the fact large social media companies have committed to protecting their users from harm, it goes largely unpunished due to difficulties in policing online public spaces. To support the automatic detection of cyber hate online, specifically on Twitter, we build multiple individual models to classify cyber hate for a range of protected characteristics including race, disability and sexual orientation. We use text parsing to extract typed dependencies, which represent syntactic and grammatical relationships between words, and are shown to capture ‘othering’ language – consistently improving machine classification for different types of cyber hate beyond the use of a Bag of Words and known hateful terms. Furthermore, we build a data-driven blended model of cyber hate to improve classification where more than one protected characteristic may be attacked (e.g. race and sexual orientation), contributing to the nascent study of intersectionality in hate crime.

Since the advent of the Internet, right-wing extremists and those who subscribe to extreme right views have exploited online platforms to build a collective identity among the like-minded. Research in this area has largely focused on extremists’ use of websites, forums, and mainstream social media sites, but overlooked in this research has been an exploration of the popular social news aggregation site Reddit. The current study explores the role of Reddit’s unique voting algorithm in facilitating “othering” discourse and, by extension, collective identity formation among members of a notoriously hateful subreddit community, r/The_Donald. The results of the thematic analysis indicate that those who post extreme-right content on r/The_Donald use Reddit’s voting algorithm as a tool to mobilize like-minded members by promoting extreme discourses against two prominent out-groups: Muslims and the Left. Overall, r/The_Donald’s “sense of community” facilitates identity work among its members by creating an environment wherein extreme right views are continuously validated.