Online Library

VOX-Pol Blog post.

Digital platforms were slow to build robust teams to counter threat actors, but today, many of those corporate teams have robust processes, specialized tools, and innovative approaches to countering highly adaptive adversaries. They operate in a tremendously dynamic environment where their adversaries can innovate at low cost, primarily because of the nature of the digital “terrain” where the conflict occurs. And while the actions these teams take are not kinetic, as those sometimes utilized in geopolitical conflict, the cat-and-mouse game between Trust & Safety teams and threat actors online suggests lessons that are increasingly relevant to the national security community. This article explores five factors that were key to facilitating innovation in Facebook’s approach to countering the Islamic State—and that I argue are more generalizable. They are: people, organization, legitimacy, tools, and collaboration. It also identifies lessons that can be learned from that experience. For example, we did not prioritize using a particular technology or focus experimentation in some bespoke “innovation center.” Rather, we succeeded because we were made responsible for a critical mission, were unencumbered by past process, and had the right team structured to reduce external dependencies for innovation. Basic technological innovation can occur in an ivory tower, but applied innovation requires proximity to real-world missions. You cannot expect dramatic innovation without failure and iteration in an environment of real responsibility. Fundamentally, that means that innovation requires accepting risk. The structures and incentives of Silicon Valley cannot and should not simply be grafted on to our national security infrastructure. The rewards and costs of failure are different. But military organizations should shoulder the risks associated with innovation and study the lessons of corollary efforts in Silicon Valley and the private sector more broadly.

Far-right extremists use the internet to recruit and connect with other radicalised individuals and spread their propaganda online, with them being prolific users of online forums and social media. Such online data can be analysed using linguistic tools to identify markers of radicalisation in social media, forum posts, and other extremist texts. Few studies have examined how language and sentiment online evolves in extremist forums, which provides insight into how radicalisation into extremism unfolds overtime. To address this gap, we examine how the extremist language used by Australian members of the online far-right forum Iron March changed in relation to the level of engagement on the forum, as well as assessing whether any relationship existed between online extremist sentiment and offline violence. Forum data were computationally coded and analysed using the Grievance Dictionary, which is designed to identify psycholinguistic indicators of extremism. Results indicate that while extremist language changes over time, there was no support for the suggestion that greater forum engagement results in higher levels of extremist sentiment. The results found that a high level of online extremist sentiment is associated with offline violence. Findings are linked to existing literature, and limitations in the study design are acknowledged.

This project by GNET looks at the user journeys of individuals who enter and participate in the online spaces of extremist communities. A user journey here refers to the distinct path a user may follow to reach their goals when accessing and using an online space belonging to extremist communities.

User journeys are particularly important in offering insights into the rationale and motivations of users on the one hand, and to the inner workings of extremist online communities on the other. This is vital for understanding their goals and objectives.

In selecting the ideologies for this project, we drew upon extremist communities – rather than extremist and terrorist organisations or groups – including those actors that participate in the extremist milieu and share ideas but do not necessarily operate in concert. These ideologies include those of formal and well‐defined extremist organisations of White supremacist and anti‐government extremist groups in the United States, supporter networks of Islamic State (IS), and looser communities of extremist actors, including accelerationists, incels and chan site members who operate on shared platforms, congregating around common beliefs without the connection of formal membership.

This project is a response to the growing interest in understanding how individuals enter and participate in online spaces of extremist communities. A core goal of the project was to understand the role of algorithms in leading users to extremist communities, including the changes in algorithmic recommendations that lead users to more extreme content online. However, examining these changes proved impossible due to the precautions taken by the expert contributors to the project, such as the use of separate technology and VPNs throughout their research.

The project also highlights the distinct posting behaviour and operational security protocols of different groups, usually along ideological lines.

Societal crises, such as COVID-19, produce societal instability and create a fertile ground for radicalization. Extremists exploit such crises by distributing disinformation to amplify uncertainty and distrust. Based on these developments, this study presents a longitudinal analysis across three different non-violent extremist ideologies in the UK (Islamist, far right and eco-radicals). As part of the study, public social media channels Twitter/X, Facebook, and Telegram) of Britain First, 5 Pillars, and Earth First! were analyzed using a computational language classifier of over 36,000 posts between 2016 and 2021. The increasing prevalence of conspiracy narratives, as well as violent, hateful, and threatening language among the corpus indicates that radicalization dynamics were present and heightened, pre-, during, and post-Pandemic.

Psychological theories of mobilization tend to focus on explaining people’s motivations for action, rather than mobilization (“activation”) processes. To investigate the online behaviors associated with mobilization, we compared the online communications data of 26 people who subsequently mobilized to right-wing extremist action and 48 people who held similar extremist views but did not mobilize (N = 119,473 social media posts). In a three-part analysis, involving content analysis (Part 1), topic modeling (Part 2), and machine learning (Part 3), we showed that communicating ideological or hateful content was not related to mobilization, but rather mobilization was positively related to talking about violent action, operational planning, and logistics. Our findings imply that to explain mobilization to extremist action, rather than the motivations for action, theories of collective action should extend beyond how individuals express grievances and anger, to how they equip themselves with the “know-how” and capability to act.