Online Library

VOX-Pol Blog post.

VOX-Pol Blog post.

VOX-Pol Blog post.

Digital platforms were slow to build robust teams to counter threat actors, but today, many of those corporate teams have robust processes, specialized tools, and innovative approaches to countering highly adaptive adversaries. They operate in a tremendously dynamic environment where their adversaries can innovate at low cost, primarily because of the nature of the digital “terrain” where the conflict occurs. And while the actions these teams take are not kinetic, as those sometimes utilized in geopolitical conflict, the cat-and-mouse game between Trust & Safety teams and threat actors online suggests lessons that are increasingly relevant to the national security community. This article explores five factors that were key to facilitating innovation in Facebook’s approach to countering the Islamic State—and that I argue are more generalizable. They are: people, organization, legitimacy, tools, and collaboration. It also identifies lessons that can be learned from that experience. For example, we did not prioritize using a particular technology or focus experimentation in some bespoke “innovation center.” Rather, we succeeded because we were made responsible for a critical mission, were unencumbered by past process, and had the right team structured to reduce external dependencies for innovation. Basic technological innovation can occur in an ivory tower, but applied innovation requires proximity to real-world missions. You cannot expect dramatic innovation without failure and iteration in an environment of real responsibility. Fundamentally, that means that innovation requires accepting risk. The structures and incentives of Silicon Valley cannot and should not simply be grafted on to our national security infrastructure. The rewards and costs of failure are different. But military organizations should shoulder the risks associated with innovation and study the lessons of corollary efforts in Silicon Valley and the private sector more broadly.

Far-right extremists use the internet to recruit and connect with other radicalised individuals and spread their propaganda online, with them being prolific users of online forums and social media. Such online data can be analysed using linguistic tools to identify markers of radicalisation in social media, forum posts, and other extremist texts. Few studies have examined how language and sentiment online evolves in extremist forums, which provides insight into how radicalisation into extremism unfolds overtime. To address this gap, we examine how the extremist language used by Australian members of the online far-right forum Iron March changed in relation to the level of engagement on the forum, as well as assessing whether any relationship existed between online extremist sentiment and offline violence. Forum data were computationally coded and analysed using the Grievance Dictionary, which is designed to identify psycholinguistic indicators of extremism. Results indicate that while extremist language changes over time, there was no support for the suggestion that greater forum engagement results in higher levels of extremist sentiment. The results found that a high level of online extremist sentiment is associated with offline violence. Findings are linked to existing literature, and limitations in the study design are acknowledged.

This project by GNET looks at the user journeys of individuals who enter and participate in the online spaces of extremist communities. A user journey here refers to the distinct path a user may follow to reach their goals when accessing and using an online space belonging to extremist communities.

User journeys are particularly important in offering insights into the rationale and motivations of users on the one hand, and to the inner workings of extremist online communities on the other. This is vital for understanding their goals and objectives.

In selecting the ideologies for this project, we drew upon extremist communities – rather than extremist and terrorist organisations or groups – including those actors that participate in the extremist milieu and share ideas but do not necessarily operate in concert. These ideologies include those of formal and well‐defined extremist organisations of White supremacist and anti‐government extremist groups in the United States, supporter networks of Islamic State (IS), and looser communities of extremist actors, including accelerationists, incels and chan site members who operate on shared platforms, congregating around common beliefs without the connection of formal membership.

This project is a response to the growing interest in understanding how individuals enter and participate in online spaces of extremist communities. A core goal of the project was to understand the role of algorithms in leading users to extremist communities, including the changes in algorithmic recommendations that lead users to more extreme content online. However, examining these changes proved impossible due to the precautions taken by the expert contributors to the project, such as the use of separate technology and VPNs throughout their research.

The project also highlights the distinct posting behaviour and operational security protocols of different groups, usually along ideological lines.