Online Library

The whitepaper has three parts. The first part provides some contextual background, describing the diverse range of online services utilised by terrorists and extremists and the process by which propaganda is disseminated online. The second part details industry responses. As well as referrals from users and law enforcement, it describes the use of AI for proactive detection and collaborative, cross-platform initiatives. The third part describes four issues for discussion: transparency; definitional clarity; the impact on those targeted; and, the use of online data for predictive purposes.

Over the past decade, a large number of extremist and hate groups have turned to internet platforms to inspire mass violence. Currently, there is little reliable evidence on how such campaigns radicalize targeted audiences. We provide systematic, large-scale, microevidence on the effect of Islamic State propaganda on social media. We use several machine learning algorithms to detect recruitment messages in online propaganda, identify their dissemination on Twitter, and quantify the reactions of exposed users. Analyzing content produced by the Islamic State between 2015 and 2016 shows that propaganda conveying the material, spiritual, and social benefits of joining ISIS increased online support for the group, while content displaying brutal violence decreased endorsement of ISIS across a wide range of videos. Only the group’s most extreme supporters reacted positively to violent propaganda.

Mobilization is central to the emergence, survival and success of armed groups challenging the state, and has lately expanded to new arenas with the rise of social media. Using a new dataset of rebel group Twitter use, we examined the topics contained in rebel group social media communications to understand how different messaging strategies impact civilian engagement with rebel messages. Rather than benefiting solely from direct calls to action, we found that rebel groups also increased civilian engagement through indirect messages of self-promotion. While direct appeals received more engagement than indirect appeals, their effects were tempered by audience fatigue when relied on too heavily. We additionally found that including images further enhanced the impact of a mobilizing message. These findings expanded our understanding of rebel communications and mobilization, with important implications for combating the use of social media as a recruitment tool for violent extremism.

Scholarship has highlighted the rise of political influencer networks on YouTube, raising concerns about the platform’s propensity to spread and even incentivize politically extreme content. While many studies have focused on YouTube’s algorithmic infrastructure, limited research exists on the actual content in these networks. Building on Lewis’s (2018) classification of an “alternative influencer” network, we apply structural topic modeling across all text-based autocaptions from her study’s sample to identify common topics featured on these channels. This allows us to gauge which topics appear together and to trace politicization over time. Through network analysis, we determine channel similarities and evaluate whether deplatformed channels influenced topic shifts. We find that political topics increasingly dominate the focus of all analyzed channels. The convergence of culture and politics occurs mostly about identity-driven issues. Furthermore, more extreme channels do not form distinct clusters but blend into the larger content-based network. Our findings illustrate how political topics may function as connective ties across an initially more diverse network of YouTube influencer channels.

Approaches for strategically countering or providing alternatives to hate speech and extremism online have evolved substantively in the last ten years. Technological advancement and a generation of young activists who have been socialized as digital natives have facilitated a maelstrom of both hate-based extremist content and attempts to counter this material in different guises and through diverse channels. The rate and pace of change within the tech sector, and social media growth in particular, have meant that although counterspeech is now more prevalent than ever before, it requires greater guidance and more robust public–private partnerships to effectively prevent and counter extremism online. The chapter embraces a cross-platform and international overview of some of the best practices within efforts to prevent and counter violent extremism online and discusses the future of counterspeech with recommendations for expanded innovation and partnership models.