Online Library

VOX-Pol Blog Post.

As efforts to identify and remove online terrorist and violent extremist content have intensified, concern has also grown about so‐called lawful but awful content. Various options have been touted for reducing the visibility of this borderline content, including removing it from search and recommendation algorithms, downranking it and redirecting those who search for it. This article contributes to this discussion by considering the moderation of such content, in terms of three sets of values. First, definitional clarity. This is necessary to provide users with fair warning of what content is liable to moderation and to place limits on the discretion of content moderators. Yet, at present, definitions of borderline content are vague and imprecise. Second, necessity and proportionality. While downranking and removal from search and recommender algorithms should be distinguished from deplatforming, tech companies’ efforts to deamplify borderline content give rise to many of the same concerns as content removal and account shutdowns. Third, transparency. While a number of platforms now publish their content moderation policies and transparency data reports, these largely focus on violative, not borderline content. Moreover, there remain questions around access to data for independent researchers and transparency at the level of the individual user.

The whitepaper has three parts. The first part provides some contextual background, describing the diverse range of online services utilised by terrorists and extremists and the process by which propaganda is disseminated online. The second part details industry responses. As well as referrals from users and law enforcement, it describes the use of AI for proactive detection and collaborative, cross-platform initiatives. The third part describes four issues for discussion: transparency; definitional clarity; the impact on those targeted; and, the use of online data for predictive purposes.

It is well established that jihadist groups and their supporters post URLs on online platforms to outlink to items of propaganda stored on other platforms. Industry initiatives – such as Global Internet Forum to Counter Terrorism’s inclusion of URLs in its hash-sharing database, and Tech Against Terrorism’s Terrorist Content Analytics Platform – have sought to counter this practice. These measures, together with new regulatory regimes (e.g. the EU’s Terrorist Content Online Regulation) and the growing use of decentralised services, raise the question whether jihadist groups’ propaganda dissemination strategies are perhaps being forced to evolve. This report considers whether there is evidence of such an evolution, by examining the means that three jihadist groups (Islamic State (IS), Al-Qaeda (AQ) and Al-Shabaab) used to disseminate their propaganda during a two-month period in early 2023.

Images are known to have important effects on human perception and persuasion. Jihadist groups are also known to make strategic use of emotive imagery and symbolism for persuasive ends. Yet until recently studies of the online magazines published by violent jihadist groups largely focused on their textual, not their image, content and, while the image content of these magazines is now the subject of a burgeoning number of studies, few of these compare the images used by different groups. This article accordingly offers a cross-group comparison, examining the image content of a total of thirty-nine issues of five online magazines published by four different jihadist groups. Starting with a content analysis, it shows that the images’ most common focus is non-leader jihadis. Using a news values analysis, it then shows how these images of non-leader jihadis are used to visually construct the identity of a “good Muslim.” This construct is characterized by three traits, each corresponding to a different news value: fulfilled (personalization); active (consonance); and respected (prominence). Moreover, these traits are intertwined: fulfillment comes from responding actively to the call to violent jihad, which in turn promises respect. The article concludes by highlighting some subtle differences between how the news values of personalization, consonance, and prominence are realized in the different magazines, and by discussing the implications of the “good Muslim” construct for efforts to develop countermessages.

To inform the discussion, the paper draws on the debates that have surrounded the U.K. ‘Encouragement of Terrorism’ criminal offence. Created by the Terrorism Act 2006, and recently amended by the Counter-Terrorism and Border Security Act 2019, this offence has proved controversial from its inception for two principal reasons. First, the offence expressly encompasses both direct and indirect encouragement. Critics have argued that the concept of indirect encouragement is too nebulous and gives the offence too wide a scope. Second, the framing of the offence focuses not on the purpose of the speaker, but on whether the potential effect of the statement is to encourage terrorism.