Online Library

The whitepaper has three parts. The first part provides some contextual background, describing the diverse range of online services utilised by terrorists and extremists and the process by which propaganda is disseminated online. The second part details industry responses. As well as referrals from users and law enforcement, it describes the use of AI for proactive detection and collaborative, cross-platform initiatives. The third part describes four issues for discussion: transparency; definitional clarity; the impact on those targeted; and, the use of online data for predictive purposes.

As efforts to identify and remove online terrorist and violent extremist content have intensified, concern has also grown about so‐called lawful but awful content. Various options have been touted for reducing the visibility of this borderline content, including removing it from search and recommendation algorithms, downranking it and redirecting those who search for it. This article contributes to this discussion by considering the moderation of such content, in terms of three sets of values. First, definitional clarity. This is necessary to provide users with fair warning of what content is liable to moderation and to place limits on the discretion of content moderators. Yet, at present, definitions of borderline content are vague and imprecise. Second, necessity and proportionality. While downranking and removal from search and recommender algorithms should be distinguished from deplatforming, tech companies’ efforts to deamplify borderline content give rise to many of the same concerns as content removal and account shutdowns. Third, transparency. While a number of platforms now publish their content moderation policies and transparency data reports, these largely focus on violative, not borderline content. Moreover, there remain questions around access to data for independent researchers and transparency at the level of the individual user.

Cooperation between law enforcement and tech companies is widely regarded as necessary to tackle online terrorist content. Both sectors have publicly stated their commitment to working together and there are examples of mutual cooperation. Yet there are also impediments to such collaboration, including different cultures and operating practices, and there have been high-profile instances of non-cooperation. The informality of existing collaborations has also led to concerns about censorship, mission creep and a lack of accountability and oversight.

The focus of this report is on how to resolve the impediments to closer cooperation between law enforcement and the tech sector in order to realise the benefits of mutual collaboration, while simultaneously addressing concerns about due process and accountability. The report utilises an interview-based methodology to examine the experiences and opinions of personnel from both sectors who have first-hand experience of mutual cooperation. It provides empirically grounded insights into this under-researched topic.