Online Library

Paul Wilkinson Memorial Lecture, University of St. Andrews, 16 April 2015

This article discusses the reasons why content moderation and control in cyberspace, being a primary means of online countering violent extremism (CVE), does not work as intended. It makes four main arguments. Firstly, despite years of efforts from CVE stakeholders, the propaganda of militant Islamist VEOs is still easily accessible on the Internet. This study has mapped hundreds of addresses engaged in digital jihadist activities at the turn of 2020–2021. It effectively proves that the current approach to online CVE brought few tangible effects. Secondly, these programs have been unevenly applied to militant Islamist organizations. There have been groups which represent violent extremist ideology but are not actively combated by stakeholders. As a result, these groups have been able to establish a solid foothold on the surface web. Thirdly, the efficiency of CVE strategies based on content takedowns are decreased by the relative ease of terrorists reestablishing banned communication channels, the availability of vast alternatives for online propaganda dissemination, the introduction of impractical legislation by governments, overlapping jurisdictions on the Internet and the “Streisand effect.” Finally, adequate strategies aimed at curbing digital jihad should be primarily based on offline and online activities which fall outside of the remit of CVE.

Who is likely to view materials online maligning groups based on race, nationality, ethnicity, sexual orientation, gender, political views, immigration status, or religion? We use an online survey (N = 1034) of youth and young adults recruited from a demographically balanced sample of Americans to address this question. By studying demographic characteristics and online habits of individuals who are exposed to online extremist groups and their messaging, this study serves as a precursor to a larger research endeavor examining the online contexts of extremism.

Descriptive results indicate that a sizable majority of respondents were exposed to negative materials online. The materials were most commonly used to stereotype groups. Nearly half of negative material centered on race or ethnicity, and respondents were likely to encounter such material on social media sites. Regression results demonstrate African-Americans and foreign-born respondents were significantly less likely to be exposed to negative material online, as are younger respondents. Additionally, individuals expressing greater levels of trust in the federal government report significantly less exposure to such materials. Higher levels of education result in increased exposure to negative materials, as does a proclivity towards risk-taking.

Who is likely to view materials online maligning groups based on race, nationality, ethnicity, sexual orientation, gender, political views, immigration status, or religion? We use an online survey (N = 1034) of youth and young adults recruited from a demographically balanced sample of Americans to address this question. By studying demographic characteristics and online habits of individuals who are exposed to online extremist groups and their messaging, this study serves as a precursor to a larger research endeavor examining the online contexts of extremism. Descriptive results indicate that a sizable majority of respondents were exposed to negative materials online. The materials were most commonly used to stereotype groups. Nearly half of negative material centered on race or ethnicity, and respondents were likely to encounter such material on social media sites. Regression results demonstrate African-Americans and foreign-born respondents were significantly less likely to be exposed to negative material online, as are younger respondents. Additionally, individuals expressing greater levels of trust in the federal government report significantly less exposure to such materials. Higher levels of education result in increased exposure to negative materials, as does a proclivity towards risk-taking.

As liberal democracies grapple with the evolution of online political extremism, in addition to governments, social media and internet infrastructure companies have found themselves making more and more decisions about who gets to use their platforms, and what people say online. This raises the question that this paper explores, who should regulate extremist content online? In doing so the first part of the paper examines the evolution of the increasing role that social media and internet infrastructure companies have come to play in the regulating extremist content online, and the ethical challenges this presents. The second part of the paper explores three ethical challenges: i) the moral legitimacy of private actors, ii) the concentration of power in the hands of a few actors and iii) the lack of separation of powers in the content regulation process by private actors.

It is relatively easy to identify tens of thousands of social media users who have an interest in violent ideologies, but very difficult to figure out which users are worth watching. For students of extremist movements and those working to counter violent extremism online, deciphering the signal amid the noise can prove incredibly daunting. This paper sets out a first step in solving that problem. The authors have devised a scoring system to find out which social media accounts within a specific extremist circle were most influential and most prone to be influenced (a tendency we called exposure).