Online Library

Online extremism has changed, and censorship by content removal or account suspension alone cannot counteract it. We need a radical new response, argues Louis Reynolds.

Content regulation and censorship of social media platforms is increasingly discussed by governments and the platforms themselves. To date, there has been little data-driven analysis of the effects of regulated content deemed inappropriate on online user behavior. We therefore compared Twitter — a popular social media platform that occasionally removes content in violation of its Terms of Service — to Gab — a platform that markets itself as completely unregulated. Launched in mid 2016, Gab is, in practice, dominated by individuals who associate with the “alt right” political movement in the United States. Despite its billing as “The Free Speech Social Network,” Gab users display more extreme social hierarchy and elitism when compared to Twitter. Although the framing of the site welcomes all people, Gab users’ content is more homogeneous, preferentially sharing material from sites traditionally associated with the extremes of American political discourse, especially the far right. Furthermore, many of these sites are associated with state-sponsored propaganda from foreign governments. Finally, we discovered a significant presence of German language posts on Gab, with several topics focusing on German domestic politics, yet sharing significant amounts of content from U.S. and Russian sources. These results indicate possible emergent linkages between domestic politics in European and American far right political movements. Implications for regulation of social media platforms are discussed.

In this brief, ARTICLE 19 seeks to contribute to discussions on greater regulation of social media platforms, including calls for such platforms to be considered publishers. We do so by exploring a possible model for the independent and effective self regulation of social media platforms.

ARTICLE 19 recognises that dominant social media companies hold considerable power over the flow of information and ideas online, given the vast quantities of content published on their platforms. The way in which social media companies have dealt with content issues on their platforms, especially around ‘hate speech’, has been of particular concern to many stakeholders. Under international standards on freedom of expression, however, it is a fairly complex task to decide whether a specific message can be identified as unlawful ‘hate speech’, and, as such, whether it should or could legitimately be prohibited. More generally, any restriction on freedom of expression, whatever the objective it seeks to achieve, necessarily raises a series of legal questions.

Online discussion forums provide open workspace allowing users to share information, exchange ideas, address problems, and form groups. These forums feature multimodal posts and analyzing them requires a framework that can integrate heterogeneous information extracted from the posts, i.e. text, visual content and the information about user interactions with the online platform and each other. In this paper, we develop a generic framework that can be trained to identify communication behavior and patterns in relation to an entity of interest, be it user, image or text in internet forums. As the case study we use the analysis of violent online political extremism content, which has been a major challenge for domain experts. We demonstrate the generalizability and flexibility of our framework in predicting relational information between multimodal entities by conducting extensive experimentation around four practical use cases.

This paper compares the social media posts of ISIS foreign fighters to those of ISIS supporters. We examine a random sample of social media posts made by violent foreign fighters (n = 14; 2000 posts) and non-violent supporters (n = 18; 2000 posts) of the Islamic State of Iraq and Syria (ISIS) (overall n = 4,000 posts), from 2009 to 2015. We used a mixed-method study design. Our qualitative content analyses of the 4,000 posts identified five themes: Threats to in-group, societal grievances, pursuit for significance, religion, and commitment issues. Our quantitative comparisons found that the dominant themes in the foreign fighters’ online content were threats to in-group, societal grievances, and pursuit for significance, while religion and commitment issues were dominant themes in the supporters’ online content. We also identified thematic variations reflecting individual attitudes that emerged during the 2011–2015 period, when major geopolitical developments occurred in Syria and Iraq. Finally, our quantitative sentiment-based analysis found that the supporters (10 out of 18; 56%) posted more radical content than the foreign fighters (5 out of 14; 36%) on social media.

Terror attacks have been linked in part to online extremist content. Although tens of thousands of Islamist extremism supporters consume such content, they are a small fraction relative to peaceful Muslims. The efforts to contain the ever-evolving extremism on social media platforms have remained inadequate and mostly ineffective. Divergent extremist and mainstream contexts challenge machine interpretation, with a particular threat to the precision of classification algorithms. Our context-aware computational approach to the analysis of extremist content on Twitter breaks down this persuasion process into building blocks that acknowledge inherent ambiguity and sparsity that likely challenge both manual and automated classification. We model this process using a combination of three contextual dimensions — religion, ideology, and hate — each elucidating a degree of radicalization and highlighting independent features to render them computationally accessible. We utilize domain-specific knowledge resources for each of these contextual dimensions such as Qur’an for religion, the books of extremist ideologues and preachers for political ideology and a social media hate speech corpus for hate. Our study makes three contributions to reliable analysis: (i) Development of a computational approach rooted in the contextual dimensions of religion, ideology, and hate that reflects strategies employed by online Islamist extremist groups, (ii) An in-depth analysis of relevant tweet datasets with respect to these dimensions to exclude likely mislabeled users, and (iii) A framework for understanding online radicalization as a process to assist counter-programming. Given the potentially significant social impact, we evaluate the performance of our algorithms to minimize mislabeling, where our approach outperforms a competitive baseline by 10.2% in precision.