Online Library

Online social media platforms generally attempt to mitigate hateful expressions, as these comments can be detrimental to the health of the community. However, automatically identifying hateful comments can be challenging. We manually label 5,143 hateful expressions posted to YouTube and Facebook videos among a dataset of 137,098 comments from an online news media. We then create a granular taxonomy of different types and targets of online hate and train machine learning models to automatically detect and classify the hateful comments in the full dataset. Our contribution is twofold: 1) creating a granular taxonomy for hateful online comments that includes both types and targets of hateful comments, and 2) experimenting with machine learning, including Logistic Regression, Decision Tree, Random Forest, Adaboost, and Linear SVM, to generate a multiclass, multilabel classification model that automatically detects and categorizes hateful comments in the context of online news media. We find that the best performing model is Linear SVM, with an average F1 score of 0.79 using TF-IDF features. We validate the model by testing its predictive ability, and, relatedly, provide insights on distinct types of hate speech taking place on social media.

VOX-Pol Guest Lecture Series, Spring 2023

Speaker: Dr. Erin Saltman

Title: Cross-Platform Approaches to Countering Terrorism and Violent Extremism Online

Outline: This presentation reviews advances in cross-platform solutions advanced through the Global Internet Forum to Counter Terrorism (GIFCT). GIFCT was founded in 2017 by technology platforms to give structure and support for countering terrorism and violent extremism online. This includes workstreams for (1) multi stakeholder knowledge-sharing, (2) developing action-oriented research, (3) building cross-platform tech solutions, and (4) managing shared incident response protocols. The presentation reviews the state of play in adversarial shifts by violent extremist groups online and what this means for the evolution of counter measures in the online space. It is followed up by a Q&A covering topics such as transparency, the technical aspects of the Hash Sharing Database, and the ethics of funding.

Approaches for strategically countering or providing alternatives to hate speech and extremism online have evolved substantively in the last ten years. Technological advancement and a generation of young activists who have been socialized as digital natives have facilitated a maelstrom of both hate-based extremist content and attempts to counter this material in different guises and through diverse channels. The rate and pace of change within the tech sector, and social media growth in particular, have meant that although counterspeech is now more prevalent than ever before, it requires greater guidance and more robust public–private partnerships to effectively prevent and counter extremism online. The chapter embraces a cross-platform and international overview of some of the best practices within efforts to prevent and counter violent extremism online and discusses the future of counterspeech with recommendations for expanded innovation and partnership models.

The counterterrorism and CVE community has long questioned the effectiveness of counterspeech in countering extremism online. While most evaluation of counterspeech rely on limited reach and engagement metrics, this paper explores two models to better measure behavioral change and sentiment analysis. Conducted via partnerships between Facebook and counter-extremism NGOs, the first model uses A/B testing to analyze the effects of counterspeech exposure on low-prevalence-high-risk audiences engaging with Islamist extremist terrorist content. The second model builds upon online safety intervention approaches and the Redirect Method through a search based “get-help” module, redirecting white-supremacy and Neo-Nazi related search-terms to disengagement NGOs.

The counterterrorism and CVE community has long questioned the effectiveness of counterspeech in countering extremism online. While most evaluation of counterspeech rely on limited reach and engagement metrics, this paper explores two models to better measure behavioral change and sentiment analysis. Conducted via partnerships between Facebook and counter-extremism NGOs, the first model uses A/B testing to analyze the effects of counterspeech exposure on low-prevalence-high-risk audiences engaging with Islamist extremist terrorist content. The second model builds upon online safety intervention approaches and the Redirect Method
through a search based “get-help” module, redirecting whitesupremacy and Neo-Nazi related search-terms to disengagement NGOs.

This paper introduces the concept of “imagined extremist communities,” a term that encapsulates the unique social landscape where right-wing lone actors, despite not being affiliated with organised groups, partake in a form of communal interaction. By examining the cases of Anders Behring Breivik, Brenton Harrison Tarrant, and Philip Manshaus, this paper illuminates how group-based and lone actors are more alike than what is conventionally expressed in existing research. Although lone actors are not subject to an external command like group-based actors are, the imagined extremist community functions as a “group” for lone actors and is, for all practical purposes, a corresponding alternative to a terror cell. During the radicalisation process, these individuals seek and turn to the imagined extremist community, enabling them to form a sense of belonging and identification and underscoring that these actors, although conventionally labelled as “lone,” are immersed in an alternative culture that nurtures their ideas and sustains their extremist ideology. This becomes particularly evident through their cognitive radicalisation, a process amplified by their psychological predispositions. The concept of the imagined extremist community elucidates how lone actors, especially those embracing right-wing ideologies, are subject to radical influences. Their conservative traits and psychological dispositions make them particularly receptive to the appeal of such communities.