Online Library

VOX-Pol Blog Post.

VOX-Pol Blog post.

Hate speech and toxic communication online is on the rise. Responses to this issue tend to offer technical (automated) or non-technical (human content moderation) solutions, or see hate speech as a natural product of hateful people. In contrast, this article begins by recognizing platforms as designed environments that support particular practices while discouraging others. In what ways might these design architectures be contributing to polarizing, impulsive, or antagonistic behaviors? Two platforms are examined: Facebook and YouTube. Based on engagement, Facebook’s Feed drives views but also privileges incendiary content, setting up a stimulus–response loop that promotes outrage expression. YouTube’s recommendation system is a key interface for content consumption, yet this same design has been criticized for leading users towards more extreme content. Across both platforms, design is central and influential, proving to be a productive lens for understanding toxic communication.

Did the Internet play a decisive role in Anders Behring Breivik’s violent radicalization? It has proven difficult to understand if and how the Internet influences radicalization processes leading to political violence (Conway 2012). The Internet constitutes only one out of a wide range of factors with a potential influence on radical and violent behavior. We also lack detailed empirical data about the online lives of modern terrorists.

The case of the Norwegian far-right terrorist Anders Behring Breivik offers unique insights into the online activities of a terrorist who used the Internet and social media in almost every thinkable way. Not only did Breivik compile his 1516-pages long compendium based exclusively on Internet sources. Before the attacks, he was also an active discussant on a number of mainstream and extremist Internet forums, and a highly dedicated online gaming enthusiast.

This article reviews new sources on Breivik’s Internet adventures and road to militancy. It is primarily based on Breivik’s original posts and comments on various Internet discussion forums between 2002 and 2011.[1] In addition, Breivik’s trial hearings introduced a wealth of new information regarding his use of the Internet.[2] Finally, the article draws on a collection of Breivik’s private e-mails which was forwarded by Norwegian hackers to a Norwegian journalist six days after the terrorist attacks. A synthesis of the more than 7000 e-mails was later published as a book (Stormark 2012).

A key finding in this study is that Breivik likely never discussed his terrorist plans with anyone online. Moreover, his comments on various Internet forums do not stand out as particularly when compared to typical far-right online discourse. In other words, Norwegian security authorities would likely not react to his online postings even if he was being monitored.

Breivik’s online posts also indicate that his critical views on Islam and socialism had been established long before the so-called counterjihad blogs were created. This means that these blogs may have played a less decisive role for Breivik’s early radicalization than assumed by many. Later on, however, these blogs certainly strengthened Breivik’s radical thinking, although they come across as far less radical than his own ideological statements after 22 July.[3]

Breivik’s e-mail correspondence shows that he first and foremost wanted to become a professional author and publisher. He proposed to establish a so-called cultural conservative paper journal together with Norwegian bloggers he admired, who were also critical of Islam and multiculturalism. He also tried to impress the Norwegian blogger Peder Are Nøstvold Jensen, better known as Fjordman, with his book project, but was given a cold shoulder. The fact that he was rejected by several of the people he looked up to may have had a decisive influence on his violent radicalization.

Breivik gathered all the necessary information to build his bomb online. He also financed the terrorist attacks through an online company, and used the Internet, in particular e-Bay, to buy materials such as body armor, weapons components and bomb ingredients. Breivik also systematically used social media platforms such as Facebook and Twitter for propaganda purposes.

Finally, Breivik was an extremely dedicated online gaming enthusiast. Playing online games dominated his daily life during the years leading up to the attacks. One cannot dismiss theories that the extreme amount of time spent on playing online games while being isolated from friends and relatives may have had an impact on his disposition to engage in extreme violence.

In the following sections, this article article describes Breivik’s use of the Internet and social media along four dimensions: (1) online radicalization, (2) online gaming, (3) online attack preparations, and (4) online propaganda.

Online social media platforms generally attempt to mitigate hateful expressions, as these comments can be detrimental to the health of the community. However, automatically identifying hateful comments can be challenging. We manually label 5,143 hateful expressions posted to YouTube and Facebook videos among a dataset of 137,098 comments from an online news media. We then create a granular taxonomy of different types and targets of online hate and train machine learning models to automatically detect and classify the hateful comments in the full dataset. Our contribution is twofold: 1) creating a granular taxonomy for hateful online comments that includes both types and targets of hateful comments, and 2) experimenting with machine learning, including Logistic Regression, Decision Tree, Random Forest, Adaboost, and Linear SVM, to generate a multiclass, multilabel classification model that automatically detects and categorizes hateful comments in the context of online news media. We find that the best performing model is Linear SVM, with an average F1 score of 0.79 using TF-IDF features. We validate the model by testing its predictive ability, and, relatedly, provide insights on distinct types of hate speech taking place on social media.

Social media systems allow Internet users a congenial platform to freely express their thoughts and opinions. Although this property represents incredible and unique communication opportunities, it also brings along important challenges. Online hate speech is an archetypal example of such challenges. Despite its magnitude and scale, there is a significant gap in understanding the nature of hate speech on social media. In this paper, we provide the first of a kind systematic large scale measurement study of the main targets of hate speech in online social media. To do that, we gather traces from two social media systems: Whisper and Twitter. We then develop and validate a methodology to identify hate speech on both these systems. Our results identify online hate speech forms and offer a broader understanding of the phenomenon, providing directions for prevention and detection approaches.