Online Library

What are the factors associated with the production of online hate material? Past research has focused on attributes associated with seeing and being targeted by online hate material, but we know surprisingly little about the creators of such material. This study seeks to address this gap in the knowledge, using a random sample of Americans, aged 15–36. Descriptive results indicate that nearly one-fifth of our sample reported producing online material that others would likely interpret as hateful or degrading. We utilize a logistic regression to understand more about these individuals. Results indicate that men are significantly more likely than women to produce online hate material. This fits with the broader pattern of men being more apt to engage in deviant and criminal behaviors, both online and offline. Other results show that the use of particular social networking sites, such as Reddit, Tumblr, and general messaging boards, is positively related to the dissemination of hate material online. Counter to expectations, the use of first-person shooter games actually decreased the likelihood of producing hate material online. This could suggest that violent videogames serve as outlet for aggression, and not a precursor. In addition, we find that individuals who are close to an online community, or spend more time in areas populated by hate, are more inclined to produce hate material. We expected that spending more time online would correlate with the production of hate, but this turned out not to be true. In fact, spending more time online actually reduces the likelihood of doing so. This result could indicate that individuals who spend more time online are focused on a particular set of tasks, as opposed to using the Internet to disseminate hate.

This paper was presented at the 2nd European Counter Terrorism Centre (ECTC) Advisory Group conference, 17-18 April 2018, at Europol Headquarters, The Hague. The views expressed are the authors’ own and do not necessarily represent those of Europol.
In a speech delivered at the United Nations General Assembly in September 2017, the U.K. Prime Minister Theresa May called on social media companies to do more to remove and block terrorist content from their platforms [1]. In the speech she stated that the average lifespan of online propaganda from the so-called Islamic State (IS) was 36 hours. For such content to be disrupted effectively, she claimed that this figure needed to be reduced to one to two hours. This has since come to be known as the ‘golden window’: if terrorist material can be detected and removed within one to two hours, its spread will be prevented.

VOX-Pol Blog post.

It is relatively easy to identify tens of thousands of social media users who have an interest in violent ideologies, but very difficult to figure out which users are worth watching. For students of extremist movements and those working to counter violent extremism online, deciphering the signal amid the noise can prove incredibly daunting. This paper sets out a first step in solving that problem. The authors have devised a scoring system to find out which social media accounts within a specific extremist circle were most influential and most prone to be influenced (a tendency we called exposure).

As liberal democracies grapple with the evolution of online political extremism, in addition to governments, social media and internet infrastructure companies have found themselves making more and more decisions about who gets to use their platforms, and what people say online. This raises the question that this paper explores, who should regulate extremist content online? In doing so the first part of the paper examines the evolution of the increasing role that social media and internet infrastructure companies have come to play in the regulating extremist content online, and the ethical challenges this presents. The second part of the paper explores three ethical challenges: i) the moral legitimacy of private actors, ii) the concentration of power in the hands of a few actors and iii) the lack of separation of powers in the content regulation process by private actors.

Who is likely to view materials online maligning groups based on race, nationality, ethnicity, sexual orientation, gender, political views, immigration status, or religion? We use an online survey (N = 1034) of youth and young adults recruited from a demographically balanced sample of Americans to address this question. By studying demographic characteristics and online habits of individuals who are exposed to online extremist groups and their messaging, this study serves as a precursor to a larger research endeavor examining the online contexts of extremism.

Descriptive results indicate that a sizable majority of respondents were exposed to negative materials online. The materials were most commonly used to stereotype groups. Nearly half of negative material centered on race or ethnicity, and respondents were likely to encounter such material on social media sites. Regression results demonstrate African-Americans and foreign-born respondents were significantly less likely to be exposed to negative material online, as are younger respondents. Additionally, individuals expressing greater levels of trust in the federal government report significantly less exposure to such materials. Higher levels of education result in increased exposure to negative materials, as does a proclivity towards risk-taking.