Online Library

Governments and companies are turning to automated tools to make sense of what people
post on social media, for everything ranging from hate speech detection to law enforcement
investigations. Policymakers routinely call for social media companies to identify and take
down hate speech, terrorist propaganda, harassment, “fake news” or disinformation, and
other forms of problematic speech. Other policy proposals have focused on mining social
media to inform law enforcement and immigration decisions. But these proposals wrongly
assume that automated technology can accomplish on a large scale the kind of nuanced
analysis that humans can accomplish on a small scale.

Social media platforms have recently seen an increase in the occurrence of hate speech discourse which has led to calls for improved detection methods. Most of these rely on annotated data, keywords, and a classification technique. While this approach provides good coverage, it can fall short when dealing with new terms produced by online extremist communities which act as original sources of words which have alternate hate speech meanings. These code words (which can be both created and adopted words) are designed to evade automatic detection and often have benign meanings in regular discourse. As an example, “skypes”, “googles”, and “yahoos” are all instances of words which have an alternate meaning that can be used for hate speech. This overlap introduces additional challenges when relying on keywords for both the collection of data that is specific to hate speech, and downstream classification. In this work, we develop a community detection approach for finding extremist hate speech communities and collecting data from their members. We also develop a word embedding model that learns the alternate hate speech meaning of words and demonstrate the candidacy of our code words with several annotation experiments, designed to determine if it is possible to recognize a word as being used for hate speech without knowing its alternate meaning. We report an inter-annotator agreement rate of K=0.871, and K=0.676 for data drawn from our extremist community and the keyword approach respectively, supporting our claim that hate speech detection is a contextual task and does not depend on a fixed list of keywords. Our goal is to advance the domain by providing a high quality hate speech dataset in addition to learned code words that can be fed into existing classification approaches, thus improving the accuracy of automated detection.

Does social media lead vulnerable individuals to resort to violence? Many people believe it
does. And they respond with online censorship, surveillance and counter-speech. But what
do we really know about the Internet as a cause, and what do we know about the impact
of these reactions? All over the world, governments and Internet companies are making
decisions on the basis of assumptions about the causes and remedies to violent attacks. The
challenge is to have analysis and responses firmly grounded. The need is for a policy that is
constructed on the basis of facts and evidence, and not founded on hunches – or driven by
panic and fearmongering.
It is in this context that UNESCO has commissioned the study titled Youth and Violent
Extremism on Social Media – Mapping the Research. This work provides a global mapping
of research (mainly during 2012-16) about the assumed roles played by social media in
violent radicalization processes, especially when they affect youth and women. The research
responds to the belief that the Internet at large is an active vector for violent radicalization
that facilitates the proliferation of violent extremist ideologies. Indeed, much research
shows that protagonists are indeed heavily spread throughout the Internet. There is a
growing body of knowledge about how terrorists use cyberspace. Less clear, however, is
the impact of this use, and even more opaque is the extent to which counter measures are
helping to promote peaceful alternatives. While Internet may play a facilitating role, it is not
established that there is a causative link between it and radicalization towards extremism,
violent radicalization, or the commission of actual acts of extremist violence.

Today, the Commission presents guidelines and principles to increase the proactive prevention, detection and removal of illegal content inciting hatred, violence and terrorism online. The increasing availability and spreading of terrorist material and content that incites violence and hatred online is a serious threat to the security and safety of EU citizens.

This study, commissioned by the European Parliament’s Policy Department for
Citizens’ Rights and Constitutional Affairs at the request of the LIBE Committee,
provides an overview of current approaches to countering terrorist narratives. The
first and second sections outline the different responses developed at the global
and European Union levels. The third section presents an analysis of four different
approaches to responding to terrorist narratives: disruption of propaganda
distribution, redirect method, campaign and message design, and government
communications and synchronisation of message and action. The final section
offers a number of policy recommendations, highlighting five interrelated ‘lines of
effort’ essential to maximising the efficiency and effectiveness of counterterrorism
and countering violent extremism strategic communication.

This report contains findings from a study that investigated extreme far right groups’
usages of social media. This was a collaborative project building on an existing
partnership between the Departments of Linguistics and Criminology at Swansea
University, and on the development of a new partnership with the social media
analytics company ‘Blurrt’ (www.blurrt.co.uk). The project was funded by the
CHERISH-DE multidisciplinary research centre at Swansea University
(http://www.cherish-de.uk/) and the School of Arts & Humanities at Edith Cowan
University (https://www.ecu.edu.au/schools/arts-and-humanities).
This report provides an overview of the aims, methodology and key findings of this
project. The project was conducted between January and August 2017. It drew upon
data from two social media platforms (Facebook and Twitter), collected over a 10-
week period (January-April 2017), and concerned two extreme far right groups:
Britain First and Reclaim Australia. Further publications will be appearing in due
course and those interested in hearing more about the project should contact the
report’s authors (details on p.23).