Online Library

Social media is a modern person’s digital voice to project and engage with new ideas and mobilise communities—a power shared with extremists. Given the societal risks of unvetted content-moderating algorithms for Extremism, Radicalisation, and Hate speech (ERH) detection, responsible software engineering must understand the who, what, when, where, and why such models are necessary to protect user safety and free expression. Hence, we propose and examine the unique research field of ERH context mining to unify disjoint studies. Specifically, we evaluate the start-to-finish design process from socio-technical definition-building and dataset collection strategies to technical algorithm design and performance. Our 2015–2021 51-study Systematic Literature Review (SLR) provides the first cross-examination of textual, network, and visual approaches to detecting extremist affiliation, hateful content, and radicalisation towards groups and movements. We identify consensus-driven ERH definitions and propose solutions to existing ideological and geographic biases, particularly due to the lack of research in Oceania/Australasia. Our hybridised investigation on Natural Language Processing, Community Detection, and visual-text models demonstrates the dominating performance of textual transformer-based algorithms. We conclude with vital recommendations for ERH context mining researchers and propose an uptake roadmap with guidelines for researchers, industries, and governments to enable a safer cyberspace.

As the physical territory held by the group known as the Islamic State diminished in 2016-2017, concern
about of the status of the group’s “virtual” caliphate increased. This report focuses on one aspect
of that virtual caliphate: the production of visual propaganda by the group’s ofcial media bureaus.
Using a dataset of more than 13,000 pieces of ofcial visual propaganda distributed from January 2015
to June 2018, this report examines how the production of such pieces has changed over this timeframe
in terms of the number of pieces distributed, the geographic dynamics associated with the production
of propaganda, and the content featured in these products. Through the course of this examination,
several key findings emerge:
Ofcial visual propaganda production has decreased significantly: According to the CTC’s collection
criteria, August 2015 represented the high-water mark for the production of ofcial visual propaganda,
with 754 releases. The low-point occurred in June 2018, with 44 releases. This represents
a 94-percent decrease in visual propaganda production. It is important to note that this decrease
does not account for non-visual production such as text-only tweets.
Despite the decrease, fluctuation in visual propaganda production is likely to continue: At the
macro level, production rebounded slightly in January 2018 before falling of again. This follows a
more sustained rebound in production that occurred in late 2016. At the local media bureau level,
increases and decreases have occurred quite frequently.
Since July 2015, 100 Islamic State media operatives have been announced as being martyred:
Among many reasons for the decrease in propaganda production, one revealed by this report is
the number of media personnel who have been killed. In the first quarter of 2016 alone, 20 such
personnel were eulogized in the group’s propaganda.
Islamic State videos (excluding Amaq and Furat Media Establishment) have been increasing in
length since January 2015: In the first five months of 2015, the average length of an Islamic State
video was a little over six minutes. In the first five months of 2018, this number had increased to
approximately 16 minutes 30 seconds. This may suggest a decreased ability to create narrowly
tailored and targeted videos.
The Islamic State’s media bureaus inside of Iraq and Syria present a worrying sign for the future:
During 2016 and after the liberation of parts of Iraq from formal Islamic State control in December
2017, production of ofcial visual products from Iraqi media bureaus declined. Since that point,
however, production coming specifically from Iraq has rebounded slightly, highlighting the group’s
resilience and potential future threat in the region.
The Islamic State’s media bureaus outside of Iraq and Syria are producing more propaganda as
a proportion of the group’s overall ofcial visual output than ever before: Due to both an overall
decline in production of ofcial visual releases inside Iraq and Syria and a small increase among
some bureaus outside of Iraq and Syria, most notably the Khurasan bureau, the Islamic State’s
media bureaus outside of Iraq and Syria have surpassed 20 percent of overall ofcial visual output
in six of the last nine months. This level of non-Iraq and Syria production had not occurred once
in the preceding 32 months.
The theme of Islamic State ofcial visual releases is overwhelmingly military as opposed to non-military:
In the first quarter of 2015, 53 percent of the group’s ofcial visual releases were non-military
in theme. In this first quarter of 2018, this number had fallen to 15 percent.

VOX-Pol Blog post.

An empirical study by the Israel Internet Association analyzes the manner and quality of response by various social media platforms to 447 verified and confirmed reports of harmful content. These reports were submitted to the platforms by the Association’s Internet Safety Hotline, recognized by most platforms as an official Trusted Flagger, after meticulously confirming that all content reported explicitly violated the relevant platforms’ content policies.  The study considers all reports verified and sent to the platforms by the Hotline team between October 7 and December 7, 2023 (the first two months of the Israel-Hamas war) through official channels, the response received from the platforms to these reports, and actions taken or not taken by the platforms in response to the reported content. Parameters evaluated include: response time, the nature and quality of response as a function of content type, and whether the platforms’ actions matched their public commitments to addressing online threats in the wake of the Hamas attacks.

The findings are troubling, and show that response times were exceedingly slow, with little to no action on weekends. Handling of content requiring manual review or contextual understanding was notably poor, revealing significant differences between content types and across different platforms. We show that the same platforms have succeeded in similar situations in other countries, suggesting that success is possible when appropriate resources are channeled into achieving it. We present detailed recommendations as to how to remedy the present situation, and emphasize that responsibility lies not only with the platforms but also with governments and legislators, who permit these companies to operate with minimal transparency and accountability.

Gender equality plays a pivotal role in combating terrorism and violent extremist in the global arena. This chapter provides a brief overview of evolution, definition and overall goals of gender mainstreaming and analysis as a tool for understanding the unique needs of men and women to achieve gender equality at the institutional level by developing policies, implementing programs and reviewing security implications. Since the 21st-century battlefield is complex, kinetic and multi-dimensional, it requires a multi-disciplinary approach to find solutions to such issues. Using technological platforms, it is possible to supplement means to understand, foresight and address such complex dynamics. A focus on the duality of information technology and its interplay with social media for recruitment, preventing the spread of misinformation and even intercepting channels of communication, plays a vital role in combating terrorism and violent extremism. From a policy standpoint, offering education and training and providing easy access to information platforms along with other similar initiatives, will assist in gender equality and in development policies, programs and strategies.

Society faces a fundamental global problem of understanding which individuals are currently developing strong support for some extremist entity such as ISIS (Islamic State) – even if they never end up doing anything in the real world. The importance of online connectivity in developing intent has been confirmed by recent case-studies of already convicted terrorists. Here we identify dynamical patterns in the online trajectories that individuals take toward developing a high level of extremist support – specifically, for ISIS. Strong memory effects emerge among individuals whose transition is fastest, and hence may become ‘out of the blue’ threats in the real world. A generalization of diagrammatic expansion theory helps quantify these characteristics, including the impact of changes in geographical location, and can facilitate prediction of future risks. By quantifying the trajectories that individuals follow on their journey toward expressing high levels of pro-ISIS support — irrespective of whether they then carry out a real-world attack or not – our findings can help move safety debates beyond reliance on static watch-list identifiers such as ethnic background or immigration status, and/or post-fact interviews with already-convicted individuals. Given the broad commonality of social media platforms, our results likely apply quite generally: for example, even on Telegram where (like Twitter) there is no built-in group feature as in our study, individuals tend to collectively build and pass through so-called super-group accounts.