Online Library

The Digital Services Act (DSA) requires designated Very Large Online Platforms and Very Large Online Search Engines (VLOPs and VLOSEs) conduct assessments of the systemic risk of the dissemination of illegal content through their services- and to mitigate such risk where it arises. With regulators, researchers, civil society, and the general public finally due to see the public versions of the first systemic risk assessments conducted by VLOP and VLOSE providers, robust evaluation of the DSA’s systemic risk management approach to the dissemination of illegal content is crucial.  

CERRE’s Benchmarks for Evaluating Management of Risk of Terrorist Content Dissemination issue paper contributes to this evaluation by examining the systemic risk management approach to the dissemination terrorist content. Terrorist content is a specific subset of illegal content under the DSA that is also the subject of a complex nexus of other EU legislation, including the TERREG and several transnational initiatives. The determination of terrorist content poses a significant challenge, as authorities and platforms grapple with the use of sophisticated evasive techniques by terrorist actors, including the use of coded content or “beacons”, while also seeking to respect the fundamental rights of platform users.  

In this issue paper, author  Sally Broughton Micova explores the key definitional issues surrounding terrorist content and maps the ecosystem of actors involved in its risk assessment, mitigation, enforcement, and dissemination. Based on detailed analysis, the paper develops a series of benchmarks and concomitant metrics for the systemic risk evaluation of terrorist content. 

This paper explores the relationship of transphobia and other forms of harassment found across the events of the Gamergate hate movement through the development of an interactive social network analysis. With the social network being derived from hundreds of events tagged by hand, special consideration is given to the positionality and biases of its authors and how they affected this specific interpretation of Gamergate events. Informed largely by the transgender perspective of its first author, this paper draws particular conclusions around the propagation of transphobia in online hate movements, such as its intersectionality with other ideological cornerstones of Gamergate.

Cet article explore la relation entre la transphobie et d’autres formes de harcèlement observées lors des événements du mouvement de haine Gamergate à travers le développement d’une analyse de réseau social interactive. Avec le réseau social dérivé de centaines d’événements étiquetés à la main, une attention particulière est accordée à la position et aux biais de ses auteurs et à la manière dont ils ont influencé cette interprétation spécifique des événements de Gamergate. Principalement informé par la perspective transgenre de son premier auteur, cet article tire des conclusions particulières sur la propagation de la transphobie dans les mouvements de haine en ligne, comme son intersectionnalité avec d’autres piliers idéologiques de Gamergate.

This study explores the significant impact of the internet on terrorism and extremism by focusing on three key areas: cyberterrorism, social media usage, and attack planning. Cyberterrorism has emerged as a major threat, with incidents such as the WannaCry ransomware attack highlighting its potential to disrupt critical infrastructures. The study reveals that the low cost and difficulty in tracing cyberterrorists contribute to its rising prevalence. Furthermore, social media has become a vital tool for extremist groups in that it facilitates global communication, recruitment, and propaganda dissemination. Events such as the Christchurch mosque attacks illustrate the challenges of monitoring extremist content online. The internet also aids in attack planning by providing discreet communication channels, easy access to materials, and simplified travel arrangements, as seen in the 9/11 and Manchester Arena bombings. Additionally, the chapter discusses how technological advancements have enhanced security agencies’ capabilities, leading to more effective prevention and response strategies. The findings highlight the dual role of the internet as both a facilitator of terrorism and a crucial asset in countering these threats.

Severe online hate and harassment increased four points across the board in the past year, which was dominated by an unprecedented surge in antisemitism online and offline in the wake of Hamas’ brutal attack on Israel on October 7. Decreases in platform enforcement and data access and new threats of hate and disinformation from generative AI tools all potentially contributed to this year’s findings.

ADL conducts this nationally representative survey annually to find out how many American adults experience hate or harassment on social media. Since 2022, we have also surveyed teens ages 13-17. This survey was conducted in February and March 2024 and asked about the preceding 12 months.

Key Findings

• Severe harassment overall went up: 22% of Americans experienced severe harassment¹ on social media in the past 12 months, an increase from 18% in 2023 (including an increase in physical threats from 7% to 10%).
• Harassment by disability: People with disabilities were more likely to be harassed than non-disabled people, 45% compared to 36% respectively for any harassment and 31% vs. 19% for severe harassment.
• People with disabilities were more likely to be harassed than in the year before, 45% compared to 35% for any harassment and 31% vs. 20% for severe harassment
• Harassment for disability spiked: Reasons for online harassment in the past 12 months remained stable, except for disability, which spiked from 4% to 12%, despite the proportion of disabled respondents remaining similar.
• LGBTQ+ people were the most harassed of the marginalized groups surveyed: LGBTQ+ people experienced increases in physical threats (from 6% to 14%), while transgender people, as a subgroup, reported severe harassment to a higher degree from last year (from 30% to 45%).
• Jewish adults were more likely to be harassed for their religion (34% of those harassed compared to 18% of non-Jews) and 41% changed their online behavior to avoid being recognized as Jewish. Nearly two-thirds (63%) felt less safe than they did last year.
• Platforms: Facebook remains the most common platform where harassment was experienced at 61% of harassment, while the incidence of harassment rose on WhatsApp (from 14% to 25%) and Telegram (from 7% to 13%).

Key Recommendations

Allow researchers inside the black box: The federal government should broaden data access for researchers and follow California’s lead and require technology companies to standardize transparency reporting and broaden data access.

Address hate on messaging apps: Messaging platforms should strengthen anti-hate policies and invest in tools to combat hate and harassment.

Support targets of hate: Platforms should implement recommended reporting tools and features that reduce hate and improve abuse reporting.

Invest in trust and safety: Tech companies should increase trust and safety resources (human and automated) to ensure platforms are enforcing their rules around hate speech and violence.

Drawing on a survey of 13,302 online Australians, this study examines the characteristics and behaviours of respondents who viewed child sexual abuse material (CSAM) and fringe or radical content online, or both. In the past 12 months, 40.6 percent of respondents had viewed fringe or radical content and 4.5 percent had viewed CSAM. Among respondents who viewed CSAM, 64.7 percent had also viewed fringe or radical content, while 7.1 percent of those who viewed radical content had also viewed CSAM. Respondents who viewed only CSAM or only fringe or radical content were similar to one another. Respondents who viewed both were more likely to be younger and male and had higher rates of criminal justice system contact and diagnosed mental illness. Their online activity, including the platforms used, also differed.

This chapter addresses the impact and discomfort of researching extremist digital subcultures. It provides reflections about the visual and personal dimension from two researchers investigating online extremist far-right and incel content, using memes as a case study. Through visually stimulating images, humor, and narratives, memes can normalize and desensitize extremism and violence, not only for the consumers but also for the researchers. We compare our research experiences to help prepare others who plan to study extremist content online by identifying themes of dehumanization and provocation, internalization and enabling harmful humor, and hyperawareness in everyday life. We give specific suggestions for handling these challenges and conclude by discussing the importance of adopting a reflexive approach for orienting our positionality in the research process.