Online Library

This report aims to provide an in-depth analysis on the state of extremism and hate on TikTok. It is the culmination of three months of research on a sample of 1,030 videos, equivalent to just over eight hours of content, posted on the social media platform. These videos were used to promote hatred, as well as glorify extremism and terrorism.

ISD set out to examine the state of hate and extremism on TikTok in two ways. The first objective involved analysing how individuals or groups promote hateful ideologies and target people on the platform based on numerous protected attributes such as ethnicity, religion, gender or others. Second, using the same framework, ISD investigated how features on TikTok like profiles, hashtags, share functions, video effects and music are used to spread hate.

This report seeks to start a conversation around how platforms like TikTok can improve their own practices to protect users from harm. Additionally, it underscores the clear need for independent oversight of such platforms, which currently leave users and the wider public open to significant risks to their health, security and rights.

— The reporting, recording and incidence of online hate speech has increased over the past two years.
— While the number of people personally targeted remains relatively low, large numbers of people are being exposed to online hate speech, potentially causing decreased life satisfaction. In particular, an increasingly large number of UK children (aged 12-15) report that they are exposed to hateful content online.
— Online hate speech tends to spike for 24-48 hours after key national or international events such as a terror attack, and then rapidly fall, although the baseline of online hate can remain elevated for several months. Where it reaches a certain level, online hate speech can translate into offline hate crime on the streets.
— Hate crime, including hate speech, is both hard to define and hard to prosecute. A patchwork of hate crime laws has developed over the last two decades, but there is concern the laws are not as effective as they could be, and may need to be streamlined and/or extended – for example to cover gender and age-related hate crime. The Law Commission is currently reviewing hate crime legislation, and has separately completed a preliminary review of the criminal law in relation to offensive
and abusive online communications, concluding there was “considerable scope for reform”.
— According to a recent survey by Demos, the public appreciates the difficult trade-off between tackling hate crime and protecting freedom of speech, with 32% in favour of a safety first approach, 23% in favour of protecting civil liberties, and 42% not favouring either option.

The United States is losing the information war to terrorists like ISIS and Hezbollah. Earlier this year, the administration rebranded the office responsible for counter messaging, but little seems to have changed. A strong, effective information offensive to counter the violent ideology being pushed by ISIS and other terrorists is long overdue. This hearing will give members an opportunity to press the administration’s top public diplomacy official on how the U.S. can be more effective.

VOX-Pol Blog post.

In recent years, Hezbollah has used social media to recruit Israeli Arabs and West Bank-based Palestinians to attack Israeli targets. A recent innovation in terrorist tactics has given rise to “virtual entrepreneurs,” which to date have been largely associated with the Islamic State’s online recruitment efforts. Hezbollah’s virtual planners, similar to those in the Islamic State, use social media to establish contact with potential recruits before transitioning to more encrypted communications platforms, transferring funds, and issuing instructions to form cells, conduct surveillance, and carry out terrorist attacks. Online recruitment presents a low-cost option that offers plausible deniability for Hezbollah. While every virtual plot led by Hezbollah that targeted Israel has been foiled thus far, Israeli authorities spend time and resources disrupting these schemes at the expense of other more pressing threats. By digitally recruiting Palestinians to attack Israel, Hezbollah and its patron Iran are seeking to cultivate a new front against Israel amid rising regional hostilities.

Disrupting the emergence and evolution of potentially violent online extremist movements is a crucial challenge. Extremism research has analyzed such movements in detail, focusing on individual- and movement-level characteristics. But are there system-level commonalities in the ways these movements emerge and grow? Here we compare the growth of the Boogaloos, a new and increasingly prominent U.S. extremist movement, to the growth of online support for ISIS, a militant, terrorist organization based in the Middle East that follows a radical version of Islam. We show that the early dynamics of these two online movements follow the same mathematical order despite their stark ideological, geographical, and cultural differences. The evolution of both movements, across scales, follows a single shockwave equation that accounts for heterogeneity in online interactions. These scientific properties suggest specific policies to address online extremism and radicalization. We show how actions by social media platforms could disrupt the onset and ‘flatten the curve’ of such online extremism by nudging its collective chemistry. Our results provide a system-level understanding of the emergence of extremist movements that yields fresh insight into their evolution and possible interventions to limit their growth.