Online Library

Over the last five years, Pakistan has introduced various measures aimed at regulating terrorist content online, including the 2020 Citizen Protection (Against Online Harm) Rules which directly targets content posted on social media, and the 2016 Prevention of Electronic Crimes Act which prohibits use of the internet for terrorist purposes.

These regulations supplement the Anti-Terrorism Act of 1997 (ATA) that provides the baseline legal framework for counterterrorism measures in the country. The ATA does not specifically target terrorist use of the internet, however, it considers the dissemination of digital content “which glorifies terrorists or terrorist activities” to be an offence – under section 11W. The same section also prohibits the dissemination of content that incite to hatred or “gives projection” to a terrorist actor.

Harmful and illegal online content have been regulated in Australia since the late-1990s via the Broadcasting Services Amendment (Online Services) Act of 1999, which established the legislative framework for online content regulation in the country.

Online regulation and content moderation in the United States is defined by the First Amendment right to freedom of speech and Section 230 of the Communication Decency Act 1996, which establishes a unique level of immunity from legal liability for tech platforms. It has broadly impacted the innovation of the modern Internet, causing global effects beyond the US. Recently, however, the Trump Administration administered an executive order directing independent rules-making agencies to consider regulations that narrow the scope of Section 230 and investigate companies engaging in “unfair or deceptive” content moderation practices. This shook the online regulation framework and resulted in a wave of proposed bills and Section 230 amendments from both government and civil society.

Canada’s approach to online regulation has, so far, been characterised by its support for tech sector self-regulation as opposed to government-led regulation of online content. However, concerns over foreign interference in Canadian politics and online hate speech and extremism, have led to public discussions considering the introduction of a legislation on harmful online content, and the possibility to make tech companies liable for content shared on their platforms.

When toxic online communities on mainstream platforms face moderation measures, such as bans, they may migrate to other platforms with laxer policies or set up their own dedicated website. Previous work suggests that, within mainstream platforms, community-level moderation is effective in mitigating the harm caused by the moderated communities. It is, however, unclear whether these results also hold when considering the broader Web ecosystem. Do toxic communities continue to grow in terms of user base and activity on their new platforms? Do their members become more toxic and ideologically radicalized? In this paper, we report the results of a large-scale observational study of how problematic online communities progress following community-level moderation measures. We analyze data from r/The_Donald} and r/Incels, two communities that were banned from Reddit and subsequently migrated to their own standalone websites. Our results suggest that, in both cases, moderation measures significantly decreased posting activity on the new platform, reducing the number of posts, active users, and newcomers. In spite of that, users in one of the studied communities (r/The_Donald) showed increases in signals associated with toxicity and radicalization, which justifies concerns that the reduction in activity may come at the expense of a more toxic and radical community. Overall, our results paint a nuanced portrait of the consequences of community-level moderation and can inform their design and deployment.

The United Kingdom has set out an ambitious online regulatory framework in its Online Harms White Paper, aiming to make the UK “the safest place in the world to be online” by countering various online harms ranging from cyberbullying to terrorist content. This is yet to come into effect, but the UK has approved an interim regime to fulfil obligations under the European Union Directive, which the UK needs to comply with during Brexit negotiations. The UK also has extensive counterterrorism legislation criminalising the viewing and sharing of terrorist content online.