Online Library

Polarization is implicated in the erosion of democracy and the progression to violence, which makes the polarization properties of large algorithmic content selection systems (recommender systems) a matter of concern for peace and security. While algorithm-driven social media do not seem to be a primary driver of polarization at the country level, they could be a useful intervention point in polarized societies. This paper examines algorithmic depolarization interventions aimed at transforming conflict: not suppressing or eliminating conflict, but making it more constructive. Algorithmic intervention is considered at three stages: what content is available (moderation), how content is selected and personalized (ranking), and content presentation and controls (user interface). Empirical studies of online conflict suggest that not only could the exposure-diversity intervention proposed as an antidote to ‘filter bubbles’ be improved: under some conditions, it can even worsen polarization. Using civility metrics in conjunction with diversity in content selection may be more effective. However, diversity-based interventions have not been tested at scale, and may not work in the diverse and dynamic contexts of real platforms. Instead, intervening in platform polarization dynamics will likely require continuous monitoring of polarization metrics, such as the widely used ‘feeling thermometer’. These metrics can be used to evaluate product features, and can potentially be engineered as algorithmic objectives. While using any metric as an optimization target may have harmful consequences, to prevent optimization processes from creating conflict as a side effect it may prove necessary to include polarization measures in the objective function of recommender algorithms.

This research employs the concept of social capital to compare the efforts that a range of tech companies have claimed to take to counter polarization and extremism and build resilience on their platforms. The dataset on which our analysis is based is made-up of a purposive sample of official blog posts from three ‘older’ (i.e., Facebook, Twitter, YouTube) and three ‘newer’ (i.e., TikTok, Discord, and Telegram) technology platforms. The selected posts focused on companies’ efforts to make their platform safer, build community resilience, counter violent extremism and/or polarization, or mentioned related topics such as countering hate organizations, radicalization, or misinformation. Revealed were seven themes incorporating, to a greater or lesser extent, the three main types of social capital (i.e., bonding, bridging and linking). These themes were granting user powers, strengthening existing communities, provision of information and education, building community, enhancing user rights, keeping users safe, and building trust and relationships with users. Analysis of these showed that while creation of all three types of social capital was apparent, similar to previous studies, bridging capital dominated here too; while there were some discrepancies between social capital generating activities and their framings on ‘older’ versus ‘newer’ platforms, other factors, including platform size and company values are likely to be equally or more important; and, finally, that companies attempts at generating online social capital can have negative as well as positive impacts with regard to countering polarization and extremism.

This paper examines the need and possibility for developing online resilience-based approaches in response to COVID-19 vaccine conspiracies, often linked to the far right. Examining three datasets collected between December 2020 and April 2021, this paper details conspiracy narratives that have developed around COVID-19 vaccines, with specific focus on understanding the deployment of the idea of a planned pandemic or so-called ‘#plandemic’. This is then used to consider where existing resilience-based approaches to countering off-line polarisation and extremism might posit an appropriate online response. The article identifies four key #plandemic framings of COVID-19 vaccines — as control, as reset, as unnecessary and as unsafe — and analyses how these themes are constructed, to find that they are often created through hostile and confrontational interaction with other users. Based on these findings, the conclusion suggests companies shift their focus away from ‘negative’ approaches to content moderation (e.g., content removal) and towards resilience-building responses that cultivate flexible individual identities, build community support networks, and/or engage users with national and supranational democratic structures, as a more effective response to the sharing of online conspiracies.

The rise of QAnon presents researchers with a number of important questions. While emerging literature provides insights into how QAnon exists online, there is a dearth of theoretical engagement with the questions of why it exists, and what conditions brought it into being. This paper seeks to address this gap by contextualizing QAnon as an ontological phenomenon underpinned by anxiety, and inquiring into the identity formation strategies employed by the movement. Applying the basic precepts of discourse theory and discourse analysis to a representative canon of QAnon content, it finds that, like other formations of collective identity, QAnon is premised on interconnected dynamics of ontological fulfillment that cannot be explained away by pointing to ‘the algorithm’ or ‘madness’. Nor can it be tackled effectively by the content takedowns and de-platforming strategies currently employed. The paper concludes with a call to explore more empathetic engagement with conspiracy adherents, arguing that until we (re)discover a more inclusive, agonistic politics, QAnon and other fantastical conspiracy movements will continue to arise and some may metastasize into violent action. New forms of resilience to (online) polarization can be built on this principle.

A counter-intuitive finding emerges from an analysis of Arabic Twitter posts from 2014 to 2015: Twitter participants who are negative toward the Islamic State of Iraq and the Levant (ISIS) are also more likely to hold negative views of the United States. This surprising correlation is due to the interpretations of two sets of users. One set of users views the United States and ISIS negatively as independent interventionist powers in the region. The other set of users negatively links the United States with ISIS, often asserting a secretive conspiracy between the two. The intense negativity toward the United States in the Middle East seems conducive to views that, in one way or another, cause citizens to link the United States and ISIS in a conspiratorial manner.

A partir de un análisis de publicaciones en árabe en Twitter correspondientes al período 2014–2015, surge una conclusión contraria a la intuición: Quienes participan en Twitter y tienen una actitud negativa con respecto al Estado Islámico de Irak y el Levante (EIIL) también tienen más probabilidades de ostentar una opinión negativa con respecto a los Estados Unidos (EE. UU.). Esta sorprendente correlación se debe a las interpretaciones de dos grupos de usuarios. Uno de dichos grupos tiene una visión negativa de los EE. UU. y del EIIL, ya que los considera poderes intervencionistas independientes en la región. El otro grupo de usuarios establece una vinculación negativa entre los EE. UU. y el EIIL, y suele afirmar que existe una conspiración secreta entre los dos. La intensa negatividad con respecto a los Estados Unidos en el Oriente Medio parece propiciar opiniones que, de una u otra manera, hacen que los ciudadanos establezcan una vinculación conspirativa entre los EE. UU. y el EIIL.

Une conclusion contre-intuitive ressort d’une analyse des publications Twitter en arabe de 2014–2015: les participants à Twitter qui ont une opinion négative de l’État islamique en Irak et au Levant sont également plus susceptibles d’avoir une opinion négative des États-Unis. Cette corrélation surprenante est due aux interprétations de deux ensembles d’utilisateurs. Un ensemble d’utilisateurs voit négativement les États-Unis et l’État islamique en Irak et au Levant puisqu’il les considère comme des puissances interventionnistes indépendantes dans la région. L’autre ensemble d’utilisateurs les associe négativement en affirmant souvent qu’il existe une conspiration secrète entre eux deux. L’intense négativité à l’égard des États-Unis au Moyen-Orient semble propice à des points de vue qui, d’une manière ou d’une autre, amènent les citoyens à établir un lien conspirationniste entre les États-Unis et l’État islamique en Irak et au Levant.

The role of online communities of the extreme-right in the radicalization of individuals involved in political violence has received increased public attention due to recent tragic events around the world. In this policy brief, we provide a systematic account of the psychological processes underpinning the formation and transformation of these communities. Our analysis is built on the premise that these communities can be understood as ideologically driven psychological groups, and as such their collective beliefs, values, and norms are key to understanding their actions. Drawing on findings from research into an Australian extreme-right online community, we show how these collective beliefs, values, and norms can change in the aftermath of particular socio-political events in the offline domain. We conclude by proposing strategies that can be used to direct policy and recommendations for research in the area.