By Daniel E. Levenson
Successful ideologues and leaders in extremist movements have long been aware of the power that misleading, but powerfully Manichean, pseudo-historical narratives can have on audiences. The foundation of this propagandistic material is often rooted in self-serving mythologies which justify the scapegoating of others and validation of the deeply felt grievances that potential followers have already internalized. The British Israelism and Christian Identity movements offer an excellent example of the use of this approach, and violent Islamist groups such as the so-called “Islamic State” have also engaged in a good deal of revisionist history as well. Whether in the form of pamphlets, radio broadcasts, books or magazines, the technology of the day has often played a key role in the production and dissemination of this material, and today a new generation of violent extremists seem poised to use generative AI to the same ends. In order to meet this challenge, we must recognize that AI offers potentially truly unique opportunities for propagandistic invention and alchemy.
Senders and Receivers Speaking the Same New Language
One problem is not merely that bad actors can, do, and will continue to exploit technology for malicious purposes, but that there is a convergence between senders and receivers who find themselves in a digital world where fringe ideas and mainstream information seem to mix endlessly. Furthermore, extremists who understand the ins and outs of synthetic media also likely realize that they don’t have to create perfectly convincing material in order to have an impact. If the images and text narratives they produce tap into underlying grievances and fears, that is likely good enough. Antecedents to this approach in print are plentiful. Take, for example, the Turner Diaries, a violent extremist fantasy that presents itself as a “found document” offering an alternative vision of reality, which real-world terrorists have found inspirational. J.M. Berger has noted that this work is one in a long line of similar publications predating the US Civil War. Part futurist projection, part how-to manual, books like The Turner Diaries did little to hide their overtly racist, anti-Semitic, and anti-government aim, although, as Berger notes, the author did noticeably switch to over racial epithets in the latter half of the book, when readers may have been more desensitized to the hate-based plot of the novel.
An Insidious Mix
What is particularly concerning about the potential use of generative AI in this space is that it can conceivably allow for a more insidious, dissembling approach to the creation and dissemination of extremist narratives. By virtue of its ability to blend fact and fiction, a shift in tone or content from subtle to overt extremism may be practically imperceptible on the part of the viewer. Unlike in printed material or on openly extremist websites, where users are likely to initially encounter obviously hateful imagery and language, it may be easier for viewers of AI-generated material to be drawn in before they realize the true nature of the content they are consuming.
Stakeholder Mitigation
Stakeholders keen to understand how AI may be used to create inherently extremist narratives which are more likely to go viral, should consider two important elements.
Desensitization
The first is the ways in which human beings can be desensitized to violence and extreme ideas – a number of scholars have done excellent work in this area, and arguably this is a good place to start. The hardcore extremists who are primed to consume and spread problematic content do not really need to be persuaded of the value (irrespective of the veracity) of a compelling story or set of images. If it resonates, then they have every reason to use it as a vehicle to spread their message. It is arguably within efforts to curtail the unhelpful behavior of non-extremists who might be attracted to something that appears novel or which they think might have a kernel of truth in it, that offer the best hope of limiting the spread of this material.
Doubt
The second is the previously mentioned evolution of new modes of online interaction, with producers and consumers of synthetic media locked in a dialogue that is bound to take us to some very strange places. Such a landscape is ripe for the exploitation of extremists, who will be able to take advantage of technological advances around synthetic media, as well as concepts like “The Liar’s Dividend,” which articulate the emerging threat to trust and faith in institutions that at the heart of civil society.
Proposed Core Attributes of Effective AI-Generated Narratives
Taking our cues from the literatures on desensitization and the social media habits of individuals who seem to share this material regardless of their own belief in it, we can then begin to think about AI safeguards which may inherently limit the generation of material which has what we might think of as three ideal features of AI-generated extremist propaganda:
(1) It taps into a particular kind of grievance that blames, implicitly or explicitly, an outgroup
(2) It uses an algorithm to cobble together a narrative that has referents in the real world, but reframes historical events and figures to serve the agenda of the in-group
(3) It uses machine learning to identify paths to desensitization with respect to digital media, and incorporates them into the narrative
There are undoubtedly other factors which help determine how effective an AI-generated or augmented extremist narrative may turn out to be, but starting with these three and developing both technical and policy approaches that limit the efficacy of extremist efforts in this area, is a good place to start.
Daniel E. Levenson is a PhD student in Criminology at Swansea University, where he is focused on the intersection of Artificial Intelligence (AI) and terrorism. He holds an MA in Security Studies from the University of Massachusetts at Lowell, and an MLA in English and American Literature and Language from Harvard University. Daniel can be contacted through his website at www.danielericlevenson.com
IMAGE CREDIT: PEXELS
Want to submit a blog post? Click here.