Seeing Eye to Eye: The Delphi Method & Potential Benefits to Mulistakeholderism in the P/CVE Space

By Ninian Frenguelli

The Seeing Eye to Eye: Developing Sustainable Multistakeholder Communities (SE2E) project was developed and funded through the 2022 Terrorism and Social Media (TASM) Conference sandpit event. The project aim is conducting empirical research into how various stakeholders view and experience multistakeholderism in countering terrorism and violent extremism online (TVE) as part of the larger umbrella of “preventing and countering violent extremism,” or P/CVE. A primary animating question of the sandpit team as we developed the project was: “How can a shared responsibility framework for robust cross sector work be developed?” See the first and second post in the series here and here.

The counterextremism landscape is changing rapidly both off and online. For example, actors are meeting in Facebook groups, moving to Telegram channels, and then organising offline events. Moreover, new technologies are constantly emerging and changing the contours of the threat and response milieus. The Metaverse poses unique challenges for countering extremism in the virtual space, due to the new ways users can interact with each other. Generative AI including “deep fake” image and sound, as well as text technologies poses unique challenges with the scale of propaganda and manifesto development and dissemination. And block chain technologies along with decentralised, distributed network structures present challenges to existing moderation and accountability strategies. The Covid 19 pandemic has seen far-right and QAnon conspiracy theories and anti-government movements merge in new ways that require new approaches and methods to understand, assess, and respond effectively.  Challenging violent extremism and terrorism requires the long-term, and efficacious collaboration of researchers, practitioners, government agencies, policymakers, legislators and regulators, technology industry players (especially “very large online platforms and search engines,” or VLOPS), law enforcement, and educators. Employing the Delphi method (or adapted versions) can help start and sustain such collaborations.

What is the Delphi Method?

The Delphi method is a consensus-oriented research method where groups of experts are asked their opinions on a topic in iterative rounds, with the feedback from each round being fed back to the other participants to facilitate the group of experts reaching a consensus on the topic. This blog discusses how the method has been used in other fields to improve collaboration and its limited use to study different aspects of the preventing and countering extremism (P/CVE) and terrorism and violent extremism (TVE) online to suggest that increased utilization of the method within the P/CVE and TVE online milieus offers intrinsic benefits to multistakeholder projects.

Using Delphi studies to improve cross-sector collaboration

The Delphi method has been used extensively in work regarding multistakeholder collaboration. One study conducted assessing collegiality practices between nurses utilised the Delphi method to create best practice guidelines that were based on survey responses from a range of experts in nursing, including registered nurses, paramedics, and midwives. A Delphi study of counselling identified a gap between academic research into counselling best practice, and the counselling approaches applied by practitioners. It identified several reasons for the disconnect between academics and practitioners such as a decline in collaboration between scientists and counsellors, and counsellors being unprepared to apply research findings to their work. The project team decided that a community based participatory research framework would work best to address these issues and used the Delphi method to identify the competencies that experts believe are required for building sustainable engagement between researchers and practitioners.

And in a study aimed at fostering better academy-industry collaboration – highly relevant and applicable to the broad P/CVE arena where industry and the academy need to collaborate – the a “fuzzy” Delphi was used to examine barriers to multistakeholder collaboration. The study found that industry sponsorship of research collaboration was the most important factor for the success of projects, followed by the research and development expenditure of universities, and then the amount of government funding available to support projects.

The Delphi method as a means of studying extremism and radicalisation

The Delphi method has been previously used in the P/CVE space to assess expert opinion on the global trends that would relate to future radicalisation and terrorism, and to  investigate key policy issues for preventing violent extremism in the EU, identifying education as a key area for CVE policy to focus on. In the psychiatric investigation of lone actor terrorism, a Delphi study lasting two years found that deradicalisation efforts should focus on reducing feelings of resentment and anger towards society. While a 2011 Delphi study by the RAND corporation assessed expert opinion on counterterrorism capabilities of the Mexican defence sector, as well as the presence of terrorist or insurgency groups operation in Mexico, among other criminal risk and preventative factors. The 2011 broad study has since been used as a starting point for additional security-based research carried out by RAND corporation.

Adapted forms of the Delphi model have also been used in the P/CVE and TVE online arenas. A Singapore based study used a modified Delphi technique to examine markers of radicalisation in social media posts. A review of terrorism risk assessment tools was carried out to identify risk factors which were then presented to experts of terrorism for evaluation. After three rounds of consensus-focused discussion, the experts identified 12 factors and the observable indicators of these. For instance, perceived grievance was identified as a risk factor and examples of injustices against the respective “in-group” being publicised on social media was found to be an observable indicator. Another study investigating risk factors for terrorism used the Delphi method combined with a literature review to move beyond identifying indicators to create an integrated framework model for assessing a group’s risk of terrorism.

The Delphi method has also been used to troubleshoot issues preventing effective counter terrorism measures, such as the lack of an internationally accepted definition of the topic. In one study of this topic, terrorism experts were given vignettes describing current and potential future scenarios and asked respondents how likely it was that their countries would classify the events as terrorist today, and in the year 2040. The findings of this Delphi study generally pointed to increasing vigilance against terrorism, with participants anticipating definitions expanding and counterterrorism policies becoming tougher and diversified to confront the changing nature of threats due to improvements in technology.


Using the Delphi method to investigate and understand the current state of multistakeholder collaboration in P/CVE offers the potential to create collaborative responses to issues that affect the field as a whole. Delphi studies—full or adapted—are good for identifying issues, practices, or other factors that inhibit or, conversely, that facilitate collaboration. Furthermore, conducting Delphi studies can also be the first step in building an effective, multistakeholder collaboration. The SE2E project was partly inspired by the Delphi method and believes that applying this method in this way could be a very fruitful strategy for future P/CVE research and intervention endeavours.

Ninian Frenguelli is a PhD candidate at Swansea University studying gender in the online extreme right. His PhD maps the different attitudes towards gender and gendered issues that are expressed by online extreme right actors. On Twitter: @ndfrenguelli

Image Credit: FREEPIK