Methodological problems in online radicalisation

By Joe Whittaker

There seems to be near-ubiquity between discussion of radicalisation to violent extremism and the Internet. Despite this, the study of online radicalisation remains under-researched and as a result ill-understood. This is, perhaps, surprising given the vast attention in the media that is given to the online presence of groups such as Islamic State, and the tens of thousands of foreign fighters who have joined them; the implicit assumption being that many became radicalised via content they had interacted with online. A large part of the reason for under-research in the field is not a lack of interest or desire, but a number of factors which make meaningful research in the field difficult. Below, I will briefly outline three of the biggest methodological problems facing the field of online radicalisation – the problem of correlation and causation, the problematic online/offline dichotomy, and the vast amount of “poor” and “noisy” data.

Correlation, causation, and underdetermination

It takes only a cursory glance to observe that today, the Internet has a high degree of prevalence in most cases of radicalisation to terrorism. Gill et al. (2015) found that in 61% of cases there was evidence of online activity related to the ultimate attack or conviction and from 2012, 76% used the Internet to learn about some aspect of their terrorist activity. Similar results have been found by Gill & Corner (2015) and Gill et al. (2017) seemingly confirming anecdotal evidence from the likes of Sir Norman Bettison, who remarked that “the internet [seems] to feature in most, if not all, of the routes of radicalisation” (Home Affairs Select Committee 2012, 16).

However, this rare empirical research takes us little closer to understanding whether there is a causal connection between the Internet and radicalisation. Even if the Internet was present in every single case of terrorism, which may occur in the future as social life becomes even more connected with the online sphere, it would merely underdetermine the relationship between the two. Philosopher of science Willard van Orman Quine made the most renowned contribution to this problem in 1951:

“The totality of our so-called knowledge or beliefs… [is] underdetermined by its boundary conditions, experience, that there is much latitude of choice as to what statements to reevaluate in the light of any single contrary experience. No particular experiences are linked with any particular statements in the interior of the field, except indirectly through considerations of equilibrium affecting the field as a whole”. (Quine 1951, 42-43)

Quine is suggesting that it is impossible to test a single hypothesis in isolation of a host of background (or auxiliary) hypotheses and that any evidence generated from empirical testing may be insufficient to offer a conclusion over other competing theories. To offer a hypothetical example, it may seem intuitive to suggest that if a greater use of the Internet is correlated to a higher chance of being involved in a terrorist incident, that it is evidence for a causal explanation of the Internet as a driver, rather than a facilitator of radicalisation. However, it could just as easily be suggested that becoming radicalised often makes actors seek like-minded people, and they do this by the use of the most effective method of communication – the Internet. The underlying point is that correlation suggests that two phenomena are connected, and it could be that either causes the other, or even that there is an entirely separate causal explanation which links the two correlated factors.

For obvious reasons, it is both ethically and practically impossible to conduct a laboratory-style experiment and test the dependent variable of radicalisation against use of the Internet on test participants while controlling for other variables. Academics will have to search for more novel methods if they wish to posit, or disprove a causal relationship.

Online/offline dichotomy

The very nature of discussing ‘online radicalisation’ can assume a dichotomy that is problematic. The little empirical data that is available suggests that, while the use of the Internet is extremely prevalent in becoming an extremist, actors regularly engage in both domains (Gill et al. 2017). In fact, there are very few cases in which an actor has radicalised solely online (Ibid.). Part of the grounding for this dichotomy stems from a belief that radicalisation on the Internet operates on a different ontological plane than it does offline. This can be seen in the surprising number of academics and practitioners who refer to the offline domain as the ‘real world’(Silber & Bhatt 2007; Weimann & Von Knop 2008; Hussain & Saltman 2014; O’Hara & Stevens 2015; Home Affairs Select Committee 2012; Holt et al. 2015 – to name but just a few), a phrase which misses the point – the Internet, and the Web 2.0 in particular, is a social space which interacts and compliments offline interactions. Maura Conway makes this point well:

Today’s Internet does not simply allow for the dissemination and consumption of “extremist material” in a one-way broadcast from producer to consumer, but also high levels of online social interaction around this material. It is precisely the functionalities of the social Web that causes many scholars, policymakers, and others to believe that the Internet is playing a significant role in contemporary radicalization processes. (Conway 2016, 4)

Although there is a degree of pedantry in signalling out research for using ill-judged terminology, the wider point is that the online domain cannot be studied in isolation from its offline counterpart (and vice versa). Although it seems to be the case that identities and habits can differ greatly online,(Aresta et al. 2015; Krasodomski-Jones 2017; Gössling & Stavrinidi 2016) they are not separate, but interconnected with their offline counterparts.

Difficulty generating and interpreting data “poor data” – “supply vs demand” – “noise”

The access to good quality data is a problem not just for online radicalisation, but the wider field of Terrorism and Extremism Studies. Rich data, as described by Nate Silver, is “data that’s accurate, precise and subjected to rigours quality control”. Generating rich data is a problem to varying degrees in most Social Sciences and Humanities, but in Terrorism Studies, scholars often have little-to-no access to extremists to try to ascertain their motivations and must often utilise open-sourced secondary data. From this, many problems of data collection pertain. For example, in empirical research by both Bakker (2006) and Horgan et al. (2016), data collectors had to code for a hard ‘yes’ or ‘no’ when looking through open-source data. The lack of readily available rich data means that the (correct given the circumstances) high burden of proof may have often not been met for cases in which it ought to have done. The effect of this is that when empirical research comes to certain conclusions, we can have less confidence in it than we otherwise could.

For online radicalisation, this problem takes on a new element. It remains as difficult to obtain empirical evidence of the process in which an actor becomes radicalised – what von Behr et al. (2013) call the ‘demand-side’, often having to rely on fragmentary open-sourced, or occasionally closed-source, data. However, this represents the minority of research. Instead, scholars tend to opt for the ‘supply-side’ of online radicalisation; using the extraordinary reach of the Internet to analyse data that is being generated by and for extremists. In other words, the former tries to assess how people become radicalised, while the latter assesses whatwould-be radicals may see on the Internet. Clearly, knowledge about online radicalisation will only progress with some combination of both. However, the small amount of research that comes from relatively “poor” data on the demand-side takes the field down a cul-de-sac. Despite how much research is conducted, for example, on the social media strategy of IS, there is a limit on what can be ascertained about the process in which people go through in radicalising.

A further difficulty in ascertaining a potentially causal effect of the Internet on radicalisation is that the available data is extremely noisy. The “pathway” to radicalisation, according to different theorists, can seemingly involve so many different factors, such as group deprivation, identity conflict, and personality characteristics (King & Taylor 2011); or different stages, such as pre-radicalization, self-identification, and indoctrination, and “jihadizsation” (Silber & Bhatt 2007); or the twelve “mechanisms” of radicalisation of McCauley & Moskalenko (2008). It is difficult, if not impossible, to decipher whether the Internet is a ‘signal’ in radicalisation, or whether it is just noise. As noted above, correlation does not mean causation. It is one thing to note the prevalence with which the Internet is used in a trajectory to extremism, yet another altogether to make value judgements about why the Internet was particularly important in certain cases and not in others, or why self-identification or a period of personal crisis were important, or even, how those three potentially overlapping concepts can even be separated from each other. A large part of the problem with noisy data is evaluation. To evaluate whether theory is well backed-up by evidence, it is helpful to continually test it – as any natural scientist would do – to get immediate feedback and to judge whether the posited hypothesis is correct. As we have already seen, data is difficult to collect and often incomplete, which makes it very difficult to test our hypotheses.

In sum, online radicalisation studies suffer from a number of methodological problems that prove a stumbling block to further meaningful research. These problems are not exclusive to this field: the problem of underdetermination underpins all scientific endeavours; the online/offline dichotomy is present in all fields that pertain to the Internet; and data collection difficulties underpin Terrorism Studies (as well as many other fields). However, each problem takes on a new light when considered in the context of online radicalisation. For those, like this author, who have committed themselves to investigating this subject further, making a contribution to tackling these obstacles will underpin the ability to conduct consequential research in the future.

Joe Whittaker is a joint-PhD Candidate at Swansea University and Leiden University. You can follow him on Twitter @CTProject_JW


Aresta, M. et al., 2015. Portraying the self in online contexts : context- driven and user-driven online identity profiles. Contemporary Social Science, 10(1), pp.70–85. Available at:

Bakker, E., 2006. Jihadi Terrorists in Europe: Their characteristics and the circumstances in which they joined the jihad, The Hague.

von Behr, I. et al., 2013. Radicalisation in the Digital Era: The use of the internet in 15 cases of terrorism and extremism, Available at:

Conway, M., 2016. Determining the Role of the Internet in Violent Extremism and Terrorism: Six Suggestions for Progressing Research. Studies in Conflict & Terrorism, pp.1–22.

Gill, P. et al., 2017. Terrorist Use of the Internet by the Numbers. Criminology & Public Policy, 16(312827), pp.1–19. Available at:

Gill, P. et al., 2015. What are the roles of the Internet in terrorism?, Available at:

Gill, P. & Corner, E., 2015. Lone Actor Terrorist Use of the Internet and Behavioural Correlates. In L. Jarvis, S. Macdonald, & T. M. Chen, eds. Terrorism Online: Politics Law and Technology. Abingdon, Oxon: Routledge, pp. 35–53.

Gössling, S. & Stavrinidi, I., 2016. Social Networking , Mobilities , and the Rise of Liquid Identities Social Networking , Mobilities , and the Rise of Liquid Identities. Mobilities, 11(5), pp.723–743. Available at:

Holt, T. et al., 2015. Political radicalization on the Internet: Extremist content, government control, and the power of victim and jihad videos. Dynamics of Asymmetric Conflict, 8(2), pp.107–120. Available at:

Home Affairs Select Committee, 2012. Home Affairs Committee Roots of Violent Radicalisation, London.

Horgan, J. et al., 2016. Actions Speak Louder than Words: A Behavioral Analysis of 183 Individuals Convicted for Terrorist Offenses in the United States from 1995 to 2012. Journal of Forensic Sciences, 61(May 2015), pp.1228–1237. Available at:

Hussain, G. & Saltman, E.M., 2014. Jihad Trending: A Comprehensive Analysis of Online Extremism and How to Counter it.

King, M. & Taylor, D.M., 2011. The Radicalization of Homegrown Jihadists: A Review of Theoretical Models and Social Psychological Evidence. Terrorism and Political Violence, 23(4), pp.602–622.

Krasodomski-Jones, A., 2017. Talking To Ourselves ? Political Debate Online and the Echo Chamber Effect, London.

McCauley, C. & Moskalenko, S., 2008. Mechanisms of political radicalization: Pathways toward terrorism. Terrorism and Political Violence, 20(3), pp.415–433. Available at:

O’Hara, K. & Stevens, D., 2015. Echo Chambers and Online Radicalism: Assessing the Internet’s Complicity in Violent Extremism. Policy and Internet, 7(4), pp.401–422.

Quine, W.V.O., 1951. Two Dogmas of Empiricism. The Philosophical Review, 60, pp.20–43.

Silber, M.D. & Bhatt, A., 2007. Radicalization in the west: The homegrown threat, Available at:

Weimann, G. & Von Knop, K., 2008. Applying the Notion of Noise to Countering Online Terrorism. Studies in Conflict & Terrorism, 31(789181513), pp.883–902.

Leave a Reply