Online Library

Digital platforms were slow to build robust teams to counter threat actors, but today, many of those corporate teams have robust processes, specialized tools, and innovative approaches to countering highly adaptive adversaries. They operate in a tremendously dynamic environment where their adversaries can innovate at low cost, primarily because of the nature of the digital “terrain” where the conflict occurs. And while the actions these teams take are not kinetic, as those sometimes utilized in geopolitical conflict, the cat-and-mouse game between Trust & Safety teams and threat actors online suggests lessons that are increasingly relevant to the national security community. This article explores five factors that were key to facilitating innovation in Facebook’s approach to countering the Islamic State—and that I argue are more generalizable. They are: people, organization, legitimacy, tools, and collaboration. It also identifies lessons that can be learned from that experience. For example, we did not prioritize using a particular technology or focus experimentation in some bespoke “innovation center.” Rather, we succeeded because we were made responsible for a critical mission, were unencumbered by past process, and had the right team structured to reduce external dependencies for innovation. Basic technological innovation can occur in an ivory tower, but applied innovation requires proximity to real-world missions. You cannot expect dramatic innovation without failure and iteration in an environment of real responsibility. Fundamentally, that means that innovation requires accepting risk. The structures and incentives of Silicon Valley cannot and should not simply be grafted on to our national security infrastructure. The rewards and costs of failure are different. But military organizations should shoulder the risks associated with innovation and study the lessons of corollary efforts in Silicon Valley and the private sector more broadly.

The old military aphorism that the enemy gets a vote is oft forgotten in both Silicon Valley and Washington, D.C. This cliche is worth keeping in mind as Congress debates adjustments to Section 230 (230) of the Communications Decency Act. For starters, Silicon Valleys persistent inability to ground products in the knowledge that some users will deliberately abuse them, and the unsurprising abuse that results, motivates many in Washington to adjust 230s liability protections. But policymakers, intent on taming platforms, must not inadvertently empower the real dangerous actorsterrorists, child predators, hate groupswho will abuse any technology and any legal recourse created by adjustments to 230.

VOX-Pol Blog post.

At the dawn of mass access to the internet, Douglas Rushkof wrote Cyberia: Life in the Trenches of Hyperspace. In his book, he observed a very special moment in our recent history in which it was possible to imagine the path ahead, before most of what daily users of the internet now
experience even existed.

VOX-Pol Blog post.

A new wave of growing antisemitism, driven by fringe Web communities, is an increasingly worrying presence in the socio-political realm. The ubiquitous and global nature of the Web has provided tools used by these groups to spread their ideology to the rest of the Internet. Although the study of antisemitism and hate is not new, the scale and rate of change of online data has impacted the efficacy of traditional approaches to measure and understand this worrying trend. In this paper, we present a large-scale, quantitative study of online antisemitism. We collect hundreds of million comments and images from alt-right Web communities like 4chan’s Politically Incorrect board (/pol/) and the Twitter clone, Gab. Using scientifically grounded methods, we quantify the escalation and spread of antisemitic memes and rhetoric across the Web. We find the frequency of antisemitic content greatly increases (in some cases more than doubling) after major political events such as the 2016 US Presidential Election and the “Unite the Right” rally in Charlottesville. Furthermore, this antisemitism appears in tandem with sharp increases in white ethnic nationalist content on the same communities. We extract semantic embeddings from our corpus of posts and demonstrate how automated techniques can discover and categorize the use of antisemitic terminology. We additionally examine the prevalence and spread of the antisemitic “Happy Merchant” meme, and in particular how these fringe communities influence its propagation to more mainstream services like Twitter and Reddit. Taken together, our results provide a data-driven, quantitative framework for understanding online antisemitism. Our open and scientifically grounded methods serve as a framework to augment current qualitative efforts by anti-hate groups, providing new insights into the growth and spread of antisemitism online.