Triggers for humanitarian crises, such as mass genocide, are difficult to predict in advance due to their rare occurrence and the variety of different contributing factors, some of which can change with little advance warning. The ability to accurately estimate the likelihood of genocide and mass atrocities could help better co‑ordinate responses and prevent some of the trauma and devastation caused by these crimes.

The Early Warning Project (EWP) tried to address this challenge by improving the early warning system for mass atrocities using a novel combination of crowd forecasting, expert ranking and statistical modelling.

The project was divided into three phases. During the first phase, experts in the field participated in an annual comparison survey, where they ranked pairs of countries by choosing which is more likely to experience a new mass killing.

The results from this survey informed the selection of 17 ‘higher risk’ countries, which the EWP tracked in real time using ‘crowd forecasting’. Crowd forecasts are calculated by aggregating many individual judgements about the likelihood of events, ranging from the outcomes of political elections to the winning teams for international sporting events. Participants can update their estimate over time depending on how different factors that influence the outcome evolve. In the EWP, crowd forecasting took place over the course of a year. Previous research on crowd forecasting has suggested that a non-specialist crowd can predict geopolitical events more accurately than individual intelligence analysts.

Alongside these human predictions, the EWP calculated a risk assessment score using statistical algorithms, one of which relies on a classical machine‑learning method called random forest. The algorithms generated their estimates based on more than 30 different variables from historical datasets, ranging from basic facts about the country, such as population size, to more specific measures of attitudes on human rights and civil liberties.

The EWP produced a ranked list of more than 160 countries, based on their likelihood of experiencing a mass killing, in order to better target preventative action by governments and charities. Mass atrocities are rare events that have little historical precedent and so the EWP’s approach ensures that weak signals from the crowd consensus predictions help address gaps in the statistical risk assessment and expert recommendations. The project is an example of combining different complementary capabilities of humans and AI to inform decision‑making.