About this report

This report provides 13 case studies of the collective intelligence initiatives developed and run by the Accelerator Labs in their first year of operation. It shows how the Labs have started to use collective intelligence design to accelerate work within complex systems (five case studies), monitor the environment in real time to fill data gaps (two case studies), create new forms of accountability and governance (two early-stage case studies), and mobilize distributed problem solving (four case studies, focusing on the COVID-19 crisis). The case studies provide an overview of what each Lab did and the methods used. It summarizes some of the successes to be built on, and common challenges to be addressed, as the Labs enter their second year of implementation. This report accompanies a second publication, Collective Intelligence for Sustainable Development: Getting smarter together,¹ which provides an overview of the use of collective intelligence methods within global development.

Who is this report for?

This report is for UNDP's senior leaders, its regional representatives and country office teams around the world. It’s also for the Accelerator Labs themselves and the funders and partners who helped make these initiatives happen. Finally, we also hope it will be read by development professionals from across the sector who are keen to learn about the work of the UNDP Accelerator Lab Network in its first year.

Methodology

The case studies were identified by reviewing public blogs published by the Accelerator Labs that were tagged with the following labels: 'crowdsourcing', 'collective intelligence' and 'citizen science'. Additional examples were crowdsourced using the Network's internal knowledge management platform. Shortlisted Labs took part in semi-structured interviews via video and email with a Nesta researcher.

Authors Aleks Berditchevskaia, Kathy Peach, Gina Lucarelli, Mirko Ebelshaeuser

Published 13 May 2021

About the UNDP Accelerator Labs The United Nations Development Programme (UNDP) Accelerator Labs is the world's largest and fastest learning network on wicked sustainable development challenges. Co-built as a joint-venture with the Federal Ministry for Economic Cooperation and Development of Germany and the Qatar Fund for Development, the Network covers 115 countries, and taps into local innovations to create actionable insights and reimagine sustainable development for the 21st century.

Learn more at acceleratorlabs.undp.org or follow us at @UNDPAccLabs

About Nesta Nesta's Centre for Collective Intelligence Design helps organizations and communities to tackle complex social problems in new ways. We do this by working with our partners to identify how they can make better use of diverse perspectives, new sources of data and digital technologies. Our work draws on a wide variety of different methods and disciplines, from crowdsourcing to AI. Our goal is to help groups of people to become smarter together – creating new collectively intelligent systems that are able to address 21st century challenges.

To learn more, see https://www.nesta.org.uk/project/centre-collective-intelligence-design/ or email the team at [email protected]

Creative Commons This work is copyright Nesta licensed under a Creative Commons Attribution NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0), unless it says otherwise. We hope you find it useful.

Acknowledgements We would like to thank the following UNDP Accelerator Lab staff for their generosity in sharing their time and stories with us: Matias Acosta, Luis Cervantes, Drasko Draskovic, Arijana Drinic, Cristian Gil Sanchez, Ehsan Gul, Khavi Homsombath, Paulina Jimenez, Ievgen Kylymnyk, Veronica Moreno, Lorena Moscovich, Oleksiy Moskalenko, Luana Natali, Nguyen Tuan Luong, Gift Ntuli, Peter Nyanda, Sofia Paredes Chaux, Phan Hoang Lan, Maniphet Phengsavatdy, Gabriela Rios, Shamiso Ruzvidzo, Phynuch Thong.

We are grateful to Eduardo Gustale, Peter Baeck and Erika Antoine for their support in the research process, valuable insight and feedback on this report.

ISBN 978-1-913095-34-5

How to cite this report: Berditchevskaia, A., Peach, K., Lucarelli, G., Ebelshaeuser, M. (2021). Collective Intelligence for Sustainable Development: 13 Stories from the UNDP Accelerator Labs.

See also the associated second flagship publication: Peach, K., Berditchevskaia, A., Mulgan, G., Lucarelli, G., Ebelshaeuser, M. (2021). Collective Intelligence for Sustainable Development: Getting Smarter Together.

Foreword

The COVID-19 pandemic has triggered a once-in-a-generation global swell of home-grown ingenuity. Communities are adapting, improvising, and looking beyond this unparalleled socio-economic challenge. Their responses have included everything from a foot-operated handwashing machine invented by a nine-year old boy in Kenya, to using the power of crowdsourcing to rapidly produce 3D-printed Personal Protective Equipment, to tracking the spread of the virus via a mobile app in Cabo Verde. The challenge now is how to sustain and scale-up this unprecedented innovation surge to accelerate progress on the Sustainable Development Goals (SDGs). Part of the solution lies in harnessing the intelligence distributed across local communities and organisations. That will help us to better understand complex problems and unlock new solutions to immense global challenges - from tackling our climate emergency to boosting the protection and restoration of our planet's precious ecosystems and biodiversity.

This collective intelligence approach is founded upon the principle that an enhanced capacity to solve problems is created when people work together through the mobilization of a wider range of data, ideas, and insights. And advances in technology and data science hold the potential to tap into collective intelligence on a larger scale than ever before. In this rapidly evolving field, the United Nations Development Programme (UNDP) has teamed-up with Nesta's Centre for Collective Intelligence Design to generate a clearer vision of how collective intelligence approaches can speed up progress on the SDGs. To this end, this new publication series, Collective Intelligence for Sustainable Development: Getting Smarter Together analyses and compares the methods and tools employed by over 200 private and public sector organizations in this discipline. Collective intelligence is a core method employed by the UNDP Accelerator Labs Network – the world's largest and fastest learning network on sustainable development challenges, which now serves 115 countries. That includes, for instance, utilising real-time data sources and ethnography to gain new insights into the first-hand experiences of women and men who live on the frontlines of climate change. The accompanying report, 13 Stories from the UNDP Accelerator Labs details more of these pioneering approaches deployed by the Labs - from using big data to improve waste management in Lao PDR, to combining multiple datasets to tackle gender-based violence in Mexico.

The hands-on experience of the United Nations in every corner of the world since the COVID-19 pandemic hit has acutely demonstrated how local communities possess the bright ideas and capacity to tackle both local and global, existential challenges. We now need to fully harness the knowledge of the almost eight billion people on the planet - and disseminate their often-surprising solutions and innovative approaches. The Collective Intelligence for Sustainable Development series is more than a mere signpost on our path to building the future of development. It serves both as an invitation and a practical guide for local, national, and international development practitioners to make increased use of real-time knowledge creation, collective action and evidence-driven decision making. And ultimately, collective intelligence will serve as a vital tool to help shape a greener, more inclusive, and more sustainable planet.

Achim Steiner Administrator, United Nations Development Programme (UNDP)

Introduction

The UNDP Accelerator Labs

In January 2019, the UNDP launched a global Accelerator Lab Network with a bold mandate: to embed collective intelligence principles into sustainable development work and to experiment with a wide range of methods. Starting with ethnographers, data scientists and social innovation teams across 78 countries, the Labs were envisaged as 'the world's largest and fastest learning network on sustainable development challenges'² with a mission to rethink development for the 21st century. Collective intelligence was introduced as a core practice for the Network. Each team received a basic introduction to collective intelligence, and a playbook of tools and activities to help them design their own experiments that used collective intelligence methods. By establishing the Accelerator Labs, UNDP created a global infrastructure to operationalize the testing and use of collective intelligence for the Sustainable Development Goals (SDGs).

What is collective intelligence?

At its simplest, 'collective intelligence' can be understood as the enhanced capacity that is created when people work together, often with the help of technology, to mobilize a wider range of information, ideas and insights (Figure 1). Collective intelligence emerges when these contributions are combined to become more than the sum of their parts.

Over centuries, every society has relied on collective intelligence - sharing knowledge, culture and tools to better manage crops, combat diseases, anticipate weather patterns and much more.

Since the start of the digital age, however, collective intelligence has really evolved. There are now thousands of digital tools helping us to pool ideas in entirely new ways, and connect people across huge distances. In the 19th century, it took almost 70 years to crowdsource the 400,000 words that made up the complete first edition of the Oxford English Dictionary. A modern-day equivalent, the English Wikipedia, receives more than 1.9 edits per second and sees about 200,000 new pages created per month.³

Digital technologies are also enabling us to generate new sources of data. We can use satellite imagery or mobile phone data, for example, to create new intelligence used to inform collective action. Many of these new data sources are generated through the 'digital exhaust fumes' of human activity, or through the active contribution of volunteers. Most of this new data is made useful by legions of people helping to label the datasets that are used to train machine models and to interpret their meaning.

Increasingly, artificial intelligence (AI) is being applied in combination with collective intelligence methods - to augment and optimize the mobilization of human intelligence. This is mainly through algorithms that increase the speed and efficiency of data processing at scale. But there are other emerging uses of AI, such as modeling and visualization, which are helping to push the boundaries of how groups work together to understand complex problems such as rapid urbanization and the localized impacts of climate change.

Combining the capabilities of people, data and digital technologies for solving problems is the next frontier of 21st-century collective intelligence.

How this is done, however, matters as much as what is done. To use collective intelligence well means being informed by its fundamental key principles (see Figure 2).⁴ This includes a recognition of the need to draw on diverse views and perspectives to come to better decisions, and the need to be mindful of biases and power relations. It also means acknowledging the importance of data empowerment - enabling people to use data to advocate for their rights, take collective action or hold governments to account. As a practice, collective intelligence is informed by many of the same values that have driven the use of participatory decision making and action research involving local communities in development since the Participatory Learning and Action approaches of the 1990s. It's critical to hold on to these foundations even as technology and new data sources change the way collective intelligence is deployed.

Collective intelligence employs a wide range of different methods. These range from deliberation to crowdsourcing and web scraping (see the Glossary for a full list). Although some of these methods remain relatively new to the world of development, their use has been slowly increasing alongside more traditional forms of intelligence, from official data to ethnographic research.

Leveraging collective intelligence for the SDGs: the case studies

During the first full year of operation, the UNDP Accelerator Lab Network experimented with a rich variety of collective intelligence methods, tackling issues from depopulation to waste management. Perhaps unsurprisingly, many Labs started with relatively mature methods, such as challenge prizes, geographic information systems (GIS), crowdmapping and citizen science.

The availability of open tools and resources, and the established communities of expert practitioners in national innovation ecosystems, were important enablers. For example, in Tanzania, the Lab used the OpenStreetMap platform to carry out crowdmapping activities, while in Argentina, the Lab partnered with a global citizen science project to build DIY sensors for measuring air quality. Other Labs experimented at the boundaries of collective intelligence practice, exploring methods that have rarely been tested in real-world settings. In Mexico, the Lab launched two such initiatives that use exploratory data analysis of open government data and non-traditional datasets to evaluate what works in public sector spending and programs.

The Labs' first year coincided with the global COVID-19 pandemic, and many teams pivoted their portfolio to help manage the impact of the crisis in their countries. Four of the case studies presented here concern COVID-19-response directly, while others had to adapt quickly as a result of changing circumstances, for example by moving in-person activities to online platforms.

Although it's still early days, the Labs' combined portfolio of experiments in collective intelligence design provides a glimpse of how collective intelligence approaches can serve multiple purposes to advance the SDGs. Opposite, we summarize the 13 case studies, grouped into four 'use cases' - practical ways in which people are using collective intelligence approaches to achieve ambitions relating to the SDGs. In the pages that follow, we provide a brief introduction to the main features of each use case, as well as the detailed accounts of the relevant Labs' initiatives.

Real-time monitoring of the environmental conditions

• Argentina: air quality
• Ukraine: open burning and environment

Working with complex systems

• Lao PDR: waste management
• Serbia: depopulation and labor markets
• Tanzania: waste management and informal settlements
• Viet Nam: waste management and circular economy
• Zimbabwe: food security

Distributed problem solving

• Bosnia & Herzegovina: ideation and innovation
• Colombia: COVID-19 response
• Ecuador: COVID-19 response and informal economy
• Guinea-Bissau: partnership and civil society action

New forms of accountability and governance

• Mexico: gender based violence
• Mexico: performance evaluation system

Over the last ten years, governments worldwide have increasingly committed to regular monitoring and sharing of data about the state of the environment, from biodiversity to air and water pollution. However, for most countries, the scale of environmental issues and the pace of change threaten to outstrip the monitoring resources that currently exist.⁵ Collective intelligence methods like citizen science and in-situ or remote sensing methods are gaining traction worldwide amongst institutions and researchers tasked with environmental protection. Environmental monitoring projects led by citizens typically bring together professional scientists and groups of local volunteers, and sometimes use satellite or sensor data.

These partnerships between concerned citizens and experts are particularly relevant for SDGs 11, 12, 14 and 15. They help to create more granular and up-to-date datasets about the state of the environment, which can inform the design of more locally responsive interventions. By involving community volunteers in collecting and interpreting data, they also help those affected by pollution to see system dynamics and take action against environmental degradation.

Case study: Argentina

• Cheaper, more granular data for localized government response

Case study: Ukraine

• Empowering communities to take action with novel data

Argentina

Using DIY hardware and citizen sensing to measure changes in air quality

Methods - participatory sensing, citizen science, open source hardware

People - students, citizens, scientists, Ministry of Environment and Sustainable Development, city-level administrations

Data - citizen-generated data on air quality

Technology - DIY air quality sensors

SDGs - 11, 17

What is the problem?

The WHO has estimated that around seven million premature deaths globally are caused by air pollution every year.⁶ Air quality monitoring is important for ensuring that pollution is brought down within limits that are acceptable for health and wellbeing. These measurements are typically supported by expensive fixed-location sensors and stations that sample at the macroscale level. Although the data from these sensors is very accurate, they are costly to maintain and calibrate. They also fail to capture the fluctuations in air quality across cities or the typical exposures to pollutants as citizens move around.⁷

What did the UNDP Accelerator Lab and partners do?

Using participatory citizen science and low-cost sensors, the Lab set out to understand of differences in air quality across different parts of Buenos Aires. The team established a partnership with open-seneca,⁸ a worldwide network of citizen science pilots that measures air quality using low-cost sensors. They ran two workshops where 80 local university students learned to build the air quality sensors, helping them to develop new skills and learn about the impact of air quality on health. Each sensor cost just US$150, in contrast to the US$50,000 cost of official sensors. Then, using an open call, the team recruited 20 volunteers to take part in the data collection. Over a period of seven weeks, these 'citizen sensors' collected data about daily changes to air quality around Buenos Aires by carrying the DIY sensors on their bikes. Each user regularly uploaded their data to the open-seneca platform where it was aggregated to produce a city-level visualization of air pollution (Figure 3).

What was the benefit of using collective intelligence for this issue?

The main benefit is a more granular understanding of air pollution. The aggregated data was shared and discussed with the Ministry of the Environment and Sustainable Development and the Buenos Aires City Government. Although official air quality data has shown that the city generally remains within the limits recommended by the WHO, the citizen-generated data helped to demonstrate localized hotspots (such as traffic junctions) where pollutants exceeded acceptable levels. Using collective intelligence helped the team to generate new data on air quality cheaply and quickly. Since the pilot, officials from the city administration and Ministry of Environment and Sustainable Development have worked with the project team to explore other measurements that could be obtained through citizen sensing, such as noise and temperature.⁹

What next?

The Lab has launched similar air quality pilots in other cities across Argentina in partnership with local authorities. These include the provinces of Mendoza, Cordoba and Tucuman. In these locations, citizen science will be used to create a much needed primary data source about air quality.

What does this experience tell us about collective intelligence design?

The project helped the Lab to understand the value of identifying specific policy needs early on in the project design to ensure that the results of a project can be taken up by the government. By working together with the Ministry of Environment and Sustainable Development, local governments across Argentina, scientists and citizens, they learned to tailor follow-on projects according to the priorities of each location. For example, in Buenos Aires they'll use temperature and humidity measurements obtained by the sensors to design urban 'climate corridors', while in more rural locations, they'll focus on measuring the impact of local practices such as open burning on air quality. This is helping the Lab to ensure that the data created through the citizen sensors has a clear route to impact.

Case study: Ukraine

Combining satellite data and crowdsourcing to map open burning in rural communities

Methods - GIS, crowdmapping, collective sensemaking

People - rural community groups, local politicians, Center for Innovations Development (NGO partner with GIS expertise)

Data - satellite data, citizen-generated data on open burning and composting

Technology - information dashboard

SDGs - 11, 12

What is the problem?

Open burning of waste is a widespread practice in Ukraine, with deep roots in culture, the agricultural economy and household habits. Official reports estimate that between 36,000-56,000 fires occur each year with devastating impacts on local air quality and the natural balance of ecosystems. Burning waste also leads to economic losses which can amount to millions of US dollars.¹⁰ Despite the scale of the problem, the government lacks granular regional data about open burning or insight into the uptake of sustainable waste management practices, such as composting.

What did the UNDP Accelerator Lab and partners do?

To better understand patterns of waste management practices across the country, the Lab set out to create an up-to-date map of open burning and composting in rural communities using a combination of open satellite data with GIS and crowdsourcing. By engaging with grassroots groups at all stages of design and implementation, they hoped to inspire regional groups to take action against open burning in their communities. Working with the Center for Innovations Development, the team created a prototype dashboard that visualized the data. The dashboard was shared with community groups over a series of ten online meetings due to COVID-19 restrictions. The sessions were attended by more than 200 members of grassroots groups, as well as policy makers from local authorities, with approximately 20 groups taking part regularly. Some meetings focused on interpreting GIS data and using open source tools, while others examined the health impacts of open burning and air pollution through group deliberation. The team facilitated open discussions through chat and video features, as well as using collaborative workspaces for attendees to document their understanding of the issue and suggest solutions to specific problems.

What was the benefit of using collective intelligence for this issue?

During the community meetings, the regional groups developed the skills to analyze open source data and use GIS themselves. The GIS dataset provided a starting point for the participants to develop a shared understanding of open burning, which catalyzed practical discussions about what could be done. This has enabled them to start taking action locally, tailoring their plans to specific burning activity in their regions. For example, the community of Zasulska identified that the majority of local fires occurred in corn fields, while in Torchynska the data helped residents to link open burning to agricultural and grass fields. The Lab also crowdsourced the location of 367 previously unknown composting sites, many of them maintained by individual households or small groups. These were integrated into the data dashboard to help local activists assess their feasibility as an alternative to open burning. While incomplete, this data provides a starting point for scaling citizen-led composting practices that are otherwise excluded from official statistics.¹¹

Footnotes

Real-time monitoring of environmental conditions

What next?

The Ukraine Ministry of Environmental Protection and Natural Resources Ministry of Environment has expressed an interest in using the open burning dashboard to inform future programs on air quality and disaster risk management. The team are currently working with the government to identify funding that would allow them to scale up and integrate the prototype into official monitoring infrastructures.

What does this experience tell us about collective intelligence design?

The team openly explored novel datasets together with participants. Doing so, allowed them to build trust on a sensitive issue, while developing shared goals and a collective understanding. This was notably lacking when it came to official data sources, which the community groups regarded with suspicion. The team also noticed that by moving the meetings online in response to COVID-19 restrictions they substantially increased diversity by bringing in people from different sectors (civil society, and local and national authorities and businesses) and different parts of the country. Both written and video contributions were more diverse, a key design principle for collective intelligence, which helped the Lab to facilitate more informed and wide-ranging discussions.

Working with complex systems

From rapid urbanization to climate change, development policy makers are increasingly grappling with complex systems. Tackling these issues often requires change at multiple levels from policies and institutions to individual behaviors. Collective intelligence approaches that combine multiple data sources can help local governments and development organizations to: better understand the changing dynamics of complex systems; design more effective interventions; and monitor their impact in real time.

Methods that mobilize inputs from underrepresented groups or those beyond the fray of ‘usual suspects' help to shine a light on parts of systems that can otherwise be overlooked. During their first year of operation, the UNDP Accelerator Labs have demonstrated how they have experimented with collective intelligence methods to help generate new insights into complex systems and issues – from air pollution to employment and waste management. Five of the case studies are presented here, categorized under three main benefits that all lead to improved understanding of a complex system.

• Case study Lao PDR, Serbia
  • Benefit Novel insights into patterns of human behavior within a complex system
• Case study Tanzania
  • Benefit Cheaper, faster data for localized government response
• Case study Viet Nam, Zimbabwe
  • Benefit Making the contribution of the informal sector visible to the system

Lao PDR

Combining big data with qualitative data to understand community waste behaviors

• Methods GIS, ethnography, microsurveys
• People community groups from three villages, student volunteers, DCYA (NGO partner), GIS consultant
• Data satellite data, citizen-generated data of open burning
• Technology mobile app for crowdmapping
• SDGs 11, 12

What is the problem?

Vientiane, the capital city of Lao PDR, produces more than 1,100 tons of waste daily. ¹ Formal waste collection services are provided by a mix of private and public actors, and come with a charge to local communities. It's estimated that less than a third of the city's million residents access waste management services, leading to the dumping of waste and open burning. Recently, the municipality has made efforts to improve waste collection coverage and introduced regulations that aim to curb the use of harmful waste disposal practices. However, open burning remains common among residents, filling the streets of the capital with thick grey smoke that causes adverse environmental and health impacts.

What did the UNDP Accelerator Lab and partners do?

The Lab wanted to use collective intelligence to better understand what motivates communities to burn their waste. Using a combination of GIS mapping with satellite data, ethnography and town hall discussions in communities, they set out to create a better understanding of open burning in three villages around Vientiane. The team first created an Air Quality Index with GIS mapping to call attention to the effects of open burning on air quality. Although satellite images helped to highlight the hotspots where burning takes place, the team realized this gave no insight about the reasons for the instead activity. Working with the local NGO, Disadvantage Children and Youth Development Association (DCYA), the Lab tapped into 20 volunteers to run community meetings that would help to fill these data gaps. The Lab used a combination of group deliberation, ethnography and microsurveys to enrich the mapping data with qualitative insights about the drivers of waste behavior in the villages. Working with community leaders, they organized six town hall meetings about waste management for the residents and local authorities. During the meetings, the villagers discussed their waste disposal practices and the environmental impact of open burning.

What was the benefit of using collective intelligence for this issue?

The GIS mapping of open burning hotspots provided up-to-date data about the scale of the problem, improving existing official data on community level waste management practices. The interviews with local residents surfaced gaps in the waste management system that can be addressed by the municipality, such as the need to reduce the price associated with waste collection and to extend collections to hard-to-reach locations. Through community meetings and microsurveys, the Lab learned more about the frequency of open burning by residents as well as their motivations for continuing to do so. For example, residents reasoned that it was difficult to bag certain types of biomass, such as branches, grass and leaves, so it was easier to burn them. These insights helped the Lab and partners to better understand some of the human drivers behind the patterns of waste accumulation in the community and the open burning highlighted by the maps. During town hall discussions, the team discovered that some residents were already using alternative waste disposal practices, such as personal composting systems, to help them manage their organic waste. Here, collective intelligence helped to uncover a locally appropriate solution. These types of solutions can be overlooked when external innovators work on a problem.

What next?

As a result, the village leaders and local authorities are exploring options for centralized composting sites at local schools, which could serve multiple households. Learning about novel data sources and analyses used by the Lab has proved particularly useful to UNDP environmental experts. The insights from this project have already fed into the wider portfolio of environmental work being carried out by UNDP Lao PDR on local air quality. New methods have helped visualize the problem space and create a shared understanding for future action.

What does this experience tell us about collective intelligence design?

A key learning was the importance of designing community-led data collection efforts together with communities to ensure solutions can be taken up locally. In the early stages, the Lab was keen to involve village residents in mapping local open burning practices using a simple app. However, after some initial prototyping they discovered that most participants didn't have access to smartphones and were reluctant to map burning events due to local sensitivity around the issue. By partnering with community leaders and the NGO, DCYA, the team built relationships with local residents over time, helping them to overcome barriers of distrust, and surface novel insights and potential solutions. This demonstrates the importance of having trusted intermediaries and expanding the network of stakeholders that UNDP teams typically work with.

Serbia

Using Google search and LinkedIn data to map labor market trends

• Methods web scraping, combining data sources, challenge prize
• People Serbian working-age diaspora, innovators
• Data LinkedIn skills data, Google search data, mobile phone data, Facebook data, satellite data, job ads data
• Technology n/a
• SDGs 8

What is the problem?

Serbia has one of the 10 fastest-shrinking populations in the world, with an 18.9 per cent decrease in population predicted between now and 2050 ² and approximately 120,000 people reportedly leaving the country each year to pursue work opportunities in other countries. This pattern of depopulation has a significant impact on the country's labor market, through the loss of the youngest people from the workforce and a growing deficit among both high- and low-skilled workers. By better understanding the education and professional opportunities sought abroad, the government aims to develop policies to stimulate their return in order to bring their networks and skills back home. Governments struggle to track the dynamics of modern economic migration in a precise way, especially when national statistics offices rely on traditional and infrequent data collection practices such as the census.

What did the UNDP Accelerator Lab and partners do?

To overcome the limits of traditional labor market data, the Accelerator Lab used collective intelligence to map and understand the drivers of outward migration in real time. The team integrated novel data (made available by the World Bank) about Serbian users of LinkedIn and Serbian language work-related Google searches. The LinkedIn data was anonymized to preserve privacy, but contained valuable information about the locations, industries and skills of Serbian professionals.

The Lab also launched a data challenge with a US$50,000 prize to surface other unusual data sources that could reveal further insights about the labor market and depopulation, or triangulate the findings from the other data. Winning ideas included using Facebook Ads to identify emigration trends, and analyzing mobile phone data to better understand activity, connectivity and mobility of people within Serbia.

What was the benefit of using collective intelligence for this issue?

The LinkedIn data has helped the team to identify trends in the skills lost and industries affected by relocation of working-age Serbians. The data revealed that research, international affairs and financial services sectors were heavily affected, while some of the top skills lost included genetic engineering and AI. Through Google search data, the team were able to uncover microtrends in the geographical distribution of the Serbian diaspora. For example, the data highlighted a significant outflow of doctors and healthcare practitioners from Serbia to Germany and Austria. Emerging findings from the data challenge have confirmed that Germany and Austria were the main destination countries for professionals, while Montenegro, Slovenia and Croatia accounted for migration of seasonal workers.

What next?

The team are currently exploring how these insights can form part of a broader portfolio to shape concrete policy interventions. An example of this is using multilateral agreements and incentives to encourage the reciprocal exchange of medical professionals between Serbia to Germany and Austria. They're also planning to create a centralized platform to share the results from the data challenge. The site will feature multiple near real-time dashboards on skills and sector trends to inform the decisions of job seekers and the policy makers working on skills retention.

What does this experience tell us about collective intelligence design?

When tapping into private sector datasets, it's important to remain flexible and anticipate what might be possible with a 'minimum viable product' approach. The Lab spent many months exploring publicly available LinkedIn datasets ³, which were limited in their granularity. Higher-level aggregation of data helps to protect the privacy of users of the platform, but also prevents the disaggregation of results by certain important characteristics, such as gender and age. By refining their research questions, they were able to get the most value from the available data whilst avoiding lengthy and expensive negotiations with a private company. This highlights the challenge of undertaking collective intelligence projects that draw on both proprietary and publicly available data.

Tanzania

Crowdmapping informal urban infrastructure to improve waste management

• Methods crowdmapping, mobile survey
• People local university students, Urban Water Supply and Sanitation Authority, Open Map Development Tanzania (NGO partner)
• Data satellite data, citizen-generated data (roads, buildings)
• Technology OpenStreetMap, OpenDataKit ⁴ (open source data collection software)
• SDGs 11

What is the problem?

Mwanza is Tanzania's second largest city and one of the fastest developing urban centers in sub-Saharan Africa. Official data from the city administration estimates that 357 tonnes of solid waste are produced by the city each day. ⁵ The municipal waste management system is supported by four private companies who act as aggregators and five community-based waste collection services. However, the city faces numerous challenges that threaten to overwhelm the current waste management system including low investment, a lack of recycling facilities and the growth of informal settlements with unmapped waste management infrastructure.

What did the UNDP Accelerator Lab and partners do?

The Lab used satellite data and crowdmapping to create an overview of the production and collection of waste in Mwanza's Buhongwa ward, a peripheral neighborhood with many informal settlements. They partnered with OpenMap Development Tanzania (OMDTZ), to run a virtual mapathon with ten local university students using the Humanitarian OpenStreetMap platform. Over multiple sessions in the space of two weeks, students labeled buildings, roads and waterways in the Buhongwa ward, as well as identifying possible trash sites. The resulting dataset was verified by the team at OMDTZ.

What was the benefit of using collective intelligence for this issue?

Through the virtual mapathons, the Lab generated a novel dataset about the core infrastructure of Buhongwa ward, mapping over 26,000 features such as roads, waterways, buildings and trash sites. The data has revealed the existence of new buildings that had previously not been registered by government officials. This information is being used by the city's Urban Water Supply and Sanitation Authority to add waste collection points to serve the previously unmapped informal settlements.

What next?

The team has been using the OpenDataKit ⁶ to enrich their data by mapping extra features like the location of waste dumpsites with local residents. Alongside this, they plan to collect qualitative insights about residents' perceptions and behaviors towards waste through community meetings. This data will be integrated into the existing infrastructure map produced by the remote volunteers and will help to inform the city's future waste management services.

What does this experience tell us about collective intelligence design?

Using the virtual mapathon approach and an existing platform for task allocation, the team were able to easily generate a large amount of novel data in the space of just one month, including labeling and validation. Mapathons or short duration 'blitz' methods are often used in participatory science to help fill data gaps. This can be a useful approach for collective intelligence projects that don't have, or can't maintain, long-term volunteer engagement.

Viet Nam

Mapping the role of informal waste workers with sensors

• Methods participatory sensing, GIS
• People informal waste workers, Evergreen Labs (social enterprise partner), GIS experts
• Data satellite data, citizen-generated data (routes taken), surveys of informal waste workers
• Technology GPS sensors
• SDGs 11, 12

What is the problem?

In Viet Nam, the government has set the target of recycling at least 15 per cent of waste by 2025. However, most households don't separate their waste at source and deposit both recyclable and landfill waste together at roadside and official dumping sites. Official waste management services are provided by a complex ecosystem of both public and private actors. But when it comes to recycling, it's the hundreds of informal waste workers who make a significant contribution helping to divert waste from landfills across the country. These workers, mostly women, separate out materials such as cardboard, metal and plastic from other waste and take them to collection centers. Their contribution remains largely invisible. This makes it difficult for the government to fully understand the recycling opportunities and vulnerabilities of the waste management ecosystem, which is vital for achieving its policy target.

What did the UNDP Accelerator Lab and partners do?

The Lab used participatory sensing, GIS and field interviews of informal workers to better understand their role in the waste management system in Da Nang city. The team recruited nine waste workers, ⁸ attaching GPS trackers to their bicycles, ⁹ to map the routes they took through two districts: Hoa Vang (a rural district) and Ngu Hanh Son (an urban district). A local social enterprise, Evergreen Labs, helped the team to add detail about local waste infrastructure to these maps. This included marking the locations of dumping sites (both official and illegal), recycling collection centers and larger waste aggregators. ¹⁰ Conducting interviews with informal workers also helped the team to understand the concerns and drivers of behavior.

What was the benefit of using collective intelligence for this issue?

Overall, the team created 39 maps (see Figure 5 for an example) of the waste-picking routes. By marking 165 dumpsites and 56 aggregators on these maps, they created the only overview of formal and informal waste management infrastructure in the surveyed areas. In combination with the GPS route tracking, the data revealed that waste workers covered official dumpsites, as well as smaller alleyways and roadsides which aren't served by municipal collection services. Through the interviews they discovered that workers relied on 'waste contacts', such as individual households and businesses who reserved waste for them. Interviews also brought to light the self-perceived upsides of this type of work, notably the flexibility of the job, which allows informal workers to fit their work alongside other responsibilities, such as caring for family. The data generated was used to inform the ongoing work of the UNDP's Climate Change Unit, which has partnered with women's unions across the country to create recommendations for local politicians about integrating informal workers into the official municipal waste management system.

What next?

The team will use storytelling workshops to share the findings of the study with the informal workers who took part in the route-mapping and interviews. These workshops will bring informal workers who typically operate independently, together with others in the community, including local women's unions and scrap owners, to develop new collective strategies for economic resilience. This is an important step in collective intelligence design: closing the feedback loop to ensure data and insights empower rather than extract. Towards scale, the Lab and wider UNDP team have started working with DNES, a government-run incubator in Da Nang, to establish a new hub for the circular economy. ¹⁴ This initiative will help to stimulate innovation that addresses issues of waste management and recycling more broadly across the country.

What does this experience tell us about collective intelligence design?

Making informal activities visible risks increasing power disparities and tensions between underrepresented groups and official institutions. These challenges can manifest in a reluctance to engage with international organizations and provide data or information. Initially, the Lab found it challenging to recruit waste workers to take part. They overcame this challenge by creating incentive structures that appealed to participants (small financial rewards) and conducting interviews in collection centers where the women typically took their breaks. Adapting the design to suit the routines of the workers made it easier for them to participate, and helped build relationships over time.

Zimbabwe

Combining datasets to identify supply trends in informal food markets

• Methods mobile microsurveys, combining datasets, data visualization
• People vendors, suppliers, producers trade associations
• Data qualitative data about experiences, historical data about daily market prices and volume of stockopen burning
• Technology Viamo, Flow Immersive, Magic Leap
• SDGs 2

What is the problem?

The informal sector accounts for close to 90 per cent of the economy in Zimbabwe. Due to its invisibility, informal economic activity isn't well understood, which makes it difficult to mitigate against market disruptions. For example, the country's food production and distribution industry has a high proportion of informal actors including growers, distributors and sellers. In the wake of the COVID-19 pandemic, Zimbabwe faced sudden food shortages. But it was difficult to identify the exact sources and drivers of this disruption due to data gaps about the ecosystem that supports food production.

What did the UNDP Accelerator Lab and partners do?

To draw attention to the informal economy's contribution to food supply chains, the Lab used a combination of non-traditional data sources and qualitative insights from informal actors. The Lab worked with three major national trade associations ¹¹ to obtain historical datasets on the volume and pricing of food supplies to informal markets. Although the associations held many datasets about the daily changes to market prices between them, the data was not interoperable and hadn't previously been combined. The Lab combined this data into an immersive virtual reality data visualization ¹² to help communicate insights about trends, market vulnerabilities and associated behaviors more effectively to decision makers.

The team also enriched this data with insights from a qualitative survey of more than 3,000 vendors, distributors and suppliers from across the country. Using the interactive voice tool, Viamo, ¹³ they sent voice response questionnaires to vendors in three different cities in Zimbabwe (Mutare, Harare and Bulawayo), as well as several smaller border towns.

What was the benefit of using collective intelligence for this issue?

By combining these novel datasets, the team were able to analyze trends and fluctuations within the entire food supply ecosystem in Zimbabwe. For example, they identified a consistent annual trend of a drop in sales in January, which emerged from surveying the daily market transaction data at the national level. In 2020, unlike previous years, this seasonal decrease continued into March and April due to the introduction of lockdown measures, which further exacerbated the loss of earnings experienced by vendors. The data also helped the team to quantify the market impact of droughts caused by climate change on important crops, such as maize. By making the data from different associations interoperable, the team helped to establish a new standardized data infrastructure for understanding the value chain of fresh produce. This data capture system is the only overview of the food production system in all its complexity. It provides a foundation for trade associations and politicians to continue tracking activity in the future.

Footnotes

What next?

The team shared the visualized data¹⁵ with the trade associations, vendors and policy makers to help them better anticipate regular changes in the market, as well as emerging trends linked to crises like COVID-19 and climate change. This has led the Lab to design a broader portfolio around urban food poverty, NextGenCities, ¹⁶ together with government, local councils and vendor associations. As part of this program, they plan to make the data capture system they developed for tracking daily market prices openly accessible. They've also shared their experience of working on this issue with Lab colleagues in Eswatini and Botswana to explore scaling the model to neighboring countries.

What does this experience tell us about collective intelligence design?

It's difficult to build communities or establish new networks from scratch, and often it isn't necessary. Partnering with trade associations helped the Lab to gain the trust of informal workers who they struggled to engage with in the initial phases of the project. The partnerships also extended their reach geographically, allowing them to generate insights using national scale data across a range of different locations. For collective intelligence to succeed, tapping into existing coalitions and networks generates meaningful insights and change.

Distributed problem solving

Just a few months after the UNDP Accelerator Lab Network was established, the COVID-19 pandemic struck. Many of the Accelerator Labs quickly reoriented their work – focusing particularly on generating new, local solutions that responded to the challenges each country was facing. For example, many countries faced shortages of personal protective equipment (PPE) among healthcare staff¹⁷ and citizens, due to a lack of established local manufacturing and supply chains. It was also difficult for development organizations and public institutions to keep up to date with what was already being tried in response to the pandemic by communities and other actors.

The Network used a variety of collective intelligence methods – from creating open innovation platforms to using challenge prizes, peer-to-peer exchange, and collaborative solutions mapping. The benefit of these approaches was that when speed was critical, they were able to rapidly generate a wide range of solutions from within the community. This increased the likelihood of finding something that was appropriate to local needs. Four examples of the different approaches taken by the Labs are provided in the case studies below.

Case study

Bosnia & Herzegovina

Benefit

Surfacing a wide range of solutions quickly

Case study

Colombia, Ecuador

Benefit

Tapping into local expertise to solve a specific problem

Case study

Guinea-Bissau

Benefit

Supporting the innovation ecosystem

Bosnia & Herzegovina

Crowdsourcing ideas for COVID-19 response to foster civic engagement

Methods crowdsourcing ideas, competition, peer-ranking

People students, citizens, entrepreneurs, Bit Alliance (membership organization of local IT companies), industry (e.g. Deloitte, PWC) and government (e.g. Ministry of Environment and Tourism)

Data citizen-generated data (ideas)

Technology CovIDEJA2020 (ideas platform)

SDGs 3

What is the problem?

From the early weeks of the COVID-19 pandemic, it became clear that the impacts would be widespread and long lasting. In Bosnia & Herzegovina, there were no existing online spaces for public discourse or to coordinate civil society responses to the pandemic. This made it difficult for those working in the public and private sectors to understand the priorities of citizens and where solutions were most needed or relevant.

What did the UNDP Accelerator Lab and partners do?

The online ideation platform, CovIDEJA2020, ¹⁸ was launched by the Accelerator Lab in April 2020. Its purpose was to mobilize citizens to contribute novel solutions to COVID-19-related issues that could be implemented within a short time.

The Lab partnered with 28 organizations, including local IT companies, international organizations (such as Deloitte) and the Ministry of Environment and Tourism. They invited ideas across 14 different topic areas ranging from economic interventions to help citizens manage the crisis, to health interventions and social support. Members of the public could submit ideas over a period of two weeks and all ideas were ranked by both an expert jury and a public vote through online voting.

The final selection was made by combining the choices of experts and citizens to ensure that winning ideas were both viable and popular. Of the 109 ideas submitted, six were selected to receive a financial reward and tailored support towards implementation, including mentorship from industry partners and pitching sessions to country-wide innovation networks. Several of the winners were invited to participate in the UNDP Boost Innovation Accelerator Program, to support their professional development.

What was the benefit of using collective intelligence for this issue?

The winning ideas included a novel approach to vaccine development, a mask recycling scheme and apps that supported the delivery of education and mental health services. Using this open innovation approach helped the Lab to surface existing solutions that could be transferred to the COVID-19 context (e.g. a new vaccine technology previously developed for tuberculosis). It also helped them to support ideas which addressed the problems that really mattered to citizens. For example, the team that pitched to develop D-App – a mental health support platform – identified the need to support citizens' mental health long before the subject attracted the attention of policy makers. D-App¹⁹ was launched in late 2020, facilitated by the attention, funding and mentoring that they received through CovIDEJA2020. In the first month, D-App attracted 1,500 active users and facilitated 250 successful connections between people and therapists in the region.

What next?

The CovIDEJA2020 platform is being developed further to act as a repository of ideas for the innovation partners and a collaboration hub for the public. Some of the partners have continued to work with the winners on developing their ideas. This is the case for Sharklab Center for Marine and Freshwater Biology who are supporting one of the winners, Dženan Kovačić, to carry out proof of concept experiments on the efficacy of his vaccine technology. Building on their experience of the ideation challenge, the Lab has started to use collective intelligence more widely across their portfolio. For example, since CovIDEJA2020 they created a public awareness campaign that used crowdsourcing to design public health messages about the disposal of infected materials, e.g. face masks.

What does this experience tell us about collective intelligence design?

Approximately two thirds of all 70 entries were submitted in the 'Act Now!' category rather than the future-facing 'Re-imagine the Future'. This suggests that citizens may be especially motivated to propose solutions for immediate problems whose effects are already being felt.

CovIDEJA2020 also helped the Lab to learn about the challenge of negotiating longer-term support from their partner organizations upfront. Sustainability support for the innovators remains a challenge. While all of the CovIDEJA2020 partners were surprised at the high quality of the ideas and the popularity of the competition among citizens, only a few of the partner organizations committed resources to supporting winners beyond the initial prize funding. Defining impact pathways together with partner organizations in advance could help to ensure that more winning ideas are taken up.

Colombia

Stimulating an innovation ecosystem to make PPE for COVID-19 response

Methods open innovation, challenge prize

People entrepreneurs, students, maker community, industry and government partners

Data citizen-generated data (solutions)

Technology community platform

SDGs 3, 9

What is the problem?

In the wake of the COVID-19 pandemic, the Colombian government introduced strict lockdown measures to curb the spread of the disease. These measures had significant socio-economic impacts, but the country had a shortage of PPE, especially face masks, making it difficult to return to economic activity.

What did the UNDP Accelerator Lab and partners do?

To address this issue, the Lab set out to prototype robust, closed face shields that could be produced and distributed quickly. This helped lockdown measures to be relaxed, reduced the spread of the disease and created new supply chains to establish a PPE market. To surface the best designs from a wide range of innovators in the shortest time, the team organized a challenge prize. The Life Helmets Challenge was a collaboration among government, the maker community, academia and industry partners through a dedicated online platform. After launching the prize, innovators had ten days to submit their ideas. To ensure widespread and diverse participation, the team used a communication campaign across social media channels and targeted mailing lists, as well as online demonstrations and FAQ sessions. They invited ideas for PPE for five target groups: medical staff, patients, homeworkers, children and the broader population, and ran two rounds of the challenge over four months. They received more than 400 proposals.

What was the benefit of using collective intelligence for this issue?

Some of the winning designs came from unexpected sources such as students, DIY makers, a doctor and a taxi driver. By working with industry partners, the team were better able to identify production constraints and the viability of proposed solutions in order to support the winning teams to develop their proposals. Following the completion of the challenge, the Life Helmets platform was turned into a hub for sharing open source designs and a marketplace for the winning solutions. ²⁰ Members of the public and companies use the platform to buy and sell face shields for personal use. UNDP Colombia has purchased and distributed more than 27,000 face shields across the country, including in remote locations where communities have struggled to access PPE.

What next?

The success of the challenge prize model for COVID-19 response has been recognized by external stakeholders, including the Peruvian government and USAID, who have sought advice from UNDP about implementing similar initiatives. The team is also mapping regional manufacturing chains to support the production of life helmets and other PPE by small suppliers throughout the country and to ensure more even distribution.

What does this experience tell us about collective intelligence design?

When organizing challenge prizes in response to a crisis, it can be tempting to prioritize speed rather than taking time to research evaluation criteria or provide up-front guidance about intellectual property. The Lab learnt that getting this right at the outset helps to improve the quality of submissions and increases the likelihood that solutions will be taken up in the long run. Communicating clearly about selection and evaluation processes is also important for maintaining participation and ensuring that participants' expectations are aligned to the project goals. There's also an opportunity to scale up good ideas that come from challenge prizes. The key to achieving this is developing shared infrastructures and practices for the innovation community that forms around the prize, so they start to become a peer-to-peer network.

Ecuador

Peer-to-peer learning for economic empowerment during COVID-19

Methods peer-to-peer exchange, microtasking, community of practice

People 27 freelance seamstresses in Quito, DIY Club (NGO partner organization)

Data n/a

Technology WhatsApp

SDGs 3, 8

What is the problem?

In the wake of the COVID-19 pandemic, Ecuador, like many of its neighbors, faced significant shortages of PPE to protect healthcare workers due to the increased demand and lack of local suppliers.²¹ At the same time, workers in Ecuador's informal sectors, including seamstresses, struggled to generate income. In Quito, these workers found it difficult to adapt their skills to the changing circumstances and struggled to find new sources of work.

What did the UNDP Accelerator Lab and partners do?

Together with the social entrepreneurship DIY Club and 27 local seamstresses, the Lab helped to establish an online peer-to-peer skills exchange program across Quito. The group was largely self-organized and peer-led, with members running virtual training sessions for one another to diversify their skillsets, as well as supporting each other by sharing sewing materials and tools. In the early days of the program, the group received guidance on producing clothing compliant with biosafety standards to enable them to contribute to the pandemic response. They distributed the work amongst themselves using microtasking arranged through their WhatsApp group. Over the first four months of existence, the group members initiated and ran 16 virtual training sessions, amounting to 90 hours of upskilling through peer learning.

What was the benefit of using collective intelligence for this issue?

In return for a small financial incentive from the UNDP, the group produced 800 full sets of biosafety-compliant PPE, helping to fill a vital supply gap in the early days of the pandemic when Ecuador's healthcare system was overwhelmed.²² This PPE was donated to local hospitals. The group went on to produce over 2,000 face masks, which are being sold online, and they have become providers to the UN System. In addition to exchanging skills and resources, they provided social support for each other when individual members faced adversity and health challenges due to the pandemic. By working closely with the group, the Lab became aware of additional barriers that informal workers and small-scale entrepreneurs face which prevent them from securing stable work. These include the inability of workers to fulfill large orders at short notice or absorb out-of-pocket payments for large quantities of materials before delivery, as well as invoicing limits for workers classified as 'artisans'. The group shared upfront material costs and negotiated for changes to the contracting thresholds.

What next?

Following the completion of the initial training scheme with DIY Club, the members co-founded the textile association 'Without Borders' in order to improve their long-term sustainability as a community of practice. The Accelerator Lab and DIY Club have extended the scheme to support a second group whose economic security was also badly hit by the pandemic, 'Mujeres de Frente' – an existing network of vulnerable women.

What does this experience tell us about collective intelligence design?

The project helped the team to mobilize skills and assets that were already available in the community. By working with the DIY Club and informal (and otherwise invisible) communities of practice, the Lab discovered a source of skills that was vital to the pandemic response. Using collective intelligence also ensured that they foregrounded the design principle of empowerment, so they not only addressed the PPE shortage but also helped the seamstresses to establish an association for longer-term economic viability.

Guinea-Bissau

Mapping grassroots innovations to identify gaps and coordinate pandemic response

Methods peer-to-peer exchange, collaborative platform, solutions mapping

People citizens, CSOs, grassroots associations, international organizations, RENAJ (a network of youth associations), UNDP Governance Unit

Data citizen-generated data (existing solutions and new ideas)

Technology Na Nô Mon platform

SDGs 3, 10, 16, 17

What is the problem?

For many years, international development organizations and civil society actors in Guinea-Bissau have struggled to coordinate actions in their efforts to support local communities and strengthen social cohesion. The absence of a dedicated space for this exchange to take place was particularly evident during the early months of the COVID-19 pandemic, where stakeholders struggled to understand local needs and to map existing grassroots efforts to mitigate the impacts of the pandemic.

What did the UNDP Accelerator Lab and partners do?

The UNDP established the Na Nô Mon platform²³ as a shared resource to help international and local development actors identify unmet needs across the country. It also functions as a coordination tool for more effective response to the pandemic and in the longer term. There are close to 400 individual members, representing around 100 civil society organizations (CSOs) and grassroots associations. The platform provides up-to-date guidance on COVID-19, and allows members to engage in dialogue, though community rooms. Members can also share resources and events related to sustainable and inclusive development.

When setting up the platform, the UNDP deliberately worked with organizations with ties to underrepresented groups across the nine regions of Guinea-Bissau. For example, the team partnered with the National Network of Youth Associations, RENAJ, to help deliver training on the use of the platform and build engagement with younger people and communities at large. By partnering with the women's radio in Bafatá they increased the participation of younger women. The platform has also been used as a coordination hub for a small grants scheme that supports grassroots innovations. This has helped grassroots groups, who are otherwise isolated, to connect to a wider community of innovators, while getting more visibility and recognition for their work.

What was the benefit of using collective intelligence for this issue?

In only the first six months, the platform became a community hub and resource center to support civil society action across Guinea-Bissau. Members of Na Nô Mon have mapped COVID-19 response case studies from different parts of the country and these are shared on the Solutions section of the platform to be searchable by others. The platform lists the expertise and interests of members, helping individuals and organizations working in development to establish collaborations with others more easily. In August 2020, the platform was used by 22 CSOs to convene a 'Civil Society Forum for Consultation'. This series of online and in-person dialogues aimed to create a common agenda and principles for collaboration – both during and beyond the pandemic.

A central town hall (Djumbai) was held in the capital city and groups from other parts of the country contributed to the discussions via the Na Nô Mon platform and Zoom. Using the platform meant that voices from different regions across the country were included in discussions and prioritizing issues. This resulted in the publication of a Common Agenda for Peace and Stability in December 2020 – which represented diverse views and had multilateral buy-in. For example, participants from regions outside the capital drew attention to local challenges, such as conflicts arising from limited socio-economic opportunities or the use of rural lands. The Common Agenda for Peace and Stability is now being used by the members to guide their interventions.

What next?

As a result of creating the most complete open repository of COVID-19 response efforts in Guinea Bissau, the team has been praised by the country's High Commissioner responsible for pandemic responses. The Commissioner's office has invested US$250,000, through the African Development Bank, for the team to allocate small grants for members and projects identified through the platform. Additionally, the recently launched Accelerator Lab has continued to expand the Solutions section of Na Nô Mon by working with local NGOs to geo-tag and map community projects throughout the different regions of Guinea-Bissau.

What does this experience tell us about collective intelligence design?

Before creating the platform, the team spent many months building relationships with other international and civil society organizations to establish a common foundation of trust and collective goals. This was vital for understanding the needs of the community and securing buy-in from those who were expected to use the platform. The process highlighted the challenge of introducing collective intelligence approaches without a shared understanding and the capacity to deliver collective intelligence within the local development ecosystem. Committing to a robust training and dissemination program, using a network of 18 community advocates, has helped to address this. Throughout the training sessions, the UNDP also collected feedback about the accessibility of the platform, which has been used to continuously improve the user experience for Na Nô Mon members.

New forms of accountability and governance

Evaluating what works is a vital part of governance and at the center of SDG 16. Yet monitoring the implementation and impact of government policy or programs is often neglected by innovators, who can be preoccupied by designing new services or solutions.

Case study

Mexico: gender based violence

Benefit

Evaluating the relative success of different policy measures

In Mexico, the Accelerator Lab is trailblazing new approaches to address this gap, using novel collective intelligence methods (such as text mining of new data sources) to help the government better evaluate what's working.

Case study

Mexico: performance evaluation system

Benefit

Identifying common barriers to effective policy implementation

Two early-stage initiatives from Mexico provide a glimpse of a potential niche for the UNDP zAccelerator Labs to use collective intelligence to advance SDG 16, working in partnership with national governments.

Mexico

Combining multiple datasets to understand gender based violence (GBV) in public spaces

Methods combining data sources, GIS, crowdmapping, mapping positive deviance

People GIZ Data Lab, GIZ Mexico, Center for Command, Control, Computing, Communications and Citizen Contact (C5) of Mexico City, Digital Agency for Public Innovation, Ministry of Women, women in Mexico City

Data open government datasets, non-public government datasets about GBV reports, [citizen-generated maps of safe locations]²⁴

Technology n/a

SDGs 5, 16

Footnotes

What is the problem?

In Mexico, more than half of the homicides of women in 2019 (52 per cent) occurred in public spaces. ²⁵ High rates of femicide have drawn increased international attention. In the face of this growing problem and public pressure, the Government of Mexico City declared a public alert about GBV in November 2019. ²⁶ Following the alert, it worked to increase the safety of public spaces in the city by introducing safer public pathways and panic buttons to encourage reporting of incidents. The impact of these different interventions on rates of violence in the city is still unclear and policy makers are keen to understand where they have proved most successful.

What is the UNDP Accelerator Lab planning to do?

In collaboration with GIZ Data Lab and GIZ Mexico, the Lab is combining open data about urban infrastructure (such as public transport) with closed government datasets about the use of new panic buttons. They hope this will lead to a better understanding of the urban characteristics associated with GBV in public spaces across Mexico City. They will also work with datasets that tap into residents' perceptions of urban safety, such as sentiment analysis of social media and crowdmapping with local women's groups. By mobilizing these different sources of intelligence, they will test whether collective intelligence can help them to identify the urban spaces where women are safest (using a method known as data driven positive deviance²⁷). Cross-correlating their findings with the known locations of the government's safer pathways will also help the government to evaluate which policy measures have been most successful, in order to guide future interventions.

What does this experience tell us about collective intelligence design?

The Lab has benefited from the government's long standing support of open data practices and existing data infrastructure within the public sector. However, negotiating a Memorandum of Understanding for the use of non-public government datasets can still take many months. To overcome this, the Lab has started by working with open datasets to develop a proof of concept (see Figure 8). Thinking creatively about how they can already work with existing available data, as they wait to finalize data sharing agreements, has given the team flexibility and time to experiment with new methods.

Case study Mexico

Text mining with natural language processing (NLP) to identify barriers to policy implementation

Methods combining data sources, NLP, [crowd labeling]²⁸

People policy makers, [volunteers], data scientists

Data free-text entries by policy makers, open government data

Technology n/a

SDGs 16

What is the problem?

In 2010, the Mexican government introduced a performance evaluation system across all departments to keep track of the impact of public programming and spending. The system, based on the Logical Framework methodology from USAID, is intended to increase accountability and improve results of public service delivery. Over the years it has been used by civil servants across the government to keep track of their progress against a set of indicators unique to each public spending initiative. The evaluation includes open text entries where civil servants write a brief justification to account for goals that weren't met. To date, there has been no analysis of this rich text dataset which spans many different public services over the last ten years – despite its potential to surface novel insights about common barriers to implementation which could improve the design of future programs.

What does the UNDP Accelerator Lab plan to do?

The Lab is mining the text entries using NLP²⁹ to cluster and rank dominant themes across this vast public service dataset. By training an AI model to compare text entries with a set of predefined common causes and identify novel themes, they're aiming to improve the reporting process for civil servants in the future. The algorithm is enabling them to analyze and make use of civil-servant generated data that's otherwise too unstructured for centralized analysis. Eventually the team hopes to build a hybrid collective intelligence model, where the quality of evaluations is improved by combining NLP classification and inputs from civil servants in real time.

What does this experience tell us about collective intelligence design?

Tackling a complex problem, like improving the effectiveness of government policy implementation, isn't something for which there is a single silver bullet solution. The text-mining process will help the government to extract lessons at a speed and scale that was not previously possible. However, it will need to be implemented alongside other efforts (such as changes to the user interface of reporting software) to make the existing program evaluation system more effective. This will help to ensure that the insights from the model are integrated into the machinery of government and affect future program design. The Lab is already considering how to achieve this. Ideas include involving civil servants in verifying the model and developing skills across government teams to enable them to work with the new system in the long term.

Seven key takeaways from applying collective intelligence

Working with people to create change

Move beyond the usual suspects to build new coalitions for the SDGs

The Accelerator Labs have demonstrated the benefit of engaging with existing networks to expand the potential reach, quality and impact of their collective intelligence efforts. For example in Zimbabwe, by partnering with trade associations, the Lab was able to make visible the contributions of 20,000 informal workers in the food production value chain from all across the country. Prior to this, the team had struggled to connect with informal vendors in local markets in Harare. Many other Labs have also worked with intermediaries who already had established relationships with local communities. This may be particularly important for collective intelligence design efforts that involve vulnerable communities and where the engagement is short term, remote, or there is little time to build trust.

The Labs have also worked effectively with CSOs to leverage specific technical skills. In Ukraine for example, the Lab worked with an NGO who had experience of data visualization, while in Tanzania, the Lab partnered with an organization who had the expertise to support an online mapathon with local students.

The Labs have also gone beyond the usual suspects to tap into new sources of innovation skills. For example, when working on the Life Helmets Challenge in Colombia, the Lab engaged with local maker communities to help them attract individuals with the necessary skills. And in Argentina, the Lab connected with an existing open source hardware community to support students building DIY sensors. Mobilizing new networks and partnerships is critical for tapping into diverse sources of distributed know-how to drive collective action for the SDGs.

Embracing new ways of working with data

Combine 'big' data and 'thick' data for contextualized insights

Official data can lack granularity of insight and can often be out of date. Novel sources of data, such as satellite data, can help address this. As the Labs found, however, big data might help you see patterns in what is happening, but it might not help you understand why. Collective intelligence methods can bridge these gaps by generating both quantitative measurements and qualitative insights that offer complementary perspectives on an issue.

In Lao PDR, for example, the Lab analyzed satellite data to help map open burning hotspots. However, this data missed the reasons behind the persistence of open burning in these locations, many of which were served by municipal waste collection services. To fill these gaps, the team supplemented the data with ethnographic methods, including observations and interviews, as well as discussions during community town halls. In Vietnam, the Lab used on-the-ground GPS sensing to map routes taken by informal waste workers in combination with interviews about the barriers they faced and patterns of working. This helped the team to better understand their contributions to the waste collection system.

Use novel data and technology to help build relationships and trust with communities

Grassroots communities can often question the motivations of institutional actors and the credibility of official data. While this challenge applies to development initiatives more broadly, it's especially critical when it comes to the collective intelligence methods, which by definition rely on mobilizing insights and action with, and by, communities.

As the Labs discovered, involving communities in generating and analyzing data can establish a shared and objective source of facts about an issue. This can help stakeholder groups to build trust and then propose solutions, with communities taking ownership of the results. An example of this is the collective intelligence work carried out by the team in Ukraine. The digital dashboard visualizing the locations of open burning from satellite images was considered a more trustworthy data source by community members than official reports. The Lab worked with rural communities to interpret this data, leading to new insights about how open burning patterns varied regionally. By learning how to work with the data themselves, these communities had more faith in its accuracy and were motivated to take action as a result.

This experience is also echoed in Cambodia, where the Lab found that the accelerated adoption of digital tools like Miro, due to COVID-19 restrictions, enabled more diverse people to take part in meetings. In addition, anonymous participation led to a higher quality of contributions.

Establish responsible data stewardship and remain flexible during data partnerships

Negotiating access to private sector and other closed data requires high levels of technical proficiency and can take many months. Involving communities or volunteers in data collection complicates considerations around privacy and ethics. These common challenges can be off-putting for organizations lacking in-house expertise, and steer them away from experimenting with some of the most novel data-driven collective intelligence methods.

Some of the Labs have demonstrated the viability of creative solutions for overcoming these data partnership challenges. For example, the team in Mexico started working with publicly available datasets to develop a proof of concept while they negotiated a memorandum of understanding with the government. In Serbia, the Lab reprioritized research questions to avoid making requests for sensitive or personal information, allowing them to work with publicly available LinkedIn data.

When collective intelligence initiatives work with underrepresented or vulnerable groups, they need to take extra care that data collection isn't extractive and is designed to benefit the communities involved. In Lao PDR and Viet Nam, the Labs worked with underrepresented communities and invested significant resources in outreach and engagement. This is an important aspect of creating a collective intelligence design and shouldn't be skipped. Experience also shows that partnering with trusted intermediaries (such as in Zimbabwe where the Lab worked with trade associations) can also be an effective route. Developing a central or regional support function for data stewardship that can provide guidance about different models for data partnerships and ethical data practices, may encourage local teams to work with new data sources.

Building pathways to impact

Emphasize the potential of collective intelligence for agile policy-making and program design

A number of the UNDP Accelerator Labs have demonstrated the value of collective intelligence approaches to inform more agile, localized and responsible governance. In Tanzania, for example, the team set out to understand the waste management practices in a semi-rural ward of Mwanza city, which had seen a rise of informal settlements as a result of urbanization. By using crowdmapping they created an up-to-date map of the infrastructure of this fast-changing neighborhood and identified potential sites of waste accumulation. These maps are already being used by the city's Urban Water Supply and Sanitation Authority to understand how to adapt their waste collection services. Ukraine's Ministry of Environmental Protection and Natural Resources is now aiming to scale the Lab's data dashboard for open burning as a monitoring tool for estimating air quality and disaster risk. In Argentina, officials from the city administration and Ministry of Environment and Sustainable Development are planning to use the temperature and humidity measurements obtained by the sensors to design urban 'climate corridors'. Although these examples are still at early stages, they provide a glimpse of how the Labs are helping governments make complex systems visible and understand problems closer to real time – enabling them to respond more effectively to localized issues.

Develop clear ‘hand-off' mechanisms and routes to impact for prototypes

Many of the Labs have focused on creating prototypes using new methods. Some of these have generated interest from their national governments who have taken them on or are helping to scale what was developed. A good example of this is in Argentina, where the Lab has worked with both national and city level governments to adopt citizen sensing for environmental monitoring. Clearly identifying a 'sponsor' within the UNDP or government from the outset, and ensuring those stakeholders' needs are factored into the design from the get-go, may help increase the likelihood that insights or prototypes find traction.

In Argentina, the Lab adapted the design of citizen science activities to collect environmental measurements that were more relevant to local urban planning decisions. In other cases, the prototypes have been taken up by the UNDP country office. For example in Zimbabwe, the analysis and visualizations produced by the Lab are being used to adapt the ways the country's long-running Resilience Building Fund³⁰ works with data for annual reporting. While in Viet Nam, the Lab chose the locations for their waste management project in collaboration with UNDP environmental experts. This ensured that the resulting insights could be used to complement the ongoing work of the Climate Change Unit in these districts.

However, many of the case studies weren't developed with a clear path to adoption or use. This is a common challenge for innovators tasked with socializing new methodological approaches. ³¹ The Labs could overcome this by spending more time upfront articulating the change they want to create and involving key stakeholders early in the design phase.

Invest in building capacity for the wider ecosystem

The most successful collective intelligence initiatives within the UNDP Accelerator Lab Network have been achieved by partnering with local NGOs who already have some methodological expertise in working with non-traditional data sources or participatory methods. For example, in Ukraine and Tanzania, both waste management projects depended on GIS expertise provided by their local civil society collaborators. Other Labs turned to experienced researchers based at universities, working with them on a consultancy basis. This was the route taken by the team in Viet Nam to tap into GIS expertise and by the team in Mexico to identify an NLP researcher. These cases highlight the challenge that Labs face when it comes to finding appropriate skills. The Labs have also found it necessary to invest time in helping to build understanding of collective intelligence amongst the organizations and stakeholders they work with.

For collective intelligence methods to thrive, the entire development sector will need to invest in developing these 21st-century innovation skills locally, as well as across borders. One potential source of future talent is students. In Tanzania, Lao PDR and Argentina, the Labs all incorporated training and workshops with local students into their project designs, so students could then undertake crowdmapping and ethnography, and build DIY air pollution sensors. Involving groups of students helped the teams to complete vital project tasks, including data collection and hardware assembly, whilst helping to build future capacity in the local ecosystem. UNDP country teams and Accelerator Labs should continue to cultivate skills in-house, as well as providing external training and workshops on collective intelligence methods. This will expand the pool of opportunities for collective intelligence methodologies to be applied at scale.

Three lessons at the network level

Aligning cultures and incentives for procurement and reporting

The Labs were set up to work in an experimental and exploratory way. This has at times conflicted with the UNDP's standard project management and budget setting processes, which work on the basis of pre-approved annual plans. Complex procurement and partnership processes have sometimes made it harder for the Labs to move fast or flexibly with projects. Likewise, standard impact reporting measurements - designed for traditional development projects – aren't necessarily relevant for small teams working at speed, primarily generating new insights. These challenges aren't unique - they're faced by nearly all innovation teams working within established bureaucracies. It may not be feasible to redesign all aspects of UNDP's established systems and processes for the Labs, but UNDP should be prepared to have an honest conversation about the tensions. Collaboratively reappraising existing rules and procedures to enable innovation cultures to flourish should be rewarded and recognized.

Sharing learning throughout the Network, and beyond

Across the Network, the Labs are experimenting with a wide range of methods and sharing their experiences as they learn. This is enabling the Labs to fast track the uptake of collective intelligence methods throughout the Network, as well as within the wider ecosystem of actors and partners with whom they collaborate.

Regular global meetings of Lab teams, shared internal repositories and WhatsApp groups have enabled insights to be exchanged and successful collective intelligence interventions to be replicated – particularly in countries from the same region. For example, Labs in Western Balkans and Eastern European countries that face similar depopulation challenges have been learning from Serbia's experience of combining novel data sources to map regional trends. Meanwhile, lessons from the experiments started in Zimbabwe on the informal economy were shared with neighboring Eswatini and Botswana.

Collective intelligence is a mindset that can be learnt

Embedding collective intelligence methods within development organizations requires developing new skills, capabilities and mindsets. The experience of the UNDP Accelerator Labs over their first 12 months demonstrates that these can start to form relatively quickly when practitioners are given an institutional mandate.

Lessons from the case studies in this report and beyond show that collective intelligence skills aren't necessarily about just developing deep expertise in one method. They require an openness to explore a range of methods and work with a broader mix of partners to bring in more diverse perspectives and new forms of data to solve problems.

During our research, we also uncovered a broader suite of collective intelligence practice being implemented across the Network as part of daily operations. For example, the team in Cambodia has delivered online multi-stakeholder workshops, where participant identities were anonymized to encourage equal contributions. While in Pakistan, the Lab uses both online analyses and offline ethnographic methods to carry out 'deep listening' to diverse voices. This helps them shape their projects based on the needs and priorities that are revealed. These activities, and many others across the Network, don't reach the scale of the case studies described in this report, but they demonstrate that when it comes to collective intelligence, 'learning by doing' can start small.

Appendix

Glossary of collective intelligence methods used by the UNDP Accelerator Labs

Citizen-generated data Citizen-generated data is a broad category that includes any information that can be collected from people either by active involvement (e.g. videos, reports, ideas – usually using digital platforms) or passively (e.g. transactions data, call detail records, wearables).

Citizen science Citizen science is any process where scientists and (usually unpaid) volunteers work together to collect or process scientific data or observations. Citizen science unlocks new resources for research, experimentation and analysis by opening the process to anyone.

Collaborative platform A collaborative platform enables a loosely defined group of participants to come together to create a product or service, which is typically then made available to contributors or non-contributors alike. It draws on principles of open collaboration and open source software development, whereby openness and transparency in product development improve participation, trust and collective memory.

Collective sensemaking Collective sensemaking refers to the process by which members of a group create a plausible understanding of a problem by exchanging interpretations and information to decide on a course of action.

Combining data sources Combining data is a process of bringing together two or more different datasets to unlock new value or generate new insights that would not otherwise be exploited. It may involve partners entering into an agreement to exchange data for a specific social cause. These datasets may include data that's passively generated by people (e.g. call detail records), or actively contributed (e.g. citizen reporting).

Community of practice A community of practice is a group who share a concern and learn by interacting with one another regularly. In pursuing their practice area, participants help each other, and build up a repertoire of shared resources (experiences, tools and solutions) while working together.

Crowd forecasting Crowd forecasting is a method that asks small or large groups to make predictions about the future. Individual predictions are aggregated using statistics into a consensus crowd forecast. It's inspired by research which showed that small crowds of non-experts can often forecast political events more successfully than individual experts.

Crowdmapping Crowdmapping is a type of crowdsourcing which gathers data from different sources (including social media, text messages or geographic data) to provide real-time, interactive information about issues on the ground. Crowdmapping can create detailed almost real-time data in a way that a top-down, centrally curated map may struggle to replicate.

Crowdsourcing Crowdsourcing is an umbrella term for a variety of approaches that source data, information, opinions or ideas from large crowds of people, often by issuing open calls for contribution. It can help bring new ideas to light that hadn't previously been considered, or to gather expertise from people who have specialized knowledge or understanding of an issue.

Data visualization Data visualization encompasses a range of techniques for representing information, from complex graphs to mind-maps and tools for illustrating complex concepts or ideas. They can be static or dynamic, and may also be connected to databases, or updated manually. Data visualization can help to bring complex or messy data to life in new ways, aiding collective awareness and understanding of the issues that matter.

Geographic information system (GIS) A GIS is a system designed to capture, store, manipulate, analyze, manage and present all types of geographical data.

Microsurvey Microsurveys are a short, abbreviated form of surveying which typically take the respondent only a few minutes to complete. Microsurveys are often delivered by mobile phone, text message or a digital platform. Benefits include a much faster turnaround, and higher frequency of results, compared to traditional surveys.

Natural language processing (NLP) NLP techniques can be used to carry out automated analysis of user-generated text from sources like social media, to better understand what issues matter to people, translate languages or simulate language.

Open data Open data is the raw data that's gathered by people or an organization and published in an electronic format that machines can read. It's then shared online and allowed to be re-used by others instead of keeping it private.

Open source repository An open source repository is a digital repository where content (e.g. code, text or DIY designs) can be stored and freely downloaded with few restrictions on use. Many open source repositories aid collaboration by providing a space for uploading documentation, monitoring and version control.

Participatory sensing Participatory sensing is where citizens use lightweight, cheap sensors to collectively monitor the environment around them. It also deepens their understanding of the issue, educating participants and empowering them to act.

Peer-to-peer exchange Peer-to-peer exchange refers to the process of sharing information horizontally to build and maintain a community, collect data, connect people or send alerts. Platforms for this vary, ranging from messaging platforms to online forums or collaborative platforms. Some rely on the internet but others do not (e.g. SMS or mesh networks).

Remote or in-situ sensing Collecting information from satellites or physical sensors recording actions and physical changes (e.g. traffic cameras, weather sensors, ambient sensors, wearables or drones). This data can provide cheap, real-time measurements of anything from pollution to crop yields.

Web scraping Web scraping is a method for extracting unstructured data from across the web, such as company websites or social media. Where official datasets are costly to gather and infrequently updated, web scraping can provide more timely insights into social or economic trends.

Photo credits

Cover: First row, left to right, Annie Spratt, witness.org, Dennis Kummer, Ryoji Iwata, Niels van Altena; second row, left to right, UNDP Colombia, UNDP Colombia; third row, left to right, Ehimetalor Akhere, UNDP Zimbabwe, UNDP Viet Nam; fourth row, left to right, UNDP Lao PDR, Thos Robinson/Getty Images for Weill Cornell Medicine, United Nations; fifth row, left to right, Nathan Sousa, Unsplash, Omar Tursić, Matias Jacobi; sixth row, Majkl Velner; page 4, UNDP; page 10 and 11, UNDP Argentina; page 12, Andrea Leopardi; page 13, Nathan Sousa; page 15, Moritz Bruder; page 17, Hermes Rivera; page 18, UNDP Lao PDR; page 19, Boudewijn Huusmans; page 20, Nikola Aleksic; page 23, Hudo Ramos; page 25, 26, and 27, UNDP Viet Nam; page 28, David Cashbaugh; page 30, UNDP Zimbabwe; page 32, Markus Winkler; page 34, UNDP Bosnia & Herzegovina; page 35 and 37, UNDP Colombia; page 38, UNDP Ecuador; page 40 and 41, UNDP Guinea-Bissau; page 42, Tiago Fernandez; page 45, UNDP Mexico; page 46, stockcam; page 58, United Nations; back cover: Benjamin Jones.

Design

Ahoy Studios

empowering experimentation

The UNDP Accelerator Labs are thankful to our founding investors: the Federal Republic of Germany and the Qatar Fund for Development. Additional support is provided by the Ministero dell'Ambiente e della Tutela del Territorio e del Mare. We are actively looking for more partners to enable the evolution of the UNDP Accelerator Lab Network.

acceleratorlabs.undp.org

Footnotes: