How we used mixed methodology to harness the collective intelligence of teenagers
As part of the UK government’s national strategy to tackle obesity, a ban on marketing unhealthy food and drinks products online is proposed to take effect in January 2023. With the bill currently going through Parliament, but facing opposition from industry, we wanted to know how much it really could reduce children’s exposure. At Nesta, our healthy lives mission aims to study and change food environments so that we can make healthy eating easier. We know that food and drinks marketing has a significant influence on food preferences and enjoyment and that the majority of food marketed to young people is unhealthy. With teenagers aged 13-16 years spending, on average, four hours online each day, this marketing plays a significant role in health outcomes. But our review of evidence showed that there were gaps in the research on the scale and nature of the online marketing that teens are exposed to. With some types of marketing excluded from the ban, we also wanted to learn if the Government’s proposals would go far enough.
The Centre for Collective Intelligence Design at Nesta creates solutions that bring together diverse groups of people, new sources of data and technology to solve problems and get smarter together. This issue of online advertising was a perfect candidate for a collective intelligence data collection project, due to the personalised and targeted nature of online advertising which means that each teenager’s digital food environment is different. By combining many pieces of data from a range of people through a collective effort we knew we could start to see a more complete picture of how teens are exposed to unhealthy food marketing online. That’s why we chose to use a collective intelligence method called citizen social science – a form of participatory research that engages citizens in social research. From early on, we knew we wanted to work with teenagers across the country to crowdsource examples of the types of unhealthy food and drink marketing they saw. The challenge was how to do it in a fun, engaging and informative way.
Designers from Nesta’s design practice and CCID took the lead with helping to answer some of our questions through user research and testing. We started with interviewing teens from different parts of the country to understand their experiences of, and opinions towards online marketing. Through September and October 2021, we also conducted workshops with young people from the youth advocacy organisation Biteback 2030 and used their insights to shape the language and visuals we used for data collection. The involvement of young people in the design process enabled us to build and prototype simultaneously.
After exploring a range of ideas, the solution we landed on was a gamified crowdsourcing experience, with a “mission” consisting of a series of daily challenges over the course of a week. We started by inviting teenagers to become “Marketing Detectives” with the goal of uncovering the truth about the food and drinks marketing they were seeing online. We designed the experience through a platform which worked on desktop and mobile (reskinning an existing digital ethnography platform called Incling for convenience), and asked the teens to screenshot and share the examples of marketing they saw each day for a week from Instagram and TikTok or even gaming consoles. We developed an introductory experience, which included a survey and video to help the teens understand the different types of marketing they might encounter. A forum was also created for the teens to express their views and share their experience of participating in the project. We worked really hard to provide timely and personalised feedback on the platform (following the evidence about what keeps people motivated in citizen science projects). At the end of the experience, teens were given an option to be connected with Biteback 2030 and join its campaigns for a healthier food environment. We also provided each participant who completed the majority of the missions’ (even if they hadn’t spotted any marketing) with a £20 cash reward to recognise their time and contribution.
Throughout the project, Nesta’s Evidence and Experimentation team offered guidance on study design and did a lot of the analysis. Because our primary aim was to generate new evidence about the digital food environment and likely effectiveness of the proposed ban, it was important for us to have diverse representation among participants by geography and socioeconomic status. So, unlike most citizen science projects, we took the decision to recruit participants through an external agency, Roots Research – which gave us greater control over our sample in the shortest possible time frame, and enabled us to manage ethical issues of working directly with teenagers. A secondary aim of our research was to learn more about the effects of participating in a collective intelligence project on the teens themselves, so we designed pre- and post-study surveys, as well as quizzes to test the teens’ ability to identify online marketing.
At the end of 2021 we launched the Marketing Detectives platform and over three weeks, 284 teenangers collected 4,879 examples of online food and drinks marketing. To make sense of this messy citizen-generated data, we again had to draw on collective intelligence – but this time of a different crowd. For this part of the project we worked with a crowd-labelling platform called 1715 Labs (a spinout of the famous citizen science platform Zooniverse). Between three and five different people from1715 Labs labelled each piece of data that had been submitted by one of our teens – helping to categorise each image into the different types of marketing we’re interested in, identify the brand or product that is featured, as well as the platform it was spotted on. The benefit of having up to five sets of eyes on each image was, ultimately, accuracy. This is because even a highly skilled person won’t assign the right labels 100% of the time, but aggregating multiple classifications from different people will result in a much more accurate result – an effect known as the wisdom of the crowd.
"I really enjoyed the task. I felt like a real detective and this task made me notice how much food ads I come across everyday when using the internet."participant
“Some of the marketing is dead obvious but some is sneaky. When it's not the brand itself, but an ambassador for a brand that can be difficult to spot.”participant
We found that nearly three quarters of all the marketing that teenagers saw online was for unhealthy food. And that over 60% of all the marketing the teens reported was for paid-for product advertising – which would be covered by the legislation if it is implemented. We can now more confidently say that the bill is a good step forward in cleaning up the digital food environment for our children, even if there is still room for further improvements. We also discovered that teens from lower income households reported about 50% more unhealthy food and drink examples than teens in higher income households. We’ve published the full results and will be sharing them widely with policy makers, and with the teens who took part.
“We can now more confidently say that the bill is a good step forward in cleaning up the digital food environment for our children, even if there is still room for further improvements.”Nesta's healthy life team
A key principle of collective intelligence design is data empowerment, not data extraction. Next our designers will be working with Biteback 2030, to support the teens to use the research findings to design and run their own creative campaigns – we can’t wait to see what they come up with.