Data

Data is everywhere. Much like physical infrastructures such as energy, transport and water, the global data infrastructure underpins our lives. An unprecedented amount of information is being created and shared online, offline, voluntarily, and unconsciously. Artists are using it as the basis of their practice—as a subject and as a material; businesses are using it to create new and improved products and services; people are using it to seek guidance on their lifestyle choices.

The potential of data to affect and reflect our lives is immense. Data is more than just spreadsheets, it's a technique for gaining deeper understanding of the world around us with an abundance of stories waiting to be discovered. Data becomes really powerful when you bring together from different data sources and use it to surface new insights. This is only possible when we open up our data and encourage others to do the same.

Data is a key learning theme from the Digital R&D Fund for the Arts. Data is influential when used correctly, and in context. Learning how to get the best from your data is the first step. Understanding what data to share, what data you could collect, and how you can combine it with data from other sources is the next.

Julie Freeman is Art Associate at the Open Data Institute, Artist at Translating Nature and a Doctoral Student, in Media and Arts Technologies at Queen Mary University of London.

Insights First, Data Second

Data is the next big thing in the arts. The problem is that a lot of us don't really know what to do with it. So we don't really do anything with it. Or even bother collecting it in the first place. It's a bit of a hassle to gather the stuff after all. And it's definitely a bit of a hassle to analyse it.

Written by Suzy Glass, a producer with a specialism in both digital and sited work.

Even if we do recognise the worth of our data and make the time to gather and analyse it, if we're a freelancer or a small organisation—as much of the sector is—it can feel like our contribution is insignificant since the data can only tell a little story. So perhaps it's unsurprising that the cultural sector is data-shy. We don't generally spend money to generate datasets in order to inform our decision making, certainly no more than just enough to satisfy minimum reporting requirements.

Cultural policy consultant John Knell believes that the secret to cracking the data conundrum is to avoid the D-word entirely. Instead, he suggests we focus in on the idea of insights. We need to embrace the idea of initiating enriching conversations, both internally with colleagues and more outwardly facing with peers and of course audiences.

We're fetishising the wrong bit of the equation. According to Knell, the assumption that organisations care about data, want to gather data and then want to do something with that data is flawed. In fact, many arts organisations have serious hurdles to leap over in the form of both expertise and resource shortages. In his opinion, 80% of us are data shy, 15% are data driven and 5% are data ready. It just doesn't make sense to ask people to jump straight in the data deep end.

Knell says we can all embark on a big data revolution, but only if we focus on generating insights that can act as a starting point for conversation. We need to give ourselves the opportunity to focus on results rather than data per se. We're all interested in critical and peer review, and if we create the appropriate platforms we can then use data in a way that reflects our critical practice.

Knell is currently working with a consortium of Manchester cultural organisations to develop a new approach to measuring the quality and value of cultural activity. Culture Counts is supported by the Digital R&D Fund for the Arts and plans to be the first system in the world to draw together comprehensive public value metrics within an electronically automated data collection platform.

Culture Counts is dubbed a Big Data project. But initially the team were simply trying to answer this question: what is a quality cultural experience? Their quest led them towards developing indicative outcomes, and subsequently a series of metrics.

The platform uses these metrics to gather real-time data from artists, their peers and the public. It then combines this with more traditional data relating to attendance, funding, and box office. Taken together, these two types of data allow the platform to deliver comprehensive value analysis and reporting on a continuous basis. This in turn enables data-driven decision-making.

Culture Counts uses a standard language across all of its surveys. By standardising measurements, it can benchmark across a number of organisations and over time. Reports are produced in real-time, so organisations can see feedback as it's actually being fed back. The platform builds beautiful and useful graphs illustrating exactly what you want and need to measure. You can easily share the reports and export the graphs and data. You can even share your saved templates with others.

This is where the magic starts to happen. Suddenly we in the cultural sector are working together to create big datasets rather than working in small individual silos. Moreover, we're working together to define appropriate metrics for evaluating quality. There's also a network effect, with Culture Counts becoming a sector-wide tool that enables reflection. We become part of a community with the potential for self-reflection.

"We can all embark on a big data revolution."

Data is now not just an advocacy tool, it's not just about audit. It becomes a useful driver of critical review, it allows us to understand how our work will be accepted by our peers and audiences. We become a live community, working with our sector-wide data to evaluate our worth. The importance of data isn't the data by itself. It's the possibility of self-awareness and self-reflection that it brings to bear to make us better.

What Do You Want Your Data to Do?

Databases are one of those things that appear neat and tidy... until you start working with them. The organisations of the Newcastle and Gateshead Cultural Venues network are creating a 'data commonwealth' in which customer data is fully shared, enhanced and used strategically for cross-promotional purposes and audience development. We spoke to Andrew McIntyre, Lead Research Partner, to capture his top lessons.

Written by Tandi Williams, a consultant dedicated to making research work for the arts.

The eight organisations, which include Dance City, Baltic, Centre for Life, Live Theatre, Northern Stage, Sage Gateshead, Seven Stories, Tyne & Wear Archives & Museums and Tyneside Cinema have been working with arts consultancy Morris Hargreaves McIntyre to share customer data and 'mine' the information they have on their audiences to greater effect. It's proving to be an exceptional business plan, with almost 15,000 people already subscribed to the consortium's new deals platform, The Insider. But the journey to get there has been far from easy.

"People think, 'If I'm on Tessitura, and they use Mailchimp, it can't be done." But workable solutions can be found," says McIntyre, who believes that a fear of technical and legal issues surrounding databases is holding the sector back.

"We created a new legal structure," he explains, "which allows us to cooperate without anyone breaching their obligations." Instead of pooling customer data, the consortium created a registry of nine separate databases. A campaign followed, where each organisation invited its customers to opt-in to The Insider to receive exclusive offers and deals.

Ahead of launching The Insider, the consortium used an external provider to 'clean' their data, checking for missing postcodes and spelling mistakes. It ended in disaster. "Something went wrong in the database structure, and they scrambled some of the data, causing a lot of problems," McIntyre explains. "Cleaning is a huge expense and a huge risk. We'll do our own accuracy and validity checks from now on."

On the other hand, the team consider money spent commissioning a security test of their online server to be worth every penny. "If our online form was hacked, it would be very embarrassing, not to mention the legal implications."

Committing to a launch date too early in the project put enormous pressure on timelines. Testing was rushed, and failed to pick up a bug in the system for removing 'duplicate' entries in the databases. "Some people in our test group received 8 or 10 invitations!" McIntyre says. "Most people responded quite well, but we learned an important lesson."

They took a long time rewriting software and rebuilding the database, followed by extensive testing. "After that, we sent 50 emails at a time, until we were happy to send it to everyone."

The combination of outgoing emails and incoming traffic, as people responded to the offer, also overloaded the server. "We had to massively increase the bandwidth, and adopt a throttling system to deal with the traffic."

McIntyre concludes: "We should have set a tentative launch date, which we could move to allow for the necessary testing, including checking for dependencies in the settings.”

Working across nine different organisations created challenges for communication.

The core team often took time to reassure partners how sharing data would benefit the whole consortium, and invested in a dashboard to visualise the results. "As soon as people can see the new customer data on the dashboard, misunderstandings evaporate."

McIntyre believes it would have been valuable to involve more people upfront, including technical staff, and box office managers who field enquiries from the public. "Circulating contact details of all relevant team members and preparing FAQs can support strong communication throughout the project."

Budgeting is never easy, but innovation projects need generous contingencies. "We have spent more than double our project management allocation," McIntyre says, "and the technical partners have also gone way over on testing and validation."

Despite the challenges, the team have maintained an R&D mentality, viewing obstacles as 'learnings' rather than mistakes. "None of this stuff we had anticipated, but we have treated the whole thing with the spirit of adventure."

Thanks to all their hard work, this adventure has a happy ending, with one successful campaign and more planned. MHM are also working towards The Insider New Zealand, where they won't have to learn the hard way.

"The core team often took time to reassure partners how sharing data would benefit the whole consortium, and invested in a dashboard to visualise the results."

Data is Worth the Trouble

Data is raw information: the letters, numbers and symbols flowing over the internet and through our organisations, systems and personal devices, connecting us to each other, to content, to objects and places.

Written by Sophie Walpole, a digital media consultant in the arts and entertainment sectors.

In its raw state data is messy, high in volume but low in value. Diving into data can be daunting for arts and culture practitioners because it requires people to talk about culture in numbers, it requires scientific, analytical skills to shape and extract value from it and it takes engineering discipline to apply it to storytelling and creative experimentation. Used well, however, data can drive audience engagement and enable business opportunities.

Every Digital R&D Fund project had to address an aspect of data manipulation, whether testing new forms of access for audiences, exploring new business models, creating new cultural products or providing evidence of usage. Through collaboration and research, projects have been able to innovate, to use data to solve problems in new ways and to demonstrate positive impact.

Using personal data to drive audience engagement is common practice in larger cultural organizations but capturing, storing and reusing such data poses a challenge for smaller bodies on tight budgets with no physical space. Several projects sought to find nimble ways of capturing audience data in real time and on different devices. NoFit State Circus and National Theatre Wales, frustrated by the limited amount of data held about their audiences (because it was locked in third-party systems and bound by data protection) worked with Joylab to find out if data could be captured in playful ways. They also looked at how data could be better stored, accessed and analyzed.

Other projects sought to drive engagement through new forms of discovery based on shared data. Albow set out to harness the power of social networks in enabling people to share, discover and support live events via a listings network called Thisison. The Newcastle and Gateshead Cultural Venues network (DanceCity et al) created a 'data commonwealth' in which audience data was cleaned, enhanced and shared to introduce audiences to the diverse range of activities in their area.

Data can also open up new business opportunities, unlocking financial value through commerce, ticketing and membership or through providing evidence of public value for funders and stakeholders. Datasets being collected and analyzed in this sector are increasingly diverse, from the familiar tabular data (website visits, ticket sales) to network data (from social media platforms), text (user feedback from surveys) and video, audio and sensor data. Big Data and Open Data in particular present opportunities to create robust, standardized metrics and stories which highlight the scale of an event or its impact—the public or social value of an organisation—in a meaningful way.

Culture Counts, for example, aims to create the world's first system to gather real-time data from artists, peers and the public and combine it with traditional data on attendances, funding and ticket sales in order to deliver continuous, comprehensive, value analysis and reporting which will drive internal planning and, in time, funding decisions.

Arts Data Impact (ADI) on the other hand, embedded the first ever data scientists-in-residence for the arts at the Barbican, English National Opera (ENO) and National Theatre, to interrogate internal data sources alongside national data and to create tools, audience insight and organizational learning for sharing.

Finally, data can be used as creative material, enabling new forms of artistic expression and innovation. Talking States harnesses tiny pieces of data – QR codes and atoms of media—to inspire audiences in physical locations. In Bristol, Knowle West Media Centre ran a programme with young people experimenting with live, open data. Their research question was simply 'Can arts organisations engage young people in creative data projects to reach new audiences?' and they produced a useful data visualization toolkit which is now available online.

"Data can be used as creative material, enabling new forms of artistic expression and innovation."

Common challenges emerge across all the projects—the need for more open standards for cultural data, lack of in-house analytical skills, the pain of integrating technical platforms and systems, the risks of privacy breaches—but these are issues facing all organisations in the digital age. Open data standards will soon prevail and tools for interpreting data will become cheaper and more familiar.

Arts organisations may never need to be 'data-driven' but they need to be 'data-savvy' as the rewards can be substantial once you know where to find the right data and how to interpret it.

Open Up Your Data

Open data is at heart a simple concept. It means providing some of your own information or content in such a way that other people can use it to create new value—for themselves, for audiences and for you.

As someone with experience running major open data projects in the arts, I've found that both parts of the term can feel intimidating. The word open can make us feel that we're losing control or giving away something confidential. And the word data can make us feel it's just all a bit too technical for us. Hopefully by the end of this article, your concerns will have softened and you'll have the information you need to at least understand open data and its potential. And you may even start an open adventure yourself.

The first two key questions are what kind of data do we want to open up and to what end? The data we are talking about can take any number of formats. Examples of cultural sector data which have been opened up in the past include event listings, collections information, text, audio, photography, video and non-sensitive audience data such as attendance figures. Some venues have even opened up their energy consumption data. The key factor is that if it is information you own, and you have the permissions to openly share, then it is within scope.

Written by Rohan Gunatillake, former Innovation Lead at the Edinburgh Festivals Innovation Lab. Find our more about the Festivals Listings API at: api.festivalslab.com

But why bother in the first place? Other than the intrinsic value of openness itself, there are five compelling reasons for open data in the arts. The first is that if third party developers and organisations can build on your data they can create new value as a result, such as selling more tickets to your events by using your data in a third party listing app that has much better search functionality than your website does. The second reason is that it can save time and money. If you are already providing some kind of data to partners then doing it more automatically through an open data system can save a lot of manual work. Thirdly it is great for audiences. If you are sharing your listings, collections or media content data it means that more people will have access to it, either in its original form or when remixed. The fourth reason is that your information becomes easily accessible and valuable for researchers. And finally, open data can be used as the fuel for creative work.

The Festivals Listings open data initiative was one of the main projects during my time with the Edinburgh Festivals Innovation Lab. Now in its fifth year, it is the largest open data project in the performing arts, providing up to date listings data from the twelve major Edinburgh Festivals across the year. While publishing an Excel spreadsheet of information on a website does technically qualify as open data, that is not the most useful format to manipulate. Therefore the festivals have created what is known as an API or Application Programming Interface which means that a third party developer can take your data as a direct feed and plug it into whatever application they want to in a relatively hassle-free way.

A key element of the project was establishing the guidelines for how that data could be used. It is a common concern that when we put our data out 'in the wild', it will then be used for purposes that we might consider inappropriate. This is easily avoided through ensuring that all users register and agree to appropriate licensing terms. Then should any issues arise access can be revoked at a click of a button.

Setting up the Festival Listings API took effort both from the organisational and technical perspective, but it has paid significant dividends. The key benefit has been that the API is serving both internal and external stakeholders. It is used to not only power festival websites and apps, it also serves media partners and other organisations who in the past would manually receive spreadsheets, thereby saving lots of work-time and improving data quality.

The other main benefit has been that providing the data in this way has led to the festivals working with independent developers and small-scale companies that they would not have otherwise had access to. This was first catalysed by the inaugural Culture Hack Scotland hosted by the festivals back in 2011 which saw creative and practical outputs including stunning data visualisations, a dating website built around Fringe shows and an innovative mobile prototype which has since been developed into the official mobile site of the Edinburgh International Book Festival.

"Setting the guidelines for how your data is used means you retain control."

Make Data Meaningful

Using data meaningfully is an ambitious goal for any organisation, and unfortunately it doesn't yet come with an instruction manual! But five strategies from Digital R&D project Arts Data Impact are beginning to bear fruit.

Written by Tandi Williams, a consultant dedicated to making research work for the arts.

Led by The Audience Agency, Arts Data Impact (ADI) is a collaboration involving creative technologist Anthony Lilley of Magic Lantern Productions and the Head of the School of Creative Arts & Technology, University of Ulster, Professor Paul Moore.

They're placing two data experts inside three national arts organisations (Barbican Arts Centre, English National Opera and National Theatre) to explore the potential for data science to help build new audiences from the inside. Meaningful use of data begins with a culture of curiosity, and organisation leaders have a critical role to play. Not everyone will become an analyst overnight (nor should they), but everyone should understand the potential and be open to discovering new insights.

Moore, lead research partner on ADI, says inviting guest speakers can help give data the right profile within your organisation, and take fear out of the equation: "We need to debunk the myth that data is uncreative, impersonal, or a threat to artistic decision making."

Other approaches include asking your team leaders to talk about how data has informed their ideas to dispel the notion that they are guided by intuition alone (what Moore calls the 'intuition myth').

Data can inform, inspire and improve decision-making processes and planning cycles, including artistic programming, but only if we give it a chance. It's worth trying things like data workshops at your annual strategic planning days, and scheduling insight presentations at board meetings.

"We need to debunk the myth that data is uncreative, impersonal, or a threat to artistic decision making."

As Moore explains: "The real strength of data is to look for patterns that can tell you where to go." For the best results, use data to look forward, explore meaning and guide future decisions, rather than retrofitting insights to decisions that have already been made. Developing data skills in individuals and teams is complex, particularly as today's data expert is not just an analyst, but a strong communicator too.

Nesta's Skills of the Datavores research, which studies 45 leading uk data businesses, found that the skills needed to create value from data are not just analysis and coding, but business sense and creativity. The data-scientist-in-residence (DSIR) is one model that can help kick things off. It recommends partnering with universities, and participating in conferences and hacks, which could help you recruit the data all-rounders needed.

Another element to this is reflecting data skills in job descriptions of senior staff, and building it into annual performance reviews.

"Data experts are sometimes viewed as necessary geeks, put in the corner and asked to churn out reports, with no understanding of how or where the stories might be told," Moore says.

To maximise impact, organisations should empower data managers to work across the organisation, ensuring they have the access and seniority needed to be effective influencers. "Vast swathes of data never get to the board, because delivery structures just aren't there," says Moore, who also cautions against closed office doors and layers of hierarchy.

Open access to data is another tactic, preventing any single person or team becoming a 'gatekeeper'. But, it's a fine balance. "If you have too many people working with data," Moore says, "you can get data paralysis."

Turning insights into stories that captivate and inspire is an artform, but one the cultural sector is well equipped to master. "If you can take a statistic, and make it human, or better still, make it fun," says Paul, "then you get people to begin using it."

Different communication styles work for different people, so data experts need to be using verbal, written and visual techniques to deliver messages. Paul believes the right data tools, such as well-designed infographics or dashboards can improve accessibility and be the key to instigating change.

Whether you do all of these things, or just one, the main thing is to experiment and be agile. as Paul says: "Sometimes things aren't going to work, but it helps to have the courage to make decisions quickly. And if you fail, learn from it and move on."

Soon enough you may be writing your own instruction manual.

Making Digital Work

Visit the Digital R&D Fund for the Arts website at artsdigitalrnd.org.uk for a wide range of resources to help organisations of different sizes and types on their own digital innovation projects.

Theme Films

Engaging short films on the four learning themes of mobile, accessibility, data & business models.

Product Toolkit

The processes and tools you need to take an idea through to a successful product.

Print Magazine

Limited edition publication full of inspiration and opinion, also available as a PDF.

Research Reports

Fascinating accounts from the R&D Fund projects from England, Scotland and Wales on the successes and challenges of their work.

Digital Culture

Results from this major 3-year study tracking how arts and cultural organisations in England use technology.

Magazine Features

Interviews, profiles and guides from a range of people and organisations working in arts, technology, research and beyond.

The Digital R&D Fund for the Arts is a £7 million fund to support collaboration between organisations with arts projects, technology providers, and researchers. It is a partnership between Arts Council England, Arts and Humanities Research Council and Nesta.

Thank you to all the contributors and project teams who have made this guide possible.

Editor

Jane Audas

Design

Ryan R Thompson / Rydo

Special thanks

Julie Freeman Anna Dinnen Rohan Gunatillake Clara McMenamin Tim Plyming Emma Quinn

Further Resources

Culture Counts

The platform to measure and communicate the value that artists and arts organisations create. www.culturecounts.cc

The Datavore

Research into a significant group of uk companies using data to drive business decisions. www.nesta.org.uk/news/datavores

Counting What Counts

A report detailing the case for greater use of data in the arts. www.nesta.org.uk/publications/counting-what-counts-what-big-data-can-do-cultural-sector

The Open Data Institute's Guides to Open Data

www.opendatainstitute.org/guides

Using Data to Creatively Engage Young People

Toolkit from Knowle West Media Centre. www.datatoolkit.org.uk

Let's Get Real

Culture 24's research into improving digital engagement. www.weareculture24.org.uk/projects/action-research

Culture Hack

Making creative prototype projects from cultural data. www.culturehack.org.uk

Related Digital R&D Fund for the Arts projects

• Art Fund
• Audience Agency
• Black Radley Culture
• Cheltenham Festivals
• Dance City
• Firestation Arts
• FutureEverything
• Home
• Imperial War Museums
• Knowle West Media Centre
• Live @LICA
• Museum of Design in Plastic
• National Holocaust Centre
• Tyne and Wear Archives and Museums

artsdigitalrnd.org.uk