Step inside the virtual orchestra

As part of Nesta’s involvement in the Audience of the Future project, I am fortunate to be working with some of the world’s leading performing arts companies as they explore innovation in their fields, focusing particularly on immersive technologies.

In this series of blogs, I’m looking at different art forms, considering their historical development and potential future trajectories as they respond to changes in media and audience tastes and expectations. The first blog was on theatre, and this second blog takes a look at music, specifically classical music. Music is unique compared to other genres of art, in that it is entirely free of form and, in theory, less restrained by the limitations of space than, say, a theatre production or a piece of visual art. The music writer David Toop captures this and the flexibility it affords to us, as both creators and consumers of music, in the below quote:

"As the world has moved towards becoming an information ocean, so music has become immersive. Listeners float in that ocean; musicians have become virtual travellers, creators of sonic theatre, transmitters of all the signals received across the aether"

His book Ocean of Sound, written in 1995, foretold of the role music plays in our lives today. Music is everywhere; in our pockets (thanks to Apple and Spotify), on telephones when we’re on hold, in shopping malls and train stations, television shows and advertisements alike. However, before music could be recorded, it was a purely live experience, one that required people to create sound themselves or at least to actively choose to attend as an audience member. Considering the gramophone was invented in the early 20th century and home ownership of record players only really took off in the postwar period, it’s fair to say that recorded music is a relatively recent phenomenon. As the great musical innovator, writer and thinker David Byrne (of American band Talking Heads) wrote,

"Before recorded music became ubiquitous, music was, for most people, something we did...All those experiences were ephemeral - nothing remained except for your memory...You couldn’t hold it in your hand. Truth be told, you still can’t."

So how can we re-establish the practice of listening actively and elevate music back to a place where it isn’t merely a soundtrack to our daily lives, but an experience which we seek out? The answer may lie in new technologies, especially immersive media such as virtual and augmented realities. One genre which is leading the way in this field is classical music, and whilst the stereotypical portrayal of classical music is one of musicians in penguin suits, hushed concert halls and anachronistic audience etiquette, it has also championed live music alongside recordings, and there are a number of ensembles who are using technology to create new forms of live experiences for audiences. We are lucky to be working with one of the leaders in this field, the Philharmonia Orchestra, on the Audience of the Future performance demonstrator.

The Philharmonia Orchestra has a long history of innovation - it was formed as a recording orchestra in 1945 and has more recently recorded music for films and video games. Their Universe of Sound project enables audiences to walk through a ten-room virtual orchestra, transforming the audience experience from a passive seated experience to an active one, creating original ways for audiences to experience Gustav Holst’s The Planets, one of the classical music canon’s most recognisable compositions.

Through their involvement in Audience of the Future, Philharmonia are experimenting further with this concept, using spatial audio recording techniques to give audiences more options and agency in how they experience classical music. I interviewed Luke Ritchie, Head of Innovation and Partnerships at Philharmonia Orchestra, to look at innovation in orchestras and talk about what this could offer to audiences in the future.

Hi Luke, thanks for sitting down with me today to talk. Let’s start with a straightforward one - why do you think it’s important for orchestras to innovate?

I can start with the hard-nosed commercial reality that we’ve kind of settled into a live concert business model which is fundamentally a loss-making business model. It’s ultimately dependent on a form of patronage, and that’s fine - art has always survived on that - but, like other art forms, it doesn’t mean we have a right to exist. This model is based on substantial public subsidy which is likely to decrease, so just looking at it from a business perspective, unless they can find a way to connect with new audiences, they’re at risk.

So that’s from a business perspective. From a creative perspective I think it’s completely normal for an orchestra to grow and adapt to its environment, so in a positive way we’ve always been innovating and evolving. The form has always changed and it’s a myth that it’s been static up to this point. In fact, the Philharmonia was created as a response to the emergence of vinyl and recorded music, which changed the way we listened to music and the way we record it and that moved into how we filmed for music, and then we have music for film and for gaming, and now we’re looking at innovation in the live concert format.

From what you are saying, it sounds to me that orchestras have therefore been evolving and innovating in response to technological change for quite a long time. Do you consider the integration of emerging tech into live performance a revolutionary moment in performing arts history, or is just the next step in this long evolutionary process?

At the moment it’s hard to say. There’s a broader commercial perspective we shouldn’t lose sight of, which is that the fate of a musician was ultimately pretty brutal, except for this brief golden period where you could bottle the live performance and mass produce it at scale, with low marginal costs, to a massive market [through recordings on vinyl, tape and CDs]. Suddenly, musicians were wealthy, and well-fed and got to play music all the time. That period has now disappeared or is disappearing. Even as late as the 90s, many orchestras were making a lot of money from recordings and most of them don’t now. The habit of buying music has collapsed. I used to spend days a week in a record store and would buy CDs regularly, so people who liked music were happy to spend a big chunk of their disposable income on it, whereas now I pay £10 a month for unlimited music. That’s the real business trend and you don’t want to lose sight of that. There are new forms of media [such as immersive technologies] appearing all the time, but from a commercial perspective, we have no idea if they’ll have an impact yet. I think they will, but I don’t know when.

That’s from a commercial perspective, but from an artistic point of view, it is transformative obviously, because of the audience experience it gives you, which is immersive and puts the audience member inside the orchestra. For me, it’s when you give the audience member agency, to move across narrative, across time, through the orchestra or to take the orchestra with you somewhere else. And that’s the big shift for me.

S: And perhaps this is the major difference between current technological developments and previous innovation?

L: I think so. I think it’s helpful to look at the example of the screen, and how it has evolved. You used to go to the cinema and there was a little screen, then the screen got bigger, then there was a big TV in your home, which got bigger and thinner, and now there’s a phone and with VR you’ve now moved ‘inside’ the phone. And the frame has disappeared and it’s all around you. And we’re doing something similar with orchestras.

There’s something particular to orchestral music, which is that, going all the way back to the idea of court music, it has always been a very immersive live experience. It’s the greatest scale of ensemble playing in terms of the richness and the sheer number of people involved, so it’s never been very well suited to flat screens [which have been the preferred media of most technological innovation so far]. It is far better suited to concert halls or to new immersive environments which we are exploring now. VR isn’t just a fad. I needed to be persuaded by seeing audiences respond to it. People love it, they respond emotionally, they cry, and what we’re doing at the moment is still super simple. I’m also impressed by its [technological] progress. Even comparing an Oculus Quest with a Samsung Gear from three years ago the change is amazing. Jump forward ten years and it could be completely bonkers.

Would it be fair to say, therefore, that for Philharmonia technological innovation and audience development work hand in hand?

What I’m finding is that immersive technology is really good at opening up the orchestra to new audiences. The orchestra has a real depth of expertise now, there is institutional experience and memory in these areas. The big immersive experiences that you might see in an art gallery or at a festival are quite exclusive, with artistic ambitions but not necessarily with the audience experience in mind. We have developed experiences which can go on tour in a truck or land inside a giant marquee on the town high street, taken to people on their own terms. We recently completed our latest Audience Development project The Virtual Orchestra, which toured the UK, with a dedicated Audience Development Manager designing a programme around our immersive work, which has really dovetailed well with the technology.

It has also been surprising though, that classical music fans, often quite traditional core audiences, are the ones who respond most emotionally to the [tech rich] experiences. Maybe it shouldn’t have surprised me, but these are often people who we still communicate with us via post.

Let’s talk more about the audience experience, then. What do you think the big challenges will be for creating a compelling immersive experience that people are loyal to?

There are two big themes being talked about a lot within immersive experiences at the moment. One is storytelling and the other is shared experience, and I understand those, but I don’t need anyone to tell me a story at all when I’m listening to symphonic music, or any music. Although orchestral music is an inherently powerful shared, communal experience,, there’s also something glorious about being alone with the music in an immersive experience. Looking at the example of taking VR experiences into schools, if you’re in school surrounded by you peers, and it isn’t really ‘cool’ to like classical music, when you put the headset on you’re on your own and no one can see you and you can enjoy without distraction. It’s similar to a concert in a way, with the lights down.

And perhaps concerts aren’t really a communal experience in their traditional format anyway?

There are such deep, cultural, unspoken rules [within the concert format]. This is my own bias of course, but I like tech, I like the playfulness of it and I get excited by it. On balance, immersive tech is still really clunky, but I take a long view of that, and it demonstrates how effective it is, that people are still willing to wear it. I think the social side of it will come [as the technology improves]. However, I think that one of the problems with the ephemera of these technological innovations is that it’s constantly changing, so it’s hard [to develop a ritual] around audience behaviour. I used to collect physical music but if Apple [Music] goes the way of, say, Netscape, then what will I have to show for it?

Going back to the format or the medium of live performance, I don’t understand how people are supposed to listen to a piece of classical music just once. Someone asks me what I think about it after [a concert] and I sometimes have no idea! Most of the music I love, I either didn’t like or struggled with at first, and you rely on those peer networks to give you recommendations.

What does it take to create a risk-taking and innovative mindset in an orchestra? Are there any particular ingredients to success that you can share from your experiences with Philharmonia?

In our organisation, it’s been leadership from the top. The real catalyst was a conductor who cared about it and an MD who supported it and enabled me to jump on it. Embedded in that is a willingess to take risk, artistically, technically and financially. Esa-Pekka Salonen (principal conductor and artistic advisor of the orchestra) brought the players along with him as well, which is an important barrier to overcome. He understands this technology to a degree that it actually means something to him. He’s very good at being an agent of change.

Another key thing is that it’s a ten year, long-term commitment. You won’t get results by only thinking a year into the future, and you also won’t gain much from only doing things once. Something I would champion is in-house expertise, something as simple as having one person whose role it is to think about this stuff.

Finally, where do you see the future of orchestras, concert halls, audiences in 2050?

My gut feeling is that change is accelerating, so predictions are really hard to make. Talking to the Head of Innovation at Disney recently, they said that they think ahead 10 years and then ask what they will be doing in the next 18 months. I think that’s quite good. I think in 10 years VR will be incredible but maybe not as a vast a leap as we think. Mixed reality will be even more improved. Therefore live concert experiences or rather, experiencing concerts live from your home, as if you were there, could be possible, except you could be sitting in the orchestra instead of in the audience!

Author

Shoubhik Bandopadhyay

Shoubhik Bandopadhyay

Shoubhik Bandopadhyay

Insights Manager, Arts and Culture Finance team

Shoubhik produced primary and secondary research to help the team and its partners better understand the impact of their work and identify opportunities for future development

View profile