We held a really productive workshop with the recipients of our big data for innovation research grants last week at Nesta. There were different views in the room about whether we should be using the term 'big data' or not. Is 'big data' just a marketing phrase for what used to be called business analytics, as Mike Lynch said at an evening reception at Nesta.
"The problem with the phrase big data is that it's grown up and it's been grabbed by lots of people. In the late sixties, there was no way that computers could understand the real world, it's too complicated.
"So you took problems and you made them simple enough for a computer to understand, you laid out information in rows and columns and defined it very precisely and that became a relational database...
"In the seventies they sold you a database. In the eighties they sold you a relational database. Then they decided they had to sell you another database, so they said you needed a data warehouse. And then you had to do business intelligence and that was another database. And then I think the next one is data analytics. And now just in case you haven't got enough databases, they say you need big data and so you have to buy another database. So the thing is that there's a marketing angle to all this."
He's not wrong.
A Google Trends look at search terms including 'big data', 'data warehouse', 'relational database' and 'business intelligence' shows the steady rise of big data to take over from all the others. Using data and information to manage your business is certainly not a new thing, and 'Big Data' is the marketing label of choice to apply to your current business information product.
So is there any substance behind the hype?
One of the things I liked about 'Big Data', the book published this year by Kenneth Cukier and Viktor Mayer-Schonberger is that they successfully separate technology changes such as faster processing and networks, from the process of 'datafication', capturing more activities with data.
They have a nice example of Navy shipping logs that became 'datafied' in the 1800s, allowing better planning of routes. This is not something that has to be dependent on technology, but it can certainly be supported by technology. When it comes to creating data where there was none before, the most useful technological changes are not processing power or giant database software, but sensors for automating data collection, and massive online networks, for crowdsourcing data or knowledge from many people (think Wikipedia and Zooniverse).
There is a large part of the diverse set of changes known as 'Big Data' that is about incremental innovation and improvement in businesses and government. The potential of these improvements is vast, simply because the number of current activities they can be applied to represents huge economic value. Mike Lynch is right - many of them represent the next incremental step on a very long path of management and efficiency tools.
But we think that in looking around for completely new data sources that might improve things, there may also be the opportunity for much more radical innovation, and completely new approaches. This is an area we are starting to explore through the research grants on new data sources we funded earlier this year, and something we're going to continue to pursue.