The CIO of the UK’s national weather service has issued a dire forecast for the future of big data.
At the Cloud World Forum, held in London this week, Met Office’s Charles Ewen warned that the rapidly increasing amount of data being stored in the cloud by companies and governments will soon mean it won’t be technologically possible for these data to be shared.
In short order, he cautioned, the concept of “open data” will be an impossible promise to see through. There will simply be too much stuff for the cloud to accommodate and no one will have the infrastructure to oversee the kind of grand-scale information sharing to which we’ve grown accustomed.
As such, truly open data strategies will go the way of the dodo.
“The future will be bigger and bigger data,” Ewen predicted. “Right now we’re talking about petabytes. In the near future, it will be tens of petabytes. Then, soon after, it’ll be hundreds of petabytes. And then we’ll be off into imaginary figure titles.”
In response, he speculated that keepers of vast amounts of data will shift their roles to become digital “curators” of information, undertaking acts of purposeful selection and distribution in ways they deem most useful to end-users. That, he adds, would require a reorganization of companies’ processes such that a thoughtful selectivity functionality is introduced.
Ewen concluded by saying that the world’s current expectations of endlessly open data are based on “misguided views” about cloud technology’s capabilities to host and provide access to huge amounts of data.
Put too much faith in the capacity of the cloud, in other words, and the truth will eventually rain down.