29 May 2013


On Big Data and its discontents

Data will save us. All we need to do is measure the world. When we have quantified everything, problems both technical and social will melt away. That, at least, is the promise of “Big Data” – the buzzphrase for the practice of collecting mountains of data about a subject and then crunching away on it with shiny supercomputers. The term has lately become so ubiquitous that people make wry jokes about “small data”. But Big Data is not only something geeks do in the science lab or the start-up company; it affects us all. So we had better understand what its plans are.

Miraculous things can already be accomplished. By analysing web-searches tied to geographical location, Google Flu Trends can track the spread of an influenza epidemic in near-real time, thus helping to direct medical resources to the right places. Another of the company’s services, Google Translate, is so effective not because it understands language to any degree, but because it holds huge corpuses of written examples in various tongues and knows statistically which phrase of the sample text is most often translated by which phrase in the target language. Meanwhile, aircraft and other complex engineering projects can be made more reliable once components are able wirelessly to phone home information about how they are functioning. This mammoth store of telemetry data can be analysed to predict part failures before they happen.

But Big Data is not just an approach that improves uncontroversially useful systems. It’s also a hype machine. IBM, for instance, offers to furnish companies with its own Big Data platform, the PR material for which is a savoury mix of space-age techspeak and corporate mumbo-jumbo. “Big data represents a new era of computing,” the company promises, “an inflection point of opportunity where data in any format may be explored and utilised for breakthrough insights – whether that data is in-place, in-motion, or at-rest.” It sounds rather cruel to disturb data that is “at-rest”, presumably power-napping, but inflection points of opportunity wait for no man or megabyte.

Another platform capable of changing the game, albeit in an unhappily permanent manner, might be the Big Data skiing goggles marketed by the tech-shades manufacturer Oakley. Rather as the computerised spectacles known as Google Glass promise to do for the whole world, these $600 goggles project into your eyes all kinds of fascinating information about your skiing, including changes in speed and altitude, and can even display incoming messages from your mobile.

Of course, it might happen that while reading a titillating sext from a co-worker you ski at high speed into a tree. And so the goggles are sold with a splendidly self-defeating warning on the box: “Do not operate product while skiing.” Clearly, all the information all of the time is not always desirable. Still less so when Big Data’s tendrils move out from cool gadgets or website tools into our personal lives, the workplace and government – notwithstanding the Panglossian boosters of global datagasm.

Read the rest at the New Statesman.