Save the data

This week we learned that Glasgow (my home town) has been volunteered to host a pilot project to collect and use urban data more efficiently. The idea is to link up real-time info about transport, traffic and energy to make delivery of related public services more responsive. Even in a medium-sized city like ours, the amount of relevant data my fellow citizens generate would seem to make this a huge job.

According to some data I just found on the internet, 90% of all the data that exists was created in the last two years. The challenge of sorting through all of this stuff, and condensing or consuming it for some useful purpose — for research, for commerce or just for fun — is becoming one of the great communications challenges of the age. Essentially, the job of any professional communicator has always been to sort through a lot of data and transform it into useful information: in journalism this means bringing out the salient facts of a story, in marketing the key selling points of a product or service, in PR a bit of both. But how do we go about sorting through the deluge of data that exists now?

  

JISC's rather good newsletter has some ideas about how we as individuals and institutions might go about this daunting task. I liked point 6, though it doesn't seem a particularly risky claim to suggest the visualisation of information will increase. The project to display interactive information about every bomb dropped on London in WWII, explored on the next page of the newsletter, is a fascinating example.

It's worth remembering that not all of this is new. This Economist article from 2007 on the history of graphic displays of information contains some examples that are just as impressive and eye-catching now as when they first appeared centuries ago. It's hard to think how today we could better illustrate the disaster that was Napoleon's campaign in Russia than by Minard's chart, while Playfair's beautiful depiction of wheat and labour prices seems to anticipate the Manhattan skyline. The dismal science can be fun too.

The JISC piece also leaves out the problem of institutional bottlenecks: right now I'm trying to explore visual.ly and easel.ly, but the browser on my work computer is obsolete and my attempts to get IT to upgrade it have so far proved futile. The potential for big data is exciting, but any big organisation generating the stuff is also liable to be sluggish in working out how to deal with it.  Big is not always beautiful, as this Guardian article explains. Mountains of data are of little use if your local council only consults 42 people about a major planning decision, then ignores them and does what it wants anyway.

One final point: as with all technology, the information revolution merely magnifies existing trends in society and human nature to produce new opportunities and new dangers. Might the disturbing extent to which American politics has become polarised be at least partly a feature of the ease with which readers, viewers and listeners can now avoid accessing opinions they find unfavourable?