Data data everywhere but not a scrap of sense
It was exhilarating to see the enthusiasm around the GovHack event as hordes of developers enjoyed pulling together data sets in new and innovative ways. It is certain that it will provide enthralled users with not only access to, but also insight from, the resulting information combinations.
It was also heartening to see Pamela Fox provide some best proactive tips for developers and data owners in her post stressing the value of structure and standardisation where possible. But I was reminded yesterday in a discussion about social software how much of our total information is now in an unstructured format, where the value lies in the ability to understand the context and meaning of data and its relationship to other information which is not supported in a nice neat way.
This became apparent at the Public Sphere event that Pia Waugh championed earlier this year where everyone struggled to consolidate the extremely valuable – but vast and unmanageable – variety of input in all sorts of different forms. Oral, written, blog posts, tweets, videos,… and many more.
The team did a great job at pulling together a useful summary and set of recommendations but I was left thinking that the increasing torrent of data is leading to diminishing returns as individuals initially try to monitor the real-time fire hose of information and secondly, as they pause to reflect, analyse, and try to derive value from a range of inputs.
So, what am I saying here? Basically that the agenda of Gov 2.0, and of the whole project of providing transparency and openness in government data, cannot be met unless we deal with the challenge of finding the “jewels”, the “gems” in the unstructured data itself. Surely, given that we have a range of companies working with us on the Gov 2.0 project, and we have recognised that utilising the power of semantic technologies is going to play a big part in allowing us to address this issue, would it not be sensible and timely to integrate some of the processes that are already being developed into the way the Gov 2.0 Task Force itself operates – the whole mantra of “eating our own dog food”. A radical thought but perhaps with some merit.