Cloudy days for data, Part 3 (conclusion)

After a bit of a delay I’m going to wrap up this series tonight.  In the last segment, I finished up describing the struggle between file independence and data management, ending with the usefulness of metadata and touching on Cory Doctorow’s dissension on that topic.

I’d rather that readers take some time to read Doctorow’s interesting thesis than for me to repeat too much of the content here, but suffice to say he pokes seven disturbing holes into what he terms “meta-utopia” and as I said previously it isn’t easy to disregard his points.  The most troublesome to me concerns the problem of crafting agnostic, universal schemas, which tend to be two-dimensional and thus create difficulties for diverse parties trying to arrive at design consensus.  Doctorow correctly points out that data sets and needs tend to be richer than that, extending easily into three dimensions (my extrapolation) in order to properly define any object.  Just as databases are better than flat files, data cubes are better still– and there has been work along those lines with XML.

Doctorow ends on a somewhat promising note by admitting that, flaws and all, metadata is still useful.  It’s just that those choosing to rely on it as part of any information management or publishing solution need to be aware of the pitfalls going in (his article should be part of the solution design requirements in my opinion).

Cloudy days for data, Part 2

Inspired by blog articles from Michael Dominic K. and Henri Bergius, in part one of this series I touched on the conflict between the order of databases and the freedom of files.  The latter are fiercely hoarded and guarded by information workers who in many cases don’t really care about the object so much as the information it conveys.  But what are files, and why does this issue of managing their chaos even exist?

The modern electronic file is a data or readable text object meant to store and/or convey useful information.  It can constitute one or more records (I’ll even consider a graphic image as a record in this context).  The beauty of a file is its common independence from the confines of enterprise data stores.  This is especially true in the case of files based on open source and highly-accepted commercial formats; they are often accessed by a multitude of viewers or renderers.

But that independence comes at a cost.  The further a file object lives from a data management system, the more risk it can introduce and encounter.  Versioning, detail accuracy and unapproved copying are just some of the risks.   Files also consume storage space often disproportionate to the value they add, especially as they are wastefully replicated in myriad emails and hard drives.  The cheapness of onsite storage has helped exacerbate this, removing old constraints that once encouraged conservation over ease of access.