Superabundance of Data

28 March

Data superabundance

 Todays blog deals with the information overload. How is it affecting us as individuals, costs to business to try to store and make sense of it and a cultural cost to us as living beings. Are you spending huge amounts of time just trying to organise your paperwork? The term for this overload is "superabundance of data".

Quote

From gatherers to analysts - the new insight value chain? | Blog
We are moving from a world where information was scarce to one where it is becoming superabundant, a world where a megabyte (2 to the power of 20 bytes) used to be a worryingly large amount of information to analyse but where we are now starting to worry about dealing with Yottabytes - that 2 to power of 80 bytes!

Quote

Technology: The data deluge | The Economist
EIGHTEEN months ago, Li & Fung, a firm that manages supply chains for retailers, saw 100 gigabytes of information flow through its network each day. Now the amount has increased tenfold. During 2009, American drone aircraft flying over Iraq and Afghanistan sent back around 24 years’ worth of video footage. New models being deployed this year will produce ten times as many data streams as their predecessors, and those in 2011 will produce 30 times as many.

Lets get this into perspective:

The Pointed End of the Spork » Data Superabundance — the coming storm
“ .In 2006, the amount of digital information created, captured, and replicated was … about 3 million times the information in all the books ever written... The digital universe in 2006 could be likened to 12 stacks of books extending from the Earth to the sun.  (A Forecast of Worldwide Information Growth Through 2010 — IDC, March 2007)

Examining Ethics

Quote

Technology: The data deluge | The Economist

Examples abound of databases being stolen: disks full of social-security data go missing, laptops loaded with tax records are left in taxis, credit-card numbers are stolen from online retailers. The result is privacy breaches, identity theft and fraud. Privacy infringements are also possible even without such foul play: witness the periodic fusses when Facebook or Google unexpectedly change the privacy settings on their online social networks, causing members to reveal personal information unwittingly. A more sinister threat comes from Big Brotherishness of various kinds, particularly when governments compel companies to hand over personal information about their customers. Rather than owning and controlling their own personal data, people very often find that they have lost control of it.

The best way to deal with these drawbacks of the data deluge is, paradoxically, to make more data available in the right way, by requiring greater transparency in several areas. First, users should be given greater access to and control over the information held about them, including whom it is shared with. Google allows users to see what information it holds about them, and lets them delete their search histories or modify the targeting of advertising, for example. Second, organisations should be required to disclose details of security breaches, as is already the case in some parts of the world, to encourage bosses to take information security more seriously. Third, organisations should be subject to an annual security audit, with the resulting grade made public (though details of any problems exposed would not be). This would encourage companies to keep their security measures up to date.

How about the affects on us as individuals

Quote

http://www.namingandtreating.com/?p=16774
The ’superabundance’ of data is changing not only the amount of available content we access online, but also our ability to store all of it. This statement also seems to reflect what may be going on with us psychologically right now as we feel overwhelmed and exhausted, trying to define how much content is enough or worth keeping. It represents a combined process of content and cognitive/emotional overload. As we over saturate ourselves in content, time and better mechanisms for searching and disseminating content will determine how effective we can be in using the data we collect.

Other References

disruptorMonkey: Data Superabundance, | Deduplication could be the answer to "too much data"