Big Data Bytes of the Week: The End of Big Data?

The end of Big Data? Based on his discussions with CIOs, reports Derrick Harris at GigaOm, Opera Solutions’ CEO Arnab Gupta “thinks the analytics market will crest around the end of next year as CIOs face enormous data spikes.”  Is this what he means by “turning Big Data into Small Data?” Apparently saying “crest” is a very convincing way to get $84 million, but does he really believe that the Big Data flood is going to start tapering off next year?

Participants in the O’Reilly Strata conference on Big Data think not, as Marshall Kirkpatrick at ReadWriteWeb reports under the convincing title “Bankers go bonkers over Big Data’s future”: “Huge opaque markets are about to become transparent because of new regulations and that means a whole lot of new data available for analysis. Scalable processing of that data will require outsourcing, giving birth to new industries. Millions of people will need to be trained to deal with all this.”

Indeed, outsourcing could be good way to raise a glass or two of Big Data, for those who are not going bonkers and are in general slow to catch up with new developments (most enterprises?). But “outsourcing” is outmoded, so Brad Howarth at CIO Magazine employs the more fashionable “data-as-a-service” (DaaS), discussing companies such as Infochimps and Factual ho do all the dirty data work for you. What really got my attention, however, was the quote from Dennis Claridge, cofounder of Melbourne-based doubleIQ, summing up nicely the state of data today: ”[Enterprises] don’t make as much use of their information as they could, and it gets lost over time. Data’s never valued correctly in an organisation.”

In contrast, says DJ Patil on O’Reilly Radar, “I’ve found that the strongest data-driven organizations all live by the motto ‘if you can’t measure it, you can’t fix it’ (a motto I learned from one of the best operations people I’ve worked with). This mindset gives you a fantastic ability to deliver value to your company by: