3 Big Data Milestones

BigData_freeLicensesIf you were asked to name the top three events in the history of the IT industry, which ones would you choose? Here’s my list:

June 30, 1945: John Von Neumann published the First Draft of a Report on the EDVAC, the first documented discussion of the stored program concept and the blueprint for computer architecture to this day.

May 22, 1973: Bob Metcalfe “banged out the memo inventing Ethernet” at Xerox Palo Alto Research Center (PARC).

March 1989: Tim Berners-Lee circulated “Information management: A proposal” at CERN in which he outlined a global hypertext system.

[Note: if round numbers are your passion, you may opt—without changing the substance of this condensed history—for the ENIAC proposal of April 1943, Ethernet in 1973, and CERN making the World Wide Web available to the world free of charge in April 1993, so that 2013 marks the 70th, 40th, and 20th anniversaries of these events.]

Why bother at all to look back? And why did I select these as the top three milestones in the evolution of information technology?

Most observers of the IT industry prefer and are expected to talk about what’s coming, not what’s happened. But to make educated guesses about the future of the IT industry, it helps to understand its past. Here I depart from most commentators who, if they talk at all about the industry’s past, divide it into hardware-defined “eras,” usually labeled “mainframes,” “PCs,” “Internet,” and “Post-PC.”

Another way of looking at the evolution of IT is to focus on the specific contributions of technological inventions and advances to the industry’s key growth driver: digitization and the resulting growth in the amount of digital data created, shared, and consumed. Each of these three events represents a leap forward, a quantitative and qualitative change in the growth trajectory of what we now call big data.

The industry was born with the first giant calculators digitally processing and manipulating numbers and then expanded to digitize other, mostly transaction-oriented activities, such as airline reservations.  But until the 1980s, all computer-related activities revolved around interactions between a person and a computer. That did not change when the first PCs arrived on the scene.

The PC was simply a mainframe on your desk. Of course it unleashed a wonderful stream of personal productivity applications that in turn contributed greatly to the growth of enterprise data and the start of digitizing leisure-related, home-based activities. But I would argue that the major quantitative and qualitative leap occurred only when work PCs were connected to each other via Local Area Networks (LANs)—where Ethernet became the standard—and then long-distance via Wide Area Networks (WANs). With the PC, you could digitally create the memo you previously typed on a typewriter, but to distribute it, you still had to print it and make paper copies. Computer networks (and their “killer app,” email) made the entire process digital, ensuring the proliferation of the message, drastically increasing the amount of data created, stored, moved, and consumed.

Connecting people in a vast and distributed network of computers not only increased the amount of data generated but also led to numerous new ways of getting value out of it, unleashing many new enterprise applications and a new passion for “data mining.” This in turn changed the nature of competition and gave rise to new “horizontal” players, focused on one IT component as opposed to the vertically integrated, “end-to-end solution” business model that has dominated the industry until then. Intel in semiconductors, Microsoft in operating systems, Oracle in databases, Cisco in networking, Dell in PCs (or rather, build-to-order PCs), and EMC in storage have made the 1990s the decade in which “best-of-breed” was what many IT buyers believed in, assembling their IT infrastructures from components sold by focused, specialized IT vendors.

The next phase in the evolution of the industry, the next quantitative and qualitative leap in the amount of data generated, came with the invention of the World Wide Web (commonly mislabeled as “the Internet”). It led to the proliferation of new applications which were no longer limited to enterprise-related activities but digitized almost any activity in our lives. Most important, it provided us with tools that greatly facilitated the creation and sharing of information by anyone with access to the Internet (the open and almost free wide area network only few people cared or knew about before the invention of the World Wide Web). The work memo I typed on a typewriter which became a digital document sent across the enterprise and beyond now became my life journal which I could discuss with others, including people on the other side of the globe I have never met.  While computer networks took IT from the accounting department to all corners of the enterprise, the World Wide Web took IT to all corners of the globe, connecting millions of people. Interactive conversations and sharing of information among these millions replaced and augmented broadcasting and drastically increased (again) the amount of data created, stored, moved, and consumed. And just as in the previous phase, a bunch of new players emerged, all of them born on the Web, all of them regarding “IT” not as specific function responsible for running the infrastructure but as the essence of their business, data and its analysis becoming their competitive edge.

We are probably going to see soon—and maybe already are experiencing—a new phase in the evolution of IT and a new quantitative and qualitative leap in the growth of data. The cloud—a new way to deliver IT, big data—a new attitude towards data and its potential value, and The Internet of Things (including wearable computers such as Google Glass)—connecting billions of monitoring and measurement devices quantifying everything—combine to sketch for us the future of IT.

[Originally published on Forbes.com]

About these ads
Posted in Big Data Analytics, Big Data Futures, Big Data History, Data growth | 2 Comments

Big Data Friday: Borasky’s Law

  • Murphy’s Law: Anything that can go wrong, will go wrong.
  • O’Toole’s Corollary: Murphy was an optimist.
  • Sturgeon’s Law: 95 percent of everything is crap.
  • Mencken’s Law: Nobody ever went broke underestimating the intelligence of the American public.

Borasky’s Law: Sturgeon and Mencken were optimists, too.

Source: What Hath Von Neumann Wrought?

Posted in Big Data Friday | Leave a comment

The Economist’s Kenneth Cukier on Big Data

Kenneth Cukier, Data Editor for The Economist talks to Fiammetta Rocco, The Economist’s Editor for Books and Arts about Big Data.  See my review of Cukier’s (and Viktor Mayer-Schönberger’s) Big Data: A Revolution that Will Transform How We Live, Work, and Think.

Posted in Big Data Analytics, Predictive analytics, Privacy, Risks, Social benefits | Leave a comment

Grok CEO at GigaOM Structure 2013 (Video)

Rami Branitzky, CEO, Numenta (now called Grok): 45% of big data is high-velocity data. Since the advent of the smart meter utilities have seen 3000X growth in the data they store but only half a percent of the data is being acted upon or being processed.

Posted in Big Data Analytics, Big Data Velocity | Leave a comment

Data Explosion: An Internet Minute

Internet MinuteToday, the number of networked devices is equal to the global population. By 2015, the number of networked devices will be twice the global population.

Source: Intel

Posted in Big Data Analytics, Big Data Futures, Data growth, Infographics | Leave a comment

Moju Labs’ Smart Album: The Killer App for Google Glass?

BuzzFeed: This photo of a man showering with Google Glass will haunt you for the rest of your life

BuzzFeed: This photo of a man showering with Google Glass will haunt you for the rest of your life

Mok Oh wants to create a “Smart Photo Album.” I think it may well be the killer app for Google Glass.

The most talked-about gadget of the moment, proudly displayed by only a handful of carefully selected  early users, The Glass has many detractors, skeptics, and adulators. John C. Dvorak thinks Google Glass is a “world-class hoax.” Google’s Executive Chairman, Eric Schmidt, finds it “a little weird,” Reuters reports. But Robert Scoble threatens never to take them off.

Saturating the cyberwaves with grainy photos (including of himself showering with the Glass), Scoble predicts: “If Google Glass is as big a deal as I think it will be, humans will generate much more data than they do today.” I’m not sure how big a deal Google Glass will be, but I recently made a similar prediction, expecting “a new quantitative and qualitative leap in the growth of data.” The Internet of Things is the main culprit in this new acceleration of data growth, and the “Things” generating and consuming data 24/7 include wearable devices such as Google Glass, the Pebble, and Apple’s rumored iWatch.

The nascent debate around Google Glass (including its privacy implications and whether it is different from a smart phone) has centered on how easy it is to take pictures with it. It’s a continuation—and possibly a step change—of the trend that started with the integration of digital cameras with mobile phones. Putting a camera in our pocket has resulted in the more than 300 million photos Facebook users upload every day. We take four times more pictures per day than we did ten years ago, says 1000memories Blog.

“In the midst of the 3.5 trillion photos that have ever been taken,” says Jonathan Good of 1000memories, “it’s easy to forget that the shoebox or album of old photos we have at home is incredibly fragile and special.”

Continue reading

Posted in Big Data Analytics, Big Data Futures, Google Glass, Internet of Things, Machine Learning, Predictions, Social benefits, startups | Leave a comment

David Smith on Predictive Analytics and Big Data (Video)

David Smith, VP  Marketing and Community Relations, Revolution Analytics, on predictive analytics, big data, and real time at the O’Reilly Strata conference.

Posted in Big Data Analytics, Predictive analytics, Revolution Analytics | Leave a comment