The Tom Davenport Guide to Big Data

DAvenport_BDBig Data at Work: Dispelling the Myths, Uncovering the Opportunities, is a new book from Tom Davenport, a veteran observer of the data analysis scene. It’s a required reading for managers that need a straightforward, hype-free introduction to big data, a clear and clarifying “signal” in the incredible noise around the confusing and mislabeled term. If Viktor Mayer-Schönberger’s and Kenneth Cukier’s book was last year’s definitive text on the subject for general audiences, Big Data at Work is the 2014 definitive guide to starting and managing the big data journey in small and large organizations.

Davenport discusses in the book the experiences of the early adopters of big data, how to develop a strategy and a plan of action regarding big data, what skills a data scientist needs, and how big data will change traditional management behaviors. He also provides a “manager-focused” overview of big data technologies, explains what is needed to succeed with big data, and outlines lessons learned (and some “lessons not learned”) from the experiences of startups, online firms, and large companies. He also offers the concept of “analytics 3.0” to describe how companies can combine the best of small data and traditional analytics with the big data approach.

This last bit, Davenport’s attempt to suggest an evolutionary path forward, while discussed only briefly as the books’ conclusion, is emblematic of the core strength of the entire book. No breathless talk about the “big data revolution” here (I think the word “revolutionary” is mentioned only once or twice—how refreshing).

In a world and business environment constantly replenished with ideas, tools, and entities that are genuinely new, it is important to distinguish the new from the old and understand what is old in the new. A good grasp of past developments is key to developing a better and more useful guidance regarding what steps to take and what to expect. This is especially important for established enterprises and their executives, who need to understand how the new phenomenon relates to their previous investments in related ideas that were “revolutionary” and “transformative” just a few years ago (before big data we had analytics, business intelligence, and data mining, to name just a few predecessors).

Tom Davenport is in a unique position to do just that in the context of big data. When he says that big data is “perhaps the most sweeping change in what we do to get value from data since the 1980s,” he not only provides the best definition of “big data” I have yet to encounter, but also demonstrates his intimate understanding of its evolutionary nature. He has been observing, since the 1980s, the constant trend—the ever-growing deluge of digital data—and the incremental changes in our ability to manage it better. In addition, he has been one of a handful of influential thinkers who have tried to understand the impact of this data deluge on the practice of management.

At the forefront of guiding managers through the complex and changing interrelations between information technology and management for many years—from business process re-engineering to enterprise resource planning (ERP) to knowledge management to business analytics—Davenport in this book writes the new, still-unfolding chapter in this history. He correctly points to “online firms” (e.g., Google, Facebook) as the originators of everything big data, not only in terms of the new tools and technologies and “the function of data science” they have developed but also in their new attitude to data and its analysis and their new, data-driven management practices.

There is one more dimension to the inventiveness of these firms which Davenport does not discuss as such but is, I believe, a very important part of what “big data” means and what impact it may have in the coming years. Google, Facebook, and other web-natives did not follow the traditional IT purchasing decision practices of other enterprises. They created their IT infrastructure on their own and did not buy it from established IT vendors. Similar to the way the new “data warehousing” and “enterprise resource planning” technologies of the 1990s, driven by a new attitude towards data and its mining, gave rise to a new type of IT buying decision favorable to vendors focused on only one element of the IT infrastructure (e.g., Oracle, SAP, EMC, Cisco), so did big data alter the way IT is bought (or developed in-house) and managed in web-native companies. Is this a dimension of big data that is going to be adopted by other companies as they adopt its other facets such as a new attitude towards data and new management practices? Is big data going to fundamentally change the IT landscape and the practice of IT?

Maybe these are questions to be examined in Davenport’s next book which no doubt, given his publishing history, will come out in 18 months or so. Other possible topics for discussion in the next book (which could be about the new Next Big Thing—the Internet of Things) could be the enterprise-related challenges of big data that are talked about only briefly in Big Data at Work.

For example, data privacy issues that executives must understand and have a good grasp of their potential solutions. Privacy is part of the larger issue of “data governance,” the comprehensive set of data and risk management policies and processes that every enterprise today must establish and follow—and few have. Recently, Varonis Systems (VRNS), a data governance provider (about which I wrote last year), had the first successful big data (and for that matter, tech-related) IPO of 2014.

Another topic that deserves a more detailed discussion is the ideology of big data, especially the misguided belief that the collection of data is a goal in itself and that the data can speak to us and answer questions we never knew we should have asked. At least this reader would have liked to hear more from the level-headed Davenport about his important warning in this regard: “Sifting [through a big pile of data] without a purpose can become very expensive and time-consuming. It’s far better to have a hypothesis in mind—particularly before gathering a lot of data, and even before analyzing it.”

There is everything required in Big Data at Work, however, to get any reader started on the big data journey, from busy executives to students who would like to understand what role it will play in their future career. “It’s not how much data you have, but what you do with it that counts,” says Davenport. This is not only a great summation of the topic of big data, but also of the book, a practical and highly useful guide to a “phenomenon… of substantial importance to many organizations” and individuals.

[Originally published on]