New Research Reports on Big Data

Two new research reports on big data flash out its early impact on enterprise IT.

David Newman, Research VP at Gartner and a member of its Enterprise Architecture team, warns in Big Data Disruptions Tamed With Enterprise Architecture that “Big data will disrupt your business. Your actions will determine whether these disruptions are positive or negative.“ Newman sees three areas where big data has the biggest impact and three corresponding strategies:

• Enabling decision makers to quickly spot patterns across different data types, which requires a data-savvy business strategy to turn information advantages into competitive advantages.
• Highlighting cultural issues to business and IT leaders, which requires incentives and metrics to overcome silos and lack of trust.
• Exposing talent gaps and introducing new interdisciplinary and senior-level roles, which require organizations to attract and retain specialists and managers with deeper analytical skill, and requiring IT leaders to acquire and apply new tools, techniques and architectures for analyzing, visualizing, linking and managing big, complex datasets.

Given Newman’s enterprise architecture expertise it is particularly interesting to see what he has to say about what he thinks is the “significant impact” of big data on the discipline of enterprise architecture (EA): “For the EA practitioner, the balance shifts from traditional methods that focus on optimization, standardization and efficiency, to lightweight approaches that focus on harmonization, externalization and growth. Big data also overturns traditional information architectures — from a prior focus on data warehousing (data storage and data compression) toward data pooling (flows, links and shareability).”

What Newman doesn’t say but I think is a fair conclusion from his discussion of EA and big data, is that external data sources play a significant role in enterprises’ big data strategies and will radically change the concept of “enterprise architecture.” With big data, the “architecture” covers much more than the enterprise, encompassing all the relevant data flows, links, industry-wide shared data, and sources of public data.  A great example, mentioned in the Gartner report, is the Pistoia Alliance, “a precompetitive information commons” formed by companies in the pharmaceutical industry.  Based on such efforts, “Gartner contends that enterprises will cooperate to co-create big data pools, then compete to extract value from them.” Indeed, that’s what big data is all about.

The other new report is the 1st Big Data Insight Group Industry Trends Report, focusing on “understanding the business benefits and strategic implications of big data.” It is based on a survey of 300 UK-based “senior decision-makers” and its key findings provide a glimpse into the current state of big data:

• 67% are either unfamiliar with big data or are still in the education process;
• A third of the organizations (33%) surveyed have either implemented or are looking to implement big data projects;
• At this early stage, the majority of organizations (80%) have not quantified the benefits of their projects;
• The ownership of big data initiatives currently rests predominantly with the IT department (47%); and
• Less than a quarter of respondents (23%) believe they will require new personnel to execute a big data strategy.

The last finding in particular is in stark contrast to the general consensus regarding the dearth of big data skills and Gartner’s advice to “produce resource planning deliverables to identify gaps for interdisciplinary roles (such as data scientists) and senior-level positions (such as chief data officers).”

Even more interesting, “a significant proportion (31%) appear confident they can execute a big data strategy with their existing staff.” The Big Data Insight report suggests that this is because people are doing “tech-led pilots and that IT people are the ones who are experimenting with the technologies. As such many are unaware if they will require new personnel for a business focused strategy.”

But big data is about better business decisions and will require not only hard-to-find data scientists and creation of new roles such as Chief Data Officer, but a new business mindset led by business executives with a fundamental grounding in and experience with big data analytics. The Big Data Insight report makes a great point about where the emphasis in any enterprise’s strategy should lie: “What shines out from these findings is that big data is not simply about new tools and technologies to deal with increasing amounts of data. It is about taking an intelligent approach to using that data to answer clear, predefined, business orientated objectives from which an organization can reap the rewards.”