Big Data and Intuition: Status Update

BD_accenture0914Two new reports on big data and big decisions were released last week by Accenture and PwC. Both reports shed new light on the impact of big data on enterprises today, and how it is changing the process of decision making by senior executives.

An Accenture worldwide survey of senior technology and business executives has found that companies that have completed at least one big data project are happy with the results. 60% of executives said their companies have successfully completed a big data implementation, 36% haven’t pursued a big data project yet and 4% were currently pursuing but hadn’t finished their first big data project. 92% of executives from companies that have completed a big data implementation are satisfied with the results and 89% rated big data as “very important” or “extremely important” to their businesses’ digital transformation.

Other highlights include:

  • Executives see tangible business outcomes from big data in finding new sources of revenue (56%), enhancing the customer experience (51%), new product and service development (50%), and winning and keeping customers (47%).
  • Challenges in implementing big data are security (51%); budget (47%); lack of big data implementation talent (41%); lack of talent for big data and analytics on an ongoing basis (37%); and integration with existing systems (35%).

In welcome news to the graduates of the numerous data science and business analytics programs, nearly all (91%) companies expect to increase their data science expertise, the majority within the next year. 54% of executives said their companies have already developed internal technical training opportunities for their employees and most organizations also tap outside expertise. Only 5% of respondents said their company used only internal resources for their big data implementations.

The report also reveals that “many companies have different definitions of big data.” Indeed, in answer to the question “Which of the following do you consider part of big data?” responses varied from “Large data files” (65%), to “Advanced analytics or analysis” (60%) to “Data from visualization tools (50%).”

Regardless of how they define big data, 89% believe big data will revolutionize business operations in the same way the Internet did. 85% feel big data will dramatically change the way they do business and 79% agree that “companies that do not embrace big data will lose their competitive position and may even face extinction.” More than anything, in my opinion, these answers reflect the persuasive powers of a buzzword, inherent in its dual, attention-getting role: The threat of “disruption” and the promise of a “revolution” (two words used to great effect by the report).

The PwC report, titled “Gut & Gigabytes: Capitalising on the art & science in decision making,” is based on an Economist Intelligence Unit (EIU) worldwide survey of 1,135 senior executives. It clearly defines big data as “the recent wave of electronic information produced in greater volume by a growing number of sources (i.e. not just data collected by a particular organisation in the course of normal business).”

Intuition has got a bad rap in the age of big data and the most interesting finding of this report is that 30% of executives admit that intuition is what they most relied on when they made their last big decision. An additional 28% relied on other people’s intuition (“advice or experience of others internally”). Only 30% said that “data and analysis (internal or external)” is what they relied on for their last big decision and another 9% relied on “financial indicators.”

The reliance on intuition and experience is based on… experience. 46% of executives said that relying on data analysis has been detrimental to their business in the past. They are concerned about quality, accuracy and completeness of data and find it difficult to access useful data.

Still, 64% of the executives surveyed said that big data has changed decision-making in their organizations and 25% expect it will do so over the next two years. And 49% of executives agree that data analysis is undermining the credibility of intuition or experience, compared with 21% who disagree. Says the report: “In reality, however, experience and intuition, and data and analysis, are not mutually exclusive. The challenge for business is how best to marry the two. A ‘gut instinct’ nowadays is likely to be based on increasingly large amounts of data, while even the largest data set cannot be relied upon to make an effective big decision without human involvement.”

[Originally published on Forebs.com]

Advertisements

About GilPress

I launched the Big Data conversation; writing, research, marketing services; https://whatsthebigdata.com/ & http://infostory.com/
This entry was posted in Big Data Analytics, Intuition, Stats, Surveys. Bookmark the permalink.

4 Responses to Big Data and Intuition: Status Update

  1. Great posting and very timely indeed. I read both of these consultancy reports as well and am pleased that the mainstream of consulting is finally focusing on the real challenge (Hint: It’s nothing to do with Big Data, but is one of Leadership) the lack of acumen & expertise in evidence-based decisioning by the C-Suite in most Enterprises. Analytics will never become pervasive, much less truly impactful unless the Senior Executive Team can “Walk the Talk”, not just cheerlead from the top down. For the Culture of the Organization to rally around Analytics and the exploitation of data and adapt to use them in their daily endeavors they must be lead by Executives who not only believe, but can demonstrate their own abilities. It is a true imperative and I have focused much of my recent postings on this challenge calling it “The Data Leadership Nexus” (www.infomgmtexec.me). Check it out if interested.

    Like

  2. Pingback: Big Data and Intuition: Status Update | kwalitisme

  3. Pingback: The Three Pillars to improve Apple Pay: Data Science, Cloud Computing and Security | Diary of a Data-Driven Product Manager

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s