The promise of big data still looms, but execution lags

When something is hyped as much as the notion of big data, there’s bound to be disappointment when results don’t meet expectations right this second.

That realization — that implementation of big data analytics and related technologies hasn’t matched expectations — is a common thread across a recent spate of research reports. While corporate execs now “get” the possible impact of aggregating and analyzing all the data their companies generate, very few companies have realized that potential.

Most adoption remains at pilot or test stage

A new McKinsey Quarterly report acknowledged that earlier predictions that retailers would parlay big data analytics to boost operating margins by more than 60 percent, and that the healthcare sector could likewise use the technology to slice costs 8 percent, haven’t played out.

While massively scaled companies like [company]Amazon[/company] and [company]Google[/company] use data analytics to wring out significant costs, data analytics success at most legacy companies is limited to a few test projects or narrow pieces of the overall business. Very few of those accounts “have achieved what we would call ‘big impact through big data,’ or impact at scale,” according to McKinsey.

CEOs have an inflated view of their own big data projects

According to a just-released survey of 362 executives worldwide, there is a pronounced disconnect between what the CEO of a company perceives is happening with that company’s big data efforts and what the rest of the organization sees.

From the key findings of the work conducted The Economist Intelligence unit for [company]Teradata[/company],:

While 47 percent of CEOs believe that all employees have access to the data they need, only 27 percent of all respondents agree that they do. Similarly, 43 percent of CEOs think relevant data are captured and made available in real time, compared to 29 percent of all respondents. CEOs are also more likely to think that employees extract relevant insights from data – 38 percent of them hold this belief, as compared to 24 percent of all respondents and only 19 percent of senior vice presidents, vice presidents and directors.

So, while there is indeed a ton of available data, it’s still hard for most companies to parlay it into useful insights. More than half (57 percent) of the total respondents said their companies do a poor job in this respect, although almost everyone agreed that access to data and the ability to wring actionable insights out of it are critical.

Paranoia isn’t driving enough action

A whopping 60 percent of 226 execs surveyed by [company]Capgemini[/company] Consulting, in yet another report, said that big data will “disrupt” their industry in the next three years but only 13 percent of them have big data implementations in production. And, less than a third of those responding (27 percent) described their own big data initiatives as successful; 8 percent described them as very successful.

So while progress has been made — as Derrick Harris and Fast Forward Labs’ Founder Hilary Mason noted on this week’s Structure Show podcast — executives definitely know what Hadoop is now and have a grasp of what big data can do, much heavy lifting still needs to be done to realize the benefits of analyzing all that data.

For more on this topic of the latest and greatest technologies and how they are put to use — and the cognitive dissonance between big data hype and real-world adoption — will be discussed by top names in the field at Gigaom’s Structure Data event  in March.

Big data implementations status


To hear Fast Forward Labs’ founder Hilary Mason on how that company combs academia, open-source and “outsider art” sources for big data applications, check out the second half of the podcast below:

[soundcloud url=”” params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]

Financial services and the public cloud: Go or no go?

Most financial services companies officially forbid the use of public cloud (aka Amazon Web Services) completely. But the forward thinkers among them — like State Street — keep their options — and minds — open about such deployment in the future.

BYOD bringing more value and challenges in a Post-PC world

Life is hard enough for the modern CIO, with BYOD taking over personal computing and cloud challenges making life more complicated. But there are plenty of benefits that accompany this shift, according to panelists at Structure Europe.

Amazon takes aim at IO bottlenecks

High-performance web applications are often hamstrung by sluggish IO. Amazon is now attacking that problem with new Provisioned IOPS block storage volumes and the ability to direct-connect those volumes to select EC2 compute instances over a 1000 Mbit/sec pipe.

Cloud providers step up consulting services

At this stage, most companies know some of the benefits of cloud computing. But many still aren’t sure what applications and data should make the trip first. That’s why Rackspace and other cloud providers are providing more consultative services and lining up systems integrators.

A CTO’s take on cloud

Joe Coyle, CTO of global integrator Capgemini, sees a lot of cloud pitches from all the major technology vendors — and God knows they all have a cloud strategy. Here’s what he thinks of the current state of the market.

Cloud security: Better than you think

Security concerns are always cited as the primary inhibitor to wider cloud adoption by businesses, but Capgemini CTO Joe Coyle says the security model for the cloud is already here and working just fine, thank you very much.