Why Big Data Will Need Big Gear

Hardware rarely comes up in discussions about big data, save for those centered on data warehouse appliances. But the omission hardly means hardware is irrelevant. In fact, big gear might become a big deal as companies look to bolster the performance of their big data systems.

Why Big Data Startups Should Take a Narrow View

One of the statements that struck me most from Structure: Big Data was CA CTO Donald Ferguson’s notion that big data represents a “very promising” opportunity for startups, particularly those targeting specific target use cases. I think he’s right, particularly with regard to the latter part.

Today in Cloud

I continue to watch the Data Marketplace trend with interest, and touched on some of the issues in two¬†weekly updates here on GigaOM Pro last month. Two marketplaces with very different perspectives are those from Infochimps and Microsoft. I’ve spoken separately with Microsoft’s Moe Khosravy and Infochimps’ Flip Kromer, and even recorded one of my early conversations with Flip as a podcast. At last week’s Structure Big Data, Stacey Higginbotham was lucky enough to get them both together on stage for a chat. Clearly aiming high, Flip’s response to a question about the data marketplace opportunity was to suggest “We’re the next Google.” It’s easy to dismiss such a flippant remark as hubris, but it would also be unwise to underestimate the scale of this opportunity. Infochimps, Microsoft, and a host of others are racing to stake out their part of the emerging data web, and to make sure their share is as big as possible.

Meet Mapr, a Competitor to Hadoop Leader Cloudera

Mapr, a stealth-mode start-up with about 30 employees is developing a version of Hadoop and plans to compete with the likes of Cloudera. The company is likely to launch later this year and has been funded by Lightspeed Venture Partners and NEA.

The Hurdles for Moving Big Data ‘Round the World

Underlying all the useful applications, like Hadoop, that have emerged out of the big data ecosystem, there’s a fundamental assumption: The data that companies want will be able to be accessed when companies want and need it, explained Michelle Munson, CEO and co-founder of Aspera.

Reducing Data Latency Leads to Faster Decisions

Estimates say that 90 percent of all data was created in the last two years alone. That staggering figure can lead to analysis paralysis for some organizations, but those that can sift through, analyze and take action on information faster than others will enjoy competitive advantages.

Data Science Toolkit Brings Big Data Analysis to the People

Pete Warden got famous for scraping 220 million Facebook profiles and then analyzing this data to unearth U.S.-wide user connection trends. Now he wants you to be able to do the same with a new web service released at GigaOM’s Structure Big Data conference today.

How Google Uses Data to Make a Better Google

Google may have more distributed data than any other company but it still takes user input to create smarter machines. Google’s Voice Search speech recognition, for example, began to improve when the service started to train itself and improve accuracy through the use of end-user data