Why the big data startup boom will likely be short-lived

There has been a remarkable flowering of companies over the past year or two, all riding a wave of developer and investor enthusiasm for the loosely defined concept of “big data.” But given that the big data startup market is probably overvalued and headed for a lot of consolidation, these new companies’ days might be numbered.

Today in Social

Derrick Harris writes how EMC is wrapping its big data analysis tools in social media to boost enterprise collaboration. The idea is that social media will aid in discovery: employees other than business analysts directly charged with the task will trip over data findings they can use for business decisions. Opening up big data to non-scientists was one of the topics that came up at a recent GigaOM Pro webinar as well. That theme – and deeper dives into social enterprise collaboration – will be all over our Net:Work conference on the future of work today. I’ll be there, so drop by the Pro booth and tell us how we’re doing and help guide future research directions. If you can’t be there, check into our live feeds, and the next best thing is this analysis anthology.

Today in Cloud

Derrick Harris provides a summary of a Facebook presentation this week, in which the company shared some insights into operating at scale. 60 million queries every second, on a sharded architecture configured to keep as much of that traffic as possible away from (expensive) queries of the database. Open source software plays a critical role in keeping Facebook running, and all of it is heavily optimised to fit the company’s rather unique environment. And underneath a lot of it? The humble MySQL database.

Today in Cloud

There’s been some coverage today of Cisco’s latest Global Cloud Index, which predicts that traffic to and from the cloud will account for one third of all traffic passing through data centers by 2015. Derrick Harris highlights the report’s interest in 2014, “the year cloud-based workloads will surpass traditional-data-center-based workloads in total volume.” Sean Ludwig pulls out some of the headline figures early in his piece, stressing that “global traffic generated by cloud computing services will increase a staggering 12 times by 2015 compared to cloud traffic in 2010, while data center traffic will increase at a less-showy-but-still-impressive four times by 2015.” With typical irreverence, The Register is quick to point out that “Cisco is engaging in a bit of marketeering by calling it a ‘cloud’ index.” They have a point; this is about far more than the cloud. Still, the numbers make for some interesting reading. Is your network ready to handle the load?