Today in Cloud

My Weekly Update about COTS solutions in webscale companies has generated much reaction (mostly negative), with much of it stemming from the condensed version posted on GigaOM. However, I’d like to hear what Pro readers think, based on the full version available here. Far from an endorsement of proprietary or commercial solutions, the post really is meant to question whether it’s impossible that they could work within web-based companies that need scalable data solutions. Maybe I’m way off base, but it doesn’t seem inconceivable that commercial vendors could reduce the need to develop backend technologies in-house, thus freeing brain power to focus on the core product. I’d love to hear what you think.

Why Big Data & Real-Time Web Are Made For Each Other

[qi:gigaom_icon_cloud-computing] There’s been a lot of discussion lately about the real-time web and the problems it poses for incumbent search companies and technologies. Fast-moving trends and the availability of up-to-the minute updates mean that purely historical answers are missing crucial information. Dealing with constantly growing information streams causes performance and scalability problems for existing systems and calls into question the mechanisms for compiling, vetting and presenting results to users.

While these challenges may sound new, game-changing performance and scalability problems are also being faced in the more traditional realm of data analytics and large-scale data management. Driven by network-centric businesses that track user behavior to a fine degree, there has been an explosion in the speed and amount of information that companies need to make sense of, and an increasing pressure on them to do so faster than ever before. What needs to be recognized is that the inadequacies of existing systems in these two seemingly different environments stem from the same source — infrastructure built to handle static data simply doesn’t scale to data that is continuously on the move. Read More about Why Big Data & Real-Time Web Are Made For Each Other

Cloudera CEO: Hadoop Will Go Beyond Web Apps

Hadoop is going to find potential markets in any industry where there are large data sets that need complex analysis,” Mike Olson, chief executive officer and one of the four co-founders of Cloudera, the startup that’s commercializing the open-source software framework Hadoop, told me earlier today. I spoke with Olson after the Burlingame, Calif.-based company said it had raised $6 million in new Series B funding from Greylock Partners and Accel Partners.

From the analysis of point-of-sale data to genomics data, the opportunities for Hadoop are plentiful, Olson said. Bioinformatics and pharmaceuticals are emerging as big opportunities for Hadoop, which first found favor with Web 2.0 and social networking applications looking to optimize advertising. Read More about Cloudera CEO: Hadoop Will Go Beyond Web Apps