Hadoop: Why build a cheaper mousetrap?

Hadoop represents a paradigm shift in computing. This affordable and scalable framework for distributed processing of massive data sets will enable unimaginable computing applications in the future. So what’s behind the unprecedented adoption of Hadoop across major vendors and the dozens of new Hadoop-focused startups emerging on a regular basis? Why are some of the best minds in our industry focused on building a cheaper mousetrap?

Analytics appliances can provide advanced and highly performant SQL analytics and will outperform nearly any SQL engine running on top of Hadoop for nearly all analytics workloads. However, these multimillion dollar systems are out of reach to all but the largest corporations.

The low barrier to entry for advanced analytics is driving Hadoop adoption. Yet  vendors focused on organizing data in Hadoop into rows and columns are too large to count. Why must all data be structured, and why must all data be accessed via SQL? Distributed computing demands a new approach to looking at data, not just adapting last century’s approach.

Processing power and storage are cheap. It’s time to rethink data analytics and focus on reducing the labor and time needed to go from raw data to insight instead of merely focusing on processing time. With a computing engine like Hadoop that scales horizontally, accelerating processing time is simple: Throw more hardware at the problem. Humans remain the most costly aspect of an analytics project. We must begin focusing our efforts on maximizing their productivity.

Read more about Splunk solutions for Hadoop.

Advertisement