Hadoop through the years: A GigaOM retrospective

A few years before we had a Structure:Data conference dedicated to big data — and, by proxy, Hadoop — GigaOM spotted Hadoop’s promise and began trying to spread the word about and advance the discussion around this groundbreaking technology. Now that Hadoop is 10 years old (give or take), we thought now would be a good time to look back on how Hadoop has influenced our events and editorial over years. This is the final installment in our four-part Hadoop anthology that has already covered its birth, present and future.

Think about this like Hadoop’s greatest hits, but know that there will be more to come. Although the big data discussion is moving away from Hadoop somewhat, it’s still an integral — if not the integral — part of the discussion around data infrastructure. We have two great panels on Hadoop at our Structure:Data conference March 20-21 in New York (which include participants from Facebook (s fb), Platfora, Continuuity and EMC’s (s emc) Pivotal Initiative (whose leader Paul Maritz will also be speaking), among others) and will keep up with all things Hadoop and data for the next 10 years.

The biggest news

  1. Hadoop-focused startup Cloudera raises $5 million (March 15, 2009)
  2. Friends on the move: Hadoop, AOL & PayPal  (Aug. 10, 2009)
  3. Survey: Hadoop is great, but challenges remain (Sept. 29, 2010)
  4. Yahoo suggests MapReduce overhaul to improve Hadoop performance (March 17, 2011)
  5. Meet MapR, a competitor to Hadoop leader Cloudera (March 24, 2011)
  6. EMC makes a big bet on Hadoop (May 9, 2011)
  7. Exclusive: Yahoo launching Hadoop spinoff this week (June 27, 2011)
  8. Microsoft’s Hadoop play is shaping up, and it includes Excel (Feb. 28, 2012)
  9. VMware aims for Hadoop on VMs with ‘Serengeti’ project (June 13, 2012)
  10. Cloudera makes SQL a first-class citizen in Hadoop (Oct. 24, 2012)

The best analysis

  1. The data mining renaissance (April 10, 2009)
  2. Is Hadoop champion Cloudera the next Red Hat? (Oct. 2, 2009)
  3. Meet the big data equivalent of the LAMP stack (Aug. 1, 2010)
  4. As big data takes off, the Hadoop wars begin (March 25, 2011)
  5. Hadoop’s civil war: Does it matter who contributes the most? (Oct. 7, 2011)
  6. 5 low-profile startups that could change the face of big data (Jan. 28, 2012)
  7. What it really means when someone says Hadoop (Feb. 6, 2012)
  8. Hadoop jumps through hoops, becomes mainstream (March 3, 2012)
  9. Why the days are numbered for Hadoop as we know it (July 7, 2012)
  10. A few stats, rumors and stories on Hadoop’s rapid growth (Nov. 9, 2012)

The coolest users … aside from Yahoo





The smart grid world

Obama for America

Yelp BloomReach Ancestry.com
LinkedIn Quantcast Disney
Orbitz Klout Twitter
The medical world Climate Corporation Skybox Imaging
Tumblr Intuit @Walmartlabs
Zions Bancorporation LivePerson The enterprise security world

Taking Hadoop to the stage

The Hadoop Meetup (May 1, 2008)

Cutting (center) flanked by Baldeschwieler and Om Malik at GigaOM’s Hadoop Meetup in 2008.

Cutting (center) flanked by Baldeschwieler and Om Malik at GigaOM’s Hadoop Meetup in 2008.

Next-generation data stores (Structure 2008; start at 57:00)

[protected-iframe id=”ebdd9886a3978c4bb1384f57f8f6c1d7-14960843-6578147″ info=”http://cdn.livestream.com/embed/structure08?layout=4&clip=pla_3878143560401242134&color=0xe7e7e7&autoPlay=false&mute=false&iconColorOver=0x888888&iconColor=0x777777&allowchat=true&height=295&width=708″ width=”708″ height=”295″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]

Hadoop, NoSQL and webscale data (Structure 2009)

[protected-iframe id=”4a1b159df6053976350a66f29cd0b524-14960843-6578147″ info=”http://cdn.livestream.com/embed/gigaomtv?layout=4&clip=pla_6674443224376701134&color=0xe7e7e7&autoPlay=false&mute=false&iconColorOver=0x888888&iconColor=0x777777&allowchat=true&height=295&width=708″ width=”708″ height=”295″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]

The big data tsunami (Structure 2010)

[protected-iframe id=”a5f574e9e6bbeeb8b3cdd792ad5ab4a1-14960843-6578147″ info=”http://cdn.livestream.com/embed/gigaomtv?layout=4&clip=pla_608b142d-06e8-4670-80a8-ad3de4ff2035&height=340&width=560&autoplay=false” width=”560″ height=”340″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]

Hadoop and beyond (Structure: Data 2011)

[protected-iframe id=”d0d3b8a0f7edd44ee63bc057f68e2334-14960843-6578147″ info=”http://cdn.livestream.com/embed/gigaombigdata?layout=4&clip=pla_770fb9d2-5ee0-4094-946f-09c3a2c4431e&height=340&width=560&autoplay=false” width=”560″ height=”340″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]

What’s next for Hadoop? (Structure: Data 2012)

[protected-iframe id=”966bd390c87c1a18474d8c46a8ddfd1d-14960843-6578147″ info=”http://cdn.livestream.com/embed/gigaombigdata?layout=4&clip=pla_afb3ecbe-ca33-4a17-81eb-02a1a14ec2fe&height=340&width=560&autoplay=false” width=”560″ height=”340″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]

Mike Olson on Hadoop (Structure: Data 2012)

[protected-iframe id=”a21e3212a876ff231fc1200d1104a9b9-14960843-6578147″ info=”http://cdn.livestream.com/embed/gigaombigdata?layout=4&clip=pla_fb7a38ae-7383-49c6-9d51-bfccd8add2cf&height=340&width=560&autoplay=false” width=”560″ height=”340″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]

Analyzing data with HBase (Structure: Data 2012)

[protected-iframe id=”e4ee0350e2e414aeed70f86d35dd4ed3-14960843-6578147″ info=”http://cdn.livestream.com/embed/gigaombigdata?layout=4&clip=pla_2751ad83-df24-4f31-88ea-ec1cc956ddd5&height=340&width=560&autoplay=false” width=”560″ height=”340″ frameborder=”0″ style=”border:0;outline:0″ scrolling=”no”]