6 million downloads later, Elasticsearch launches a commercial product

Fast-growing analytics startup Elasticsearch is now is expanding its open source footprint into the commercial world with a new product called Marvel. If you’re not familiar with Elasticsearch, it’s a very popular open source technology that lets users search, analyze and visualize data contained in JSON documents in near real time. Marvel is a management and monitoring platform for Elasticsearch clusters.

CEO Steven Schuurman explained during a recent interview that Marvel is something using have been requesting for a while — and that it’s well worth the “cheap as chips” price tag. It’s free for development use and $500 for five nodes in production, and was built from the ground up to be an enterprise-grade product (i.e., it’s not a “bolted-on” feature for the open source software). Marvel lets users see performance in real time, but also saves historical data that can be used for anomaly detection and predictive analytics.

Elasticsearch has a large pool of users from which to convert paying Marvel customers, too. Since it was created a few years ago, the core Elasticsearch software has been downloaded more than 6 million times, and Netflix, Facebook, Wikipedia, Atlassian and Github are among its prominent users. When Elasticsearch’s investors started investigating its promise (the year-and-a-half-old company has raised $34 million in two rounds from Index Ventures and Benchmark Capital), they found plenty of supporters among their portfolio companies, Schuurman said.

Users aren’t afraid to pay, either. “We’re doing really well financially, in terms of selling our production support,” Elasticsearch creator and company Co-founder and CTO Shay Banon said.

Shay Banon

Shay Banon

The software is so popular, he explained, because it’s so useful. Part of that utility comes from its ability to slice and dice JSON documents by whatever information they contain (e.g., the 120 attributes in a single tweet) without the need to build predefined schema. That makes it a nice complement to data stores like MongoDB and Hadoop, the former of which isn’t designed to analyze data and the latter of which isn’t designed to be easy or run queries at millisecond speeds.

Facebook uses Elasticsearch for internal analytics, while Wikipedia uses it to power its front-end search engine.

And log analysis has actually become a pretty popular, albeit unexpected application, Banon said. In fact, the team worked for about two days straight helping Rockstar analyze server and error logs when its Grand Theft Auto 5 online release suffered problems. The Guardian used it to index web logs to figure out where traffic is coming from in real time, so it can get the information to editorial staff who can try to maximize readership in the moment.

CEO Schuurman is confident enough in Elasticsearch’s log capabilities that he’s starting to take jabs at log-management kingpin Splunk.

“For Splunk, it’s bread and butter,” he said. “For us, it’s a use case.”

An example of data visualization in Elasticsearch.

An example of data visualization in Elasticsearch.

But Elasticsearch’s ascent to developer-favorite status isn’t exactly novel. The company is one of many, including its JSON-centric peer in MongoDB, to take advantage of a boom in mobile and web applications by building businesses centered around pleasing developers. Although there’s a lot of venture capital floating around this space, not every company that gets it can win.

However, assuming it can handle the transition from open source company to software vendor, and assuming nothing nothing better comes along to displace it — not an entirely inconceivable notion in a fast-moving open source world — Elasticsearch does seem positioned to be one of the big winners.

Correction: This story was corrected at 7:30 a.m. to reflect the proper pricing for Marvel. It is $500 a year per five nodes, not $500 a per node.