The Real Reason Powerset Sold (Out)

When visiting Israel in the middle of summer, it’s generally not a good idea to go for a walk in the afternoon, even if it is along the sea. The heat and humidity sap your energy, making you feel as if you spent nearly three hours in the gym. But that wasn’t enough to stop me from writing a post about Microsoft buying Powerset for what is rumored to be around $100 million.

I’ve been unable to stop wondering why founder Barney Pell decided to take the money and run — after all, he used to turn blue in the face telling people how superior Powerset’s approach to search was. If it was so superior, Mike Masnick of Techdirt put it best when he wrote that “[T]he exit certainly falls well short of the hype around Powerset. If Powerset was actually seeing any traction at all it never would have agreed to sell at that price.”

To some extent, Mike is right, but I would add another reason: infrastructure, specifically how expensive it is to build. At our Hadoop meet-up earlier this year, Chad Walters, director of engineering at Powerset, noted that their search “requires 100 times more processing than simple keyword searching and indexing (about one second per sentence is required for processing).”

Powerset used some pretty nifty technologies to build out their system, but in order to really scale, they would have needed more money — a lot of it.

And Powerset would have had to scale; there’s no other way to compete with search’s 800-pound gorilla, Google. That’s why Microsoft is building a gigantic data center in the Chicago area focused almost entirely on search. (Which it can now use to help roll out Powerset’s search technology to a larger audience.)

This is an abject lesson for every startup looking to get into the business of search: No matter how good your algorithms are, you still have to deal with the cost of queries, which need to be low enough to be offset by some kind of advertising in order to make a profit. (The conspiracy theorist in me says that if your results are really good you won’t be able to generate enough inventory to serve up ads that bring in the dollars, but maybe I’m just too cynical.)

One of our readers believes that it is possible to build a search engine that surpasses Google’s. Nevertheless, as I’ve noted in the past, “[P]rocess-optimized infrastructure ensures that Google’s cost of executing a query keep going down” — and that allows the company to wring more dollars from the system.

Given all that, Powerset has done a good job of wringing a hundred million from Microsoft. Not that there’s anything wrong with that.

Bonus Link: Don Dodge of Microsoft explains the logic behind the deal.