Chartbeat tries to fight the smoke and mirrors in web measurement by going public with its metrics

While some progress has been made in getting web publishers and advertisers to stop thinking about pageviews or other traffic-focused metrics, and start thinking about measuring actual attention or engagement, there is still one big problem, according to Chartbeat founder and CEO Tony Haile — everyone has their own way of measuring those things, and no one is willing to share the secrets that lie inside the black box they use to do so.

In an attempt to force a change to that state of affairs, Chartbeat announced on Monday that it is making public all of the metrics, standards and methods of measurement it uses internally — including a detailed breakdown of their weaknesses and limitations.

In an interview before the announcement was made about the company opening up its measurement approach, the Chartbeat CEO said he wants to try and get everyone in the industry to share as much of their data and methods as possible. Why? Because it will make online publishing — and the advertising-based business model that continues to fuel it — better in the long run, he said.

I’m in the business of change, I actually want this industry to get better, and I think what will help the industry do that is if we can all be a bit more open and transparent about these things. We’re going to get out there and say this is exactly how we do everything.

Openness builds trust in online numbers

Last month, Chartbeat became the first measurement company to have a metric based on attention accredited by the Media Rating Council, the body that determines what standards can be used to buy and sell advertising online. Now, the company has put the documents and reference material it supplied in order to get accredited — a process that took more than six months worth of interviews and in-depth review of every aspect of the company’s business — and putting them all online where anyone can examine and/or learn from them.


Haile admits that he is fighting an uphill battle when it comes to openness. Historically, measurement and analytics companies have kept their methods to themselves as much as possible, and so have the media and publishing companies that make up most of their clients. In many cases, that’s because the numbers they used were little more than educated guesses at best, and outright smoke and mirrors at worst. As Chartbeat put it in describing its move:

The lack of clarity around how metrics are actually measured, their methodologies, advantages and limitations has led to immense confusion. This opacity has led to publishers and agencies each choosing the vendor that spins the data into the best story, regardless of accuracy, and then fighting over who’s right. That’s a lousy way to do business.

The Chartbeat CEO says he believes that online publishing needs to throw off the covers and get rid of the smoke and mirrors if it wants to succeed as an industry, and so he has decided to make his company an example. Hopefully others will see that openness is better, he said — in part because it allows clients and agencies to trust the numbers they are getting more than if it is a black box. And if everyone shares their data and methods, then everyone benefits.

But couldn’t other companies — especially other competing providers of attention metrics and analytics — simply copy everything that Chartbeat is doing by reading through all the documents it has put online, without opening up about any of their own metrics or methods? Haile admits that they could, and that his decision is risky. But he says that he believes the gamble is worth it, if it helps the industry as a whole succeed. And maybe if he opens up, clients will start asking other companies they work with to do so as well.

Post and thumbnail images courtesy of Thinkstock / Tay Jnr