We are drowning in data about readers and attention, but which metrics really matter? You won’t like the answer

Thanks to the web and real-time measurement tools, the media industry has gone from having virtually no hard data on readers and attention to an embarrassment of riches — not only can we measure what people click on, but we can measure how far down the page they got when they were reading, whether they posted a comment, which social networks they came from, and a hundred other pieces of data. The only problem is that this is very much a double-edged sword.
New York Times media writer David Carr recently looked at some examples of media companies that are rewarding their writers based on traffic statistics and other measurements, including The Oregonian — whose efforts I wrote about here. But is paying your journalists based on pageviews or other metrics a smart way to align their incentives with your goals as a business, or does it poison the well when it comes to enhancing or encouraging creativity?
This fear of well-poisoning has even led some outlets — including The Verge and MIT’s Technology Review — to deny their journalists access to the statistics about readers and attention, because they’re concerned that it might distort their judgement about which stories to cover or how much time to devote to them. But then how do writers know whether their work is reaching an audience?

Be careful what kind of incentives you use

Grumpy Cat
In a piece he wrote for the American Journalism Review this week, Chartbeat CEO Tony Haile (who is also an adjunct professor of journalism at Columbia) looked at both sides of the data sword. One danger of using the wrong metrics to reward your journalists, he noted — as I also tried to point out in a recent post — is that you wind up incentivizing the wrong thing, and that can take your site far away from what its original goals were:

“What we choose to measure defines what we aspire to. It is thus unsurprising that when a newsroom prioritizes a measure of link performance, that newsroom learns how to optimize links, but not content. Instead of metrics teaching the primacy of good content, we get clickbait and slideshows.”

BuzzFeed founder Jonah Peretti, who some might think would be a natural proponent of traffic-boosting statistics given his site’s focus on viral content, has also talked about the downside of this phenomenon. “If you are a slave to the numbers, then you start creating more stuff like that… and pretty soon you will have a site full of trash and salacious garbage,” the BuzzFeed CEO said in an interview with Capital New York last fall.
As Haile put it in a similar piece he wrote for Time magazine earlier this year, media companies are “getting a lot wrong about the web these days — we confuse what people have clicked on for what they’ve read. We mistake sharing for reading.” As absurd as it sounds, according to some of Chartbeat’s research, whether someone shares a link to a particular story or blog post has virtually no relationship to whether they have actually read it or not.
[tweet 430008055263408128 hide_thread=’true’]

Who is your customer — reader or advertiser?

So if publishers and media companies shouldn’t be looking at raw pageviews — which many advertisers still seem to prefer as a measurement of how effective their ads are going to be — and they shouldn’t be looking at sharing statistics because that doesn’t measure whether people actually read the content, then what should they be looking at? Not surprisingly, that question is more complicated than it seems, because there is no single answer.
Part of the problem is that most content publishers are serving two very different masters: one is the reader, and the other is the advertiser (the exception being reader-funded sites like Andrew Sullivan’s The Daily Dish or Jessica Lessin’s The Information). As Jeff Jarvis argued in a Twitter conversation with Haile — which I have Storified here, and also embedded below — if you want to measure service to the reader then you have to look at different data.
Sites like Upworthy and even Chartbeat itself, Haile points out, choose to focus on metrics that measure something approaching “engagement” — so not only whether a page was loaded but how long the reader spent, whether they were a returning visitor, and so on. Upworthy’s version of this is called “attention minutes” (which I wrote about here) and Chartbeat calls it “engaged time.”
Unfortunately, many advertisers and brands continue to look at what they call “reach,” and that encourages a focus not on engagement but on raw numbers like pageviews, as News Corp. executive Raju Narisetti noted in a recent piece for the Poynter Institute. And the larger those numbers are, the better advertisers like them — even if that encourages gaming strategies, such as buying traffic from clickfarms, etc. And that is often exactly what happens.
[tweet 456059858354466816 hide_thread=’true’]

How do you measure engagement?

Driving pageviews is not difficult even if you don’t want to buy cheap traffic from clickfarms: all you need is someone like former Gawker writer Neetzan Zimmerman, who drove more pageviews per month than all of the other writers combined by focusing exclusively on viral content. Scott DeLong, who created a site called Viral Nova to do something very similar, built a massive traffic-generating engine singlehandedly from his house in rural Ohio.
The downside -of this kind of behavior – as Jarvis points out in a recent post at Medium that is part of a series about the future of news — is that focusing on these metrics drives journalistic organizations even farther away from what should their goal, which is not just encouraging engagement but actual interaction. Haile makes a similar point in his AJR piece, and comes to a fairly grim conclusion:

“Newsrooms are mission-driven businesses that have more complex standards of success than just profit. That complexity requires a more thoughtful approach than simply Revenue = Impressions = Pageviews. Otherwise, like the oily salesperson who wrings every last dollar out of your visit but ensures he will never see you again, media companies risk destroying their ability to build a long-term business in exchange for a few additional dollars today.”

Embedded below is a Storify of the conversation that Jarvis and Haile had on Twitter about the metrics media organizations should be paying attention to:

[protected-iframe id=”80a5f2cccb6830e5b43c4fad15786aa8-14960843-8890″ info=”//storify.com/mathewi/there-s-lots-of-data-on-digital-media-but-which-da/embed?header=false” width=”100%” height=”750″]

Post and photo thumbnails courtesy of Thinkstock / Sergey Nivens