Why the “tweets per minute” statistic isn’t all it’s cracked up to be

Last week, the internet became captivated by a phenomena called Sharknado. If you’d asked me whether Sharknado was one of the most-tweeted events of the year so far, based on my own feed last week, I might have reasonably guessed that it ranked decently high — it seemed to comprise, even conservatively, one in 10 tweets in my feed on Thursday night. I couldn’t get away from Sharknado.
But as The Atlantic notes, Sharknado didn’t even do that well in the actual ratings (drawing almost the same number of viewers as a similar show that had no Twitter hype in March). As Twitter itself noted in a blog post, a large percentage of the tweets came from a a few users. So yes, Sharknado filled my Twitter feed, but did that mean anything in terms of the show’s overall popularity and viewership? Maybe not.
Over the years, Twitter has morphed into an incredible platform where people can make conversations about something like Sharknado viral in a matter of hours. But it can be a misleading gauge of popularity, particularly the tweets per minute statistic that the company likes to use as a way to measure consumer interest and engagement with major events.
We tend to conflate Twitter popularity with general public opinion. Within the context of using tweets to measure the American public’s attitude toward the 2012 election, my colleague Derrick Harris wrote how Twitter data is certainly interesting, as it provides us with some information we couldn’t get before, and can help us measure reactions from people within certain age groups or demographics. But in general, he wrote that you’d be crazy to take Twitter data alone as the sole factor in determining how the entire general public will sway on all issues. A few of the reasons, which the Wall Street Journal outlined as well:

  • Twitter users tend to be younger and more tech-savvy than the average population.
  • Twitter can’t always connect individual tweets to specific events unless people use the right keywords.
  • Some people tweeting about an event (like a presidential debate) might not actually be paying attention if they’re busy tweeting (and similarly, avid politicos might not be doing any tweeting at all).

Even if all Twitter users were representative of the general public, and we could use sentiment on Twitter to accurately predict how the entire populace feels, it still wouldn’t be correct to use the tweets per minute statistic as often as we do.
On Sept. 8, 2011, Twitter had 100 million monthly active users, and just slightly more than a year later in December 2012, the company announced that it had doubled that number, to more than 200 million monthly active users. Of course, we don’t know exactly what a monthly active user means (especially if 40 percent of users don’t even tweet). But if the service doubled its userbase in just one year, that rate of growth makes it challenging to compare the numbers for two events unless they occur just a few weeks apart — certainly not from one election to another, four years later.
But the company continues to use the tweets per minute statistic, because it validates Twitter as a large platform where people are talking about live events — exactly the platform advertisers want to reach. Tweets per minute might be great for measuring intense spikes around a television show or a political event or a sports event (which are perfectly suited for purchasing promoted tweets on Twitter). But when it comes to measuring slow-moving sentiment on something like health care regulation, or airline complaints — even if that’s a really common use for the service — tweets per minute just doesn’t cut it.