Meet Twitter’s New Real-time Architecture

Twitter rolled out User Streams to its users for the first time this week, through limited beta offerings by third-party applications TweetDeck and Echofon for Mac. This new, streaming API has been long in the making for Twitter, but may not reach regular users until the end of the year. It’s a significant architectural change for the company that should make Twitter much faster and more reliable. TweetDeck’s Richard Barley called the upgrade “Twitter at the speed of ‘Wow!’”

Twitter clients currently poll Twitter’s REST API as often as they’re allowed to — 350 times per hour is the standard, though it now drops during periods of heavy usage. These rate limits will go away with User Streams. The streaming API enables each user’s Twitter desktop client to establish a single persistent connection to a Twitter server that pulls down new information as soon as it appears. The connection lasts until the user shuts down the client.

This addresses the way many people use Twitter — as a real-time communication and information tool. In an interview at Twitter’s office with head of product Jason Goldman, Ryan Sarver, and Raffi Krikorian I conducted for  our premium research service, GigaOM Pro, Goldman compared User Streams to a Bloomberg terminal.

Twitter will continue to use its REST and search APIs for many functions, but the company is also adding new functionality to user streams, such as real-time alerts for new follows and items favorited within a user’s network.

Sarver said user streams are expected to “have a measurable impact on reducing infrastructure demands and costs” because they will eliminate constant and unnecessary polling. Even though the streaming connection is always open, it’s event-driven, only transmitting information where there’s something new. However, Sarver admitted it’s possible that streaming could end up generating more demand than REST.

Twitter didn’t invent the concept of a persistent connection to a server; it’s commonly used for communications technologies. Chat clients like Meebo (see disclosure), for instance, make use of synchronous server connections in the browser, while enterprise mail programs like Exchange use them to deliver messages on mobile. Though it might seem unwise to take infrastructure advice from Twitter, the fact is, the service has experienced unprecedented growth and usage. The company built its service around an architecture that was designed to perform a set function without the capacity to scale. Now the company’s engineering team has to retain (and improve) that capacity and still scale. It’s a lesson other web startups should be so lucky to need to learn.

For more detail and analysis, read the GigaOM Pro piece (sub req’d).

Photo courtesy Flickr user The.Rohit.

DisclosureMeebo is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.