XKeyscore program indexes everyday internet activities, Snowden documents show

The Guardian has published a detailed look at XKeyscore, one of the key systems used by the NSA and its intelligence partners to suck in and search through people’s internet activities.
XKeyscore has previously been outlined in Brazil’s O Globo (in an article co-authored by Glenn Greenwald, who also wrote today’s Guardian piece) and Germany’s Der Spiegel, but the new article comes with a full look at a 2008 XKeyscore presentation leaked by Edward Snowden.

“Rolling buffer”

XKeyscore appears to take in a vast amount of data on a pretty indiscriminate basis and store it in a “rolling buffer” of around 3 days, unfiltered — this means intelligence analysts can query XKeyscore for full content retrospectively, to a certain extent. Metadata, which helps narrow down searches (we’re talking about a lot of data here) gets stored for 30 days, or at least it was back in 2008.
These days, the data is presumably sucked in through a variety of means, including the U.S. PRISM scheme (covering data going into and out of the U.S.) and the British Tempora program, which allegedly takes in more data than any other country’s surveillance operation.
According to Wednesday’s article, analysts can launch a search by filling in a simple webform, rather than needing to get a warrant. This lets them access “nearly everything a typical user does on the internet”, which chimes in nicely with what NSA whistleblowers have previously said about the agency building dossiers on everyone it can, including U.S. citizens.
XKeyscore looks at internet sessions and indexes email addresses by username and domain, files by filename and extensions, IP addresses and ports used, client-side HTTP traffic, phone numbers, and webmail and chat activity by username and contact lists, as well as relevant cookies.
Analysts can then search through this indexed data by email address, name, telephone number, IP address, keywords, browser type and so on. Disturbingly, the documents also suggest the user can target specific VPN providers, decrypt their traffic and identify their users.

Highlighting anomalies

According to the NSA presentation, XKeyscore also picks out what it reckons are anomalous events, such as someone using an unusual language for the region they’re in, or someone using encryption – this approach makes it possible to flag up suspicious individuals even where there is no email address or other strong “selector”.
So, for example, an analyst could search the system for all encrypted Word documents being sent from Iran, or all encrypted email being sent from that country. Someone could be flagged up for using German online when they’re in Pakistan.
The presentation, which we must remember dates back five years, claims more than 300 terrorists had been caught by that time using XKeyscore. The slideshow is marked for the attention of the so-called Five Eyes countries – the U.S., Britain, Australia, Canada and New Zealand – but Der Spiegel has previously said the Germans also get to use XKeyscore.