Truthsquad Shows Potential of Crowdsourced Fact-Checking

NewsTrust, a non-profit startup aimed at improving the credibility of the media, says that its week-long “Truthsquad” experiment — in which it tried to crowdsource the fact-checking of public statements made by politicians, special-interest groups and others — was successful enough that it plans to continue with the project, if it can find additional funding. The pilot, which saw input from individual users that was then checked by a panel of guest judges, was a joint venture with the Poynter Institute and was funded in part by the Omidyar Network and the MacArthur Foundation.

Fabrice Florin, a former executive at Macromedia who started NewsTrust in 2005, said that the response to Truthsquad — which lasted from August 2 to August 8 — was even better than the company had anticipated, based on other short-term projects launched by the startup. “Over three thousand visitors checked Truthsquad and about three hundred participants helped fact-check a dozen quotes this week,” Florin said in an email, adding that the project got a total of more than 500 responses, despite a minimum of publicity for the campaign. Although that doesn’t make Truthsquad a mainstream venture by any means, it does show that appealing to readers to fact-check quotes has potential.

The idea behind the experiment was a simple one, Florin says: to take a quote made by a public figure about an important issue and allow users to vote on whether they were right or not, as well as give their reasons for reaching that conclusion. One example was a quote from Fox News host Bill O’Reilly, a fairly polarizing figure on the political front, who said that “crime is out of control in Phoenix” as a result of illegal immigration. Another quote that Truthsquad checked was a statement by U.S. Senator Orrin Hatch that “87 million Americans will be forced out of their coverage” by new health-care legislation.

On each quote page, users can vote in support of or against the opinion in the quote, and give their reasons, as well as links to news stories or other sources of information that support their position. The right-hand column on each page contains a ranking of how many links support or oppose the statement, and after enough input, the Truthsquad editors and guest judges (whom the site has not named) rule on whether the opinion expressed in the quote is true or false. In the case of the O’Reilly comment about Phoenix, for example, the editors ruled that it was false and that “the only sources that corroborated O’Reilly’s statement seemed clearly biased,” while more trustworthy sources such as the police department and the FBI “indicated that crime in Phoenix has been dropping steadily for years.”

One thing that troubled me when I heard about Truthsquad was the potential for it to descend into the same kind of flame-war and troll-baiting commentary that so many news and opinion sites from both sides of the political spectrum often suffer. But Florin says that the company was pleasantly surprised that most comments “were generally civil, and that participants seemed genuinely engaged in this communal quest for credible information.” Based on the favorable response, the NewsTrust founder said that the company is “prepared to offer Truthsquad on an ongoing basis, and are now seeking donations and foundation support to provide it as a regular feature.”

NewsTrust has spent the past few years trying to “crowdsource” the idea of reputation for media sources, with a trust-rank for different writers and outlets that readers can vote on. It also offers a customized “news for you” kind of view that acts as a news filter or customized feed reader. The company is funded primarily by grants from the MacArthur Foundation, as well as support from the Sunlight Foundation and the Mitch Kapor Foundation, and the NewsTrust advisory board includes Craigslist founder Craig Newmark and digital-media guru Dan Gillmor.

Florin said that the impulse behind both NewsTrust and Truthsquad was the idea that “if we are to become our own editors in the digital age, we need better tools and training to thoughtfully evaluate the credibility of the information we find online.” Embedded below is a video presentation that the NewsTrust founder did at the 2009 News Literacy Conference.

Related content from GigaOM Pro (sub req’d): What We Can Learn From the Guardian’s Open Platform