September 20, 2018

Measuring Facebook’s fact-checking effort

Many have tried to figure out how exactly Facebook’s effort to cut down on misinformation is going.

BuzzFeed News does an annual roundup of the top-performing hoaxes on the social network. Newswhip regularly measures how well fake news stories perform on Facebook. And other publishers have gotten more anecdotal evidence on how the company’s efforts are going.

Last week, those ad hoc measurements got some academic backup (which Slate’s Will Oremus covers well here.)

In a working paper, researchers at Stanford University, New York University and Microsoft Research analyzed 570 sites between January 2015 and July 2018 that were identified by at least one credible source as being consistent purveyors of false news. They found that interactions with those sites rose on both Facebook and Twitter during 2016, but fell sharply for the former in the beginning of 2017 while the latter’s continued to rise. They found no similar patterns for other news sites.

At least one reason for the fall in interactions with fake news sites could be Facebook’s partnership with independent fact-checking projects, which debunk false news stories, images and videos on the platform, decreasing their reach in News Feed. The project launched in December 2016 (and is partly facilitated by the IFCN).

The partnership isn’t perfect. Daniel found in July that many stories Facebook’s fact-checking partners rate as false aren’t flagged in the platform’s system.

Facebook has worked to improve its flagging mechanism — which fact-checkers told the company was hard to use — by incorporating the ClaimReview markup, which Facebook announced it was incorporating at the Global Fact-Checking Summit in June. That should cut down on the amount of time it takes to get a fact check into Facebook’s system.

Beyond Facebook, the study’s results also highlight the need for a more robust approach to misinformation on Twitter. Daniel pointed out in August that the platform doesn’t have a similar initiative like Facebook’s fact-checking product — in spite of fact-checkers repeatedly asking for one. It’s taken smaller steps to cut down on misinformation, such as deleting bot accounts, but the working paper suggests that they don't seem to be working.

The study’s authors concede that their findings are “far from definitive.” As Alexios noted, the definition of "fake news" at the domain level — and which domains made the cut — makes a big difference on the results. The researchers also concentrated only on the United States.

With these caveats, the study provides helpful additional information to what journalists have spent months trying to figure out.


This is how we do it

This is bad


This is fun

Coming up


A closer look

  • The New York Times took a look inside Facebook’s election War Room, where dashboards show spikes in the circulation of false stories or the creation of automated accounts in specific locations.
  • At the Online News Association conference, danah boyd of Data & Society gave a keynote about how the media can avoid amplifying conspiratorial messages. Read a summary of her remarks on Medium.
  • Agência Lupa boss Cris Tardáguila co-authored "Você foi enganado," a book fact-checking claims by Brazilian presidents all the way back to 1921. Political lying has a long history, turns out.

12 quick fact-checking links

  1. Three members of Congress have asked the intelligence community to assess the potential security threat posted by deepfake videos.
  2. Israeli Prime Minister Benjamin Netanyahu is being sued for sharing a fake news story about George Soros.
  3. Stop. With. The. Post-Truth. Headlines.
  4. Still relevant, from our archives: Journalism can’t afford for corrections to be the next victim of the “fake news” frenzy.
  5. A modest proposal: Maybe Sweden should run Twitter.
  6. Do you know the difference between a goatee and a Van Dyke? If not, you’re in good company.
  7. Sometimes saying there’s no evidence is as important as saying what evidence there is.
  8. Polygraphs are in the news this week. Evidence suggests they aren’t actually any good at detecting lies.
  9. Google is still featuring results from dubious sources. This time, on celebrities’ wealth.
  10. Donald Trump Jr. tweeted a photo of CNN anchor Anderson Cooper in waist-high hurricane floodwaters, suggesting he faked the scene to make it look worse than it was. But Cooper debunked that on his show — the photo, from Hurricane Ike, was taken while he showed viewers how deep the water could get.
  11. A fact-checking training event in Kazakhstan was interrupted when police arrested a journalist visiting from Ukraine.
  12. The New York Times issued an editor’s note after a story about expensive curtains at the home of the U.S. ambassador to the United Nations. It focused too much on current ambassador Nikki Haley, even though the curtains were ordered under the Obama administration.

Until next week,

Daniel and Alexios

Clarification: This article has been updated to reflect that Donald Trump Jr. did not explicitly say that Anderson Cooper faked a Hurricane Florence live shot — he suggested it by tweeting during the storm.

Support high-integrity, independent journalism that serves democracy. Make a gift to Poynter today. The Poynter Institute is a nonpartisan, nonprofit organization, and your gift helps us make good journalism better.
Daniel Funke is a staff writer covering online misinformation for PolitiFact. He previously reported for Poynter as a fact-checking reporter and a Google News Lab…
Daniel Funke

More News

Back to News