Factually is a newsletter about fact-checking and misinformation from Poynter’s International Fact-Checking Network. Sign up here to receive it on your email every Thursday.

Less than half of Birdwatch users include sources, and many reveal partisan bent

In January, Twitter announced Birdwatch, its experimental crowdsourced fact-checking platform as a way to fight mis/disinformation. The project allows users to mark tweets as misleading and add context within “notes.” It also lets other users rate those notes based on helpfulness and sourcing.

For the last week, I’ve analyzed more than 2,600 notes made by Birdwatchers and reviewed 8,200 ratings pushed by the social media platform based on that information. And I have tested out a public algorithm the company uses to rank notes based on their “helpfulness.”

The results so far aren’t encouraging, as I found blatant misinformation receiving “not misleading” notes, context that reveals political bias and a small number of voices — with dubious Twitter feeds of their own — dominating Birdwatch activity.

I’m skeptical of farming out fact-checking to every Twitter user and an algorithm, as no automated system can compete with human fact-checkers in the search for truth. But, if Twitter is committed to improving its tool in order to fight against mis/disinformation, here are a few steps the social media platform should take:

About sources:

  • Twitter should require Birdwatch users to cite at least one source within their notes. More than half of the content I reviewed in the program did not include a single source.
  • Rely on human fact-checkers to vet Birdwatch notes. Wikipedia, another crowdsourced portal, was one of the most-cited sources on the program. Fact-checkers do not rely on Wikipedia as a reliable source.
  • Promote notes that offer fact-checked articles. URLs from professional fact-checking organizations were rare among the notes I analyzed, even though some of the topics had been largely covered by fact-checkers.

About the algorithm:

  • Stop promoting Birdwatch notes that carry misinformation or lack sources.
  • Review the vetting process so that prolific users who have shared misinformation on their own timelines are not allowed to become Birdwatch users.

About language and profile:

  • Limit the number of notes per Birdwatch user. The five most active Birdwatchers account for more than 10% of Birdwatch’s notes.
  • Employ natural language processing to weed out bias in notes. The most prolific Birdwatch users use common partisan language.

About the community:

  • Onboard Birdwatch users with basic media literacy training. Research from the Stanford Social Media Lab has shown that an hour-long online course increased participants’ ability to identify false news headlines by more than 20 percentage points.
  • Offer regular fact-checking lessons to users in the form of an email newsletter, or even regular direct messages from the Birdwatch account.

I am aware that this is just a pilot program, which will likely change considerably before it is rolled out in the rest of the U.S. and beyond. But the problems I found echo what the fact-checking community feared when Birdwatch was announced.

Still, Twitter’s transparency and commitment to improve the tool as data comes in gives me hope that we could — one day — see a somewhat successful model.

Alex Mahadevan
Senior multimedia reporter, MediaWise

Interesting fact-checks

Courtesy of Faktograf.hr

  • Faktograf: “Red Bull didn’t test positive for COVID-19”
    • Croatian fact-checkers debunked a video that supposedly shows a man trying to test an energy drink for COVID-19. In December, the same team flagged an Austrian politician attempting the same test with Coca-Cola. In both cases, the tests were incorrectly handled and their results can’t be considered valid.

Quick hits

From the news:

  • “The Value of News on Facebook,” from Facebook. The social media giant announced Thursday it would “restrict the availability of news on Facebook in Australia,” in response to a proposed law that would make tech companies pay news publishers for their content. This comes on the same day Google announced a deal with News Corp. to begin paying the media giant to display its news content.
    • RMIT ABC Fact Check, one of IFCN’s verified signatories in Australia, is no longer available on Facebook. The staff is using Twitter to direct followers to its app.
    • The Bureau of Meteorology’s page was temporarily blocked from posting on Facebook, but had its access restored just after midday local time.
    • According to The Verge, the Department of Fire and Emergency Services Western Australia, and Queensland Health had no content available on their Facebook pages either.
    • And the move drew the ire of journalists in the Philippines. Rappler’s CEO, Maria Ressa, criticized Facebook on Twitter, saying this ban will have an “impact on facts and democracy.” Rappler’s page has been taken down too.

From/for the community:

  • The Washington Post launched “#DIYFactCheck,” a series of tools to verify videos and debunk misinformation. Available via the Post’s Instagram, the guide takes readers step-by-step as it explores key questions such as how to find the original video, who posted the video, and where and when the video was filmed. It builds on the paper’s 2019 infographic — “The Fact Checker’s Guide to Manipulated Video”.
  • Institutions and governments, content creators, journalists, teachers and students in Latin America should take a look at PortalCheck. The website, launched by UNESCO, Chequeado and LatamChequea, with the European Union’s support, offers not only useful resources to fight mis/disinformation but also a list of events and activities taking place in the region. PortalCheck is available in English, Spanish and Portuguese.
  • Here’s an idea: In an attempt to promote vaccination all across India, NewsMobile has recorded and posted videos on social media featuring first-person accounts of Indians who have received the COVID-19 vaccine. Many reacted positively to the vaccination saying they felt great and had no unexpected reactions. Recordings took place in Delhi, Patna, Bhopal and other cities.

Events and training

  • Feb. 18 (Today): “Fact-check images using your cell phone.” Offered in Portuguese, by Agência Lupa, in Brazil. In this 90-minute virtual course, participants will learn how useful their mobiles can be in the battle against visual disinformation.

If you are a fact-checker and you’d like your work/projects/achievements highlighted in the next edition, send us an email at factually@poynter.org by next Tuesday.

Any corrections? Tips? We’d love to hear from you: factually@poynter.org.

Thanks for reading Factually, and a special thank you to Alex for joining us this week!

Cris, Harrison, and Alex

Support high-integrity, independent journalism that serves democracy. Make a gift to Poynter today. The Poynter Institute is a nonpartisan, nonprofit organization, and your gift helps us make good journalism better.
Alex Mahadevan is director of MediaWise at the Poynter Institute. He has taught digital media literacy to thousands of middle and high schoolers, and has…
Alex Mahadevan
Cristina Tardáguila is the International Fact-Checking Network’s Associate Director. She was born in May 1980, in Brazil, and has lived in Rio de Janeiro for…
Cristina Tardáguila
Harrison Mantas is a reporter for the International Fact-Checking Network covering the wide world of misinformation. He previously worked in Arizona and Washington D.C. for…
Harrison Mantas

More News

Back to News


Comments are closed.

  • **Limit the number of notes per Birdwatch user. The five most active Birdwatchers account for more than 10% of Birdwatch’s notes.**

    It’s currently a pilot program with a limited number of participants. It’s completely arbitrary to cap the number of notes based on a percentage. Who is to say whether those note-writers can sustain that pace over time? Instead, emphasize quality. Limit notes based on quality assessments (bad note writers get a restricted number, perhaps as low as zero).