Here’s what you should consider before using a fact-check rating system

June 8, 2017

One of the hallmarks of fact-checking is to demonstrate whether a claim has been corroborated by facts. Many fact-checking organizations use a rating system when they offer a verdict for a claim they have researched, according to a survey by the Duke Reporters’ Lab.

Here are some pros and cons to implementing a rating system, such as the Pinocchio scale used by The Washington Post’s Fact Checker.


  • Using a rating makes the article clearer to readers. Readers are bombarded with information and will appreciate being able to get directly to the point.
  • Having to assign a rating forces the reporter to reach a conclusion. Sometimes the fact-checker will struggle to adjudicate the veracity of a claim. But that is precisely why readers appreciate this form of journalism.
  • Using a scale helps build your brand and audience. Usually the most recognizable element of a fact-checking operation is its rating system. The rating also tends to travel well on social media.
  • A rating offers readers something light and engaging to go with the substantive, often intense, analysis.


  • Some fact-checks will not perfectly fit any of the ratings. Two falsehoods may be very different from one another in terms of importance, yet get the same rating.
  • The difference between what is “almost true” and what is “half true” isn’t science, even if each fact-checker at the organization tries to follow the methodology closely. Fact-checkers might find it hard to maintain consistency over hundreds of fact-checks.
  • Using a rating adds significant work to the editorial process. It is definitely simpler to publish findings without determining a rating.
  • Giving ratings is a more confrontational approach and will result in more criticism from readers and fact-checked politicians than simply discussing the facts and figures. The rating may distract critics from the analysis itself.

Taken from Fact-checking: How to Improve Your Skills in Accountability Journalism, a self-directed course by Alexios Mantzarlis and Jane Elizabeth at Poynter NewsU.

Take the full course

Have you missed a Coffee Break Course? Here’s our complete lineup. Or follow along at #coffeebreakcourse.