Journalists need to bring science and forecasting into their news coverage, despite the fact that predicting outcomes is viewed as “uncouth” in some newsrooms, statistician Nate Silver told a gathering at the Online News Association conference Friday.
Silver has achieved rock-star status in the world of data journalism thanks to correctly calling the 2012 presidential winner in all 50 states, not to mention his smart analysis of baseball and a best-selling book about why predictions fail.
A critic of stories that use numbers in inaccurate or inexact ways, Silver offered eight points that journalists should know if they use statistics:
1. Statistics aren’t just numbers. Silver’s message is that statistics have specific meanings that require critical thinking on the part of the journalists who use them.
2. Data require context. Silver cited an essay in The New York Times about the difficulty in raising children in polluted Beijing. The author noted China’s place as the world’s second-largest economy, but Silver said that given China’s population of 1.2 billion, a better statistic is its economic place in the world on a per capita basis, a measure by which it ranks 93rd globally. That makes China a poor country dealing with environmental issues, not a rich one.
3. Correlation is not causation. Articles that suggest a hot climate leads to juvenile delinquency fall into this trap, Silver suggested. Similarly, an assertion in Politico that the recent government shutdown boosted Democrat Terry McAuliffe’s chances in the Virginia gubernatorial race isn’t supported by evidence, he said. (In a post Friday, Politico said in fact its story “stated a correlation but not a confirmed causation” between the unpopular shutdown and McAuliffe’s lead.)
4. Take the average, stupid. Recent stories that reported Oreos are as addictive as cocaine failed to reflect the subtleties of the research that prompted the articles, Silver said. Doing the work isn’t necessarily difficult: Silver’s blog, FiveThirtyEight, uses a simple count of polls and averages in some of its analyses.
5. Intuition is a poor judge of probability. Eighty percent probability is 80 percent, not 100 percent, Silver noted, adding that people who deal with sports statistics or play poker tend to better understand the idea of probability.
6. Know your priors, (A.K.A, as assumptions). Journalists need to be open to testing their prior beliefs, Silver said — and if new evidence shows they were wrong, they need to be honest about that.
7. Insiderism is the enemy of objectivity. Insider information may not be reliable. A journalist whose circle is too tight may forget there is more outside of it. Silver cited forecasts made on the McLaughlin Group that he called as accurate as “monkeys throwing poop at a dartboard.”
8. Making predictions improves accountability. Silver called on journalists to be more empirical in their coverage. But he also said if journalists state what will happen in the future, they need to think such predictions through and not treat forecasting as a game.
Silver is moving FiveThirtyEight from the Times to ESPN, with plans to relaunch in early 2014. Asked if he had any concerns given ESPN’s pullout from a PBS project on NFL concussions, Silver said the network takes its journalism very seriously and he expects to have considerable editorial freedom.
Comments