Carebot, a forthcoming tool from NPR, aims to help journalists figure out how much their audiences care about their work. Pictured above is a proposed day in the life of a journalist working with Carebot to think about his/her work. (Photo credit: NPR)
(Photo credit: Brian Boyer, NPR)
For years, journalists have railed against pageviews and uniques, twin metrics that have established supremacy over much of the Internet. Their argument, all tidied up, goes like this: Pageviews and uniques tend to emphasize frivolous content (think cat videos) at the expense of time-intensive journalism. The metrics can be gamed by publishers with the money and inclination to buy traffic. And a lack of standardization throughout the industry has led to confusion among advertisers and outlets about the value of a click.
In lieu of pageviews, several organizations have called for a better standard for measuring audience attention. Early last year, Upworthy announced the debut of “Attention Minutes,” a metric that emphasizes the time users spend on sites over raw pageviews. Tony Haile, the CEO of Web analytics company Chartbeat, last year hailed the dawn of the Attention Web in an article for TIME, claiming advertisers would begin to adopt increasingly specific metrics for monitoring readers. In 2013, product scientist Pete Davies noted that Medium was also shifting to a metric that values time readers spent on site over whether they clicked on an article or not.
That camp gained another ally on Tuesday when the John S. and James L. Knight Foundation gave NPR $35,000 to build Carebot, a tool that aims to go beyond pageviews and uniques to determine how valuable stories are to readers. The stated mission of the project, spearheaded by NPR Visuals Editor Brian Boyer, is to devise metrics that change how newsrooms measure and celebrate successful stories.
“Our official team motto these days is: We make people care,” Boyer said. “We believe that visual journalism has the power to make people care. So if that’s our motto, if that’s what we tell ourselves, if that’s what we’re trying to do every day, then how do we know if we’re doing our jobs? How do we measure that success?”
The answer, according to Boyer’s team, lies with the kinds of numbers that news organizations choose to count and how they choose to calculate them. Rather than adding up the number of pageviews and uniques, Boyer’s team is building a tool that will aggregate data from a variety of sources, including Chartbeat, Google Analytics and social networks like Facebook and Twitter. Carebot will crunch those numbers following a formula devised by Boyer’s team to evaluate how much readers cared about any given story.
Although Carebot hasn’t been built yet, Boyer says that the tool will emphasize a series of metrics that include social engagement (likes, shares, comments) time spent on site and completion rate. Rather than examining raw numbers for each statistic, Carebot will base its scores on the number of engagements per pageview. By this reckoning, a story with 1,000,000 pageviews and 1000 shares would have a lower Carebot score than a story with 1,000 pageviews and 100 shares. The goal, Boyer says, is to come up with numbers that reflect how much readers love various stories produced by NPR.
“Since we can’t put our readers on an MRI machine and actually watch the happy parts of their brain tick off when they’re reading a piece, we have to approximate.”
NPR, which will likely build Carebot into a website, plans to open-source the program for use by other news organizations. Money from the Knight Foundation will be spent on extending the employment of Livia Labate, a Knight-Mozilla Fellow at NPR who wrote the grant application.
Taking user satisfaction as the primary measure for a news organization’s success might seem fanciful coming from NPR, which does not rely on advertising for the lion’s share of its revenue. But Boyer thinks metrics that focus on quality user experience will de-emphasize clutter and help decrease the number of websites filled with ugly display ads.
“I certainly don’t think the metrics we need are there yet,” Boyer said. “So it wouldn’t be a stretch to say that we’re trying to come up with the metrics that we need.”
