Looking at metrics? Here are 55 questions to consider first

We have so many ways to measure journalism. Melody Kramer wrote about 50 of them here Tuesday. She could have listed another 50 and still had plenty to include.

However, most journalism begins not with a number but with a question: What do you want to know?

Let’s take a step back for a moment. I believe that the most fundamental two questions, the ones everyone in your newsroom need to be able to answer without thought, don’t have numerical answers.

They are:

• What is our mission?
• How can we fulfill it more effectively?

I’ve worked in newsrooms where everyone has clear answers to both questions. I’ve worked in newsrooms where nobody could answer either. In my experience, the second type were the newsrooms that more slavishly pushed targets around pageviews and unique visitors, because they brought a sense of certainty and purpose where otherwise there was none.

However, once you have answers to the first questions, more questions arise:

• How well are you fulfilling that mission?
• How effective are your decisions?
• How true are your assumptions about how to fulfill your mission more effectively?

To answer these, you need to break them down into more granular questions, the answers to which can be measured using metrics such as those Melody Kramer listed. With, of course, an important caveat.

While numbers present precision, they lack real meaning.

Is a high number of unique visitors on an article a sign that people love the piece, or that a lot of people are hate-reading it and making fun of you? Does the difference matter to you or to your boss? Should it? Did that pageview spike happen because you had a particularly great piece, or because the subject has 3 million Facebook followers? Do you have a lot of comments on that video because there’s a healthy discussion happening, or is it an abuse-filled pile-on around the presenter’s race, gender or physicality?

Questions lead to questions lead to questions. That’s journalism. Let’s not forget to apply it to ourselves.

Here’s 55 questions you might want to try and answer through careful acts of measurement. Each one probably requires multiple metrics applied in combination, and some of them can’t be measured through passive analytics tools.

Journalism

1. Does our coverage connect to our journalistic mission?
2. How much does our coverage connect to a real problem our readers have?
3. Did our coverage change someone’s attitude?
4. Did our coverage change someone’s life?
5. Did our coverage change the actions of someone in authority?
6. Did our coverage move forward thinking about this topic in the field?
7. Did our coverage lead to a law being changed?
8. Did our coverage make things better for people affected?
9. Is our coverage clearly connected to our other work?
10. Is our coverage clearly connected to our brand?
11. Is our coverage easy to find, both outside our site and on it?
12. How does this piece compare with other pieces by this author?
13. Did another publication follow up on our coverage?

Readers

1. Why are people reading this piece?
2. Is our coverage reaching the readers I want to reach?
3. Is our coverage reaching as many readers as I expected it to?
4. How did readers find this piece?
5. Where did they go next?
6. Were the readers of this piece regular visitors to the site?
7. Did the readers feel informed at the end of this piece?
8. Do the readers of this piece trust us as an institution?
9. Is anyone who saw this now more engaged in, involved in or connected to our journalism?
10. Did anyone start to take an action on the page, then abandon it part-way through?
11. How many readers didn’t stay on the page long enough to read/watch/listen to the story?
12. Did any readers come back to this article and share it later?

Money

1. How can I attract more advertisers?
2. Did this encourage people to subscribe, donate or pay money?
3. Did readers see the ads?
4. Did readers click on or interact with the ads?
5. Did the ads load correctly?
6. Did any readers block the ads?
7. How much did this piece cost to create?

Community

1. Who is talking about this?
2. Where are they talking about it?
3. Did this create positive conversation?
4. Did this create negative conversation?
5. Did the people who I expected to engage on this topic actually do so? If so, where?
6. How long did it take for readers to write their contributions to the community?
7. How long did people read the community’s contributions before leaving their own?
8. Did the audience try to bring anything useful to the attention of the journalist?
9. Are journalists engaging with the audience?
10. Are our community tools making people feel safe?
11. Are our community tools making people feel respected?
12. Are our community tools making people feel listened to?
13. Do our community tools encourage positive interactions?
14. Did anyone return to the conversation at a later time?

Technology

1. What device are our readers using?
2. Did the page load quickly?
3. Did the page look good on their device?
4. Did a recent change in our technology have a positive or negative impact on any of the above areas?
5. Did they push the content to another reading place, like Pocket or Instapaper?
6. How much data are we gathering on our users?
7. What is the minimum amount of data we need to gather, and are we collecting more than we need?
8. How much data on our users are we sharing with third parties?
9. Do readers trust our technology?

What did I miss? Please add your answers — and questions — below.

Don’t overlook the excellent work in newsroom metrics and analytics that is happening right now, including Carebot, NewsLynx, and Impact Tracker.

Editor’s note: Andrew Losowsky is Project Lead of The Coral Project.

See also: Analytics 101: Understanding Digital Metrics