ProPublica makes it easier to see sources behind a story

ProPublica is debuting a new feature today that enables readers to view the reporter’s sources of information without leaving the story.

The “Explore sources” button enables people who are deeply interested in the topic to explore documents themselves. It holds the reporter accountable for the facts in his story. And — although it wasn’t intended to — it creates another layer of fact-checking.

The new feature is a result of collaboration between reporter Marshall Allen, who is using DocumentCloud extensively in his reporting, and news applications developer Al Shaw, who built a new tool to enable Allen to integrate all those notes into a story.

Shaw’s tool enables anyone to annotate a story, no HTML knowledge needed, and export it to a standard Web content management system.

“Al built this to be repeatable,” said Scott Klein, editor of news applications, “so we absolutely see it as something we’re going to use again.”

Primary sources in context

Today’s story, “Why can’t Linda Carswell get her husband’s heart back?,” looks like any other. The difference is that when the reader clicks a link labeled “Explore Sources,” about 50 passages in the story are highlighted.

Clicking any of those highlighted words or phrases causes a box to pop up displaying the portion of a document that substantiates the language.

The pop-ups provide a way for people to “see behind the scenes and explore the information further, if that’s something that interests them,” Allen said.

By clicking on highlighted text, readers can see the portion of a document that backs it up. If they click on the box, they can explore the document further.

In the course of his reporting, he created about 500 annotations in 64 documents in DocumentCloud; about 50 of them are embedded in the story.

“The story itself is what gets presented,” Allen said, “but there’s this whole meta-layer that goes into doing the story.”

DocumentCloud already enables users to embed an annotation from a document, similar to a block quote. And you can link to a document. The first approach interrupts the flow of the story; the second one takes users away from the story, to a separate Web page.

Neither approach seemed appropriate. “A better idea, rather than throwing it up on a separate page, would be to let them see the documents in context,” Shaw said.

Accountability increases

Though Allen said he’s a careful reporter already, this approach “made me very conscious that I better make sure I get this right. … If you’re inviting people to look at your source material, you have to realize they might do that.”

As he went through the story and added each link, he ended up fact-checking his work again.

“It’s really a great fact-checking tool,” he said. “When you annotate, you go, ‘Is that really what they said there?’ ”

He didn’t find any mistakes, but he deleted one line from the story after he realized that a line in the document “was not quite as black-and-white an answer as I thought it was when I wrote the story.”

Allen was well-suited for this experiment because he’s integrated DocumentCloud into his reporting rather than simply using it as a way to show people primary documents.

Rather than highlighting and making notes on paper, he does that work in DocumentCloud, which helps him when writing a story. When he used printouts, “I’d have piles of paper I’ve highlighted and noted, and when I go to write the story, I’m digging through reams of documents. When I can annotate electronically and have printed those annotations, I’ve already pulled key salient points from those documents.”

Taking the coding out of it

Shaw could have hand-coded all those DocumentCloud annotations into the story, Klein said. “I think the great spark Al had is, ‘This is a great idea; we’re going to want to do this again.’ ”

So Shaw spent about a week building a tool that enabled Allen to embed the notes himself without coding.

Once the story had been edited, Allen loaded his story into a Web application running on his computer. On one side of the application was his story, on the other side was list of annotations from his documents. Allen highlighted the text in his story and then clicked which annotation he wanted it to link to. When he was done, the application exported the story with the HTML tags that call up the annotations.

Allen said it took him about three hours to use Shaw’s tool.

The code generated by the tool can be inserted into pretty much any Web content-management system. And it can be edited afterward, just like you can edit the linked text in a story.

Klein said ProPublica plans to release the Web application (they’re calling it “the annotizer” until they can think of something better) for anyone to use, although that will take some time.

We have made it easy to comment on posts, however we require civility and encourage full names to that end (first initial, last name is OK). Please read our guidelines here before commenting.

  • Adam Smallman

    I so badly want to like this but I fear it’s more of the sort of journalism discourse disconnected from how mainstream audiences, whether in professional or consumer worlds, act and think. It’s more of the  ‘nice idea, no idea how this is going to help fund journalism’ ideas so beloved of The Guardian, NYT, others. My New Year resolution is to understand my audience better so I can sell them intelligence, including journalism, analysis, data. This is an interesting idea that appeals ostensibly to other journalists and is of no consequence to profitable audiences.

  • Anonymous

    This is a great, great tool and like Lynn said, I think it makes it even more obvious how well-researched and reported those stories are. One thing I wish it showed is which document the annotation comes from.  With dozens of documents used, it’d be nice to know if the part I’m looking at comes from an autopsy, a police report, court records or another document.

  • Anonymous

    If this catches on, it also could draw attention to the difference between well-researched material and the ever-expanding supply of thinly-sourced [or worse] stories & blogs masquerading as news sources on the Internet. Perhaps it would allow solid journalism to stand out — and to accrue the credibility and respect it deserves.

  • Lynn Connor

    ProPublica’s Explore Sources button is a big deal in journalism, I’m looking forward to seeing it in action. 
    I think it will amaze me when I see the number of annotations. The skill of the writer hides the large amount of research a journalist does when assembling a story, don’t you think? It’s like what they say about professional athletes, “He makes it look easy!” 

  • Anonymous

    Mitt Romney condemns The Church of Jesus Christ of Latter-day Saints for
    its founding of polygamy and its current practice by his church. The 
    Republican wanna-be presidential candidate’s great-grandfather had five
    wives and at least one of his great-great-grandfathers had 12.

    Polygamy was not just a historical footnote but a
    prominent element in the family tree of

    “little Willy” as his grandmother/sister used to call him.

    Romney’s grandfather, Miles Park Romney, married
    his fifth wife in 1907. That was more than sixty years after LDS church
    leaders banned polygamy and more than three decades after a federal law
    barred the practice.

    Miles, former Michigan Gov. George Romney’s father was not a son of the first married, and only legal wife,
    meaning that by US law he was officially an illegitimate bastard. That
    would make Mitt the son of a bastard, no wonder he is ashamed of his
    first name. 

    Romney’s father, George , was
    born in Chihuahua, Mexico, where church members fled in the 1800s to
    escape being sent to prison and being held responsible for there
    actions, at least Warren Jeffs was man enough to stay around and take
    his punishment.