ProPublica is debuting a new feature today that enables readers to view the reporter's sources of information without leaving the story.

The “Explore sources” button enables people who are deeply interested in the topic to explore documents themselves. It holds the reporter accountable for the facts in his story. And -- although it wasn't intended to -- it creates another layer of fact-checking.

The new feature is a result of collaboration between reporter Marshall Allen, who is using DocumentCloud extensively in his reporting, and news applications developer Al Shaw, who built a new tool to enable Allen to integrate all those notes into a story.

Shaw's tool enables anyone to annotate a story, no HTML knowledge needed, and export it to a standard Web content management system.

“Al built this to be repeatable,” said Scott Klein, editor of news applications, “so we absolutely see it as something we're going to use again.”

Primary sources in context

Today's story, “Why can't Linda Carswell get her husband's heart back?,” looks like any other. The difference is that when the reader clicks a link labeled “Explore Sources,” about 50 passages in the story are highlighted.

Clicking any of those highlighted words or phrases causes a box to pop up displaying the portion of a document that substantiates the language.

The pop-ups provide a way for people to “see behind the scenes and explore the information further, if that's something that interests them,” Allen said.

By clicking on highlighted text, readers can see the portion of a document that backs it up. If they click on the box, they can explore the document further.

In the course of his reporting, he created about 500 annotations in 64 documents in DocumentCloud; about 50 of them are embedded in the story.

“The story itself is what gets presented,” Allen said, “but there's this whole meta-layer that goes into doing the story.”

DocumentCloud already enables users to embed an annotation from a document, similar to a block quote. And you can link to a document. The first approach interrupts the flow of the story; the second one takes users away from the story, to a separate Web page.

Neither approach seemed appropriate. “A better idea, rather than throwing it up on a separate page, would be to let them see the documents in context,” Shaw said.

Accountability increases

Though Allen said he's a careful reporter already, this approach “made me very conscious that I better make sure I get this right. … If you're inviting people to look at your source material, you have to realize they might do that.”

As he went through the story and added each link, he ended up fact-checking his work again.

“It's really a great fact-checking tool,” he said. “When you annotate, you go, 'Is that really what they said there?' ”

He didn't find any mistakes, but he deleted one line from the story after he realized that a line in the document “was not quite as black-and-white an answer as I thought it was when I wrote the story.”

Allen was well-suited for this experiment because he's integrated DocumentCloud into his reporting rather than simply using it as a way to show people primary documents.

Rather than highlighting and making notes on paper, he does that work in DocumentCloud, which helps him when writing a story. When he used printouts, “I'd have piles of paper I've highlighted and noted, and when I go to write the story, I'm digging through reams of documents. When I can annotate electronically and have printed those annotations, I've already pulled key salient points from those documents.”

Taking the coding out of it

Shaw could have hand-coded all those DocumentCloud annotations into the story, Klein said. “I think the great spark Al had is, 'This is a great idea; we're going to want to do this again.' "

So Shaw spent about a week building a tool that enabled Allen to embed the notes himself without coding.

Once the story had been edited, Allen loaded his story into a Web application running on his computer. On one side of the application was his story, on the other side was list of annotations from his documents. Allen highlighted the text in his story and then clicked which annotation he wanted it to link to. When he was done, the application exported the story with the HTML tags that call up the annotations.

Allen said it took him about three hours to use Shaw's tool.

The code generated by the tool can be inserted into pretty much any Web content-management system. And it can be edited afterward, just like you can edit the linked text in a story.

Klein said ProPublica plans to release the Web application (they're calling it “the annotizer” until they can think of something better) for anyone to use, although that will take some time.