Think about all the information in government records, everything from your town's restaurant inspections to congressional earmarks. Think about all the things you need to know to access them: The law. What records exist and who keeps them. Perhaps SQL.

You may want to add psychology, persuasion and government culture to that list, according to a new book written by two public records experts. Rather than listing public records laws or detailing what agencies keep which records, "The Art of Access: Strategies for Obtaining Public Records" describes ways to get what you want, even if you're denied at first and can't afford to go to court.

"You can't bash down the door with a ram. You have to work your way through the maze," David Cuillier, one of the authors, told me in an e-mail. Cuillier, a professor at University of Arizona and the Chairman of the Society of Professional Journalists' national Freedom of Information Committee, worked on the book with Charles Davis, executive director of the National Freedom of Information Coalition.

To coincide with last week's publication of the book, the two have started blogging about these issues on a companion blog called The Art of Access. For instance, in one post Cuillier noted that you can learn a lot about what government agencies value in FOI officers by reading their job postings. (They include reducing FOIA backlog, understanding privacy laws and knowledge of exemptions to FOIA.)

Cuillier and I talked about his book Wednesday, and then corresponded by e-mail. In the edited exchange below, he told me:

  • How often public records requests are denied
  • How you can persuade officials to change their minds
  • Whether you'll have more luck playing nice or playing lawyer
  • What to do about gigantic bills to retrieve information from databases

Steve Myers: I've seen other books about public records. What's different about yours?

David Cuillier: What we tried to do that is different, I think, is put together in one place an explanation of the complete process for getting public records, with nuts-and-bolts strategies and a focus on understanding human interaction for getting an agency to provide documents without going to court.

It's not enough to know the law or where to get records. Successful requesters also need to know effective ways of asking for records, ways to overcome illegal denials, and how to deal with data. We pulled together the suggestions and insights from experts throughout the country, and included a lot of what we've learned in years of FOI training, working with requesters and our own research on access strategy. Based on our conversations with frustrated requesters, we felt a guide like this has been needed for a long time.

How often are public records requests denied? What can you do about them?

Cuillier: Based on my analysis of FOI audits conducted nationally since the early '90s, along with some other studies, I've found that on average, law enforcement will illegally deny access to a basic crime report 71 percent of the time. On average, school districts will illegally deny a request for a superintendent's contract a third of the time. That's just outrageous. I can't get away with breaking the law, but police can?

And few people have the time or resources to sue, so we outline dozens of strategies for getting those records. It takes tenacity and strategies for working through the system. You can't bash down the door with a ram. You have to work your way through the maze. When you reach one roadblock, you find your way around it. We talk about all the different strategies -- principled negotiation, hard tactics, psychological strategies, etc. Requesters don't have to threaten, shout, lie or manipulate. They just have to understand human interaction and have a lot of tenacity.

One section of the book is entitled, "Understand the nature of no." What does that mean?

Cuillier: In that section we explain the prevalence of illegal denials, including some humdinger examples, and the reasoning behind a lot of them. Then we dedicate a whole chapter to how public officials perceive the records process and the barriers they face in giving you records.

It's crucial to understand the constraints agencies work under to be more effective in getting what you need. Those folks don't come to work with horns and cloven hooves. There is a whole bureaucratic world that thinks differently than requesters. Understand that world, and you'll navigate around it much better.

How do public officials think about their records?

Cuillier: Research indicates that records custodians focus on two things:

  • Make sure the process runs efficiently, like the processing of widgets.
  • Make sure nobody gets hurt or in trouble by giving out information that shouldn't be given out.

When you understand that, you can focus the request process in a way that suits their needs and also suits yours (ultimately the public's needs). You don't need to compromise -- you come up with principled solutions that get the job done without tearing apart relationships.

A lot of (but not all) officials see records as "theirs," not the public's, and they can be very possessive and controlling of them. But it also varies depending on whether the custodian is a full-time records officer or hands out documents as part of another job.

Other factors affect access from an official's perspective, such as budget constraints, political pressure and lack of training, and suggestions from custodians on how to be more effective in getting records.

When public officials deny requests, are they simply hiding something?

Cuillier: Sometimes, but not always. I think most of the time there are other reasons:

  • Often agencies don't understand what a requester really wants. Maybe the request is vague and they think they need to go through mountains of files to get the information when the requester is looking for just one little piece of information. That's where it's important for the requester to be specific.
  • Or, sometimes there is a tiny piece of a record that is legally exempt, like a Social Security number, and the officials think they have to keep the whole record secret. They sometimes don't realize that exempt information can be blotted out.
  • Often officials are very concerned about people included in records. They want to protect people from privacy invasion or some reporter from bugging them.
  • And of course, sometimes they want to hide embarrassing facts. Some studies have found that requests from journalists drag out longer and are more often denied than requests from other requesters.

Even if you take into account the perspective of these custodians, their objectives often don't align with journalists and others who seek these records. How do you deal with that?

Cuillier: That's where you need to get psychological. Sometimes an official doesn't want to give you a record that you are entitled to because the official is a jerk, just like some requesters are jerks. Or maybe they see the world differently. So that means reporters need to apply soft tactics and, if necessary, hard tactics.

Talking with a clerk together on one side of the counter, rather than face to face, squared off across the counter, can make a difference.For example, we apply to the records process principled negotiation techniques developed by William Ury of Harvard's Program on Negotiation and co-author of "Getting to Yes." A lot of it is basic human interaction and persuasion. How many journalism programs in the country have a course on the psychology of sources? For example, something as simple as talking with a clerk together on one side of the counter, rather than face to face, squared off across the counter, can make a difference, avoiding the subconscious psychological barrier of "opposition."

We also include psychological hard tactics from the world of persuasion to deal with officials who are obstinate and unwilling to cooperate. We do not advocate manipulation or lying, just understanding what motivates people. I always tell people to use this knowledge for good, not evil, and to always be up-front and ethical.

You write about effective ways of requesting records. Tell me what the most effective method is.

Cuillier: We suggest people ask for records verbally first and make sure they know exactly what they need (such as knowing the name of the record and where it is located). If the agency balks, blows you off or ignores the request, send a written request. Depending on the situation, you can choose a friendly tone, neutral letter (like the Reporters Committee for Freedom of the Press online automatic letter generator), or a legalistic, threatening letter (like the one provided online by the Student Press Law Center).

Agencies that got the threatening letter were much more likely to respond, respond more quickly, and charge less for copies.I conducted two experiments to find out which letter is most effective, sending requests to all school districts (for superintendent contracts) and law enforcement agencies (use-of-force records) in Arizona. I found that the agencies that got the threatening letter were much more likely to respond (three-quarters responded versus half of the agencies who got the friendly or neutral letters); respond more quickly (by a day); and charge less for copies.

I predicted the friendly letter would work best, but what I found was that the legalistic letter appeared to be forwarded to the attorneys, who then saw that the records needed to be disclosed. Clerks who received the friendly letters tended to ignore them.

Now, I'm not suggesting records requesters be threatening, but I think it shows that it is important to cite the law to make it clear it is a public records request and to get the request in the hands of the right person.

When it comes to electronic records, I often hear stories in which people are told it will cost thousands of dollars in programming to retrieve information from a government database. Do you think those fees are ever legitimate? How do can you get the information without paying thousands?

Cuillier: Rarely are those fees ever legitimate. Electronic records should be cheaper than paper records, or free, not more expensive. When I did CAR work as a reporter and editor I gathered lots of databases and I never paid a dime to any agency for any of them except for one ($100 for a state health department database that they sold to everyone; it wasn't worth the fight at the time).

Sometimes they say it will take 2,300 hours of programming time and cost more than $100,000. I've seen some reporters told their data would cost millions of dollars to process. Yet, I've never seen a reporter pay millions of dollars for the data, and they almost always get what they need.

It takes patience and some tried-and-true tactics, such as asking for a backup tape of the data, which they produce anyway. Or finding out what other agency has the data and going there. Or looking at their FOI logs and finding that they produce the data for others, and then asking them to make an extra copy. Or calling the company that makes the agency's software and getting a tech to explain to the agency how easy it is to export data. Or using hard tactics to show them that it is more work to fight the request then just copy the data.

Finally, tell me the most surprising thing you learned in writing or researching this book.

Cuillier: I learned a lot about public records custodians that surprised me. As a former government reporter and someone who is pretty skeptical of authority, I came into this project seeing the process as fairly adversarial. It is, to some extent, but it doesn't have to be.

Spacer Spacer

Officials are people too, and they have their own perspectives. Most are good people. I'm not suggesting people become buddy-buddy with officials. Journalists serve as watchdogs, and our first loyalty is to the public, not to our sources. But I learned that you can get the records and serve the public interest without going to war over every scrap of paper. A lot of our fights are based on miscommunication and misunderstanding.

I imagine a lot of folks in the agency community will see this book as fairly confrontational and adversarial, and it definitely speaks for the requester and transparent government, but I hope it also will bring a lot more understanding of both sides that will help journalists and citizens get the records they are entitled to without going to court, saving everyone -- requesters and agencies -- money, time and angst.