How The Associated Press is using automation to rethink the way it does news
When The Associated Press announced last year it was using software to write investment stories, the news was greeted with a mixture of wonderment and hand-wringing from journalists who feared that automation would affect the quality of coverage or lead to layoffs at the news cooperative.
So far, those fears have proved to be unfounded. Over the next few months, the AP drastically increased the output of the writeups and announced it was making moves to automate its coverage of NCAA sports. It also announced the hiring of its first automation editor, Justin Myers.
The man at the center of this push is Lou Ferrara, AP's vice president and managing editor, who has championed automation as a powerful tool for saving its global newsroom time and money. I talked to Ferrara, who's a member of Poynter's National Advisory Board, how the technology is changing the way the company thinks about news.
Besides sports and investment coverage, what else is the AP automating?
We're automating a lot of different elements within the inner workings of the AP. As you can imagine, as a 170-year-old news agency, we had a lot of sprawling, legacy-type processes out there that consume people's time. It may be just be three hours a week or eight hours a week, but jobs get layered in this business, particularly in a news agency. And we're trying to eliminate a lot of those tasks so that we can use the resources we have to do the journalism.
We're also automating a couple technologies that will get information from government databases where there are filings to our reporters faster rather than having to have someone go in and look something up constantly, once a week or twice a week, whatever it may be. Having an automated algorithm that can ping and surface things to reporters faster gives us a competitive edge to reporting the news.
We're also looking at anything where there's structured data. The weather reports that we publish, we traditionally did those for newspapers because they wanted them for their weather pages, and we did maps and supplied all kinds of things, and a lot of that can be automated now. As the LA Times has done, we're looking at earthquakes. We're looking at them on a global scale and trying to figure out if we can automate a process to a point that it would make it faster for our reporters and editors to determine whether something should be published or more reporting should be done.
Have you given any thought to how you could automate coverage of election returns?
We have. In fact, we've not only given thought to it, we're in deep exploration on it. Because the one thing we have a lot of structured data around is elections, whether that be in the form of exit information or vote counts. And I think we're looking at whether there's a process by which we can speed that up on our own workflows. Another part of that is what new forms of storytelling or information can we glean out of this? Some of this process could be automated for that. Because you could drill into different data sources in different ways with automation to extract things that you wouldn't normally do if you were doing it from a human perspective.
Can you give me an example?
As someone who lived through the 2000 election in Florida when I was working in Sarasota, if we had automation then, I think a lot more could have been surfaced much more quickly. If you think about that time period, it was really interesting how the reporting came about — it was a lot of manual work by reporters and journalists and of course the political parties to surface that information. I think automation would have drilled very quickly into some data analysis and given us things that we wouldn't have had before. I think that could happen on a larger scale. I think also just the quickness of an ability to call a race may be enhanced by automation.
How do you decide which tasks are good candidates for automation?
What I've tried to look at with automation is time. How are we spending people's time? Journalism is a messy business in a lot of ways. It takes time to look up information and talk to people. And I want people spending their time doing the stuff that takes time to do and not doing the stuff that should be done automatically. People don't get into journalism to execute a process.
Are there administrative duties that can be automated? Things that 20 years ago assignment editors might have taken care of?
We're starting to look at that. I don't know if it's about full automation as much as it may be part of it is automation. So scheduling of staff, for example. Also looking at things like our digest — we put out a lot of digests and planners at the AP. And there's a lot of copy/pasting, emailing that consumes a lot of time.
Why has the Associated Press prioritized this push into automation?
We're realists. Media is contracting right now. Legacy media is filled with legacy operations and processes, and some of those things produce products or content that you need in the market. But that doesn't mean the process isn't right for change. And we're living in an age with technology at our fingertips and certainly there has to be a better way to do things. When you look over the history of modern media, say the past 30 years, we went from ATEX computers to desktops to laptops, Macs, pagination. When I started in the newspaper business 20-some-odd years ago, we had a backshop with paste-up. That was gone within a couple of years. I saw that disappear in North Carolina, I saw it disappear in Florida. I saw it disappear in Saudi Arabia in a span of three years. As an industry, we seem to accept that because that's production work. But we don't look at the craft of journalism in a way to say: What are we layering on with production that's taking away from doing our raison d'être for being journalists?
We were spending an inordinate amounts of time doing earnings reports. And I think everyone can agree that the content is ubiquitous the minute it comes out, and I challenge you to find any reporter that liked doing earnings reports. No one in the last year and a half has come to me and said: This person loves doing earnings reports, that's what they wanted to be in life. Nobody liked doing them. And yet we were spending a huge chunk of time.
Sports is in a similar situation. The reason sports writing was created was to cover the games and the teams. Going back to the history of baseball, the early days of golf, football, whatever. We covered the games and then the players and the coaches and the teams and it's grown. But we're probably going to have to go back and look at that, because I think we can agree: What's now in most demand in the marketplace in sports has very little to do with the game. You look at any NFL team, and people want to know everything about every individual player, they want to know about coaching decisions, they want to know about off-the-field antics or problems or challenges. They want to know about contracts. And the game happens live before us, and it also happens live on social media. So it doesn't mean we're not going to be at the games. But I think it's going to redefine sports writing for the modern era.
What do you think will not be automated in the future? Or do you think everything can be automated?
I don't think everything can be automated. Anyone who knows me knows I love reporters and journalists, no matter what format they are. I like the business we're in. And there's a reason I'm still in this business: I have a passion for it. I don't look at it as: well, I want to automate everything. But I think there's a load of stuff that can be automated and a lot of it we're hitting on.
I'll say this: I think what won't go away is bread-and-butter investigative reporting and reporting the news that no one else has. When I see the news reporters are breaking, there's a lot of stuff that isn't going to be done by a robot.
That said, though, I think for a lot of journalists, automation is going to be part of their lives. They're going to use tools that in the background are automating some process to surface information to them faster. I gave the example of automating database queries to surface information faster — that's going to be a part of modern reporting, and you have to be able to capitalize on that.
Can automation be used for things besides straightforward articles?
I think there's a huge amount of automation to be done in video and photos. Some people don't think that's possible, or there's reasons not to, but there are companies working on this right now. Anyone who's dealt with video will tell you that telling stories in that medium is a labor-intensive process. There's no getting around that. So anything you can do to make that more efficient and speed up that process and take away process work so that more of your video journalists can do video stories, that's not a bad thing.
What steps in the video editing process do you think could be automated?
When you come back from an event, you have to scroll through your video, get your quotes, edit it, package it up, put out multiple different packages. I think there's a faster way to do that. I've talked to at least two companies that are trying to save time on the production end by matching either spoken word or text with video images to make it a faster process. I think it's two or three years off, but someone is going to start figuring out that we have plenty of video in the world.
How is the AP's new automation editor, Justin Myers, changing the way you do things?
Internally, our processes really weren't reflective of what else we could automate, because we really didn't know where to look. So we've put Justin on the task of going through and tunneling in and finding these things that could be streamlined through automation and would free up people's times.
One process he did through a pinging a database — it was taking somebody about three hours a week to do this thing. And that's just gone now. He's working on a process now that we're having to spend one staffer a day on every week. And we're saying, "well, if you get rid of that, that time goes elsewhere."
How does automation affect the way the AP spends its man-hours?
I keep reading all these stories about robotics and automation wiping out jobs, and I suspect that's going to happen to some degree — there's no question. It did in manufacturing, it did to a degree in the auto industry and these things have played out, as we've seen, over time. I think in journalism you're going to have that hit as well, in terms of jobs that were once needed that are no longer needed the way they were.
But in journalism, the difference is, compared to, say, making cars, that there's a never-ending supply of news and information you need to go report. I've never worked in a newsroom where someone says, "we have enough resources." They don't exist. It's a myth. So every time you're freeing up a staffer's time, or somebody's time, that time is going elsewhere to do something that is more relevant in the modern media world we're living in. I think some jobs will go away, for sure. But I can assure you those jobs are going to be redeployed elsewhere — and different jobs will be created.
This conversation has been edited for length and clarity.