Automated (or Interactive Voice Response) Polls have received mixed reviews in the polling community. Many remain skeptical of the capability of these polls to produce reliable results. But some believe their track record suggests that the technique can be valid.
Some essential questions to ask about IVR polls include:
- How were households selected to participate? How did the poll address the federal restriction that prohibits calls to mobile phones using an autodialer? If mobile phones were completely excluded, the representativeness of the poll is questionable, since a substantial proportion of the adult population lives in a household with a cellphone but no landline.
- How were individuals in the household selected to be respondents? Some IVR surveys begin with a human interviewer to screen for respondents — say, youngest male in the household, registered Republicans — then switch to a computer. Others merely take the answers of whoever answers the phone. A drawback to the recruit-and-switch method is that many respondents hang up during the switch to IVR.
- Were respondents capable of answering all the questions to a computer? Human interviewers, while fallible, are able to repeat questions or answer categories to respondents and can ask for clarification of a vague answer. If a question is long or has several answer categories, respondents may be confused or forget the choices listed. Journalists should be able review the questionnaire used.
- How were the data adjusted? Were the data weighted to adjust for demographics, lack of mobile phones or to match the data of other polls released? It is important to know how much adjustment has been done and its impact on the poll’s accuracy.
- What is the track record for this particular company’s IVR polls, compared with other companies’ polls? Some companies that use IVR have an established track record that compares favorably with companies that use more traditional methods. Other companies’ records may be spottier.
Automated polls are subject to certain federal restrictions in the United States under the Telephone Consumer Protections Act (also known as TCPA). This legislation generally prohibits calls to mobile phone numbers using automated dialing equipment without permission of the owner of the cellphone. Fines for violations of TCPA range from $500 to $1,500 per call attempt. Through the legal concept of vicarious liability these fines may extend beyond the firm that actually makes the call into the company that commissions the research. More information about TCPA is available here and here.
Taken from Understanding and Interpreting Polls, a self-directed course at Poynter NewsU, developed in partnership with the American Association for Public Opinion Research (AAPOR).
Have you missed a Coffee Break Course? Here’s our complete lineup. Or follow along at #coffeebreakcourse.
