There is an old saying that figures don’t lie but liars figure. That’s a good thing to keep in mind when examining how some companies market the results of surveys.
OK, perhaps lying is too strong a term, but I’ve seen too many press releases that promote the results of a survey but don’t tell the entire story, and surveys whose methodology — including the questions asked and how the sample was derived — simply doesn’t pass muster. Unfortunately, all too often journalists and bloggers pick up on these press releases without critically examining the methodology or digging further to make sure that the actual data confirms the headlines.
In many cases, when a company or research organization announces the results of a survey, the press release provides a brief summary of the survey but not much detail beyond that. Before I write about a survey, I ask to see the underlying report. It should include a summary of the methodology, including how the sample was derived, the actual questions asked, and how they were answered. Sometimes, after examining that report, it’s clear to me that the methodology was flawed or the summary isn’t fully supported by the underlying data.… Read more