Is Polling Still Worth It?
I feel like I've been buried in poll numbers even more than usual, from Wisconsin governor recall results to public confidence in the economy to American Idol. But are polls really trustworthy anymore, when you have one-third of the public living cell-phone-only and most of the rest using caller ID on land-lines to help them avoid any surveys, even when they support the cause or campaign (guilty as charged!)?
Because so many associations poll members and potential members on everything from dues raises to advocacy positions, I turned to the man who knows more than almost anyone about the veracity and challenges of accurate polling: Bill McInturff, co-founder & partner, Public Opinion Strategies.
Bill, who is speaking today as part of the "Decision 2012" General Session at the ASAE Financial and Business Operations Conference, leads--along with partner Peter D. Hart--the largest polling company in the country, Public Opinion Strategies. The firm handles polling for NBC News/Wall Street Journal and works closely on polling challenges with the two primary industry associations, the Council of American Survey Research Organizations (CASR) and American Association of Public Opinion Research (AAPOR).
"You can believe poll results but still have dwindling confidence," he told me. "There's no question that with the glut of polling, credibility is a little lower, because people are hearing wider, more diverse results of what different polls are saying. And there's no question that the basic confidence they have in polling is very different than it was 20 to 40 years ago. They're certainly asking more questions about methodology.
Despite those troubles, "if it's done correctly, it's still broadly accurate," Bill says. "It's still the best way to collect customer and other information about public opinion, and people don't tire of needing that information."
It will cost them more, though, to get it. According to Bill, the price of polling has risen for three reasons: (1) "federal laws and mandates dictate that you cannot use auto-dialers for cell phone numbers--you have to call cell phones by hand; (2) cooperation rates are much lower, so you have to call more people to get a completed survey; and (3) you have to collect the data ... using increased labor costs."
To better ensure poll veracity, Bill--who was the lead pollster for John McCain during the latter's 2008 presidential bid--advises associations to "be good consumers and make sure you go through a discussion with the pollster about methodology," asking about compensation rates for cell-phone-only or other respondents, how the "convenience factor" of women answering the phone more than men is handled, and how the data have been weighted and by how much.
I'll be writing a second blog post shortly that shares Bill's responses on whether associations can trust that the viewpoints of respondents reflect those of non-respondents as well, the potential for social media to offer new surveying opportunities, and more. I invite comments about your own association's successes or challenges when polling. And maybe you can snag Bill after the session to get more of his input, too. Thanks, Bill, for sharing your insights so generously at this busy time!