June 8, 2012

Is Polling Still Worth It?

I feel like I've been buried in poll numbers even more than usual, from Wisconsin governor recall results to public confidence in the economy to American Idol. But are polls really trustworthy anymore, when you have one-third of the public living cell-phone-only and most of the rest using caller ID on land-lines to help them avoid any surveys, even when they support the cause or campaign (guilty as charged!)?

Because so many associations poll members and potential members on everything from dues raises to advocacy positions, I turned to the man who knows more than almost anyone about the veracity and challenges of accurate polling: Bill McInturff, co-founder & partner, Public Opinion Strategies.

Bill, who is speaking today as part of the "Decision 2012" General Session at the ASAE Financial and Business Operations Conference, leads--along with partner Peter D. Hart--the largest polling company in the country, Public Opinion Strategies. The firm handles polling for NBC News/Wall Street Journal and works closely on polling challenges with the two primary industry associations, the Council of American Survey Research Organizations (CASR) and American Association of Public Opinion Research (AAPOR).

"You can believe poll results but still have dwindling confidence," he told me. "There's no question that with the glut of polling, credibility is a little lower, because people are hearing wider, more diverse results of what different polls are saying. And there's no question that the basic confidence they have in polling is very different than it was 20 to 40 years ago. They're certainly asking more questions about methodology.

Despite those troubles, "if it's done correctly, it's still broadly accurate," Bill says. "It's still the best way to collect customer and other information about public opinion, and people don't tire of needing that information."
It will cost them more, though, to get it. According to Bill, the price of polling has risen for three reasons: (1) "federal laws and mandates dictate that you cannot use auto-dialers for cell phone numbers--you have to call cell phones by hand; (2) cooperation rates are much lower, so you have to call more people to get a completed survey; and (3) you have to collect the data ... using increased labor costs."

To better ensure poll veracity, Bill--who was the lead pollster for John McCain during the latter's 2008 presidential bid--advises associations to "be good consumers and make sure you go through a discussion with the pollster about methodology," asking about compensation rates for cell-phone-only or other respondents, how the "convenience factor" of women answering the phone more than men is handled, and how the data have been weighted and by how much.

I'll be writing a second blog post shortly that shares Bill's responses on whether associations can trust that the viewpoints of respondents reflect those of non-respondents as well, the potential for social media to offer new surveying opportunities, and more. I invite comments about your own association's successes or challenges when polling. And maybe you can snag Bill after the session to get more of his input, too. Thanks, Bill, for sharing your insights so generously at this busy time!


September 7, 2011

An Anniversary No One Will Forget: Associations Vary in 9/11 Treatment

So many associations are gearing up to share tributes, assess their industry's progress, and conduct community service projects in commemoration of the 10th anniversary of the 9/11 terrorist attacks that it's impractical to list them all. That said, I do want to share some of the tools, communication efforts, and creative projects in case some organizations are still pondering what their staff or members might want to do:

Created a microsite of resources. The American Psychological Association (APA) has set up a microsite with resources to "help people cope and build resistance" during the emotional days around 9/11.

Partnered for a TV special/podcast/on-demand show. APA also partnered with "Nick News With Linda Ellerbee" to co-develop a TV report called "What Happened? The Story of September 11, 2001," which ran September 1 and is available on iTunes as a free podcast and in Nickelodeon's video-on-demand offerings throughout the month. A related discussion guide helps parents and teachers talk to kids about the tragedy and tough emotions.

Developed a so-called "impact kit" for reporters--a compilation of stats, resources, and trained commentators who can discuss an event from the perspective of its impact on an industry, profession, or locality. The Insurance Information Institute (I.I.I.) has organized materials around terrorism and insurance to aid reporters covering the 10th anniversary, including prepping its board president for media interviews and promoting I.I.I.'s white paper on "terrorism risk and insurance." A strong quote in its press release will likely get good response from media: "The 9/11 attack was the largest payout in the history of insurance until Hurricane Katrina in 2005," says President Robert Hartwig said. "Insurers became the nation's economic 'first responders,' and as construction progresses on the site of the former World Trade Center, insurance claims dollars continue to play an essential and highly visible role in rebuilding lower Manhattan while also mitigating the overall economic impact of the 9/11 attack."

Conducted a 9/11-related study. A good example was released this week by CoreNet Global, an association of corporate real estate and workplace professionals. The study concludes that 9/11 "had a permanent effect on the workplace," in part by accelerating the trend toward "distributed work" conducted by workers in multiple locations. "The focus on risk management as an intrinsic strategic planning and management function also grew stronger," according to the association. "Business disruption planning became a common element for many corporate workplace and asset managers as a result of 9/11," says spokesperson Richard Kadzis. "Elements of this planning include mobile work plans for employees, facility collocation policies, redundant facilities, energy back up, business continuity plans, and off-site data storage."

Combined old-time traditional communication tools with social media tools to promote public service. The Michigan Chapter of the Council on American-Islamic Relations (CAIR-MI) has launched a billboard and Internet campaign called "Remember Through Service" to mark the day by highlighting the service of Michigan Muslims to the nation and to "provide an accurate depiction of how Muslims contribute to the broader society." Individuals highlighted include a doctor who was a first responder to Ground Zero, a Detroit police officer, an assistant prosecuting attorney, an assistant principal in an Ohio public school, a Vietnam veteran, and a volunteer doctor at a free medical clinic. You can see the billboards here[LINK TO] and related YouTube videos[LINK TO].

Volunteered like crazy. The goal is more than 50 million--that's the magic number for how many volunteers the government, community partners, and others hope to engage in community service projects such as park cleanups, mentoring, and food drives. Any organization still interested in community service projects can go to for a list of opportunities.


March 30, 2011

The Great Divide

I recently attended a presentation by Bob Wendover from the Center for Generational Studies on communicating between generations. It was a very odd experience, as the entire room was filled with "Generation Xers" (those born from the 1960s-1981) and "Baby Boomers" (those from 1940-1960). I was the only "Millenial" in the room, the group of people born between 1982 and 1999.

The presentation made me realize that we focus way too much on the generational "divides". Why? Because the presentation discussed how attention-deficit, impulsive, and unprofessional "Millenials" are. But really, I'm on the cusp, being born in 1982. I've been taught how to write a professional business letter. I remember life before the Internet. My See-N-Say had a string cord, my Slinky was metal, and my Easy-Bake Oven actually baked.

But yet formal "generational" divides lump me into a class with people such as my 14-year-old cousin, who despite living down the street from me only communicates with me via Facebook status messages and abbreviated chat-speak text messages. He asks his friends if they want to hang out.. and despite them living down the street, what he means is "let's play Xbox from our respective homes while talking on the phone". He and I have pretty much nothing in common at all.

The reason I share this is to warn you NOT to look too much into age or generation stereotyping. After all, our grandparents are joining Facebook, and my mom has an iPhone while my husband can hardly text message. Be sure you're communicating in every way with every member, instead of making generalizations based on age. You know what they say about assumptions.

| | Comments (7)

April 9, 2010

Crowdsourcing without the crowd knowing about it

Professor and author Ian Ayres delivered the Friday morning keynote at Digital Now, urging associations to adopt more statistical regression and randomized testing to better inform their decisions. If you're not, he says, "you're screwing up."

Why? Because, even if you think you're an expert, humans simply aren't good at predicting outcomes in situations that involve multiple driving factors.

In fact, Ayres picked the title of his book, Super Crunchers, via a randomized test through Google AdWords. Some searchers saw an ad for "Super Crunchers," and some saw an ad for "The End of Intuition," (Ayres' personal choice). More people clicked on "Super Crunchers," and so that title won.

I've seen other authors who have openly crowdsourced the titles of their books. Ayres did it blindly. The crowd didn't know it was being measured.

Of course, associations have their own built-in crowds, and Ayres says associations should more actively test anything and everything. "You routinely get a 5 percent to 10 percent lift in whatever numbers you care about when you do randomized testing," he says.

Just a few measurements associations can test:

  • Member acquisition or renewal probabilities
  • Member lifetime value
  • Any and all marketing copy
  • Website design

Ayres says the web has made statistical testing much more accessible because making adjustments is easy and cheap, as is gaining a large sample size.

My key takeaway from Ayres' presentation is that associations should trust numbers more (and get their boards to trust them, as well). We often overreact to complaints from members or feedback from evaluations. Broader statistical analysis of how members behave, rather than what a few of them tell you, can let you know whether the ones who are speaking up are representative of the full membership or merely outliers.

| | Comments (6)

December 29, 2008

Cheap, Easy, Effective: A Different Kind of Education Tool

The Ethics Resource Center has a cool twist on the usual e-update to stakeholders: a regular series of e-mailed PowerPoint charts or graphics on specific topics titled EthicsStat.

This week’s subject is on “Global Reporting” and shares data in an easy-to-absorb color chart on reported employee misconduct in various countries, urging leaders to “keep their fingers on the global pulses of their organizations.”

Considering that many people learn best through visual representations of data rather than straight narrative, this appears to be a smart, unique approach that would work well for many organizations that regularly share research or trends info with stakeholders.

| | Comments (1)

August 20, 2008

Frank Fortin Talks Social Technographics for Associations

Following up on a post I did on the Association Marketing Springboard, here is a short video from the Annual Meeting of Frank Fortin, communications director for the Massachusetts Medical Society, explaining how he is using Forrester's Social Technographic Survey to better understand his association's social media successes and failures. Over the next six months, Frank and his team intend to apply the lessons they've learned from Groundswell and their own experience to transform the Massachusetts Medical Society's social spaces online.

Trouble viewing the video? Click here.

Here's one interesting point Frank made that didn't make it into the video. It's not just the question, it's how you ask it. For example, Some of your members might not be familiar with RSS, but they might be using it on sites like iGoogle, Google Reader, Bloglines, NetNewsWire or some other aggregator. Are we making assumptions about our members' social media aptitude simply because we're asking the wrong questions--or the right questions in the wrong way? It's something to think about.


April 8, 2008

Quick clicks: Common sense

I've been saving a variety of links to share today:

- Tom Peters suggests that you run your ideas and proposals by an "ombudsman for common sense." I wonder how many bad ideas could have been turned into good ideas with a little common sense at the beginning of the project ...

- Speaking of good ideas, Rick Johnston at the CAE Weblog recently shared one: instead of a focus group, hold a "listening lab" guided by your customers or members and their concerns.

- Jeff Cobb at Mission to Learn learned five good presentation tips from a series of 60-second lectures.

- I'm pleased to note that Joan Eisenstodt now has a blog, Good Stuff From Joan Eisenstodt, at the Meetings Collective. Those of you who read the listservers where Joan participates or have heard her speak know how interesting it is to listen to what she has to say. (Thanks to Sue Pelletier for linking to Joan's blog first!)

| | Comments (1)

February 7, 2007

In the Eye of the Beholder


In conjunction with its 150th anniversary, the American Institute of Architects asked Harris Interactive to develop a survey of 2,000 ordinary Americans to find out our favorite buildings and structures. They were shown photographs and asked to rate 247 buildings nominated by 2,500 architects in various categories. The architects were surprised at the results.

The Empire State Building topped the list. Other than the Bellagio Hotel and Casino, no building of the past decade made it in the top 30 in the poll. Only two in the top 20 were built in the last 35 years: The Vietnam Veterans Memorial (10) and the World Trade Center (19). Some architect favorites are notably missing, including Ludwig Mies van der Rohe's Seagram's Building in New York and Louis Kahn's Salk Institute in La Jolla, Calif. (See the complete list here. The association plans to post the results in blog form to get even more feedback at

"The Bellagio -- I can't believe it," Edward Feiner, a director of the Washington, D.C., office of top corporate architects Skidmore, Owings & Merrill, which has five buildings on the list, tells Alex Frangos in a Wall Street Journal article. "The Bellagio is tasteless."

In the last AIA survey of architects in 1991, Frank Lloyd Wright's Falling Water house in Bear Run, Pa., topped the list. It's No. 29 on the general-public survey. Architects ranked Thomas Jefferson's University of Virginia as No. 2. It's not on the new list at all.

After I stopped laughing, I realized the example makes an important point for all trade associations - even if you know your members and know what your members like and admire and feel about your industry and what it stands for, are you sure that John Q. Public feels the same way?

| | Comments (3)

November 17, 2006

Traditionally, associations had a competitive advantage over commercial publishers because they represent both the content experts (authors) and the buyers (members). Plus, association publications could be road-tested and peer-reviewed, representing best practices if not the industry standard.

If that's your niche, hold on to your hat.

In case you missed the story in the Wall Street Journal Online edition earlier this week, London-based publisher Pearson is teaming up with Wharton and MIT's Sloan School to create a business book authored and edited by a "wiki" online community. More than 1,000 have already signed up.

The book will be called We Are Smarter Than Me. "One goal of the WeAreSmarter project," the online WSJ reports, "is to see how a wiki can organize and balance material provided by experts such as consultants and professors and managers who are using the techniques in their own business."

Like the nonprofit online encyclopedia, Wikipedia, this effort distills the wisdom of many, scrubbing out personal opinion through community-enforced rules.

Fans of James Surowiecki's 2004 best seller, The Wisdom of Crowds, will recognize the shift from forecasting to best practice.

Are you harnessing the wisdom of many to revolutionize your publication program?

| | Comments (1)

October 26, 2006

The baby bottle measure

ASAE & The Center recently published 138 pages that started to define what being a remarkable association is (7 Measures of Success: What Remarkable Associations Do That Others Don't).

If you think his schtick is insightful—which I do much more often than not—then you'd have to agree that Seth Godin is a remarkability maestro. His Purple Cow: Transform Your Business by Being Remarkable (my edition is 142 pages) should be at the top of the list for anybody aspiring to be anything other than mediocre.

In case you don't have time to read those 280 pages, check out this 33-word post on Seth's blog, "What it means to be remarkable." He left of off eight words: "If not, then why do you do it?"

| | Comments (1)

October 25, 2006

Getting to know your members

I’ve been reading an advance copy of Made to Stick, by Chip and Dan Heath—a study of how and why some messages “stick” in the popular consciousness and others don’t. One story they tell is about a brand manager at General Mills who revitalized the Hamburger Helper line.

When she began her work, she was given reams of sales and marketing research data (the “death binders,” she called them). But none of that was as valuable to her as what she gleaned by actually watching customers “in the field.” Seeing a parent put a meal together while holding a baby on one hip and watching small children pick through unfamiliar foods at dinner gave her a whole new perspective.

A related question was raised earlier this week during ASAE & The Center for Association Leadership’s CEO Think Tank on 7 Measures of Success. Attendees were asked whether staff at their associations visited members at work. Scattered hands were raised around the room—it certainly wasn’t a majority. And come to think of it, in my past experience in association management, I’ve never had the opportunity to observe a member at work. I regularly spoke with them about their jobs; I wrote articles on the issues facing their profession. But none of that was the same as actually shadowing a member as he or she supervised construction of a bridge or sampled for chemicals at a manufacturing plant.

Clearly, quantitative research has its place. But I think we would be much more inspired to come up with solutions to members’ daily challenges if we were able to see them, up close and personal.

| | Comments (1)

September 28, 2006

Research vs. intuition

Based on Andy's excellent comment to my post on my likes and dislikes of 7 Measures, I wanted to clarify that I am not antiresearch. Far from it actually.

Andy makes the point that the ideal scenario is a blending of research and intuition to make decisions. I think that's what I was trying to say, though I think perhaps not well enough. I think the relationship between the two is that research—things like membership surveys, reader surveys, and benchmarking surveys—should absolutely be performed and does provide valuable information, which, in turn, helps people form intuition. Many, many other things help form intuition, too. Some of the important things that come to mind are having conversations with members and other constituents (and then using those conversations to start more conversations with your staff or volunteers to see if they're hearing the same things); looking at the trends in the world around you, staying informed not just of the local, national, and international news (and please, go beyond U.S.-based sources for your international news), but of trends in marketing, technology, diversity and generational issues, economics, and yes (one of my personal weaknesses), even pop culture; and noticing how other organizations are changing.

Having said all that, I don't back down from my assertion that research should not directly inform decision making. Rather, research should be one factor considered in making decisions.

Andy also brings up the notion that research is now faster, more efficient, and less expensive than it used to be, all of which is true—kind of. The other side to that coin is that it is harder than ever to perform classic research that has statistical meaning. The scholarship on research is still coming to terms with the tools that are faster, more efficient, and less expensive. Do the tools introduce a bias into the results based on who is willing to use them? And if so, what is that bias and can you account for it? One small example that most people are familiar with is that phone surveying had been considered a pretty strong method of research assuming the rest of the research design was good. Now, with more and more people giving up landlines and going completely mobile and the prevalence of caller ID systems, there are starting to be questions about what biases are being introduced. These are things I geekily admit to being interested in (at the expense of pop culture). I don't know how it will evolve, but for right now, I treat fast & cheap research with about the same deference as focus groups. It can be a good source to help inform opinions, but as a statistical measure, I'd be skeptical.

| | Comments (4)