Surveys

Thanks to the internet, surveys are a lot cheaper than they used to be. And the prevalence of roll-your-own amateur surveys means that we all have a lot to learn.

A survey can teach your customers or it can help you learn from them.

And it might be a real survey, or it could be a census.

The traditional understanding of a survey is that the goal is to LEARN from your population and that you will ask a scientific sampling, not everyone.

You can TEACH people with a survey, though, simply by asking them questions that help them notice things they never noticed before. "Do your prefer option A or option B," might just be a way of getting people to notice that you even have an option B. 

The very act of asking a question may change the experience for the customer. One small firm I know shows prospects a book of testimonials. Then they say, "I hope that when we've completed our job for you, you'll be willing to write one too." That seed increases the likelihood that people are going to be looking for something good to say, which increases the likelihood that they'll enjoy the event.

Of course, this can spiral out of control pretty quickly. Push polling, in which faux pollsters call people up and ask them questions with patently false assumptions about competing candidates, for example, is just wrong.

But don't forget the hybrid solution, which I call a Trident survey. "4 out of 5 dentists surveyed recommend sugarless gum..." Hardly scientific, but publishing the results made dentists feel better about recommending the gum and made people with teeth happier about chewing it.

Which leads to the question of how many people you're going to ask. Professional surveyors almost never ask everyone. They carefully select a representative sample (not so easy) and invest in each interview, thus scaling the results for the whole population. That's how Nielsen works.

There are plenty of inexpensive ways to ask EVERYONE your question, though. That turns your survey into a census. A census only works, of course, when the response rate is close to 100%, because uneven response rates are going to skew your results.

Analytics is a form of census survey. You can track how everyone who visits your website behaves. Focus groups, on the other hand, are a poor use of just about anyone's resources, because they are inherently not surveys at all. Without a skilled moderator, all you get is useless (but extremely vivid) data.

So, I guess I'd summarize the survey question by identifying four kinds of surveys that are worth doing:

• Census surveys designed to teach your market, not you. The act of asking the question is a marketing tactic.
• Public non-scientific surveys (or census surveys) in which publishing your results to the group helps change the group's behavior.
• Professional surveys designed to extract really meaningful data from a small group.
• Census-based analytics in which you are extracting data about behavior from the entire group.

One last warning: round off. 98.2% is a bogus result. "Most" is a lot more accurate. The ultimate purpose of most traditional surveys is to make decisions. Alas, your audience is often the very worst group to help you make a decision. When you let a survey be presented as accurate, it becomes the silent decision maker in the room and leads, often, to mediocre products for the middle of the market.

Thanks to John for the question.