Survey Choices – How Much is Too Much?

In a recent blog post on The Market Research Event 2011, Marni Zapin discussed the dilemma of survey choices; today we want to revisit that idea and explore it a little further.

In Marni’s post, she referenced a discussion led by Sheena Iyengar, who has done extensive research on how humans respond to choice. In a nutshell, her research has found that most of us regularly claim to want more choices…but when confronted with a variety of choices, we’re usually less happy than if we had been given fewer choices to start with.

In the cases Iyengar researched, we find ourselves affected by a “decision paralysis” – where the variety of options presented to us is overwhelming, so we simply avoid making a decision.

In other words, more choices equal less action.

Other researchers have found similar results in their research – most notably, Barry Schwartz, whose book The Paradox of Choice – Why More Is Less found that eliminating choices can greatly reduce anxiety for shoppers.

So what does this have to do with online surveys?

While the two researchers above focused their research on shopping, the results of their research can clearly apply to online surveys as well. The more choices a survey respondent has to contend with, the less likely they are to take action.

In other words, when creating an online survey, you should be very aware of the number of survey choices you’re inserting into your survey. If you offer a checkbox question with a list of 24 answers, for example, it’s more likely to cause your survey respondents anxiety than a checkbox survey question with only a few answer choices.

The same can be said for essay survey questions. Let’s face it: taking a survey with a ton of essay questions is quite possibly every survey taker’s worst nightmare. It’s similar to what artists call the “blank canvas effect:” when confronted with an empty essay question that you as a survey respondent must fill out, the possibilities are endless…and that much scarier.

How many survey choices should I have?

For checkbox questions, radio questions, and other “single-list” question types, Iyengar’s research indicates that the ideal number of choices is three options. Sometimes, of course, three options won’t be enough. Too Many Survey ChoicesIn those cases, we would venture to say that three predetermined survey answers and an “Other” textbox would offer a good compromise between gathering additional information and not being too overwhelming.

For other “multi-list” survey question types (like table of checkboxes survey questions), Iyengar’s findings suggest that three columns with three options each would be ideal. More than that could easily overwhelm your survey respondents.

Why survey choices are important

Ultimately, survey creators are looking to gather data. If we put something in the way of us coming up with as much good survey data as possible, we’re doing ourselves a disservice.

The more choices a survey respondent has to contend with, the less likely they are to take action. By keeping your survey options to a minimum you can make your online surveys that much more engaging…and collect better data, too.

Image courtesy of fdecomite – Flickr, Creative Commons (Attribution)

Join the Conversation
  • Theresa

    Thank you VERY much for this article. I frequently have to ask people to limit their lists of possible responses, telling them I believe too many choices will have an adverse impact on their response rate. Now I can refer them to your article as well.

  • Kipp Chambers

    Thanks for the comment, Theresa! Glad you’re getting some use out of it!