However, what stood out on these surveys was a distinct lack of creative imagination and brand identity. There were no logos, no traces of color, no brand voices, messages or mentions of how the information will be used. One such survey was a plain white page with a string of 40-plus questions that were as poorly written as they were thoughtlessly conceived. One couldn’t help but feel like the whole survey process was an afterthought. I don’t know about you, but I’m not going to provide thoughtful feedback toward an afterthought. I relish the thought of marketers or graphic designers going into cardiac arrest upon learning such surveys were being used to represent their brands.

The brands we interact with and grow to love work tirelessly in crafting their messaging, their look, and their voice. They spend countless hours and dollars to deliver perfectly-positioned creative. They build a seamless, continuous, and consistent brand experience that extends from the customers’ first exposure to advertising all the way to their purchase and use of that brand’s product. Every touch point is carefully orchestrated to deliver the ultimate brand experience every time.

With so much effort invested in crafting the perfect customer experience and brand expression, it’s a shame when there’s one element that is only given a passing consideration — and it’s a tragedy when that element is the customer feedback process.

Measuring and improving customer experience are two of the most critical business functions driving long-term sales growth. Appearing before customers minus a well-executed survey strategy with dovetailed creative and brand voice is of the same as arriving to the championship game wearing nothing but your time-worn tighty-whiteys. It’s not pretty. Aside from compromising brand integrity, you’ll struggle to engage survey respondents in the process. The result: low response rates and unwillingness among customers to provide thoughtful feedback.

This year, I’m on a personal mission to help brands extend the seamlessness of their brand expression to their customer feedback initiatives. When they show up in front of customers, asking, “When you met our frontline employees eyeball to eyeball, how did we do?”, they’ll look and sound the way they intended: cool, composed, convincing. Let’s help the brands we love show up to the big game not only fully clothed, but ready to win. Along the way, let’s save marketers from having heart attacks because of afterthought customer touch points.

Regardless of whether you’re an owner of a small brand or an analyst at a large brand, nailing the messaging of your customer survey is critical to its performance. Before you deploy your next survey, take some time to involve your creative team or ask an outside resource for help to get it right. Your efforts will be rewarded with quality feedback and — who knows — your respondents’ gratitude for building a great survey experience.

Image courtesy of olarte.ollie – Flickr, Creative Commons (Attribution)

“Dear Boulder Resident,

[...] To get a representative sample of people living in Boulder, this questionnaire should be completed by the adult (anyone 18 years or older) in your household who most recently had a birthday.

Please have this person take a few minutes to answer all the questions and return the survey in the postage-paid envelope [...].”

Hmmm. A few minutes? Are they kidding? The survey had 170 questions and 8,500 words on it. Spending just 6 seconds on each question (and how important it is to the City of Boulder) means the survey would take more than 15 minutes. Thoughtfully considering each question could take hours.

A survey like this undoubtedly creates “survey fatigue,” but that alone is probably not its biggest problem. If done correctly there are situations where a 170-question survey can be appropriate – but that’s a topic for another article.

The biggest problem with this survey is that the authors made a number of mistakes when putting it together. Here’s the list of the most common mistakes survey authors make from my previous blog entries:

10 Common Mistakes Made When Writing Surveys

1. Having little or no understanding of the target audience
2. Providing multiple choice lists that are too restrictive
3. Requiring answers to all questions (online surveys only)
4. Asking too many open-ended questions
5. Using ranking questions incorrectly, or overusing them
6. Asking unnecessary questions
7. Asking too many questions
8. Asking two questions in one
9. Making questions too general
10. Putting too little thought and planning into writing the survey, period

The survey above has been useful in helping me refine the list of common mistakes (which supports the old adage that nothing is ever a complete failure, as it can often serve admirably as a bad survey example). To wit, I have refined the list (refinements shown in red below):

10 Common Mistakes Made When Writing Surveys

1. Having little or no understanding of the target audience and what information they will be able to provide
2. Providing multiple choice lists that are too restrictive
3. Requiring answers to all questions (online surveys only)
4. Asking too many open-ended questions (or asking open-ended questions that are not useful)
5. Using ranking questions incorrectly, or overusing them
6. Asking unnecessary questions or ones that won’t produce usable information
7. Asking too many questionsand/or including disjointed laundry-list questions
8. Asking two questions in one
9. Making questions too general
10. Putting too little knowledgeable thought and planning into writing the survey, period

Below are some examples from the survey that support this refinement of my list of mistakes.

First, an example of the type of open-ended question that is not useful:

What do you think should be the top three priorities of the Boulder City Council in 2012?

This type of question will never lead to gathering useful information, except by accident. The reason for this is the respondent has not been given any context within which to answer the question, such as: What’s on the City’s docket? What kinds of things can and will the Council address? Are there budget constraints? It is not fair to respondents to require them to guess at context. This type of question is what gives open-ended questions a bad name.

Next is an example of one of several “laundry-list” questions that were included in the City of Boulder 2011 Community Survey:

Give your respondents a break, literally. At least break the list up into smaller pieces. A more thoughtful way to present the question would be to group items into topic areas.

In summary, the main mistakes in the City of Boulder 2011 Community Survey were:

Mistakes 4 and 7 were the result of survey mistake #10: the authors did not put enough knowledgeable thought and planning into writing the survey. As a result, the data collected by this survey will not necessarily provide the information they want. There is really no way to know exactly how survey fatigue has affected the data collected. A survey of this length will certainly exclude a significant portion of the target audience.

I will address the issue of appropriate survey length in my next blog post. Happy Survey-Taking!

Image courtesy of opensourceway – Flickr, Creative Commons (Attribution)

In Marni’s post, she referenced a discussion led by Sheena Iyengar, who has done extensive research on how humans respond to choice. In a nutshell, her research has found that most of us regularly claim to want more choices…but when confronted with a variety of choices, we’re usually less happy than if we had been given fewer choices to start with.

In the cases Iyengar researched, we find ourselves affected by a “decision paralysis” – where the variety of options presented to us is overwhelming, so we simply avoid making a decision.

In other words, more choices equal less action.

Other researchers have found similar results in their research – most notably, Barry Schwartz, whose book The Paradox of Choice – Why More Is Less found that eliminating choices can greatly reduce anxiety for shoppers.

So what does this have to do with online surveys?

While the two researchers above focused their research on shopping, the results of their research can clearly apply to online surveys as well. The more choices a survey respondent has to contend with, the less likely they are to take action.

In other words, when creating an online survey, you should be very aware of the number of survey choices you’re inserting into your survey. If you offer a checkbox question with a list of 24 answers, for example, it’s more likely to cause your survey respondents anxiety than a checkbox survey question with only a few answer choices.

The same can be said for essay survey questions. Let’s face it: taking a survey with a ton of essay questions is quite possibly every survey taker’s worst nightmare. It’s similar to what artists call the “blank canvas effect:” when confronted with an empty essay question that you as a survey respondent must fill out, the possibilities are endless…and that much scarier.

How many survey choices should I have?

For checkbox questions, radio questions, and other “single-list” question types, Iyengar’s research indicates that the ideal number of choices is three options. Sometimes, of course, three options won’t be enough. In those cases, we would venture to say that three predetermined survey answers and an “Other” textbox would offer a good compromise between gathering additional information and not being too overwhelming.

For other “multi-list” survey question types (like table of checkboxes survey questions), Iyengar’s findings suggest that three columns with three options each would be ideal. More than that could easily overwhelm your survey respondents.

Why survey choices are important

Ultimately, survey creators are looking to gather data. If we put something in the way of us coming up with as much good survey data as possible, we’re doing ourselves a disservice.

The more choices a survey respondent has to contend with, the less likely they are to take action. By keeping your survey options to a minimum you can make your online surveys that much more engaging…and collect better data, too.

Image courtesy of fdecomite – Flickr, Creative Commons (Attribution)

Specifically, today we’re going to address online survey welcome pages. If you don’t use welcome pages on your surveys, there are a few key reasons why you should. First off…

They’re User-Friendly.

As Zingerman’s Deli (one of our favorite organizations, who were named the coolest small company in America by Inc.com and who wrote the book on customer service) points out, one of the most stupid ways to lose a customer is to ignore them.

If you’re a retail business, you should greet each customer when they come in the door. So why not do the same thing with your online surveys? By adding a welcome page to greet a potential survey respondent, you can convey to them that you appreciate their opinion.

Additionally, with a survey welcome page you have the ability to set your respondents’ expectations. You can outline any incentives you’re offering for their participation. You can tell them how long the survey will take, or let them know your privacy policy. Setting expectations is a good practice any time you’re asking for someone to take time out of their day to help you – and it’s especially true when they’re not seeing you in person. No matter how nice you make it look, an online survey can be pretty impersonal – so why not make it friendly?

Ultimately, letting your respondents know what to expect using a survey welcome page is one simple way to encourage them to continue on with the survey. If they know what to expect, they’re much more likely to keep going.

It’s not just about survey usability, though…

Survey Welcome Pages Have Data Collection Benefits

In addition to encouraging your readers to continue, a survey welcome page also offers a key structural benefit: they’ll help you collect better data about your survey.

Here’s what we mean:

If a survey respondent arrives at your survey and finds a page full of questions, they might decide not to take your survey. SurveyGizmo counts this as an Abandoned survey response (**for a complete explanation of how we categorize responses, check out our Abandoned, Completed, and Partial Responses Explained tutorial).

The problem is, a survey respondent could leave your survey on the first page for any number of reasons. They could have just been called into a meeting, or gotten a phone call, or perhaps their computer crashed. Either way, we’ll count that response as Abandoned.

By adding a survey welcome page, you give yourself one additional bit of survey data – because by the simple act of clicking the Next Page button, they now register as a Partial survey response. This may not seem like much information – but it gives you a little more insight into how respondents are reacting to your survey.

It also helps us troubleshoot if you have a problem – if your survey has no welcome page and shows thousands of Abandons and no Partials, that doesn’t give us as much to work with. On the other hand, if you build your survey with a welcome page and it still shows thousands of Abandons and no Partials, that suggests the issue is on the welcome page. By the same token, if your reporting shows a significant number of Partial responses that left your survey on the first page after the welcome screen, we can assume there’s an issue with your first page of questions.

Why Not Use Welcome Pages?

Of course, there are still some reasons to not use a welcome page with your online surveys. For one, if you’re embedding a survey, it will likely not produce the same experience as will a full-page survey.

Additionally, there are certain times when a welcome page might introduce some unwanted bias into your survey responses. Obviously, you should always be wary of anything creating bias – in this case, you can often avoid it by simply paying close attention to the content of your welcome page. However, in some cases it may simply make more sense to leave off a welcome page and deliver your respondents straight into the survey.

Still, it’s our experience that more often than not, adding a survey welcome page to your survey will help set your respondents’ expectations and increase your response rate. We highly recommend doing it when possible.

Images courtesy of grahamc99 & woodleywonderworks – Flickr, Creative Commons (Attribution)

And yet people still do it all the time. They build online surveys that are way too long…and they scare off their respondents in droves.

And that sort of bad survey design can have even worse effects – if your respondents feel like you’ve wasted their time, it may reflect badly on your company or organization. They may decide to ignore all online surveys from you in the future – or worse, they may decide they want nothing to do with your products or services.

But aside from those obvious problems, there’s another survey fatigue issue that can arise: Survey fatigue can affect your data.

Let’s say you’ve created a survey with survey branching or survey page jumping that shows or hides a page or asks a different set of questions based on your respondent’s answers. And let’s just say that one branch of your survey has more questions than the other. If you were an online survey respondent and you were growing impatient with the survey, don’t you think you might choose one response over the other based on which one will get you finished quicker?

Granted, you might just abandon the survey. But if there’s some sort of incentive on the line, or if you’re really far into the survey, you might go ahead anyway.

So switching back to your role as the survey creator, you now find yourself with corrupted data because people are choosing the shortest route to get them to the end of the survey.

Now, of course, you can prevent all this by out-thinking your respondents – either keeping the same number of questions per branch, or using the “Forward Only” feature in the Survey Settings menu…but still, that doesn’t solve the root problem of survey fatigue.

The key here is that your survey respondent started off with good intentions – and then the length of your survey caused the respondent to start looking for the quickest way to the end.

So all of this is to say: think about what you’re asking on your surveys. Is it essential, or just a nice-to-have? If it’s not a must-have, ditch it, and watch your abandonment rate decline.

Photo courtesy of toolmantim – Flickr, Creative Commons (Attribution)

PS – how many of you looked at that photo and couldn’t help but yawn? That’s what we thought.

As I said last time, although it’s nice to know the most common survey mistakes made, the real value is in understanding how to avoid them. You can do that by understanding why each is a mistake.

6. Asking unnecessary survey questions

It is easy when writing surveys to fall into the trap of wanting to know everything. However, as a survey writer you owe it to your respondents to only ask questions from which the resulting data will be used to take action or make a decision. Respondents can sense when you are asking a question that isn’t needed and will not be used. The two most common types of unnecessary questions are asking about something that has already been decided and asking about things over which you have no control.

7. Asking too many questions

This survey mistake appears straightforward, but is often misunderstood. I’m often asked how many questions can be asked on a survey; however, there is no magic number for the right number of survey questions. The two limiting factors are: 1) The commitment and attention span of the target audience, and 2) the resources and time the survey owner has for acting on the information received.

For example, when it comes to commitment to a survey, you can’t ask as much time of a general audience without an investment in your subject than you can of a loyal customer or a dedicated employee. If respondents are committed to your subject and are kept engaged by the survey instrument, they will spend the time it takes to complete a long survey.

The second limiting factor, the resources and time the survey owner has for acting on the information received, usually overrides the first. The only reason to do a survey is to use the information obtained. Thoughtfully using this information and making changes or improvements takes a considerable amount of time and effort. It is better to implement shorter surveys more often than to implement one survey that produces five years of work.

8. Asking two survey questions in one

This is a great way to frustrate your respondents and give you ambiguous data. For clarity, let’s look at two examples. The first is the question: “Please rate the technician’s knowledge and professionalism.” This is clearly two questions. The technician’s knowledge may be great and his professionalism lousy.

The second example, asked of a high school counselor, illustrates a more subtle way of making this survey mistake. “Do you interact with your students’ parents about college?” Again, this is really two questions: “Do interact with parents?” and, if so, “Do you talk about college?” This mistake can be fixed in either of two ways. You can ask both questions separately or you can ask the one question and adjust your response choices to include both “I don’t interact with students’ parents” and “I interact with students’ parents but not about college.”

9. Making questions too general

The problem with questions that are too general is that two respondents can sometimes answer the question the same but for completely different reasons. For example, “Do you believe wireless devices can cause health problems?” Clearly, there are many ways for people with very different views to answer this question “Yes”. One person may feel it is a remote chance while another may think it is an absolute certainty. The quality of the information obtained from a survey depends on asking focused, unambiguous questions specific to the survey objectives. A better approach for determining people’s beliefs about the dangers of wireless devices might be to ask, “Do you curtail your use of wireless devices specifically to avoid risk to your health?”

10. Putting too little time and effort into writing the survey, period

The fact is, it is easy to write a survey with lot of questions and send it out to a broad group of people. The difficulty is getting usable information that can help with solid decision-making. Every question in a survey needs to be well thought out and evaluated against the survey objectives and the target audience. Too often the results from a survey raise more questions than they answer because the questions weren’t well thought out, reviewed, tested and reviewed again. Extra effort spent writing your survey will pay big dividends when using the data.

With that in mind, I thought I’d share with you the ten most common mistakes made when writing surveys.

Keep in mind that while it is nice to know the most common mistakes made, the real value is in understanding how to avoid them. In order to do that it is important to understand why each is a mistake. I will address them one by one in this and my next article. To start, here are the first five.

1. Having little or no understanding of the target audience

This seems straightforward: how can you write an effective survey if you don’t understand much about who will be completing it? The issue is making a connection with your respondent. The survey writer should know as much as possible about the attitudes and beliefs of the potential respondent. The wrong wording can offend respondents or just steer them away from what you intended. Often, too much focus is placed on what information the survey writer wants to get back, and not enough on what information respondents can provide.

2. Providing multiple choice lists that are too restrictive

It is always a good idea to include answer options that include “don’t know” or “uncertain,” “not applicable” and “other.” Respondents become frustrated when they don’t see their response in a multiple-choice list. The idea is to weed out respondents that don’t have a clear opinion from those that do. Otherwise you risk contaminating the good responses.

3. Requiring answers to all questions (online surveys only)

Nothing is more annoying to respondents than having offered their time to complete a survey and then having trouble progressing through the survey. A few skipped responses is not going to change your results – and ultimately you cannot force respondents to answer a question. If they want, a respondent can just close their browser and forget about your survey.

4. Asking too many open-ended questions

It is good to have comment fields, but too many open-ended questions makes it appear that the survey writer did not want to put in the effort to create easy-to-answer questions focused on survey objectives. The main purposes for open-ended questions in a survey are to provide respondents an outlet for thoughts and opinions that may otherwise distract them from thinking about the questions asked, and to add richness and understanding to the quantitative results obtained.

5. Using ranking questions incorrectly (or overusing them)

The inexperienced survey writer will often overlook the fact that ranking questions are difficult for a respondent to answer and even more difficult for them to analyze and interpret. The mistake made is to assume that the best way to ask a question of one person is the best way to ask the same question of many people. If I have only one customer, then I would want that customer to rank their priorities (one to whatever). That all changes, however, when I have to consider the priorities of many customers together. Asking respondents to select their top three priorities (or two or four, etc.) creates a natural ranking when the data is summarized.

In my next post, I’ll finish my list and address the last five most common mistakes made when writing surveys.

We’ve also heard repeated requests for something else you want: improved collaboration tools. And that’s why we’re pleased to introduce an important new feature we’ve just added: Editor Notes.

In the past, your best bet for getting feedback on a survey was resorting to the tried-and-true “email flood” technique. You know what that is, right? It’s the one where you send out a survey test link to the rest of your colleagues, then wait for a few minutes until your inbox is flooded with suggestions about things that don’t work or don’t look right in the survey.

You might have also experienced the popular “passive-aggressive co-worker” scenario, where one of your colleagues goes into the survey and makes changes on their own. (And doesn’t let anyone know about it, naturally.) If you’ve experienced this particular situation, you know that it’s not very fun to login to a survey you’ve spent lots of time building, only to find that a question changed by a co-worker has messed up all your logic.

At SurveyGizmo, we’re not a fan of either of the above scenarios. And we want to make it easier for you to collaborate with your teams to create amazing surveys. Which is why we’ve launched Editor Notes!

Editor Notes are quick notes that you can add to your in-design survey. If you have a multi-user account, you can have discussions about your survey’s elements with other account users. The back-and-forth can go on as long as you want it to.

If you have a single-user account, Editor Notes still work for you! You can use them to create notes to yourself about different elements, or annotate your surveys for future reference.

Additionally, if you contact our awesome Support team with a question about a specific survey question, we can now leave you Notes in your survey about your issue. (Sometimes it’s easier to see the solution right inside your survey, rather than having to go back and forth from a support ticket.)

The collaboration possibilities are endless.

We’ve made Editor Notes really easy to use. To create an Editor Note, just click on the Add Note link at the top right of any element in the survey editor. If notes have already been added to a survey element, you’ll see a running tally in place of the Add Note link.

Editor Notes are active in your account now – go give them a try!

That being said…we’ve got new survey integrations to tell you about!

First off, there’s our new GoDaddy integration. If you need a private branded domain for your survey, you can now easily register it right from the SurveyGizmo dashboard. It’s simple to get to:

1. From the Publish tab, click on “Default Link” under the My Links and Campaigns menu.

2. Under “Sub-Type,” click on either “Branded Subdomain” or Private Domain.”

3. Click on “Account Settings.

4. Under “Private Domain,” follow the instructions to set up a GoDaddy domain from within the menu.

Once your domain is set up, we’ll incorporate it into your online survey automatically. No more messing with CNAME’s or DNS – it’s all taken care of. For those of you with Enterprise plans, this should make it much easier to whitelabel your surveys and promote your brand using private domains.

Secondly, we’ve integrated with WhatUsersDo. For those of you who don’t know, WhatUsersDo provides on-demand usability testing and user experience research for websites of all shapes and sizes. But websites aren’t the only thing that needs testing – web surveys deserve testing, too (we always recommend testing your survey using our “Send a Test Link” feature). Now, for a small fee, you can get a ton of insight into how independent users view your survey.

Here’s how it works: when publishing a survey, click on the link marked “Send a Test Link.” Normally you would use this feature to send a test survey out to up to 5 different survey respondents whom you specify. Now, however, you’ll notice that there’s another option – simply click on the link to WhatUsersDo, and you can immediately get feedback from random survey respondents around the world.

You can specify different objectives for the users you test, and ask them specific questions as the survey ends. They’ll also provide a video (with audio) of the user moving through the tasks you’ve assigned them, so you can really get the feel for how they’re interacting with your survey, and where they’re confused or running into problems.

Obviously, it’s a great way to step away from your survey and get a feel for how an unbiased user will react. You might learn that a page is too overwhelming, for instance, and decide to go back and break it up into smaller, more digestible pieces.

Both of these new integrations are just a few of the integrations we offer with our partners – from email to CRM, you can find a ton of resources to enhance and extend your surveys.

Happy surveying!

What have we done? Well, now you can get more information about your questions without having to even click the edit button and open up each individual question.

If you check out the image below, you’ll see the new Question Information at a Glance Column.
This shows you the Question ID, the Template hook names & aliases (if you have assigned any), and it let’s you toggle the required question and question numbering on/off.

Now if you decide you no longer want to require a couple of your questions, all you have to do is uncheck the boxes on the side instead of having going into each individual question. You’re survey building life just got 10 times easier!

We also color coded the feedback on any questions and pages you have. If you check out the images below you’ll see that any logic you have a on a page or question will always be shown in the color purple and any validation you have will always be shown in the color green.

We hope you like the changes we made to the editor and it helps you get your work done quicker!