Your survey is done and you are ready to go. Ready, Set, Wait! You need to you’re your survey. “Test?” You may be asking yourself, “If I have followed all the rules of great survey design, do I really need to test my survey before it is fielded?” The answer is: Yes! No matter how much care and preparation has been incorporated in creating your survey, it is important to test your survey!
If you follow these seven tips for testing your online survey, you will surely be on your way out of survey purgatory and on your way to survey heaven.
Tip One: Testing Surveys With and Without Survey Logic
It is best to do your initial testing of your survey prior to adding any logic to your survey. It is much easier to make dependent text changes and move questions and pages around prior to adding logic. It is just plain easier!
If you need to make these changes after you add logic, you may need to rebuild your logic anyway. Therefore, it is best to test and then add logic.
Once you have tested your survey without logic, you will need to add the logic in and test your survey again! Yes, we are advocating lots of testing, but wouldn’t you rather find any problems or issues before gathering responses? We thought so!
Tip Two: Have Stakeholders Take your Survey
Hopefully, by the time your online survey is complete, your stakeholders have been along for the ride, helping you develop your survey goal and objectives, as well as brainstorming the questions that you will be using.
Now that you have designed and built your survey, it is time to re-engage your stakeholders to make sure that this is the survey that you all had in mind when you started the process.
Stakeholders, and well just about everyone has lots of things on their mind these days, so before sending them the survey to take as a test run, make sure to remind them of your survey goal and the actionable results that you are hoping to derive from this survey.
Caution: This will be a time when stakeholders will request to have more questions added to your survey. Be strong. Only add questions that are inline with your survey goal and will lead to actionable results!
Tip Three: Have Someone Outside the Project Take your Survey
Up to this point, you and your stakeholders have been the only folks that have had exposure to your survey. You are really close to this project and can probably recite the survey questions and options in your sleep. It is time for somebody else to take a look!
Now there are often limitations on which people you can share your surveys with due to security and privacy issues, but if possible try to find someone who is not familiar with your survey to review it. It is best to find some folks that have no exposure to your project at all. This will give a fresh perspective on your survey and give you a feeling for how your respondents will view the survey.
Their feedback can be especially helpful in deciphering confusing language, and or identifying questions without viable answer option for all respondents.
Tip Four: Read Out Loud to Your Dog
I find that it is best to read my survey out loud, to my dog, because all dogs go to heaven. Well, actually, I like to read to my dog because he is a great listener and gives me the undivided attention that I need! If you do not have a dog to listen as you read your survey, a cat, guinea pig or friendly stuffed animal will also do!
Why should you be reading out loud? I find that reading out loud helps me read the survey in its entirety, without skipping and words or phrases. I can see how the survey flows and see if it needs any editing. It might look and sound funny, but please give it a try.
I also like to time myself reading out loud, as this gives a clear idea of how long it would truly take someone to read each option and answer my survey thoughtfully, which I hope that every respondent will do.
Most online survey tools provide you with an estimate of how long it will take to complete your survey, but it is just an estimate. If I am going to promote that length of time that it will take to complete my survey, I want to be confident in the number that I am presenting.
Tip Five: Test in Multiple Browsers
So, you tested your survey in your favorite browser—mine is Firefox right now. But every one is not using the same browser and all browsers are not the same. They often display web content differently, so you will need to do some testing. Yes, more testing!
If you know your target audience well, you may be able to predict which browser they will be using, but you just never know. It is best to test in multiple browsers to guarantee that respondents get the best survey experience and are able to navigate properly through your survey.
If you are wondering what browsers to test in, we have some suggestions. We test survey SurveyGizmo in:
- Internet Explorer (IE) 7, 8 and 9
So those are a good place to start!
It is important to note that some people and organizations are still using older versions of Internet Explorer. We have found that this is most prevalent in government, education and non-profit organizations. If these are your targets, then you may want to consider testing in IE 6, as well.
We have also seen a dramatic increase in surveys being taken in a mobile environment. So, you will want to test your survey on mobile devices like smart phones and tablets. If your survey is not mobile friendly, you do have a couple of options:
- You can create a mobile friendly version of your survey, so those taking your survey on a mobile device will receive a slightly different version of the survey then those taking it on non-mobile devices. Most online survey tools can detect which device a survey is being taken on and serve up the proper version.
- You can request that respondents take your survey in a non-mobile environment. This might sound simple, and it is, but communicating the best way to take your survey will help respondents to understand that their best survey experience will be on a desktop or laptop computer. You might be surprised at how well your respondents listen and follow this request.
To find out more about making your survey mobile friendly, click here.
Tip Six: Test Reports
So, now that your survey is ready to go there is still another item to test…..reports. Sure, you know that questions that you want to ask, but you also want to know how the data will look in summary reports and exports. It is important that the data will be usable and will be easy to analyze.
Run some test data and see how the results will look in reports and exports. You can tweak your questions, add validation and set up reports charts and graphs so that your data comes to you in a useable manner.
Tip Seven: Set Aside Time for Testing, Fixing and Re-testing
All this testing might sound like it will take quite a bit of time. Well, it does take time, but not that much time! Just be prepared to do it! It is important to set aside time in your market research plan for all types of testing.
When we scope a project we often leave time for building our survey, and having our stakeholders review it, but we forget to build in time for testing.
Often, when a test is conducted, there will be changes to make in your project. Make those changes, but be sure to test the project again, even if it was a small change.
Small changes to an online survey can make for cascading problems across your project. This may seem obvious if you are making changes that might affect logic and page order, but you should re-test even if you are changing wording to a questions. Even one word on one question can create confusion or inconsistencies in your survey.
Heavenly Bonus Tip: Send to a Sample of Real Respondents and Review
Your ready to go, but all of your testing was done in a testing environment. If you still have time in your market research project schedule, get some real, live responses and make sure that there are no snags in your survey. Double check to make sure that the reports look the way that you expect them to.
To do this, you can send your survey to a sub-set of your sample; send to approximately 10 percent of your total sample. If there are no issues with your survey, you can use the responses from this sub-set. If there are issues, you still may be able to use some of the partial data that you collected.
Test, Test, and Then Test Some More
So, if we were not clear enough, we recommend that you test your surveys, test again and then test some more.
Spending the time and the effort to test your survey will save you the anguish of having to deal with the issues that come while your survey is in the field. It will also leave you the time to rejoice in the praise, appreciation and gratitude of a job well done save you from survey purgatory and land you in survey heaven!