Survey Design Lessons from a Bad Telephone Survey
I was making dinner at home the other night when we received a call on our home phone (our line reserved for phone solicitations). I answered and the person on the other end asked if I was “Chris.” I told him that was my wife and handed her the phone.
It turns out it was a phone survey aimed at members of the local teacher’s union of which my wife is a member.
Here is a sampling of the comments I overheard from my wife during the subsequent 15-20 minutes as she attempted to complete the telephone survey.
“What are the choices again?”
“I have no idea.”
“How do you expect me to answer that, I’ve never been in that situation or even known anyone in that situation?”
“I can’t answer that, it doesn’t make sense.”
“I thought you said this survey would only take four minutes.”
“That doesn’t apply to me.”
“Can I just answer, ‘I don’t know’?”
And a couple more comments from my wife after she hung up:
“I don’t ever want to take another survey like that!”
“That caller doesn’t care anything about the survey.”
I really wonder who does these telephone surveys and how they get away with perpetuating the bad wrap surveys receive. Unfortunately, the data from these types of surveys is probably treated just as it would be from a meaningful survey.
However, as my grandmother used to say, “Nothing is ever a complete failure, because it can always serve admirably as a bad example.”
Let’s look at what we can learn from this bad survey example:
- How people feel about the survey affects the quality of the data you receive. If you are frustrating your respondent, the data they give you is compromised.
- It is important to understand your audience completely before starting the design of your survey questions.
- Know who is making the calls for your survey and work to make a respondent-friendly telephone script.
- Finally, when you look at survey results from someone else consider how the data was collected!
One of the challenges with all kinds of surveys (telephone, mail, online) is that you have no way of knowing how well the respondent understands the questions. The design must account for this.
Do you know who is on the other end of your telephone surveys?