Does Your Consumer Survey Data Paint The Whole Picture? When to Use Qualitative Versus Quantitative Market Research Questions
When analyzing your consumer survey data, are you seeing the whole picture? Or are you stuck trying to connect the dots?
If your data looks more like a pointillistic painting rather than a real live photograph than you’re not collecting enough of the right data.
Pointillism is a painting style that uses dots or points to sketch in shapes and give the viewer an artistic outline of what the artist is painting. While pointillism may create artful and artistic images, it is not a technique you want to use in your survey!
Your data points should paint a realisitic picture; not an artistic interpretation. Otherwise, your results could be skewed or misinterpreted. You need real details so you can make the right decisions.
What is the Difference Between Qualitative vs Quantitative Market Research?
If you didn’t get the data you needed in your last consumer survey, it is likely that you didn’t ask the right questions or use the right question type.
At SurveyGizmo, our customers frequently ask us which question type they should use. The answer, of course, depends on whether they are looking for qualitative or quantitate data.
If you’re not sure what the difference is, here is the short, simple answer:
- Qualitative questions will tell you Why
- Quantitative questions will tell you Who and What
Think of it this way:
- Qualitative research is subjective (as in beauty is in the eye of the beholder);
- Quantitative research is objective (as in 5 out of 10 people answered yes).
5 Survey Design Steps You Need to Consider:
To get the full picture, you will need to clearly define your objectives, consider the variables, and targeted groups so you know the best format to use when phrasing your questions.
In order to get the data you need, consider these 5 steps:
- Figure Out Your Goals
- Improve our customer services in:
a. painting classes
b. after-class one on one help sessions
c. selling custom portraits
- Increase class signups
- Provide more and different offerings next year
- Decide whether or not to:
a. increase sales representatives, and/or
b. find better sales/marketing channels
- Determine whether we can charge more for our services to cover increased costs in art supplies and staff
- Understand the Variables
To get an accurate picture, make sure you understand all the factors that affect the outcome.
In our painting example, variables might include the different types painting classes such as oil paintings, pointillism and watercolor. Customer experience would be another variable to consider if I am offering novice and advanced classes.
- Know Your Target Groups
- Collect Quantifiable Data
- Follow-up with Qualitative Questions
The first step to painting a full picture is to have a clear survey objective. Are you wondering Who, What, or Why?
This will determine whether you should launch a qualitative or quantitative survey. Or, perhaps you need a combination.
Define your goal in writing before starting to design your survey design. Be short and concise. Break down the goal into different categorizations or departments so you know where action is needed.
For instance, if your business offers a product and service, be sure to distinguish between the two when phrasing your questions. Your customers may love your product but hate your delivery or support service. Or vice versa.
A written goal might look like:
Figure out how to:
So that we can:
Using our example above, think about your painting audience. Are novice or experienced painters more likely to signup up for the painting class? If level of experience is a factor, you better ask some preliminary questions up front so you understand who is using and who is satisfied with your service.
This can be a huge factor in determining the right action you need to take. This will determine whether you need to modify your product/service, or whether you need to change your advertising message to specify who it is for?
Understanding the degree of the answer option will determine the best course of action. A simple yes/no answer option will not always suffice. Rating your answer options will let you know how drastic of a measure you need to take when making improvements.
For instance, do your customers absolutely love your product (your custom portraits, etc.) or do they think it’s good, but not worth the amount of money you’re charging? Do they love your service (your painting class) or do they think it is just Ok? Perhaps it really stinks and you need to make a drastic change quickly! Quantifiable data will allow you to act accordingly.
Revisit Step 1 (your written goals) and Step 2 (the variables) to ensure you’re asking the right questions, including all the factors surrounding the issues you want to address.
Get the whole picture. Not only do you need to gauge your customer’s response so you know how to act, but you need to get the details. Ask WHY they answered the way they did. This is where qualitative questions come into play.
Expanding upon our painting example above, you could use a quantitative question rating the portrait artist on a scale of 1-10 for the custom portrait. You could follow it up with an open-ended qualitative textbox question asking why the customer rated the portrait artist the way they did. You might discover that your master painter is talented but an egotistical maniac who does not appreciate budding artists.
If you only ask quantitative questions, you might miss a data point that touched on a separate issue that you were not aware of. Maybe your customers want a different painting class than pointillis or oil painting. Perhaps they are looking for classes on charcoal sketches and watercolors.
If you are still unclear when to use a qualitative or a quantitative question type, the following examples should help.
When to Use Qualitative and/or Quantitative Questions
Qualitative questions are great for the discovery phase of your research project. If you are attempting to identify an issue, qualitative questions will tell you WHY. Or, if you need to delve deeper into an issue, a qualitative question is the way to go.
For instance, let’s say you have a consumer survey where you ask respondents how well the product met their needs. If the respondent indicates not at all, you will likely want to know WHY. Follow-up with a qualitative question in the form of a comment box that allows them to specify why they answered the way they did.
Because qualitative questions types are collected in an open text format, it is harder to measure the results. For this reason, you want to use them sparingly. They can also create survey fatigue.
Quantitative questions allow you to measure opinions, attitudes and behaviors. These questions will tell you how many people, think, feel or behave a certain way. Quantitative data helps explain something rather than describe it.
Your boss will love it when you can show him the hard numbers. Use quantitative questions to:
- Uncover facts about your customers
- Discover the extent of consumer attitude
- Measure frequency of actions and trends
- Collect facts that support or disprove your hypotheses
- Measure consumer satisfaction of a product or service
- Identify different market segments
- Test a persuasive advertising message
- Determine how any people like an idea
Quantitative research assumes you know what you are looking for. If you are not sure what you are looking, than you are still in the discovery phase and need to be conducting a qualitative study with more open-ended questions that will help you find what you should be measuring or looking at.
If however, you have hypotheses and want to test them (like offering new painting classes at different price points), than a quantitative survey is the way to go. Of course, you can use a combination by following quantitative questions up with a qualitative question to get more details.
Your Consumer Survey Data Should Paint a Clear Picture:
With your canvas and paintbrush in-hand, you now have the color palette you need to paint the whole picture.
Remember these 5 steps paint a clearer picture with your survey data:
- Step 1: Start with a clear, concise survey objective – recognize whether you are in the discovery phase of your research project or whether you are ready to test hypotheses.
- Step2: Identify variables – Your picture will not be complete if you neglect to define the variables.
- Step 3: Know your target audience and understand groupings.
- Step 4: Use quantitative questions to tell you How and to What extent consumers behave the way they do.
- Step 5: Use qualitative questions to tell you WHY consumers behave the way they do.
Test your survey and review your test data with everyone who might use the data to make decisions before you launch your survey. Getting their input now will spare you headaches later.
If you didn’t get the data in the format you needed, then start over. The earlier you recognize flaws in your survey design and data collection, the more time you will save during analysis.
Now your ready to start painting and stop connecting the dots!
Image: Pointillism rendering vs. photograph of the Flatirons in Boulder, CO by Zach Dischner via Flickr
Quantitative Vs. Qualitative Research – When to Use Which
Using Qualitative Exploration To Create Quantitative Surveys
Did your last consumer survey give you a realistic picture? Share your tips here!