Using Marketing Surveys to Cut Split Test Run Time in Half

As a marketer you know that your website is judged by more than just its content. The look and feel of your site are key factors in keeping users on your page.

You need to make a good first impression in less than 50 milliseconds. That is all the time it takes for people to judge your website and determine if they will stay or leave. That is less than a blink of an eye!

To ensure you have the best conversion-centered design, you will need to test new header images, headlines, layout, signup forms, call to action messages, etc.

Website Testing Takes Too Long!

The most effective way to make measurable improvements to your website is to conduct an AB split test.

AB testing (also known as split testing) takes a variant of a website element and runs it simultaneously against the control (original) to an equal number or visitors to determine which one performs better.

Split testing can be daunting and time consuming. If you have several variations of an element, for example 3 header images, that means you will need to run 3 different tests. It will take you a while to collect a statistically valid sample of visitors and see a significant difference in conversion results.

Let’s say you also have 3 headlines you want to test. Now you have 8 different tests to run against the original. Each test is going to have to run for some time before you have enough results to make a decision.

You need results and you need them now! You’re under pressure to optimize leads and time is not on your side!

A multivariate test would be much faster than several AB tests but the draw back is that you can’t be sure which change had the most significant impact on results.

In internet marketing, multivariate testing is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous A/B tests performed on one page at the same time.

Use a Marketing Survey

You can significantly shorten your website testing time with a marketing survey.

By allowing users to rate images, headlines, layout, etc., you will know which ones to start with and which ones to knock off your list.

If you already have a list of web elements you want to test, you can easily turn these into a survey. Each web element should be a separate question with the variations being your answer options.

Most advanced online survey tools have special image and video question types making it easy for you to ask your users which image or logo they prefer. Here is an example:

image select question type example

You can use standard question types such as a radio button, checkbox, or Likert scales for non-image type questions. For instance you could use a checkbox question type to ask users which tagline or headline best describes your product or service. Like this example:

survey question type example

If you are not sure which website elements you should include in your survey, consider these 3 factors that have the biggest impact on first impressions and conversions:

  1. Visual appeal
  2. Clear message
  3. Strong call to acton

Visual Appeal
Start by keeping your design and layout simple. Visitors are quickly turned off by a busy page. Resist the temptation to fill up your page with a lot of images, buttons, and content no matter how nice they may be.

In a recent study, they found that users like websites that are simple and familiar and that these two factors are interrelated. If your website is simple but unfamiliar or complicated but familiar, you are likely to hurt users’ first impression and damage their expectations.

Clear Message
Start with a clear message that lets your users know they came to the right spot. Be short and concise. Don’t make them weed through an entire paragraph. And don’t use superlatives such as “the best”, “the easiest”, “the most affordable”. Superlatives raise skepticism – not interest.

Call To Action
Each web page should have a clear goal known as the call to action.

A call to action, or CTA, is a banner, button, or some type of graphic or text on a website meant to prompt a user to click it and continue down a conversion funnel. It is an essential part of inbound marketing that actively strives to convert a user into a lead and later into a customer.

Once users have absorbed your message, direct them to the next step. Their eyes should flow right to your sign-up or download button. Color contrast and visual hierarchy are two great ways to achieve this.

Distributing Your Marketing Survey

Now that you have your survey built (and you have tested it), there are several ways you can distribute it to collect responses.

One of the most common methods is an email campaign. If you maintain a list of your current customers you can set up an email campaign within your survey.

You could also embed your survey on your website. Polls are embeddable one-question surveys. This is a great way to get feedback on a single web element.

You can also embed a multi question survey if you are looking for feedback on several different web elements. Take a look at this embedded 3-question survey (feel free to take this survey):

Analyzing Survey Results

A Summary report will aggregate your data and tell you the most popular answer option for each question. In the example below we see that out of 193 responses (generated from test results), 78 preferred header image B, or 40.4% of the respondents preferred this image. Since this image option has the highest responses, I would use this image in my first split test. How to Use Email Segmentation To Improve Customer Survey Frequency Effectiveness.

In the example below we see that out of 193 responses (generated from test results), 78 preferred header image B, or 40.4% of the respondents preferred this image. Since this image option has the highest responses, I would use this image in my first split test.

For my tagline I see in the report example below (generated from test data) that of 147 responses, 103 preferred the tagline “Online Survey Software for better results”. Since 70% of respondents chose this option, I would use this tagline option in my website test.

If you have used a checkbox question type, you can use a TURF Analysis (Total Unduplicated Reach and Frequency) to determine the optimal combination of items that would best serve your audience. Using the example below we identify which 2 headlines would appeal to most of our visitors. (Note that TURF Analysis only works with checkbox question types.)

In this TURF Analysis example (based on test generated results) the headline combination of “Create online surveys that make a difference” with “Make your online surveys count with data you can act on” would appeal to the greatest portion of my audience.

Since I have 2 sections on my homepage where I use a headline, I would use these two headlines in my website test.

While I have determined which answer options I want use in my website test, I still don’t know which of these 3 elements will have the biggest impact on increasing my leads.

If however, I were conducting a mulitvariate test, I would start with the most popular answer option for all three new elements in my test against my original page.

Marketing Surveys Will Not Eliminate Website Testing

Just because you know which answer options are likely to have the biggest impact, don’t think you are done. You will still need to conduct your website test. But at least you have shortened your testing cycle by starting with the best answer options..

When testing, be sure that you have enough traffic and conversions to make a calculated decision. If your website testing tool doesn’t calculate the significant difference in results, there are calculator tools to help you determine if you have enough results to make a decision.

You will be tempted to cut your test short as soon as you see one version performing better than the other. But don’t do it! Be patient. Results can change from day to day. In fact, most split tests fail because they are stopped before a significant difference could be determined.

Website Optimization

A marketing survey can significantly shorten your website testing cycle. But know that even after you declare a winner, your website is not done. You will need to make continuous improvements and add fresh content if you want to rank at the top of organic search results. You can ask Google about that!

Give us your feedback on our Marketing Survey and let us know how you are conducting your website testing!

Written by

Sandy McKee

Sandy McKee is a digital marketer with over 10 years of experience in SEM, SEO, and social media marketing. She is a lover of books, fine food, and a mother of 2.

Join the Conversation