However, what stood out on these surveys was a distinct lack of creative imagination and brand identity. There were no logos, no traces of color, no brand voices, messages or mentions of how the information will be used. One such survey was a plain white page with a string of 40-plus questions that were as poorly written as they were thoughtlessly conceived. One couldn’t help but feel like the whole survey process was an afterthought. I don’t know about you, but I’m not going to provide thoughtful feedback toward an afterthought. I relish the thought of marketers or graphic designers going into cardiac arrest upon learning such surveys were being used to represent their brands.

The brands we interact with and grow to love work tirelessly in crafting their messaging, their look, and their voice. They spend countless hours and dollars to deliver perfectly-positioned creative. They build a seamless, continuous, and consistent brand experience that extends from the customers’ first exposure to advertising all the way to their purchase and use of that brand’s product. Every touch point is carefully orchestrated to deliver the ultimate brand experience every time.

With so much effort invested in crafting the perfect customer experience and brand expression, it’s a shame when there’s one element that is only given a passing consideration — and it’s a tragedy when that element is the customer feedback process.

Measuring and improving customer experience are two of the most critical business functions driving long-term sales growth. Appearing before customers minus a well-executed survey strategy with dovetailed creative and brand voice is of the same as arriving to the championship game wearing nothing but your time-worn tighty-whiteys. It’s not pretty. Aside from compromising brand integrity, you’ll struggle to engage survey respondents in the process. The result: low response rates and unwillingness among customers to provide thoughtful feedback.

This year, I’m on a personal mission to help brands extend the seamlessness of their brand expression to their customer feedback initiatives. When they show up in front of customers, asking, “When you met our frontline employees eyeball to eyeball, how did we do?”, they’ll look and sound the way they intended: cool, composed, convincing. Let’s help the brands we love show up to the big game not only fully clothed, but ready to win. Along the way, let’s save marketers from having heart attacks because of afterthought customer touch points.

Regardless of whether you’re an owner of a small brand or an analyst at a large brand, nailing the messaging of your customer survey is critical to its performance. Before you deploy your next survey, take some time to involve your creative team or ask an outside resource for help to get it right. Your efforts will be rewarded with quality feedback and — who knows — your respondents’ gratitude for building a great survey experience.

Image courtesy of olarte.ollie – Flickr, Creative Commons (Attribution)

These are all good questions, but unfortunately there is no ONE good answer. The “right” number of questions for a survey depends on the survey, the survey audience, and your resources for acting on the information received.

This means we need to look at the factors that determine an acceptable length for a survey and use those to guide our thoughts on survey length (for both online surveys and offline surveys).

The main factors that determine an acceptable length for your survey are:

1. The relationship between you and the respondent.
2. The relevance of your survey’s subject matter to the respondent.
3. The thought put into the survey questions and how the survey flows.
4. The likelihood that you will use the information from the survey to make a decision.
5. The resources (time, money, and people) you have to implement the results of the study in a timely fashion.

Let me explain each factor so that you can understand how each affects the acceptable survey length.

1. The relationship between you and the respondent.

As an example, a very loyal customer with extensive knowledge about your product can stay engaged through a longer survey. The stronger the relationship, the longer a survey can potentially be. On the other hand, if a marketer is reaching out to the general public for opinions on a new product with no prior relationship to the target audience, he or she needs to be looking at a keeping the survey short and focused.

2. The relevance of your survey’s subject matter to the respondent.

If you are doing a survey to get opinions on golf products from golf enthusiasts then the survey certainly can be longer than a community survey on the workings of its local government. In general, people don’t mind spending a few minutes on a survey of passing interest, but will spend quite a bit longer on a survey that covers a topic near and dear to them.

3. The thought put into the survey questions and how the survey flows.

Respondent-friendly surveys are easier to complete and thus reduce survey fatigue. While writing the survey you need to think about how much effort it will take a respondent to answer the questions. The question type (e.g. ranking questions), the content involved, and the length of the survey questions all deserve consideration when aiming for a respondent-friendly survey.

4. The likelihood that you will use the information from the survey to make a decision.

It may seem strange to think that your actions after the survey can affect how a respondent feels about your survey, but a respondent can usually tell how useful the information they are giving will be. If your respondent is wondering how the information they provide can possibly be used or is wondering why certain questions are being asked, they will likely lose interest quickly.

5. The resources (time, money, and people) you have to implement the results of the study in a timely fashion.

If a respondent sees action taken as a result of his or her input then they are more likely to complete a future survey thoughtfully. Many surveys are used to identify improvement opportunities, but budgets and/or resources may limit the ability to implement the opportunities identified. It might take years to act meaningfully on information gathered from 100-question survey, for example, even if they were all essential questions and engaging for the respondent. In a case like this, the survey should be broken into a number of shorter surveys and the results of each can then be acted on in a timely manner. This will keep your respondents interested in taking part in each of your surveys.

These are five factors to consider when deciding whether your survey is too long or not. You should also obtain feedback by surveying a pilot group from the target audience to help with the same decision.

In my next blog post, I’ll look at things you can do to enhance survey response rate.

Image courtesy of lisalou66 – Flickr, Creative Commons (Attribution)

“Dear Boulder Resident,

[...] To get a representative sample of people living in Boulder, this questionnaire should be completed by the adult (anyone 18 years or older) in your household who most recently had a birthday.

Please have this person take a few minutes to answer all the questions and return the survey in the postage-paid envelope [...].”

Hmmm. A few minutes? Are they kidding? The survey had 170 questions and 8,500 words on it. Spending just 6 seconds on each question (and how important it is to the City of Boulder) means the survey would take more than 15 minutes. Thoughtfully considering each question could take hours.

A survey like this undoubtedly creates “survey fatigue,” but that alone is probably not its biggest problem. If done correctly there are situations where a 170-question survey can be appropriate – but that’s a topic for another article.

The biggest problem with this survey is that the authors made a number of mistakes when putting it together. Here’s the list of the most common mistakes survey authors make from my previous blog entries:

10 Common Mistakes Made When Writing Surveys

1. Having little or no understanding of the target audience
2. Providing multiple choice lists that are too restrictive
3. Requiring answers to all questions (online surveys only)
4. Asking too many open-ended questions
5. Using ranking questions incorrectly, or overusing them
6. Asking unnecessary questions
7. Asking too many questions
8. Asking two questions in one
9. Making questions too general
10. Putting too little thought and planning into writing the survey, period

The survey above has been useful in helping me refine the list of common mistakes (which supports the old adage that nothing is ever a complete failure, as it can often serve admirably as a bad survey example). To wit, I have refined the list (refinements shown in red below):

10 Common Mistakes Made When Writing Surveys

1. Having little or no understanding of the target audience and what information they will be able to provide
2. Providing multiple choice lists that are too restrictive
3. Requiring answers to all questions (online surveys only)
4. Asking too many open-ended questions (or asking open-ended questions that are not useful)
5. Using ranking questions incorrectly, or overusing them
6. Asking unnecessary questions or ones that won’t produce usable information
7. Asking too many questionsand/or including disjointed laundry-list questions
8. Asking two questions in one
9. Making questions too general
10. Putting too little knowledgeable thought and planning into writing the survey, period

Below are some examples from the survey that support this refinement of my list of mistakes.

First, an example of the type of open-ended question that is not useful:

What do you think should be the top three priorities of the Boulder City Council in 2012?

This type of question will never lead to gathering useful information, except by accident. The reason for this is the respondent has not been given any context within which to answer the question, such as: What’s on the City’s docket? What kinds of things can and will the Council address? Are there budget constraints? It is not fair to respondents to require them to guess at context. This type of question is what gives open-ended questions a bad name.

Next is an example of one of several “laundry-list” questions that were included in the City of Boulder 2011 Community Survey:

Give your respondents a break, literally. At least break the list up into smaller pieces. A more thoughtful way to present the question would be to group items into topic areas.

In summary, the main mistakes in the City of Boulder 2011 Community Survey were:

Mistakes 4 and 7 were the result of survey mistake #10: the authors did not put enough knowledgeable thought and planning into writing the survey. As a result, the data collected by this survey will not necessarily provide the information they want. There is really no way to know exactly how survey fatigue has affected the data collected. A survey of this length will certainly exclude a significant portion of the target audience.

I will address the issue of appropriate survey length in my next blog post. Happy Survey-Taking!

Image courtesy of opensourceway – Flickr, Creative Commons (Attribution)

The terms “engagement survey” and “satisfaction survey” are often used interchangeably; and in my opinion, wrongfully so. They are quite different – and savvy organizations that understand the difference will always want to measure levels of employee engagement over satisfaction.

Commitment is a key component of engagement that does not exist with satisfaction. Engaged employees love their work and continually look for ways to improve their work experience, and that has implications upon your customers. Satisfied employees, meanwhile, are typically happy if things stay the same, expectations are not changed and there is certain predictability about the work environment.

I know my wife would have been quite surprised if I presented her with a “satisfaction” ring rather than a sincere proposal to share a lifetime together. The term engagement implies a joint commitment toward the long-term success of the relationship. Satisfaction implies a general sense of how one party rates the relationship.

Organizations must “satisfy” employees; it is something employers do for them. Those individuals who say they are “satisfied” may or may not be actively involved in improving outcomes and looking for ways to enhance quality and improve productivity. Satisfied employees can come to work, collect a check and go home. Back to my relationship example – would you rather have your significant other say he or she is satisfied, or would you rather hear them say they are excited and eager and 100% committed to a successful relationship?

Organizations interested in engaging their employees are seeking active involvement from their employees, and employees are seeking work environments that are most conducive for well-being, development, recognition, and effectiveness from their organization’s leaders. The commitment is shared. Both parties understand that when all employees are engaged they are actively involved in business and customer outcomes. There is a shared responsibility for success and a sense of energy, focus and commitment that all people feel for their work and the organization.

Are you satisfied with your job? That question can be answered affirmatively by many people who merely do not hate their jobs. Does your organization value you? Is your organization effective? Does your organization engage you in creatively solving problems and listen to your ideas? Does your organization spend time and money developing your skills? These questions are tougher because it measures your level of engagement. Please do not assume satisfaction surveys and engagement surveys measure the same outcomes – they do not.

How to Create Employee Satisfaction Survey Questions That Measure Engagement:

1. When creating an employee engagement survey, it’s a good idea work with experts.
2. Choose a survey with validated survey questions that are shown to reliably measure engagement. Simply picking questions from a list ignores issues of validation and reliability.
3. Avoid including random questions on topics that have very little to do with the employees’ level of engagement. For example, asking about the quality of the cafeteria food and employee parking availability is hardly linked to engagement, making these types of questions superfluous.
4. Be wise. Dig deeper and maximize your survey by asking questions that will get to the heart of the issue.

Image courtesy of LWVC – Flickr, Creative Commons (Attribution)

The topic was political. The purpose seemed to be to poll the opinions of voters. As I first answered the call, I hesitated to continue, but given that I’m in the survey business I went forward. (I usually try to participate in surveys at least until my patience runs thin.)

There was a slight introduction asking if I was registered to vote, etc., but not much else. The survey robot seemed friendly but clearly refused to laugh at any of my jokes. (That should have been a clue.)

I answered the first few survey questions without much trouble. All required Yes-No responses. Questions like: “Did you vote in the last election?” “Do you think Boulder’s City Council is doing a good job?” Then things started to get more involved. Soon there was a question I couldn’t answer without some thought. “Hmmm, I’m not sure,” I said. The robot responded, “This question requires a ‘Yes’ or a ‘No’ answer.” I hesitated again, not knowing how to answer and the robot seemed to get annoyed, replying, “This survey will end unless I receive a ‘Yes’ or a ‘No’ answer.” Suddenly, I panicked. “Yes,” I said.

Whew! That seemed to calm down the robot and the survey continued with a few more questions before there was another for which my real answer was not “Yes” or “No.” It was probably something like, “I’m not sure” or “I haven’t decided yet” or “It depends” or “I don’t understand the question.” Once again the robot insisted, “You must answer ‘Yes’ or ‘No’.” As I tried to explain to the robot why I couldn’t answer “Yes” or “No,” the line went dead and I assumed that meant the survey was over.

Lots of Questions

As I hung up the phone, a host of questions rattled through my head:

• How do the survey sponsors expect to use the data from a survey like this?
• Do the sponsors really think the data they get is unbiased?
• Who writes these surveys, anyway?
• Do the survey authors really think it doesn’t matter what frame of mind people are in when they respond to their survey? Do they even think about that?
• Do the survey authors really think all of their survey questions only have the possibility of a Yes-No response?
• Do the survey authors really think that a respondent’s true feelings, opinions, and attitudes are like items on a shelf, ready for a robot to pick up and put in a box?
• What were they really trying to do with their survey?
• Isn’t it likely that the analysis of the data will ignore the method and circumstances under which the data were collected?
• Will the survey authors give any thought to the data’s validity? Or will the survey data instantly become meaningful (in their minds) once they see numbers or graphs on a page?

That is a lot of questions that went through my mind in the few seconds it took me to put down the phone.

The Mistakes This Survey Made

Clearly, mistakes were made in putting this survey together. I think it would be instructive to go back to my last article, the 10 Common Mistakes Made When Writing Surveys, and see which mistakes from the list were made here.

Let’s take a look. Here’s my list of the common survey mistakes:

1. Having little or no understanding of the target audience
2. Providing multiple choice lists that are too restrictive
3. Requiring answers to all questions
4. Asking too many open-ended questions
5. Using ranking questions incorrectly, or overusing them
6. Asking unnecessary questions*
7. Asking too many questions*
8. Asking two questions in one
9. Making questions too general
10. Putting too little thought and planning into writing the survey, period.

(I’ve bolded the ones I think the authors of the robotic survey made. The ones with stars are the ones about which my premature exit from the survey preclude me from making a clear determination.)

Items 1-3 and 9 are all closely linked. The authors’ understanding of their target audience seems minimal because they didn’t allow for more response choices and the choices they did provide were too restrictive, even to the point of requiring every question to have a “Yes” or a “No” response. The survey questions become too general from the restrictive responses.

The end result of these four mistakes is mistake number 10. The authors did not put enough thought into who would answer the survey, how their survey method would affect the response, and how the information was to be used.

Happy Survey Taking! I’ll talk about the other survey I attempted to take in my next blog post.

Photo courtesy of Jonty Wareing – Flickr, Creative Commons (Attribution)

As I said last time, although it’s nice to know the most common survey mistakes made, the real value is in understanding how to avoid them. You can do that by understanding why each is a mistake.

6. Asking unnecessary survey questions

It is easy when writing surveys to fall into the trap of wanting to know everything. However, as a survey writer you owe it to your respondents to only ask questions from which the resulting data will be used to take action or make a decision. Respondents can sense when you are asking a question that isn’t needed and will not be used. The two most common types of unnecessary questions are asking about something that has already been decided and asking about things over which you have no control.

7. Asking too many questions

This survey mistake appears straightforward, but is often misunderstood. I’m often asked how many questions can be asked on a survey; however, there is no magic number for the right number of survey questions. The two limiting factors are: 1) The commitment and attention span of the target audience, and 2) the resources and time the survey owner has for acting on the information received.

For example, when it comes to commitment to a survey, you can’t ask as much time of a general audience without an investment in your subject than you can of a loyal customer or a dedicated employee. If respondents are committed to your subject and are kept engaged by the survey instrument, they will spend the time it takes to complete a long survey.

The second limiting factor, the resources and time the survey owner has for acting on the information received, usually overrides the first. The only reason to do a survey is to use the information obtained. Thoughtfully using this information and making changes or improvements takes a considerable amount of time and effort. It is better to implement shorter surveys more often than to implement one survey that produces five years of work.

8. Asking two survey questions in one

This is a great way to frustrate your respondents and give you ambiguous data. For clarity, let’s look at two examples. The first is the question: “Please rate the technician’s knowledge and professionalism.” This is clearly two questions. The technician’s knowledge may be great and his professionalism lousy.

The second example, asked of a high school counselor, illustrates a more subtle way of making this survey mistake. “Do you interact with your students’ parents about college?” Again, this is really two questions: “Do interact with parents?” and, if so, “Do you talk about college?” This mistake can be fixed in either of two ways. You can ask both questions separately or you can ask the one question and adjust your response choices to include both “I don’t interact with students’ parents” and “I interact with students’ parents but not about college.”

9. Making questions too general

The problem with questions that are too general is that two respondents can sometimes answer the question the same but for completely different reasons. For example, “Do you believe wireless devices can cause health problems?” Clearly, there are many ways for people with very different views to answer this question “Yes”. One person may feel it is a remote chance while another may think it is an absolute certainty. The quality of the information obtained from a survey depends on asking focused, unambiguous questions specific to the survey objectives. A better approach for determining people’s beliefs about the dangers of wireless devices might be to ask, “Do you curtail your use of wireless devices specifically to avoid risk to your health?”

10. Putting too little time and effort into writing the survey, period

The fact is, it is easy to write a survey with lot of questions and send it out to a broad group of people. The difficulty is getting usable information that can help with solid decision-making. Every question in a survey needs to be well thought out and evaluated against the survey objectives and the target audience. Too often the results from a survey raise more questions than they answer because the questions weren’t well thought out, reviewed, tested and reviewed again. Extra effort spent writing your survey will pay big dividends when using the data.

With that in mind, I thought I’d share with you the ten most common mistakes made when writing surveys.

Keep in mind that while it is nice to know the most common mistakes made, the real value is in understanding how to avoid them. In order to do that it is important to understand why each is a mistake. I will address them one by one in this and my next article. To start, here are the first five.

1. Having little or no understanding of the target audience

This seems straightforward: how can you write an effective survey if you don’t understand much about who will be completing it? The issue is making a connection with your respondent. The survey writer should know as much as possible about the attitudes and beliefs of the potential respondent. The wrong wording can offend respondents or just steer them away from what you intended. Often, too much focus is placed on what information the survey writer wants to get back, and not enough on what information respondents can provide.

2. Providing multiple choice lists that are too restrictive

It is always a good idea to include answer options that include “don’t know” or “uncertain,” “not applicable” and “other.” Respondents become frustrated when they don’t see their response in a multiple-choice list. The idea is to weed out respondents that don’t have a clear opinion from those that do. Otherwise you risk contaminating the good responses.

3. Requiring answers to all questions (online surveys only)

Nothing is more annoying to respondents than having offered their time to complete a survey and then having trouble progressing through the survey. A few skipped responses is not going to change your results – and ultimately you cannot force respondents to answer a question. If they want, a respondent can just close their browser and forget about your survey.

4. Asking too many open-ended questions

It is good to have comment fields, but too many open-ended questions makes it appear that the survey writer did not want to put in the effort to create easy-to-answer questions focused on survey objectives. The main purposes for open-ended questions in a survey are to provide respondents an outlet for thoughts and opinions that may otherwise distract them from thinking about the questions asked, and to add richness and understanding to the quantitative results obtained.

5. Using ranking questions incorrectly (or overusing them)

The inexperienced survey writer will often overlook the fact that ranking questions are difficult for a respondent to answer and even more difficult for them to analyze and interpret. The mistake made is to assume that the best way to ask a question of one person is the best way to ask the same question of many people. If I have only one customer, then I would want that customer to rank their priorities (one to whatever). That all changes, however, when I have to consider the priorities of many customers together. Asking respondents to select their top three priorities (or two or four, etc.) creates a natural ranking when the data is summarized.

In my next post, I’ll finish my list and address the last five most common mistakes made when writing surveys.

The interested reader might want to refer to Edward R. Tufte’s 2001 book, The Visual Display of Quantitative Information for a more detailed account on how to make sure your reports have “graphical integrity.” It is an excellent book.

Tips for Maintaining Graphical Integrity

1. Always include the number, N, of observations on your chart.

This simple addition to any chart (see example below*) is critical to understanding the data presented. Including the number of observations on your chart let’s the reader reconstruct your data from the percentages given. It also provides an indirect measure of the variability one might expect to see.

2. Use counts instead of percents when N is small, fewer than 10 to 15.

When you have a very small number of survey respondents it is better to plot counts than the percentage of counts (see following two charts). Even though it is possible for the reader to calculate the counts on his or her own, going to counts for small numbers helps the readability of a chart by eliminating the extra step. The human brain can read and interpret small numbers quickly and without ambiguity.

3. Don’t overload your chart or graph.

It is important that your reader be able to get information quickly and efficiently from your chart of graph. There is often a tendency to add too many items to your graph that do not add to the data’s message. Often “non-data” items (e.g. 3-D images) are included that can detract from or mask the true information. Even if you include only data on the chart, it is easy to overload it will information.

The following example shows a common mistake, which leads to a chart that is difficult interpret.

The most critical piece of the data from the chart above is the trend information. Therefore it is best to emphasize that element of the data. If, in addition, you combine the percentages for “Much better” and “Better” as well as for “Worse” and “Much worse,” the chart becomes much easier to read and interpret (see below).

4. Show all of your data.

The complete truth about a data set usually extends beyond what you can fit on a single chart or graph, but you need to strive to show all available data. For example, suppose a manufacturer comes out with a new product release and wants to see what effect the new release has had on service calls. The chart below shows the number of product service calls for the year before the release compared to the same number after the new release.

This seems like a reasonable approach but the two years of data do not begin to tell the whole story. Showing all of your available data better represents the true situation (see chart below). The two charts give substantially different pictures.

As I’ve talked about many times, graphical displays of data can be powerful communication tools. If you follow these few tips it should help you provide clear and concise information and avoid charts that are confusing or misleading to your readers. In my next article I’m going to return to the topic of writing surveys, by going over the top ten most common mistakes made when writing surveys.

*NOTE: The charts and graphs in this blog post were generated by exporting a CSV of SurveyGizmo survey data into Excel.

String Variables to Numeric

While SurveyGizmo allows for changing of variable format in SPSS, sometimes I end up with string variables that really want to be numeric. This can easily be accomplished with a short line of SPSS syntax.

Let’s say you wish to convert the variable named “String_Zip” to a numeric variable entitled “Numeric_zip”. The following code achieves this:

RECODE String_Zip (Convert) INTO Numeric_zip.

EXE.

While this is simple, the default numeric value in SPSS is a scale variable.

To help remind yourself not to take the “average” of zip code you might want to change the variable level to Nominal. This can be achieved with the following line of syntax:

Variable Level Numeric_Zip (Nominal).

As a reference, below are the other variable levels supported in SPSS

• Variable Level Numeric_Zip (Ordinal)
• Variable Level Numeric_Zip (Scale)

Numeric to String

If you need to convert a numeric value to a string there is one additional step. You must first declare the string variable. The following creates a string variable named String_Zip2 with a width of 5 characters.

STRING String_Zip2(A5).

Now that you’ve created the variable, you can convert the numeric value to it’s string counterpart.

COMPUTE String_zip2=STRING(Numeric_zip,N5).

EXE.

Dropping unused variables

Want to drop the old numeric variable from the data set? Simply use the match files command with the drop statement.

Match files file=* / DROP Numeric_Zip .

EXE.

Thank you again, Joe, for teaching us a few quick and easy tricks using SPSS. If you’re out there reading this and would be interested in becoming a SurveyGizmo survey expert, please send us an email at marketing@sgizmo.com and we would be more than happy to start working with you.

The one question to ask when interpreting trend data
There is really only one question to ask when interpreting trend data and it is, “Has a change occurred?”

After that question is answered, there are a host of follow-on questions to ask, such as, “Why haven’t we improved?” or “Why is customer service satisfaction declining?” and so on. None of these questions can be addressed until we have determined whether or not a change has occurred.

Anytime we measure something more than once, we will see variation. Variation always exists. It is neither bad nor good.

That is why rather than asking the question, “Is there a variation?” the more pertinent question is, “Has a change occurred?” Even better, we can phrase the question slightly differently to read, “Is the variation we see in the data just normal variation or has something significant changed?”

Practice looking at trend data!
We don’t expect our measurements to always be the same even when nothing has changed, but how much do we expect them to vary?

There are statistical tools that can help to answer this question, but ultimately it takes practice looking at data; looking for and recognizing patterns, asking questions, investigating the numbers behind the data, and understanding how the data were collected. There is no substitute for experience when it comes to looking at trend data, or any data for that matter.

Let’s start by looking at the three examples in Figures 1, 2, and 3 below. Tips:

1. Ask “Am I looking at normal variation or is something significant going on?”
2. Try not to focus on individual points. Instead, look at the graph as a complete picture.
3. Look for patterns or anomalies.

There doesn’t seem to be a lot going on with this data. After three increases in a row starting in 2002 we might have been excited, but would have wanted to withhold the celebrating. The satisfaction level over the ten years looks fairly flat, hovering around 80%.

This example appears straightforward. Something changed for the better after 2007! The data from years 2001-2007 look relatively flat hovering around 58%, but the satisfaction levels after 2007 clearly increased.

This data look erratic, especially when compared to the previous examples. Prior to the third quarter the response rate may have been higher and dropped, but we only have two data points to consider so not much can be said about that. Next, the variability across quarters 3 through 8 seems large next to that for quarters 9 through 14. That should be investigated. The level of the response rate does not seem to have changed.

Optional: The Has-A-Change-Occurred (HACO) Chart
For those who want a more rigorous way to answer the question about whether a change has occurred, one can look at a HACO chart, perhaps better known as a control chart. The theory and application of HACO charts is quite extensive and complicated, but I’ve found one type of chart that is both easy to use and versatile. That chart is an Individuals Moving Range (IMR) chart.

An IMR chart, like all HACO charts, has an average line, an upper limit, and a lower limit. Calculating these values is straightforward, but it is never a good idea to make blind calculations without some understanding of the tool you are using. That is a discussion for another time.

To create an IMR chart, follow these steps:

1. Sum the absolute differences between one data point and the previous point (moving ranges of size 2).
2. Divide the sum obtained above by the number of data points minus one to create the average moving range, MR_ave.
3. Calculate the average of the points, X_ave.
4. Compute the lower limit as X_ave – 2.66 * MR_ave and the upper limit as X_ave + 2.66 * MR_ave.
5. Plot the data and add the average, upper, and lower limit lines to the chart.

Let’s see how the IMR chart looks for the three trend data examples given above. They appear in succession below.

The HACO chart tells us that nothing has changed with the data in Figure 4, that a change did occur in the data in Figure 5 and that the response rate has not changed in Figure 6. What about the issue of reduced variability that we saw in Figure 3? This is an important issue, but needs to be resolved by a different type of control chart, the Moving Range (MR) chart (which is a subject for another article).

In summary, let me remind you that it takes practice to read and interpret trend data. If you are not used to looking at graphical displays of data, it can be hard to interpret what you are seeing. Next time I’ll continue to look at the power of graphical displays in interpreting data.