Discovering Different Ways to Report on Matrix or Table Data

One of the most commonly used question types among survey designers is the matrix question, also known as a table question. This question type allows the respondent to pick one attribute from a list of attributes that are rated using the same Likert scale.

Here’s how to use this question type to collect survey data, and, more importantly, how to report on the data once you’ve got it. 

Example of a Table Survey Question

Please indicate your level of satisfaction with each of the following aspects of our service.

This type of question is popular for surveys because it is easy to design and easy for the respondent to complete, both good reasons to use it. In this article, I look at the best way to report the data obtained from this question type.

Alternatives

  1. Table with Counts and Percentages:(plus)SurveyGizmo adds “Average %” at the bottom of the table.
  2. Individual Bar Charts: This requires five charts, one for each attribute. I have shown an example of what one would look like above.
  3. Stacked Bar Charts:This option essentially puts all five bar charts on the same chart. This chart was created in Microsoft Excel using the percentages SurveyGizmo gives in the original table. It is one of the chart type options Microsoft products offer and is undoubtedly a part of other charting software as well.
  4. Average* Attribute Ratings Bar Chart: This chart was created by first assigning numbers to the Likert scale levels; 5=Very satisfied, 4=Satisfied, 3=Neutral, 2=Dissatisfied, 1=Very unsatisfied and then averaging all the ratings for each attribute. The five averages are then plotted using a horizontal bar chart not unlike the vertical bar chart in alternative 2.

Advantages and Disadvantages of Tables

Alternatives 1, 2 and 3 all have a key advantage over alternative 4.

From the averages in number 4, one can’t recreate the raw data as you can with the other three alternatives. In addition, averages can be difficult to interpret. The percentages in the first three alternatives have very specific meanings.

If we look at just alternatives 1, 2 and 3, alternative 3, the stacked bar charts, has some distinct advantages. It is less cumbersome than the five charts required for alternative 2 and it is easier to read than the 61 numbers in the alternative 1 table.

With alternative 3, one can make quick comparisons between the different attributes. One measure I like to use when comparing attributes is the percentage of respondents that were either “Very satisfied” or “Satisfied” (the first two sections in the stacked bar chart).

So reporting matrix data can be pretty straight forward, but the problem gets more complicated when you want to look at the same data over time.

This leads to the goal of “one number reporting”.

By “one number reporting” I mean developing a single number that best depicts, for example, your client satisfaction.

One number reporting is always desirable (for simplicity and tracking purposes) and always challenging with the tradeoffs that are required. 





survey reporting ebook




Join the Conversation
  • Andrew Marritt

    I am always uneasy in using means with Likert scales believing that the variables should be treated as ordinal. Would love to hear any strong evidence to counter my fears. I will agree that using means is common.

    I use 2 other visualizations to add to the list, though in truth they are adaptations of two here.

    The first is using a heat matrix type approach with 1. I find that adding colour (or a scaled shape) draws the eye to the important details better than just the percentages / values.

    Second is an adaption of 3. My logic here is that of the 5 options 2 are positive and 2 negative. What the viewer usually wants to know is the balance between the positive and the negative – the size of the centre is often of least interest.

    I therefore use a stacked bar but instead of aligning as you have I centre at the neutral and present the stacked negatives to the left of the central line and the stacked positives to the right. With multiple items this seems to provide the easiest way of seeing the important information skew / distribution information quickly.

  • I feel the same way about means with Likert scales, so I cannot offer anything to allay your fears.

    Your stacked bar chart adaptation is interesting. I have not see that before, but it seems useful although the formatting seems like it would be more difficult.

    Thanks for those additions.

  • Pingback: Net stacked distribution – a better way to visualize Likert data. — OrganizationView()

  • Ed,

    I’ve written up my ‘Net Stacked Distribution’ here: http://www.organizationview.com/net-stacked-distribution-a-better-way-to-visualize-likert-data

    As you noted the formatting I’ve added a section at the bottom on how you would create such a graphic. I use Tableau but I guess the logic would hold in any graphing application.

  • Hey Andrew,

    I like it!! Very slick. I’m going to try and recreate your chart in Excel. If I’m successful (which I should be) I’ll start including it in my analyses and reporting. It will be interesting to see how people like it. (I do already use the color shades as you have, even though I didn’t include that in my article.)

    Thanks!

  • Pingback: Likert Scales and One Number Reporting | SurveyGizmo()