The Importance of Survey Question Order

It’s a rare rainy day in Boulder, CO which reminds me of one of my favorite psychology experiments.

Back in 1983 Norbert Schwarz and Gerald Clore published a study examining how mood relates to more global evaluations of general well-being. Essentially these researchers wanted to know if being in a good mood leads people to report that their life, overall, is going great (or conversely, if being in a bad mood leads people to report that their life, overall, actually isn’t so hot).

The experiment was conducted in the spring on either rainy or sunny days. A researcher in Chicago called participants randomly selected from the University of Illinois at Urbana-Champaign phone directory and asked the following questions:

1. First, on a scale of 1 to 10, with 10 being the
happiest, how happy do you feel about your life
as a whole?

2. Thinking of how your life is going now, how
much would you like to change your life from
what it is now? This is also on a scale of 1 to 10.
Ten means “change a very great deal” and one
“means not at all.”

3. All things considered, how satisfied or
dissatisfied are you with your life as a whole
these days? (with number 10 being the most

4. And, how happy do you feel at this moment?
Again, 10 is the happiest.

That’s all the questions I have. Thank you for
your time and cooperation.

For half of the participants, these questions constituted the entire interview. For the other half of the participants, the researcher asked “By the way, how’s the weather down there?” before asking the four interview questions.

Overall, participants in the study reported feeling better on sunny days and worse on rainy days (no surprise there). What is surprising however, is that when the researcher asked only the interview questions (without mentioning the weather) participants rated their life overall in a manner consistent with the current weather conditions. On sunny days they reported being happier, were less interested in making life changes, and said they were generally satisfied with their life. On rainy days participants reported being less happy, were more interested in making life changes, and said they were generally less satisfied with their life.

Interestingly, when the researcher began the interview by mentioning the weather, participants reported being happy, were not particularly interested in making life changes, and said they were satisfied with their life overall regardless of whether it was rainy or sunny. These participants answered questions similarly to the sunny day respondents in the “no weather prime” condition.

So, attending to the fact that it’s raining outside helps people separate their bad mood (short term affect) from more global evaluations of their life overall. These types of unintended carry-over effect are important to keep in mind when designing surveys. A rainy day is a good reminder that question order can sometimes make a difference in how people respond to questions.

Hope you’re having a great day (and a great life!) regardless of whether it’s rainy or sunny down there, wherever you are.

Norbert Schwarz
Gerald Clore

Join the Conversation
  • Geeks

    Hi Ethan,

    That’s so true. In London, where I live, it’s pretty much always raining! But on the rare sunny days, you obviously notice a big change in everyone’s feeling and their relationship with the others.

    I guess the result of this survey can be used in marketing activities. For example if people are looking for “change” more in rainy days, then marketing campaigns that sell a “change” product or service may be more effective. I will be interested to see such surveys in the future.

  • Pingback: Homepage