“Ed
Confidence interval and margin of error do correspond to specific metric (e.g., customer satisfaction or likelihood to recommend) as you point out but level of significance applies for a hypothesis test.
-rags”

Thanks Rags, you make a good distinction. Perhaps I can cover in a future article the topic of confidence intervals, margin of error and level of significance in a market research setting.

I received a comment early that is gone. I’ll repeat it here:

From Rags Srinivasan:
“I wonder if they mean level of confidence and margin of error when they say stat significance. A decision maker may not know the technical nuances, and that is probably why they want to hire an expert do the survey for them. I think an appropriate reply to them is not what do they mean by stat significance but what are they trying to do differently with the data and walk them thru the required confidence level or statistical significance.”

My reply was:
Thanks for your comment.

Yes, most of the time they are asking about level of confidence and margin of error, but my point is those concepts only make sense in the context of an individual survey question and in particular a test statistic and a hypothesis. There is a misconception out there that a survey can be “statistically significant” and that there is a certain sample size that will make things “statistically significant”.