As a marketer I love statistics. When used well, they can really hammer home a message. Take this one for example:
73.6% of all statistics used in marketing are inaccurate, misleading or made up.
It’s very compelling isn’t it? When you read that stat, did it resonate with you? Did it grab your attention?
Google it. It’s there. So it has to be true, right?
That’s how most of us consume statistics. It’s how so many of us in marketing use statistics to promote our ideas.
Note that I didn’t source it or qualify it. What if I had rephrased it? What if it’s no longer accurate? What if it wasn’t true to begin with?
Before you go and tweet that 73.6% statistic, you should know it’s made up. You can find it on Google because Mark Suster made it up in a Business Insider article where he talks about very similar point that I’m making here. (Thanks, Mark.)
Mark was discussing the pitfalls of statistics and how they can become folklore. I want to talk about how damaging it can be to the marketing profession and how we can protect ourselves by using statistics wisely.
Misusing Statistics Hurts the Credibility of the Marketing Profession
In a rough draft of my last article, Succeeding as an Entrepreneur, I used a common statistic that stated women account for 85% of all consumer purchases. I’ve seen this stat frequently – perhaps you have too. It appears in major publications like Inc.Com and Business Week.
I didn’t question this statistic until our CEO, Christian Vanek, called me on it. It caught his attention because it wasn’t sourced and the way it was phrased didn’t pass the common sense check in his head.
He referred me to The Wall Street Journal’s article about this very statistic, Who Makes The Call at the Mall, Men or Women? This article debunks this statistic and states that even early sources that use this figure don’t know the original source. It has become an “accepted truth”. It goes on to state that not only are the origins of this number “murky” but the number appears to be wrong.
It was very embarrassing.
My CEO found this funny, but what if a client caught this? What if you don’t have a CEO with a sick sense of humor like his? What if your CEO already sees marketing as a waste of money and catches something like this?
Worse, what if the company you work for bases a market strategy off this statistic? What if you base an ad campaign off it and spend tens of thousands of dollars?
As marketers we already have our work cut out for us in the work place. We don’t need to add fuel to the fire by misusing statistics and damaging our credibility.
Good Statistics Do Not Lie, But They Can Unintentionally Mislead
Mark Twain wrote: “There are lies, damn lies and statistics.” (He attributed this quote to Benjamin Disraeli; speaking of accurate sourcing.)
As someone that works for a company in the survey research field I both disagree with this quote and also see where he’s coming from.
Statistics, at least well-researched statistics, are not lies. In fact, as a branch of mathematics, statistics are a very valid and highly useful way to model the world around us and give market researchers insight into human behavior.
Statistics don’t lie.
The problem is the interpretation of the statistic without knowing the population, intent, question and demographics behind the statistic can be disastrous. Usually, a market statistic is specific and the researcher only intended on the statistic to be used in context of their original project.
It’s when we start reusing statistics and wordsmithing the results that a valid stat can become a lie.
One of our market researchers, Marni Zapin, told me that this happened to her in a previous job with a media company. She had supplied the company’s marketing team with some statistics about the company’s viewership to be shared in their monthly press release.
“[It became] a back and forth game, of me supplying a stat and [the PR team] ‘adjusting’ it to sound better, and me making sure that I saw the final statement to make sure that it was still true!
I supplied a stat: “Of all the viewers of [media company], 55% are looking to buy a new car in the next six months.” The draft of the press release came across my desk with the headline: Find them here! Fifty-five percent of all people that are looking to buy new cars are viewing [media company]!
They took my stat and changed it so completely that it had an entirely different meaning. Instead of being a stat about the people that were viewing [media company], the PR group changed it to be about all people that were interested in buying cars. It may sound better their way, but it was not true!”
Some refer to this as “spin marketing”. As a marketer you want to avoid this as it can damage credibility.
Protect Yourself From Misleading Statistics
Using statistics to lend authority to our message is a very powerful tool that marketers and companies use. We aren’t going to stop doing it, but we can follow some guidelines to keep your audience from misunderstanding a statistic and thereby protect your credibility as professionals.
Here’s our guide for using statistics:
- Find and Review the Original Source
- Don’t Reword the Statistic
- Learn From Wikipedia; Cite EVERYTHING
- Statistics Are Not Like Wine Or Cheese; They Don’t Age Well!
- If It Sounds Too Good to be True, It Probably Is
- Do Your Own Research
When using someone else’s numbers, find and review the original source.
Check that they used a proper methodology and that the data is statistically sound. Also confirm that the data they collected is applicable to the population you’re reporting on for your message.
“Close enough” doesn’t cut it. The group being studied has to be an exact match to the group you are reporting on. Anything else is misleading.
As marketers, we know that small word changes and order can change an entire meaning of a sentence. If you reword a statistic it is no longer valid. Remember, this is where things usually go awry even if we mean well.
Cite the author, study name, time period and additional information that is relevant to the data gathered. It is fine to add this information to the end of your article or paper, as we do for this article.
A statistic is a snapshot in time. It really can only describe results of the study at the time of that study. So look at the age of the statistic. If the statistic is older than a canned good in your pantry you’d feel comfortable eating then it’s too old! Throw it away and look for an updated source.
Do one final check to make sure that the information you are referencing is from a reliable and competent source. If the information just seems to nice and neat, there is probably some spin involved in the reporting. Look deeper.
If you want the freshest and best statistics, do your own research. Learning how to create simple but sound surveys is not rocket science. If you have a basic background in survey methodology running your own studies is very rewarding and can give you the holy grail of marketing statistics; original data!
Statistics Don’t Lie; We Shouldn’t Either
As a marketer or market researcher your credibility is on the line when sharing data. It is easy to misinterpret the data if you don’t know the story behind the numbers. Beware of misleading statistics and arm yourself with data you can stand behind.
Be sure to question and verify before sharing anything. Keep a reporters’ mentality; consider your sources and investigate before you report it.
We would love to hear how you have been duped by false data. Please share your comments here!