Product Feedback

How Data-Driven Executives Release Products With Confidence

Andrea Fryrear
6 min read

This is the second in a three part series on data-driven organizations. You may want to start by reading Data-Driven Strategy & Executives: Pitfalls and Possibilities

Launching a new product may be one of the scariest moments of an executive’s career.

No one wants to be the next Ford Edsel, a $400 million investment that was pulled after three years.

Nobody wants to be remembered like RJ Reynolds’ 1980’s bet of $325 million on smokeless cigarettes that lasted a mere four months.

More recently, of course, there’s the Zune, Joost, and Barnes & Noble’s Nook.

The halls of shame for failed products are long and crowded. So how does a modern executive navigate increasingly complex marketplaces to launch a product with confidence?

If they’re smart, they use any and all data they can get their hands on. More specifically, they use conjoint analysis.


Lowering the Risk of Launching a New Product

One of the simplest ways of ensuring that a new product has a good chance of success is by showing it to its ideal audience and getting their feedback. There are two basic ways of achieving this: actually producing the product, or describing a planned product.

In some cases, producing a minimum viable product (MVP) that’s just functional enough to share with consumers may be possible. This idea was popularized by Eric Ries’ 2007 book The Lean Startup, in which he advocated releasing a basic version of software or products as soon as they work, carefully monitoring customer responses, and then iterating on the design, features, and functionality based on that data.

This often works well for non-physical products like software, but it would have been tricky for Ford to build an MVP version of the Edsel. Consumer products almost always require heavy investment in research and production, even if you’re just making a handful to test.

And even the simplest, cheapest MVP doesn’t represent a well-defined, scientific test.

It is, rather, “an unstructured search for feedback,” which itself is still up for interpretation. The bottom line is that people are bad at articulating why they would (or would not) buy a product.

Conjoint analysis surveys overcome this hurdle by putting multiple versions of an imaginary product in front of consumers and asking which one they would buy. In some cases, you even offer them a choice of buying none of the offered versions.

Running this type of study doesn’t require you to make anything; the only investment is in finding the right audience (and gathering the right number of responses for statistical validity).

And, if your product isn’t easy to describe, you can even show photos or drawings in your conjoint questions.

If Microsoft had put the Zune up against the iPod in a conjoint study, they might have saved themselves a lot of money and embarrassment.


Choosing Your Next Feature With Conjoint Analysis

The same process applies to rolling out new features for an existing product. Before you even begin developing or researching improvements, you can put several permutations in front of the right audience and collect their unbiased feedback.

If you hope to increase customer retention with a new feature, you’d want to use existing customers as the audience for your conjoint study.

If the new feature was going to target a new audience segment, you would need to ensure they made up the respondents.

As with product rollouts, getting this type of quantitative insight before you invest time, money, and resources lets you release with a much higher level of confidence.


Conjoint Analysis for Pricing Products

One of the most important outcomes of a conjoint analysis survey is quantifying exactly how much various features (including price) affect consumer choice. So, when it’s time to price a new product or new feature, executives who use conjoint analysis know precisely how cost interacts with other factors in getting people to buy their product.

This data then allows them to run a market simulation, revealing how various combinations would perform. You can find out more in this introductory guide, but the final outcome is spreadsheet that you can manipulate to simulate the market share of different products:

You can download this spreadsheet (and learn more about the magical math that powers it) in SurveyGizmo’s Help Documentation.

Keep in mind that using conjoint analysis to determine pricing works best in evolutionary rather than revolutionary markets. Evolutionary markets are those “in which products currently exist, customers currently purchase, and products are evolving.”1 Changes to products in these markets represent subtle shifts in features, improvements rather than disruptions to the status quo, which is what you would find in a revolutionary market.

Evolutionary products make up more than 98% of the new products on the market, so chances are this is where you fall.

In these markets, customers are aware of existing products, and they can therefore conceptualize different combinations of attributes and predict their benefits.

As Tim Smith argues, “in evolutionary markets, customers hold sufficient information that is critically required to make informed statements regarding their preferences, and therefore executives can reliably conduct conjoint analysis.”


All the Research in the World Won’t Protect Some Products

As we saw in Part 1 of this series, simply having data and performing analysis isn’t enough. Executives and organizations who won’t heed the conclusions of data can derail even the most meticulous research plan.

“It’s a question of culture,” Vineet Mehra, President of Global Marketing for J&J Consumer Group says. “There is a learning curve involved in embracing analytics for the business leaders. They have to learn to trust the data and that can be a serious challenge.”



Pricing Strategy: Setting Price Levels, Managing Price Discounts, and Establishing Price Structures by Tim Smith

Start making smarter decisions

Start a free 7-day trial or speak with a member of our sales team.

Start a free trial