4 hints to avoid your customer satisfaction survey from failing

In our latest article we discuss the Goodhart’s law and what it meant for customer satisfaction measurement. In short, if satisfaction scores become a company’s objectives, the likelihood is high that results will be biased.

Contact us for your B2B market research

A very good article explains the origin of this bias (it’s called reflexivity) and even details other “laws” that point to the same effect :

• Goodhart’s Law is most succinct: “When a measure becomes a target, it ceases to be a good measure.”
• Campbell’s Law is the most explicit: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
• The Cobra Effect refers to the way that measures taken to improve a situation can directly make it worse.

The Campbell’s law explains particularly well why the Audi satisfaction measurement in car dealerships got biased. Employees are incentivized on satisfaction scores, which leads to deviant behaviors and pressure on customers.
That being said, he underlying question is “how can a satisfaction measure be developed without being at risk of becoming biased ?”. And when you read the Belfius case, you understand it’s a real challenge. Here are our 4 recommendations.

1) Build a model to explain customer satisfaction

A satisfaction survey is not something you can write on the back of an envelope. A satisfaction survey must be drafted, tested, and ideally a model developed that explains how satisfaction is build. Technically, this is often a SEM (structural equation modelling) approach that enables to detect which components do influence customer satisfaction. Look for instance at this academic article on the ASCI where you’ll find on page 8 the underlying model for customer satisfaction.

2) Give incentives for what employees can really influence

Based on the model defined above, it becomes possible to identify those areas which contribute the most to satisfaction, and train employees to influence positively those dimensions. If employees understand their role and contribution to overall satisfaction, they’ll be able to make a difference.

Only for our subscribers: exclusive analyses and marketing advice

Email address  *

Subscribe

By signing up, you agree to our Terms of use and privacy policy.

"I thought the blog was good. But the newsletter is even better!"

Esteban Hendrickx, art director

3) Never let employees administrate satiusfaction surveys themselves (especially if they get incentives)

To avoid the realization of the Campbell’s law, don’t let employee administrate satisfaction surveys themselves. If you do, they may put pressure on the respondents which will lead to biased results (even more if the employees gets incentives based on satisfaction score).

4) Let customer satisfaction be measured by a third party

The best thing to do is to let a third-party provider develop and administrate a measurement tool. Keep in mind however that this survey should be developed on solid basis so that you can really leverage the results. We see too often customers with predefined questions asking a third party to administrate the survey. This doesn’t make sense. The Art of satisfaction is really to be found in the definition of the model and the questions to assess this model.