3 June 2020 591 words, 3 min. read Latest update : 5 November 2023

Satisfaction Surveys: How to Get Worthless Results

By Pierre-Nicolas Schwab PhD in marketing, director of IntoTheMinds
Mistakes can be made that will taint the validity of your satisfaction surveys. In this article, find out how a car dealership influences customers' scores and gets results that don't reflect reality.

Customer satisfaction surveys are all very well, but you must know how to use them. I’ve often told you what to do. Let me show you an example of what not to do. This is a real-life example from the dealership of one of the world’s 3 leading car brands (Toyota).

Contact IntoTheMinds to conduct your satisfaction surveys

A misused satisfaction survey

In the example below, the company thinks it’s measuring a Net Promoter Score (NPS). However, it misuses this satisfaction survey to influence its operational and financial results. The influence it exerts on responses removes all spontaneity from the exercise. As a result, the measurement is no longer reliable.

The problem I describe in this article is widespread and can take many forms. Here, clear instructions are given to the respondent, but the biases can be more subtle:

  • omitting part of the clientele from the survey
  • adding colors to the measurement scale so that the respondent associates an unconscious color code (red – orange – green) with the nature of the score
  • captioning only part of the scale to draw the respondents’ attention.

It may be advisable to call in a third party to avoid this kind of drift. A survey company, for example, can act as a neutral partner in data collection. They can set up a methodology for accurately measuring customer satisfaction at different stages of the customer experience. By mixing different methods of collecting responses, it will also be possible to go beyond relying solely on customers who spontaneously give an opinion.

Customer satisfaction survey biased answers

Satisfied? Give us a score of 9/10 or 10/10″. This satisfaction survey in a car dealership needs to be better used, and the scores collected will not reflect reality.

Problem n°1: biased answers

The dealer must, as part of his contract with the brand he represents, provide indicators of the quality of his work. Customer satisfaction is one of these indicators. The dealer is penalized if the satisfaction score is not high enough. This is a widespread practice in many networks.

The picture below was taken in a car dealership. Result of the race: the dealer tells you what to answer in the satisfaction survey (“give us a score of 9 or 10/10”) and thus takes away all the value of the satisfaction survey. The results are distorted (biased in marketing language). The parent company then analyses the wrong figures and has the illusion that everything is fine.

This is a perfect example of Goodhart’s law in action.

Problem n°2: selected respondents

The purpose of a satisfaction survey is to get an accurate picture of reality. As we’ve seen, it’s already a bad start when we tell the customer what to answer. But it’s messed up when we select the customers to whom we send the questionnaires.

After my last visit to this car showroom, I had a dispute with the manager who wanted to make me pay for a service I hadn’t requested. The tone rose, and I was asked to leave the showroom. I have never received a satisfaction questionnaire. As a result, I couldn’t give a bad grade, and the parent company will not be made aware.


In conclusion

This real-life example shows that a poorly conducted satisfaction survey is worthless. Two major mistakes were made by the car manufacturer in question:

  1. The dealer’s income depends on the satisfaction rate: the results are influenced by the manufacturer.
  2. The car showroom can administer the satisfaction questionnaire itself: the results are faked. Bas du formulaire


Posted in Marketing.

Post your opinion

Your email address will not be published. Required fields are marked *