30 April 2014 685 words, 3 min. read Latest update : 24 March 2021

The top 3 reasons why customer satisfaction surveys fail

By Pierre-Nicolas Schwab PhD in marketing, director of IntoTheMinds
In this post I’d like to remember the main reasons what your results to customer satisfaction surveys will not be representative. Eventually I’ll complete this post with another one: a reflection about Belfius’ advertised 95% customer satisfaction rate. 3 top […]

In this post I’d like to remember the main reasons what your results to customer satisfaction surveys will not be representative. Eventually I’ll complete this post with another one: a reflection about Belfius’ advertised 95% customer satisfaction rate.

3 top reasons why customer satisfaction surveys fail

One thing you should keep in mind of customer satisfaction survey is that the design of your questionnaire will very much influence the answers you’ll get.

Among the many different factors that can trigger errors in the results, let me cite those 3 top errors that I find again and again in satisfaction surveys.

Error #1: The questionnaire is too long

I remember a satisfaction survey given by the national railway company that took me more than 20 minutes to fill in. It was four A4 pages long. Do you really think the answers to the questions on the last 2 ages (i.e. 50% of the questionnaires) will reflect what I really think ?

Actually, after 20 questions respondents are bored and will rush through the questionnaire without really reading, thinking about their answers. In scientific marketing research we actually control for fatigue (but that’s another topic). A satisfaction questionnaire is very focused (on satisfaction) and can be answered in much less time.

My top advice for designing your customer satisfaction questionnaire : 10 questions max, 5 minutes.

My #2 advice for your satisfaction survey : start directly with this question “How likely is it that you would recommend [your company] to a friend or colleague?”. It’s called the net promoter score (NPS) and it will help you capture some profound learning points.

Error #2 : questions are biased

There are many reasons why questions can be biased. We uses some actual examples when we give market research trainings (if you read French you can follow this speakerdeck link).

One obvious reason is the very formulation of the question. Example: “don’t you think it may be a good idea to provide in-company trainings on the ad hoc reconciliation of invoices in a B2B context ?”. If you say it …

In the question above the question is stated negatively (“don’t you think”), is very long, and uses several specific professional (“reconciliation”) and non-professional (“ad hoc”) words that will impede the understanding of the question.

My top advice : use only simple word and try to limit question to 10 words. If you can’t achieve it … try again.

My advice #2 : test your questionnaire on a first sample of people in face-to-face interactions to observe their reactions. You’ll see on their faces if something is not understood.

Error #3 : you didn’t ask the good questions

It may be obvious but do you actually know which questions you should ask ? Most people think they do.

With more than 10 years experience in the field I can say that nothing is less true than that. You actually don’t know what to measure (expect the NPS which is a good first question to start with) unless you have done a pre-research (of qualitative nature).

If you don’t follow the qualitative research / quantitative research sequence, you’ll end up with only a vague idea of why your customers are satisfied or not. I must admit however that it’s very tempting to sell biased questionnaires (and biased satisfaction results). Most companies outsource this task, and most employees of those companies don’t know the specificities of qualitative vs. quantitative research. They rely on third parties which don’t have the customer’s best interest in mind : they prefer to sell a pre-designed satisfaction survey (it’s cheaper) rather than having the pain of going through the very definition and design of this satisfaction survey. We’ve lost one contract because of this. We refused to sell the quantitative part without the design of the questionnaire; in the end we were more expensive (logically) and the prospect preferred to go low-cost and have a 50-question survey administrated. I let you imagine the quality of the results …

My top advice : investigate first (with qualitative method) what your customers value. Then proceed with a quantitative analysis



Posted in Marketing.

Post your opinion

Your email address will not be published. Required fields are marked *