An online survey is an essential part of any proper market research. While it has become much easier thanks to the Internet to carry out a quantitative survey than it was 20 years ago (the blessed time of paper questionnaires!), the fact remains that preparing a good questionnaire cannot be improvised. This is not an exercise that can be done on a corner of the table. In today’s article, we have compiled a list of 10 mistakes that you should avoid entirely. If you have any questions or comments about this list, please do not hesitate to contact us or leave a comment at the bottom of the page.
Summary
- unvalidated scales
- less than 300 responses
- use of social networks
- questionnaire length
- repetition of the essential questions
- focus on important issues
- language
- convoluted formulations
- vague formulation
- reward
1. Use non-validated scales
The measurement of complex concepts (such as satisfaction or loyalty for example) requires the use of so-called “measurement scales”. These scales consist of a series of questions to understand the different aspects of this concept. There are validated scales for a large number of theories. Feel free to ask us about this if you need more information or advice.
Also keep in mind that for some particularly complex concepts (such as customer experience), there is currently no precise definition and therefore no scale that is unanimously accepted.
2. Be satisfied with less than 300 responses to your online survey
Some online survey platforms maintain the illusion that market research is only a methodological formality. Do a little questionnaire on a corner of the table, send it by email to friends and publish it on your social networks, and that’s it. You will get a few dozen answers from well-meaning people who know you more or less. Everyone will be in agreement to a greater or lesser extent: “Your idea is excellent, you have to hurry up and propose it to the whole world before someone snatches it from you.
With all this, you will have forgotten that a few dozen answers will never allow you to appreciate the variation in opinions (the famous standard deviation of your statistics courses to which you had not understood anything). However, it is here, in the unanimity of opinions (or not), that the whole difference between an excellent idea and a one-off idea can be made. To capture all nuances of judgment and their variations, never draw conclusions on a sample of fewer than 300 people.
Advice from a leading customer satisfaction expert
We had the privilege of interviewing a world expert on customer satisfaction in the person of Prof. Moshe Davidow. We produced a podcast with him from which we have extracted several chapters according to the topics discussed. You will find below the section dedicated to the measurement of customer satisfaction and the recommendations to follow (in particular, questions to be included).
3. Asking your networks (social or other) to complete the questionnaire
By using your networks, you will have forgotten that not all your future customers necessarily look like the few dozen people you have interviewed. By doing so, you will have used what is called a “convenience sample”. To be simple and to get to the point, you will have interviewed the people who suit you but not the ones who matter.
The results obtained will therefore undoubtedly be biased, in other words, they will not reflect reality. Let’s be clear: this is the best way to make a mistake.
Our golden advice: rent a panel from a company that specialises in this area. This will first of all greatly simplify your recruitment task, your online survey will be accelerated (in 48 hours you will typically have 300 responses), and the results will be unrivalled concerning quality.
4. Create a questionnaire that is too long
This is a common mistake that is very easy to avoid: the long-distance questionnaire.
Before designing a long-distance questionnaire, ask yourself this question: “Would I like to complete this survey myself?”
We will reassure you right away. This mistake is made even by the most experienced. Some market research firms (perhaps under pressure from their clients) agree to develop incredibly long questionnaires that can only produce in the respondent a feeling of frustration that goes hand in hand with +1 and sloppy answers. Why risk tiring your customers with too many questions? Instead, take an empirical approach. Use a simple procedure, and start with no more than 20 questions at a time (or adjust your questionnaire so that it takes no more than 10 minutes of the respondents’ time).
Once the answers to this first questionnaire have been collected and analysed, you will be able to launch another one, inspired by your primary analyses, to refine them. If you follow our other tips, and especially if you choose to rent a panel of respondents, be aware that this method will not cost you any more than a lengthy questionnaire. In most cases, panels are paid according to the number of questions asked and the number of people interviewed.
5. Ask the most essential question only once
There are bound to be some crucial questions that go through your mind and that your questionnaire must answer: are customers satisfied with current products or services? Is this functionality what they all expect and who will make them buy without restraint what you want to put on the market? These questions are too important for you to ask them only once. Keep in mind that these questions only reflect a perspective on the problem you want to study. You may well formulate another question from a slightly different angle. By multiplying the questions on the same subject, you will be able to ensure the consistency of the answers, and thus better substantiate the points of view expressed in the online questionnaire.
6. Save the most important questions for the end
In the same context as the previous point, make sure you ask your most important questions as early as possible (when your respondents are still highly motivated), those for which the most cognitive energy is required. So, keep the basic questions (socio-demographic for example gender, age, and so on…) for the end. Respondents do not need to think to answer these questions. Incidentally, be aware that socio-demographic issues become superfluous when you rent a panel. Indeed, since these details have already been provided by the respondents at registration, there is no need to ask for them again, and you can, therefore, retrieve them from the panel provider without any difficulty.
Attention! Are your questions not too invasive?
In some cases, it may be useful for your market research to penetrate respondents’ privacy. You may, therefore, be tempted to ask sensitive questions that your respondents would not want to answer. Your completion rate (the percentage of completed questionnaires returned) will decrease, and the quality of responses to these questions will have to be reconsidered.
Avoid at all costs questions that could be perceived as intrusive by respondents. The intrusiveness of a question is not only related to the physical intimacy of the person. If, as part of a safety market research study, you ask the question “Do you own a safety-deposit box at home?”
Be prepared that the answers obtained will not entirely reflect the truth. Who would want to reveal that he/she has a safety-deposit box in his/her home?
7. The use of specialised language
When formulating your questions, be careful not to use “meta-language”, that is to say, specific terms that the respondents may not be able to understand. You must formulate your questions in such a way that they can be understood by all.
8. Convoluted formulations
As a follow-up to the previous point, it is necessary not to formulate the questions in a convoluted manner. The question must be easily understandable and must not be ambiguous. Be particularly careful with the turns of phrase used (make sure you don’t use any double negations, for example).
9. Unclear wording
Also, make sure you are precise in your wording. By using specific terms, you could evoke concepts that are more or less vague in the minds of the respondents. This is particularly true for questions involving time concepts.
When you ask someone if they frequently go to a supermarket to shop, the term “frequently” can be interpreted very differently depending on the person. A retiree may interpret “frequently” as “every day” while for a working person “frequently” will mean more than once a week.
10. Do not give rewards to respondents
When you are conducting a customer satisfaction survey, for example, the response rate can be meagre (<5%) if you do not bother to offer a small incentive. Again, ask yourself the question: “Would you spend time completing a survey if you are assured that you will not receive anything in return? The penalty, if you are stingy, maybe double:
- your response rate will be low
Let’s face it, the response rate to a satisfaction survey, for example, is generally low, but an incentive can significantly increase this rate.
- you may only attract the most (unsatisfied) customers
Who are the respondents who do not need any financial motivation to complete an online survey? Those who have an emotional overflow to spill because they have had a particularly unsatisfactory and (in rare cases) particularly satisfying experience. In other words, you may only get the opinions of extremely dissatisfied customers and those who are extremely satisfied.
What incentives for your online surveys?
That is the question that everyone is asking themselves, and unfortunately, we do not have a magic formula. There are many ways to thank your respondents for their time: a sweepstakes contest (with gift vouchers or gift boxes), a gift in kind for all respondents. The more generous you are, the higher the response rate to your online survey will be. It is not necessarily useful to offer “big” gifts. We have seen surveys double their response rate by offering some goodies to respondents. If you are launching such a survey on behalf of your company, and you have promotional gifts, do not hesitate to use them in this way. You will make some people happy!
Images: Shutterstock
Posted in Marketing.
