Quantitative research: create and test an online questionnaire

We have already dealt with the question of quantitative questionnaires, their importance, the risks, and the pitfalls to be avoided. Today, we will help you to create your questionnaire, formulate your questions and improve it overall.

Summary

• Define and understand the subject to be researched
• Preparing the questions
• Testing the questionnaire
• Conclusion

Define and understand the subject to be researched

Saris et al. (2014) rightly point out that it is essential to define the subject clearly. Indeed, the first step in drafting a questionnaire involves:

• The definition of the topic
• The definition of a framework for the research
• Definition of variables

Before you start writing down your questions, you will need to set down on paper the final objective you hope to achieve through the distribution of your questionnaire. Once this goal has been expressed, estimate the variables to be measured, based on your knowledge of the market and consumers, but also the results and conclusions of previous stages (desk research, qualitative research, tests, and so on).

The example defined by Saris et al. (2014) suggests that to assess the likelihood of voting in elections, it will be necessary to investigate age, level of education, degree of adherence to the norm (‘you have to vote’) and political interest. These variables will, therefore have to be included in the questionnaire to be validated or invalidated depending on the results.

Preparing the questions

We have already mentioned the importance of formulating the questions according to the type of response desired.

It is, of course, advisable, in the context of a quantitative questionnaire, to propose response options that are as exhaustive as possible (maximum 7 proposals to avoid the cognitive overload of respondents). The aim is also to formulate your questions in an uncomplicated manner. Respondents must be able to understand the meaning of your questions on the spot. Indeed, if they hesitate between several interpretations of the same question, it is highly likely that not all of them will answer the same question.

The risk of respondents choosing different interpretations is very high and will negatively impact the quality of your results.

So, make sure you choose precise wording. If necessary, it is also possible to supplement some questions with further information, such as:

• An introduction
• Additional information
• Definitions
• Instructions

Testing the questionnaire

Presser et al. (2004) define the pre-testing of a questionnaire as an indispensable step. Both the formulations and the design of the questionnaire (question and answer formats, the sequence of questions, architecture, and so forth) need to be tested.

Once you have answered all these questions and made the necessary adjustments (if needed), you can proceed to test your questionnaire. To do this, you launch what is called a “soft launch”, that is to say, a first launch to about 10% of your respondent panel. This initial feedback will enable you, after observation and pre-analysis, to understand whether the objectives of the research will be achieved after the full launch. At this stage, you can make changes, so take advantage of them. If a soft launch is not possible, ask your friends and family, distribute the first version on small platforms or social networks.

Research conducted by Forsyth et al. (2004) confirms the importance of testing questionnaires before launch, mainly via:

• Expert review of the questionnaire
• Evaluation of the questionnaire
• Cognitive interviews

These actions are intended to highlight problem areas in the questionnaire and which should, therefore, be improved to optimise the results. In their evaluations, Forsyth et al. (2004) indeed define that the questions that have been designated as problematic following the pre-testing of the questionnaire proved to be inconsistent during the analysis.

Once problems have been identified, they must, of course, be addressed. However, highlighting a problem does not systematically lead to the detection of a solution. Researchers on the subject also agree that it is necessary to gather detailed opinions on the difficulties (understanding the problems encountered by respondents) to update the questionnaire judiciously.

Once you have made the necessary adjustments, it would be ideal to test the questionnaire once again to ensure that the changes are significant.

Conclusion

It is essential to test a questionnaire before distributing it. The same is true for interview guides in qualitative research, which sometimes need to be adjusted after a few interviews to fit with the objectives of the research. The same applies to quantitative questionnaires.
The fixed and rigid nature of the online questionnaire means that it must be clear and complete at the very beginning to ensure quality results that meet the needs of the research. Do not, therefore, hesitate to make several amendments to your questionnaire by testing it with several people and understanding the difficulties and problems encountered by your testers.

Sources

• Saris, W. E., & Gallhofer, I. N. (Eds.). (2014). Design, Evaluation, and Analysis of Questionnaires for Survey Research.
• Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., & Singer, E. (2004). Methods for Testing and Evaluating Survey Questions. Public Opinion Quarterly, 68(1), 109–130.
• Forsyth, B., Rothgeb, J. M., & Willis, G. B. (2004). Does Pretesting Make a Difference? An Experimental Test. Wiley Series in Survey Methodology, 525–546.

Illustrations : Shutterstock

By Lorène Fauvelle