In this article, you will discover that startups want to replace respondents to qualitative interviews and focus groups with artificial intelligence (AI). We analyze this approach and explain its advantages and disadvantages.
In a previous article, I outlined the current research into replacing surveys with generative AI. The latter is now also threatening qualitative methodologies. At Websummit 2024, we met startups proposing to replace focus groups and semi-structured interviews with AI-generated avatars. I look at this approach in this article and share my thoughts with you. Spoiler alert: I see quite a few methodological problems.
Contact IntoTheMinds for your competitor satisfaction survey
If you only have 30 seconds
- Startups are beginning to propose using AI-generated avatars to simulate responses in qualitative interviews and focus groups.
- This approach has advantages in terms of budget and timing.
- However, there are several limitations: responses are limited to information the algorithm ingests. This means you cannot generate the latest information a priori.
- The algorithm cannot reliably generate real, precise information (prices, names, etc.), which is the very added value of market research.
- This type of application is not suitable for B2B research.
The tech behind this innovation
The technology is based on the use of AI to model virtual personas. The aim is to simulate the behavior of real consumers. Interactions are carried out on an online platform (SaaS). Companies can use these personas to assess products, brands, or campaigns.
The AI is trained based on real interviews conducted for each persona available on the platform.
The advantages of using AI to replace your respondents
The advantages of using AI to “simulate” respondent behavior are obvious:
- Saving time
- Saving money
The most tedious parts of qualitative research are eliminated:
- You no longer need to recruit respondents for your focus groups and individual interviews
- You can automate the question-and-answer phase
- All answers are provided in writing, so there is no need to transcribe them
- Simulate the effect of a product on several consumer profiles in parallel
If we push the limits further, we can imagine that avatar interaction files are sent directly to coding software and analyzed automatically. The qualitative research process would occur in a vacuum, and human intervention would not be needed.
But is this realistic?
Disadvantages
As you can imagine, there are many disadvantages. I am going to show you the system’s limitations.
Insights are limited to what is already known
This is undoubtedly the biggest drawback. As this is a conversation simulation, the AI behind it must have been trained with data. Therefore, this type of approach can only deliver already-known information. But markets change, and so do behaviors. So, there is a real risk that the answers you get will no longer correspond to the state of the market.
Limitations for B2B Market Research
Technology is based on the creation of personas. These are “average profiles” that summarize the overall behavior of a group of customers. This presupposes a certain homogeneity of behavior. This prerequisite leads me to question the relevance of this approach in a B2B context.
In B2B market research, the aim is most often to understand how the interviewee makes decisions. In many cases, this decision is the result of complex mechanisms involving:
- Several people within the company
- Formal processes (e.g., supplier evaluation, negotiation, price limits, etc.)
- So, a priori, there are infinite decision modalities, and it is difficult to provide an “average” answer
No accurate, real-world information
Another problem with simulating a qualitative study using generative AI concerns the reality and accuracy of the information. Whether in B2C or B2B, a study aims to provide precise answers on:
- Market share
- A price paid
- The name of the product or service used
When a customer contacts a market research agency, they expect precise information. However, this information is not exhaustively available in the training data. As a result, the answers provided by AI are bound to be imprecise.
In the context of B2B research, the data is never public (this is the essence of business secrecy). It is, therefore, impossible to expect AI to give you reliable information in this context.
The non-verbal aspects are ignored
Qualitative research is not just about what is said. A good interviewer or moderator takes non-verbal reactions into account. A non-verbal reaction can:
- Detect cognitive dissonance: what the person is saying is not in line with what they are feeling
- Better moderate a focus group by giving the floor to people who do not dare react spontaneously
- Deepen certain positions
Alternative market research techniques not applicable
Finally, AI simulation is limited to text-based interactions. This prevents the use of alternative market research techniques:
- Use of images during interviews
- Manipulation of objects during focus groups
- Creation of prototypes using design thinking techniques.
Conclusion
AI-based interview and focus group emulation technology can be useful for simulating well-known, well-documented behaviors.
However, this technology is ill-suited to:
- In highly dynamic market contexts, where customer behavior is likely to change rapidly
- When the aim is to gather precise, factual information on prices paid or products/services used
- In most B2B research
