Market research has been around for quite some time, so why do studies continue to produce non-actionable insights that do not help businesses move forward?  

Bastian Verdel: While describing research results as "non-actionable" may be an extreme characterization, the transfer of insights into marketing actions is often not straightforward. To bridge this gap, we have developed the Customer Thinking approach, which combines DesignThinking elements like prototyping workshops to facilitate the transfer of insights into actionable marketing strategies. But even more important, we use a Behavioral Economics perspective to decipher the connections between the needs and decision heuristics of the target audience and their response to products and marketing efforts.  

By understanding the drivers of behavior, it becomes easier to derive marketing strategies based on them. 

You have incorporated ChatGPT, AI, and behavioral economics into your research approach. How long have you been working with AI?  

Bastian Verdel: We have been using AI for several years now, but primarily as support for handling labor-intensive processes such as coding open-ended responses, transcribing and translating qualitative interviews. Last year, we began exploring the possibilities of Open.AI, initially in a playful way. This year, however, things have changed fundamentally, particularly as we discovered ways to analyze our own data and provide assistance on the content level. This means that we can now use AI not only to summarize interviews but also as an assistant to aid us in the analysis of qualitative research. As a result, opportunities have emerged to use AI for behavioral psychology analysis. For example, we can analyze which signs on the use of specific decision-making and evaluation heuristics emerge from the interviews.

 Sign up for the webinar (09 May, 13:00 hrs., in German)   

What sets your new hybrid research approach apart? When you say that humans still play a central role in gaining insights, what tasks does AI take on for you?  

Bastian Verdel: The core of the approach is that we do not use AI as a project management tool but rather for the content level of the studies. The hybrid approach means that we do not leave the AI with the complete answering of research questions, but as experts, we retain the interpretation authority over the data.  

However, we use AI where it is better than humans.  

Better means, for example, that it can read transcriptions much faster and find quotes on a specific topic, or that it can remember basic marketing knowledge much better since it ultimately has access to the freely available expertise of the world. But better also means that through the language model, it can recognize needs in language at least fairly consistently. However, the AI sometimes tends to hallucinate, so the evaluation of the results absolutely requires an expert's review.  

Ultimately, this combination of AI and human expertise enables us to conduct studies more quickly and efficiently, especially in the qualitative field, without compromising on quality – in fact, it even improves it.  

For international studies, for example, we see that we obtain better insights when we do not translate transcripts but work with the original language in the AI. Overall, the advantages are particularly evident in larger studies with higher case numbers or several countries. If I do n=8 UX interviews, these are also quickly evaluated in the classical way. But if, for example, I have n=24 in-depth interviews - and perhaps even in 2-3 countries - the effort involved is significantly reduced. This makes qualitative studies with larger case numbers conceivable as a replacement for Qual-Quan study setups since at a certain point, the number of interviews hardly affects the analysis effort. 

What were the obstacles in the development process? 

Bastian Verdel: There were several hurdles: first, the capacity of ChatGPT per request is limited - although this is better with GPT 4.0, it is still only sufficient for a half-hour interview. Secondly, we naturally do not want to share personal data with the AI, so we had to ensure that the transcriptions do not contain personal data. We also do not want our data and, above all, insights to be used to train the language model, and these insights are then made available to other ChatGPT users in the worst - albeit somewhat theoretical - case. Then there are smaller hurdles regarding data preparation to ensure that it is properly structured and can be processed by the corresponding tools.  

The biggest challenge, but also the most exciting, is prompt-writing.  

Asking ChatGPT to summarize an interview is generally not very helpful - instead, the tool should help us uncover the less obvious aspects. For example, we use a basic need model and analyze which fundamental needs are reflected in the responses of our respondents, or we want to identify which cognitive biases the respondents use in a particular context. These are things that cannot be directly asked of the AI, but rather we must provide it with the necessary tools or formulate the prompt in such a way that it retrieves the necessary tools (e.g., if it involves models or phenomena described in books or studies). This immediately creates the next challenge of finding a way to make this prompt available to the team - we are currently working on this. 

Some market researchers warn against the hasty integration of tools like ChatGPT since AI is far from perfect. What is your view on this?  

Bastian Verdel: The essential thing is how I use AI and what expectations I have.  

Currently, AI is not able to write a report or even a reasonably meaningful management summary based on the interviews of a qualitative study.  

However, as mentioned earlier, AI has several significant advantages over humans. This can be compared somewhat to Kahneman's Two-System Theory. There is System 1, which is incredibly fast and has powerful processing capacity but is largely unconscious, and System 2, which is much slower but can consciously reflect on and often correct the impulses of System 1. Only the interplay of both systems enables us to lead our lives and make decisions for ourselves - mostly sensibly.  

An AI language model like ChatGPT has certain similarities to System 1 - because we are "unconscious" about exactly how it works, it is very fast but also very unreflective.  

The human in their role as a research expert assumes the role of System 2 - reflecting on the results, questioning them, and ultimately deciding which results of the AI to use and which not to use. Moreover, market researchers have a crucial role to play. Language models like ChatGPT are very good at holding basic knowledge. For example, if I ask ChatGPT about the decision criteria of people when choosing a current account, it comes up with the dimensions we also see in our studies. However, if I want to know how a particular brand can deal with this or just want to know what the decision criteria of a particular target group look like, the model lacks the detailed or contextual knowledge to answer that. We bridge this gap with our studies tailored to the respective topic. 

It is known that AIs still often fail to capture emotions. Generally, measuring emotions is a tough nut to crack. What needs to happen for us to finally succeed, perhaps even with AI? 

Bastian Verdel: When it comes to capturing pure sentiment, that is indeed true. However, this is rarely a problem in our everyday work. In quantitative surveys, we have emotion measurements that are independent of the open-ended questions, and in qualitative analysis, much is revealed from the context of the language. The content level with its emotional context is more decisive for our analysis than the detached emotion itself. We want to know what excites people or possibly what leads to irritation, or whether the irritation is so great that someone is disappointed, for example - and these things can indeed be identified.  

Ultimately, in qualitative research, we do not need an exact count of how many positive or negative emotions there were. 

Who shouldn't miss your webinar? 

Bastian Verdel: I would say anyone who is interested in understanding what AI means for our work as market researchers. Especially corporate market researchers who want to get an overview of what is already possible and where the limits are. 

Sign up for the webinar (09 May, 13:00 hrs., in German)  

 

– published in German on marktforschung.de –