A New Path to Understanding Consumer Behaviour: The Role of AI in Redefining Market Research
Market research is on the cusp of a transformative era, driven by rapid advances in (generative) artificial intelligence (AI). As industries continually seek more efficient and effective methods for exploring consumer behaviour and preferences, AI technology, particularly Large Language Models (LLMs), is proving to be a revolutionary tool. This article delves into the groundbreaking applications of AI in market research and explores how AI technologies not only redefine traditional approaches but also increase the precision and speed at which market insights are gathered and analysed. Here, we will examine the critical role of these AI models in simulating consumer reactions, creating perceptual maps, and providing deeper insights into emotional recognition.
A Revolutionary Approach to Simulating Consumer Preferences
In market research, it is crucial to leverage technological advancements to gain effective and efficient insights into the needs and preferences of consumers. An innovative approach using LLMs, in this case, GPT-3.5, has shown significant potential to redefine our understanding of consumer preferences. The study conducted by Brand et al. (2023) utilises AI to simulate consumer reactions, providing insights that are both economical and efficient compared to traditional market research methods. At the heart of this research is the use of GPT-3.5, a model trained on extensive online data, including consumer reviews and discussions. This allows for new ways to obtain comprehensive consumer insights. The method involves querying this model to simulate hundreds of consumer responses. This approach reflects documented consumer behaviour patterns, such as declining demand curves and certain dependencies in purchasing behaviour (e.g., competing products). The application of a maximum "temperature" setting in GPT-3.5 is particularly noteworthy. This setting is used to increase the diversity of responses by introducing randomness, similar to capturing a broader spectrum of human behaviour. By setting the temperature to the highest level (1.0), researchers were able to mimic a more diverse and realistic range of consumer opinions.
One of the most compelling results of this research is the model's ability to derive the willingness to pay (WTP) for various products. This was achieved by directly querying GPT-3.5 and comparing the derived demand functions. These demand functions were carefully constructed by varying product features and prices, observing the model's preferences, and quantifying the implied WTP: The estimated WTP for products and their attributes not only showed realistic magnitudes but also closely matched those derived from traditional market research methods involving human subjects. For example, GPT was asked to choose between two toothpastes (prices between $2 and $6; reference price $4). In this example, a declining demand curve is apparent. It also shows a visible kink in the demand curve when the price exceeds the reference price (see fig.).
The implications of using generative AI in market research are immense. For managers, this technology offers a fast and cost-effective alternative to traditional methods such as conjoint analysis and focus groups. This can significantly accelerate the product development cycle, enable more dynamic pricing strategies, and improve market entry tactics.
AI to Enhance Perceptual Mapping in Market Research
Similar to the simulation of consumer preferences, the integration of LLMs brings a transformative approach to understanding consumer perceptions, particularly through perceptual mapping. The study by Li et al. (2023) highlights the effectiveness of LLMs, in this case, GPT-4, in creating Perceptual Maps that closely match those derived from traditional human surveys, thus streamlining market research processes and reducing associated costs. The core of this research utilises the capabilities of LLMs to simulate consumer insights traditionally gathered through surveys. By intelligently using specific prompts, researchers were able to create Perceptual Maps based on data regarding brand similarities and product attribute ratings. This approach not only aligns with traditional methods but also enhances them by enabling rapid and scalable insights.
The used prompts ranged from simple requests where the LLMs were directly asked for a numerical similarity rating between brand pairs, through Few-Shot prompts enriched with real examples from human surveys to improve response accuracy. Moreover, Role, Task, Format (RTF) prompts were employed, explicitly simulating the LLM as a survey respondent, with the task being to respond with a number from 0 to 10. For even deeper guidance and to increase data quality, researchers combined RTF structures with Few-Shot examples in an extended prompt format.The Perceptual Maps generated by LLMs were compared with those from human surveys and showed a match of up to 85% in some product categories. These maps illustrate the landscape of consumer perceptions and provide market researchers with a visual representation of how brands are positioned in the minds of consumers. The matrix used in the study represents the similarities between brand pairs, where each cell (i, j) indicates how often brand i is mentioned concerning brand j (in this case, car brands). This frequency matrix is then analysed to derive similarity values. The Perceptual Map illustrates the relative proximity between brands as perceived by consumers (see fig.).
Significantly, the study highlighted that LLMs could relatively accurately replicate human perception. This is particularly remarkable when considering the generation of brand similarity values and attribute ratings, where the responses of the LLMs were very similar to those of human participants. This ability underscores the potential of AI to reliably replicate consumer behaviour. However, it is essential to note that the effectiveness of LLMs in generating useful market research data largely depends on the availability of extensive training data. For brands and product categories well-represented in the training corpus of LLMs, the accuracy and reliability of the results are significantly high. This dependence on rich data sources means that LLMs are most effective in scenarios where ample data is available for training, which poses a limitation in cases involving niche or less popular product categories.
Practical Implications: The Emotional and Interactive Side of AI-Supported Market Research
As market research continues to evolve, the integration of emotionally intelligent AI represents a significant advancement in understanding consumer emotions and improving data collection methods. Companies like MorphCast or Hume.ai utilise AI technologies that can recognise and analyse human emotions with high precision. This ability allows market researchers to measure emotional reactions to products, services, and advertising, offering a deeper level of insight that goes beyond traditional data points. Hume.ai uses advanced algorithms and neural network models to interpret subtle facial expressions, voice modulations, and physiological responses indicating various emotional states. By accurately identifying emotions such as joy, frustration, or disappointment, this AI technology can enhance the understanding of how consumers respond to their offerings. These emotional data can be crucial for refining product features, tailoring marketing messages, and improving customer engagement strategies.
Another groundbreaking application of AI in market research is its role as an interactive interviewer. Platforms like ListenLabs use AI to conduct interviews where the technology not only asks initial questions but also intelligently responds to the interviewees' answers (see fig.). This dynamic interaction mimics the ability of human interviewers to probe deeper based on the context and course of the conversation. This can yield more nuanced insights into consumer preferences and behaviours (see also research by Chopra and Haaland (2023) on this topic).
The combination of emotional intelligence and interactive interviewing capabilities in AI transforms how data is collected and analysed in market research. AI systems equipped with these abilities can conduct interviews on a large scale and with consistent accuracy, reducing human error and biases. Additionally, the ability to analyse emotional responses in real-time allows for immediate adjustments to marketing strategies, enabling companies to quickly respond to consumer moods and market developments. It remains important to note that the effectiveness of AI in market research continues to depend on the quality of the training data and requires careful human supervision. As these tools evolve, they promise to become essential components of innovative market research strategies, enabling companies to lead in a competitive and rapidly changing market landscape.
Moreover, AI offers many more exciting opportunities to generate in-depth customer insights. For example, secondary data (e.g., consumer reviews) could be analysed (e.g., sentiment analysis) to identify, categorise, and prioritise customer issues.
We are only at the beginning of what AI can achieve in market research. With the ongoing development of technology, we can look forward to even more precise and comprehensive insights into consumer behaviour, enabling companies to operate with unprecedented speed and accuracy.
"The future of market research is not just a canvas for data but a dynamic tool that allows us to capture and anticipate the pulse of consumer sentiment in real-time. We stand on the threshold of an era where data-driven intuition is augmented by machine intelligence, and this is just the beginning." (Prof. Dr. Dr. Athena)
