Why are focus groups problematic?
When I first set out as a client-side Customer Insight Manager focus groups were frequently suggested to me as a suitable exploratory method of research. Initially this wasn’t a problem. I was new to the industry. However, over the years I gradually become more aware of their frailties. I have come across some excellent moderators. But I have also seen for myself how focus groups can be misleading and misused. The key problems I have experienced include:
- Respondents often don’t know what they want and tend to respond more favourably towards what is familiar to them rather than what might be new.
- Respondents sometimes say what they think you want them to say. I have observed the same respondent say completely contradictory things during a group. Seemingly to please the audience.
- Reactions to stimulus material is highly influenced by how the material is presented and communicated. This can be made worse if the moderator isn’t equally comfortable with all the variants.
- Respondents will over-analyse and rationalise topics (e.g. advertisements) when we are looking for their emotional response as this is often the key driver of the desired behaviour.
- A single respondent may introduce new information into the discussion, and it can completely change the direction of the discussion. Sometimes this can be useful, other times you wonder if it didn’t overly influence other respondents.
- One person can completely dominate the discussion and I have observed people changing their opinions to avoid conflict or because they mistake confidence for knowledge.
- Some respondents will not open their mouth once to contribute to the conversation.
- They are often conducted in a completely alien environment which does not match the context of the discussion. Viewing facilities can exasperate this problem as they can create a laboratory atmosphere.
- Discussion guides often have too many items for discussion and include closed questions. I find moderators are generally overly optimistic about getting through a long discussion guide or don’t suggest removing items to allow for additions to the guide.
The industry may argue that most of these issues can be dealt with by an expert moderator. But we now have increasing evidence from behavioural economics that people are often irrational in their decision making. Humans rely heavily on mental short-cuts that allow them to make fast, automatic decisions that require little cognitive effort.
This means that our conscious mind may not be aware of what drives a lot of our decisions. Such findings support the view that people post-rationalise what drives their decision making. This leads to people creating erroneous reasons for decisions when asked direct questions. Neuroscience also supports the view that the majority of human decisions are made by this largely unconscious mind.
Indeed, it appears only a small proportion of decisions are made using our slower, more cognitive mind. This uses a lot more energy than our unconscious mind. For this reason we use this second system for complex, difficult tasks that requires a lot of thought. This has major implications for how we conduct and analyse market research. But what else can we learn from behavioural economics? Here are few key observations that have implications for focus groups and market research in general:
- Everything is relative and people don’t know what they want until they see it in context. Priming, anchoring, and framing are strong influences on how we respond to stimulus. People like to compare things that are easily comparable, and the context of how we present items heavily skews how we respond to them. This is especially problematic for anything totally new.
- Social norms such as reciprocation, commitment and consistency are powerful influences on our behaviour. We are more strongly motivated by social rewards than by monetary rewards. And yet most participants in focus groups are incentivised solely with cash.
- Our state of mind affects how we respond to questions. Dan Ariely’s experiments demonstrated how men’s responses to questions are heavily influenced by how aroused they are. This begs a question for market research. If we ask a respondent questions away from the natural context of the decision does this invalidate their answers? Context is an important aspect of how we respond to stimuli and we may not gather the key insights if we divorce the research from the natural decision-making environment.
- If uncertainty exists people look to the actions of others to guide them. People herd and like to do what other people are doing or saying. ‘Group think’ can also set in if the group is too homogeneous and the moderator seeks a consensus of opinion rather than having a show of hands. This is particularly worrying for focus groups.
- People have an extreme tendency to comply with the commands of someone in authority. If a respondent in a group comes across as something of an authority on the subject under discussion you may have a problem.
- Don’t underestimate the importance of presentation. It sets expectations for how people respond now and in the future behaviour. This confirms my point about how people are very sensitive to how stimuli is presented. This may have implications for using mock up products or artwork rather than the finished design.
- First impressions count. Once a price has been established in our minds it will largely determine our perception of current and future prices. This indicates how people have a natural tendency to anchor prices from past experience and can be heavily influenced by priming.
Behavioural economics challenges the validity of relying solely on asking consumers direct questions, and highlights bias that can result from group dynamics (eg herding & group think). The most fundamental challenge for market research is that humans are using two different cognitive systems for decision making.
Thus, responses obtained from a person using their slow, rational system will be invalid if in reality people normally use their fast, unconscious brain to process and interact with the stimuli presented. And what if they switch between the two systems at various points?
The consumer’s reliance on the unconscious mind for decision making and the unreliability of traditional research methods is analysed by Philip Graves in his book Consumerology: The Truth about Consumers and the Psychology of Shopping (new revised edition, including a new preface from the author).
He suggests that the most reliable methods for understanding consumer behaviour are live tests and covert observation. Where practical I think this is excellent advice. For websites online experiments can be run relatively easily using A/B and multivariate testing. Where live tests are not feasible, there is strong evidence to suggest Predictive Markets, such as those developed by BrainJuicer, offer us a reliable alternative.
For a number of reasons we are still going to want to ask consumers direct questions, though not necessarily in a focus group format. What behavioural economics suggests to me is that we need a framework in place to increase the likelihood that our methods and analysis will uncover the key insights. This means we need to consider factors including:
- The mind system that is predominately engaged during the process.
- The state of mind and context of the process being investigated.
- How we present items to avoid priming, anchoring and framing bias.
- Existing behaviour – what habits and preferences currently exist.
- Emotional responses that might be engaged
- Social norms that may be influential
Under the right conditions direct questioning of consumers can still provide useful insights. A good example is how Proctor and Gamble investigated the potential market for Febreze. Initially Febreze under-performed and was very close to being cancelled. To identify what was holding the product back P&G sent researchers into peoples homes to observe and record cleaning behaviour.
Researchers were able to ask questions during or straight after cleaning episodes. Respondents were therefore in the right frame of mind and in a natural environment to discuss cleaning related products. What they discovered was that people were not aware of their own house odours. P&G’s strategy had been built around removing such unwelcome odours so this is explained why this wasn’t working. Further in-home observation and questioning identified how Febreze customers incorporated the product into existing cleaning habits.
This was the break through they needed as they had been trying to create a new cleaning habit for Febreze. Instead P&G was able to piggy back onto an existing cleaning habit. This resulted in a completely new advertising strategy, and P&G has never looked back. Febreze now has annual sales of over $1 billion.
For focus groups, the dynamics of groups and the environment appears to make their findings more prone to bias. As Philip Graves points out:
“The artificial nature of the research environment can also be responsible for not flagging up something that, in the real purchase environment, is unconsciously reinforced and hugely significant in determining a product’s fortune.” Philip Graves, Consumer.ology
Jonathan Ive, Apple’s Senior Vice President of Industrial Design, once said that Apple had found good reason not to use focus groups because:
“They just ensure that you don’t offend anyone and produce bland inoffensive products.”
Does this mean that the focus group is dead? I suspect not and experiments with using gamification in focus groups have shown how we can use this technique to disrupt and counter dominant group members. What this suggests is that we should look to incorporate some of the insights from behavioural economics into the design of focus groups.
Why not try running a focus group in a kitchen environment if the topic is about cooking appliances? Use role play and gamification techniques to bring out the appropriate state of mind. Observe body language and facial expressions, not just what people say. But most important of all don’t use focus groups as your default method of research, only use them when they are appropriate and can provide some true insights for your organisation.
Thanks for reading. I hope this blog has challenged a few myths and generated some useful ideas.
This post has been published on the GreenBook Blog market research website.
You can view my full Digital Marketing and Optimization Toolbox here.
To browse links to all my posts on one page please click here.
- About the author: Neal provides digital optimisation consultancy services and has worked for brands such as Deezer.com, Foxybingo.com, Very.co.uk and partypoker.com. He identifies areas for improvement using a combination of approaches including web analytics, heuristic analysis, customer journey mapping, usability testing, and Voice of Customer feedback.
- Neal has had articles published on website optimisation on Usabilla.com and as an ex-research and insight manager on the GreenBook Blog research website. If you wish to contact Neal please send an email to firstname.lastname@example.org. You can follow Neal on Twitter @northresearch and view his LinkedIn profile.