Why Are Voice of Customer surveys fundamentally flawed?

Voice of Customer Process Flow image

Voice of Customer (VoC) surveys are a popular method for organisations to monitor customer preferences and levels of satisfaction. However, how robust is the standard process of interviewing a few customers and then developing and administering a highly structured survey?

The VoC survey framework appears to be based upon an outdated and false view of people as rational, independent agents with relatively fixed preferences. Because of the way the human brain works people have limited access to the emotional, social and psychological motivations that drive much of our behaviour.

 

SixSigma Voice of Customer

source: iSixSigma.com

Surveys also ignore the power of contextual influences which means that VoC programmes are highly likely to generate unreliable and misleading results which contribute to an illusion of understanding customers.

A flawed measure:

  • Asking direct questions is unreliable because many of our daily decisions are made without much conscious thought and we post-rationalize decisions to ensure consistency with our internal model of the world. When people answer direct questions they attempt to rationalise their decisions and behaviour. They naturally construct a narrative that explains their  actions in a rational and consistent way.
  • However, this fails to capture many of the underlying motivations of human behaviour. Further, as our brains use mental short-cuts to save energy and speed up decision making, we are prone to making sub-optimal and often irrational decisions. This is contrary to how we like to perceive ourselves and naturally we don’t articulate this when answering survey questions.
  • Asking people about importance is especially misleading. Psychologists have found that relative importance of an item is heavily influenced by the ease with which we can retrieve it from our memory. This often correlates heavily with the amount of coverage an issue gets in the media. I noticed this when I managed a survey of financial advisers. The importance of financial strength always shot up after media coverage of any kind of financial crisis.
  • Our ability to recall an event is also limited as a memory is constructed from a series of brief fleeting moments from an experience. We construct a memory from visual snapshots, thoughts, feelings, smells and sounds. The perception of an experience is also influenced by a range of factors including our mood, social context, our vocabulary, the physical environment and our knowledge.

mri-head-scan

Source: Freeimages.com

  • However, the reconstruction process itself is also the product of these same factors. This means that each time we recall an event we have to piece these elements back together and inevitably our memory changes each time we recall an experience. Indeed, we frequently have false memories of events that we will passionately defend as our brain attempts to retain our internal consistency with our values and beliefs.
  • Our memory is also biased towards  remembering what happened during the most intense moment of an experience, and what occurred at the end of the episode. The length of the experience appears to have little impact on our overall satisfaction with an event.
  • Our herd instinct means we are heavily influenced by what other people in our network and beyond are doing. We are constantly copying people, especially when we find ourselves in a new or uncertain situation. Social norms and trends are also powerful forces, but again we may not  be consciously aware of these influences and so we cannot expect people to articulate this via direct questioning.

Sheep on the road image

Source: Freeimages.com

Analysis and Action:

VoC surveys also fail to deliver when it comes to analysis of results as little or no allowance is often made for the fundamental flaw in using direct questions to obtain the data.

  • Due to the survey design operational and customer facing areas often find VoC feedback to be too generic and therefore not actionable. They want to drill down to specific locations or touch points but sample size are usually not sufficient to allow for such analysis.
  • Unfortunately research executives can sometimes be pressurised into providing analysis based upon tiny sample sizes. This gets quickly circulated around the organization to support changes in service or product delivery. The fact that responses are unlikely to be an accurate reflection of real customer motivations and preferences is usually ignored until results contradict what a senior stakeholder wants the data to say.
  • Sponsors like to set targets to improve VoC scores but ratings tend to fluctuate for no obvious reason. When expensive changes to the product or service are implemented is it surprising that we may not see any change in customer satisfaction?
  • Some organisations now base changes on an understanding of our selected memory bias by focussing on the peak and end moments of an interaction. This may lead to improved VoC scores but does this really mean that we are delivering an improved customer experience?  Are we not just kidding ourselves by using human psychology to temporarily boost a flawed score?

Experiments and Observation:

Top retailers have known for decades that if you want to find out if something new works running a controlled test in a number of stores is more reliable than asking people direct questions. This was the original A/B test and is the reason why online retailers are now some of the biggest users of this experimental research design.

“Just following consumer wishes leads to replaceable products, copycat advertising, and stagnating markets.”
Stephen Brown, Professor at the Kellogg School of Management

Direct questioning of customers is limited for the reasons given above. However, VoC surveys are particularly problematic because the standard framework developed as part of the Six Sigma methodology gives it an aura of validity that the technique does not merit. This is an illusion which Six Sigma followers espouse due a lack of understanding of basic human decision-making and psychology.

 

SixSigma VoC Customer Focus

Source: iSixSigma.com

More emphasis on conducting experiments, together with observing or listening to real customer interactions would be more effective methods of research. Co-creation can also be a powerful approach to allow brands to open up a two way conversation with their customers. Direct questioning for VoC programs needs to be used sparingly and the results treated with extreme caution.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

Related post: 5 ways to get more valuable insights from your Voice Of the Customer programme.

Recommended reading:

image

Consumerology: The Truth about Consumers and the Psychology of Shopping (new revised edition, including a new preface from the author)

  • About the author:  Neal provides digital optimisation consultancy services and has worked for  brands such as Deezer.comFoxybingo.com, Very.co.uk and partypoker.com.  He identifies areas for improvement using a combination of approaches including web analytics, heuristic analysis, customer journey mapping, usability testing, and Voice of Customer feedback.  By  aligning each stage of the customer journey  with the organisation’s business goals this helps to improve conversion rates and revenues significantly as almost all websites benefit from a review of customer touch points and user journeys.
  • Neal has had articles published on website optimisation on Usabilla.com  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to neal.cole@outlook.com. You can follow Neal on Twitter @northresearch and view his LinkedIn profile.

Is Digital Marketing A Zero-Sum Game?

 

I recently read an inspiring book, Predatory Thinking, by Dave Trott.  It’s a highly readable and engaging book full of brilliant anecdotes and short stories. It made me reflect on life and work. For me it has some great insights that are very salient to both digital marketing and website optimization.

Marketing is a zero-sum game, you can’t win new customers unless you take them away from someone else. This means your content and proposition needs to be compelling, and persuasive, as otherwise your visitors may well sign up or purchase from another website. It may seem obvious but you only need to look at a selection of websites and many don’t communicate clearly what makes them unique from their competitors. Many copy the standard template of carousel and product images without much thought to include persuasive content. Design is a framework to build and structure content, but it is content that engages and persuades.

“Because marketing, like war, is a zero-sum game. If you want something you have to take it from someone else.”
Dave Trott, Predatory Thinking

But that doesn’t mean continually adding new elements to your website or proposition. You can’t have everything at once. When you add something you also need to consider taking something away as otherwise you are in danger of ending up diluting your value proposition and confusing visitors. Testing different messages on separate landing pages can assist this process as it helps maintain a single-minded proposition.

Digital marketers spend a lot of time trying to formulate the right messages for their landing pages. However, as Dave Trott reminds us ordinary people:

  • Don’t notice stuff
  • Are only interested in one thing at a time
  • Are conditioned to filter out distractions (e.g. banners)
  • And will probably do the opposite of what you want them to do.

The insight here is that first and foremost we need to think about “how do we even get noticed”. We need to workout how to get the visitor’s attention and not bombard them with multiple messages or distracting graphics that don’t nudge them towards our goal. Look at your bounce rates and time spent on pages to see if you are being noticed.

“£18.3 billion is spent yearly in the UK on all forms of advertising. 4% remembered positively, 7% remembered negatively, 89% not noticed or remembered”.
Dave Trott, Predatory Thinking

“What’s in it for them?” Marketing is about getting peopleto do something that we want them to do. This means that content needs to be persuasive and not just communicate what we want to tell visitors. Further, people are not rational agents and so are far more likely to respond to a message that engages their imagination than focusses on rational reasons.

Don’t assume our visitors know what we know. As experienced
digital marketers we understand the websites and brands we work on much more than the average user. It’s important to take a step back and accept we won’t see our website like a customer. We know how to navigate to the account page to change privacy settings, but a first visitor may not know it exists. Maybe they don’t even care.

“We can’t believe the world isn’t exactly the same way for
everyone else, as it is for us.” Dave Trott, Predatory Thinking

You need to get closer to your visitor’s view of the world by observing how they behave and listening to what they say about your website and brand. Find out why they came to your site and what tasks they were looking to complete. But also what may have prevented them from completing their task and what frustrations they have about their experience. There are so many tools on the market to obtain customer feedback there is no excuse for not capturing visitor opinions.

The role of advertising:

Advertising has the potential to give you an edge over your
competitors, but it can’t turn a core non-user into a core user. A person has to be in the market for your product in the first place to have the potential to convert. Similarly if your landing page or advert can grab their attention you have the potential to influence visitors.

However, the best you can hope for is to create a propensity to convert. Most of our visitors won’t convert because we don’t tick all their requirements at this point in time. Some may return to our site if they remember us or if something grabs their attention that
makes them believe we can help them achieve a current goal.

The key implication from the book for website optimization
is that you need a compelling, single-minded value proposition that is communicated using simple and imaginative messages. A useful framework for evaluating your website is the Wider Funnel’s Lift
Model. This demonstrates visually how you can’t rely on reducing distractions, anxiety and creating urgency etc, if you don’t have a strong value proposition. Your brand simply won’t get off the ground.

image of Widerfunnel Lift Model of website optimization
Source: Widerfunnel.com

 Recommended Reading:

By Dave Trott – Predatory Thinking: A Masterclass in Out-Thinking the Competition

Related posts: 

Website optimisation toolbox – Over 30 categories of tools for increasing your conversion rate.

How should you prioritise your A/B testing ideas? – A framework for evaluating your A/B testing ideas.

Which A/B testing tool should you choose? – Considers the criteria for selecting an A/B testing tool and looks at a survey of user rating for 9 of the most popular tools.

You can access links to all my posts on my index page.

The Lift model framework from Wider Funnel

Thank you for reading my post and if you have time please browse my other articles.

  • About the author:  Neal provides digital optimisation consultancy services and has worked for  brands such as Deezer.comFoxybingo.com, Very.co.uk and partypoker.com.  He identifies areas for improvement using a combination of approaches including web analytics, heuristic analysis, customer journey mapping, usability testing, and Voice of Customer feedback.  By  aligning each stage of the customer journey  with the organisation’s business goals this helps to improve conversion rates and revenues significantly as almost all websites benefit from a review of customer touch points and user journeys.
  • Neal has had articles published on website optimisation on Usabilla.com  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to neal.cole@outlook.com. You can follow Neal on Twitter @northresearch and view his LinkedIn profile.

Can You Get Visitor Heatmaps, Form Analytics, Online Surveys, User Testing & more for €29 A Month?

image of Hotjar.com homepage

I recently attended a conversion conference and was discussing the use of visual behaviour tools for gaining visitor insights. These tools can be expensive and so I was shocked to hear that a start-up is offering such a product from just €29 a month.

When I next logged onto my laptop I Googled Hotjar.com and sure enough they are currently in beta with a product that offers unlimited sites and users. However, as well offering your standard heatmaps and sessions replays, they also allow users to serve feedback and exit polls, online surveys, and live chat. What other tool offers all these options?

  • Heatmaps
  • Feedback polls
  • Surveys
  • Conversion funnels
  • Form analytics
  • Visitor recordings
  • Recruit user testers

This seemed too good to be true but I managed to find an interesting review by Stacey from a digital marketing agency. Although only a sample of one, it’s useful to know that Stacey hasn’t uncovered any bugs or problems with the product. She is excited about the prospect of having a single tool that does so much more than heat maps and session replays. Other reviews I have read are also positive so far.

Apart from its low cost, the ability to provide survey and user testing as well as visual behaviour analytics is a potential big advantage for Hotjar. I’ve not had the opportunity to test the product so I can’t comment on how well the product performs all these different needs. However, there is certainly the danger that it could be perceived as trying to pursue too many markets at once and being masters of none. The feedback I’ve seen so far does not support this and so maybe this won’t be a major obstacle.

The level of support and responsiveness will also be key. These tools are great when they are working correctly and you know what you are doing. But if heatmaps are not displaying a page correctly or you can’t figure something out this is when you need to be able to get hold of someone to help. With potentially a much larger customer base than some of its competitors will Hotjar be able to deliver the level of support needed? I suspect this will be a key issue for customers considering switching from existing suppliers and an important area for Hotjar to prove itself in.

It will be fascinating to see how Hotjar gets on in an increasingly competitive market for online customer insight tools. Thanks for reading my post and let me know if you have any experience of Hojar or other visitor insight tools that you would like to share.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

Recommended reading:

If you would like a step-by-step approach to website optimisation I suggest reading:

 

image

Website Optimization: An Hour a Day

  • About the author:  Neal provides digital optimisation consultancy services and has worked for  brands such as Deezer.comFoxybingo.com, Very.co.uk and partypoker.com.  He identifies areas for improvement using a combination of approaches including web analytics, heuristic analysis, customer journey mapping, usability testing, and Voice of Customer feedback.  By  aligning each stage of the customer journey  with the organisation’s business goals this helps to improve conversion rates and revenues significantly as almost all websites benefit from a review of customer touch points and user journeys.
  • Neal has had articles published on website optimisation on Usabilla.com  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to neal.cole@outlook.com. You can follow Neal on Twitter @northresearch and view his LinkedIn profile.