Why Do People Prefer Gut Instinct to Research?

By | 6th April 2014
Behavioural economics tells us that people become overconfident and ignore their ignorance as they develop expertise

Image Source: Dilbert

Behavioural Economics and Gut Instinct:

Business people pride themselves in their decision making and many  businesses embed market and competitor research into this process. However, business people often prefer their gut instinct as they are prone to the same human frailties as everyone else. This can discourage the use of research and experimentation as a way of gaining feedback and learning from our mistakes.

Behavioural economics suggests that as people gain experience and knowledge in their area of expertise they have a tendency to become overconfident and complacent about their ability to understand the past and predict the future. Our brains assume that we are living in a simpler, more predictable world than is really the case.

Image of Dilbert cartoon where boss has gut instinct - behavioural economics indicates people rely on their gut instinct for many decisions

Image Source: Dilbert.com

 

This is one of the most useful insights of behavioural economics and yet professionally it is a difficult truth to acknowledge when we like to be seen as an expert our field. Indeed, we are sometimes informed that decisions have been made on the advice of an ‘expert’ as if this guarantees the quality of the decision. Behavioural economics though tells us that experts are often no better than non-experts at predicting the future.

As humans we are certainly prone to the illusion of understanding. Our minds create narrative fallacies from our continuous attempt to make sense of the world.

We notice the small number of unusual events that happen rather than the multitude of events that failed to occur. We focus on the attributes of survivors and ignore the influence of the process that shaped them.

Our memory is selective and biased by the workings of our mind. We construct vivid accounts of the past based on memories that change every time we recall them but believe they are a true reflection of past events.

We suffer from a tendency to like (or dislike) everything about a person. This  helps generate a simpler and more coherent representation of the world than is really the case. We fill gaps in our knowledge about a person using guesses that fit our emotional response.

Short-term emotions are probably the most powerful force in our decision making arsenal. Many of our judgements and decisions are directly influenced by feelings of liking and disliking rather than rational deliberation.

We hate uncertainty and suppress ambiguity because inconsistencies slow our thought processes and interfere with the clarity of our feelings. People are attracted towards confidence and we prefer decision makers that demonstrate such qualities above someone who may be equally competent but wants to think through a decision before giving an answer.

Image of Dilbert cartoon where boss wants to use his gut rather than read document

Image Source: Dilbert.com

 

People are heavily influenced by the What You See Is All There Is (WYSIATI) rule. We naturally work with the fragmentary information that we have access to as if it were all there is to know. The paradox is that it is easier to construct a consistent story when you have little knowledge. People make fallible guesses from incomplete information by making a leap of faith about how things should work.  Steven Pinker points out our only defence is that it worked sufficiently well in the world of our ancestors.

“Our comforting conviction that the world makes sense rests on a secure foundation: our almost unlimited ability to ignore our ignorance.” Daniel Kahneman, Thinking, fast and slow.

This can lead us to frame our choices too narrowly and consequentially reduces our options. Research and working collaboratively can help by widening our horizons and introducing new insights to challenge our perception of the topic.

We are also very good at changing our beliefs after an un-predicted event without being aware of it. We often unconsciously adjust our view of the world and find it difficult to recall what we believed before the event. Hindsight bias leads us to evaluate the quality of decisions by the nature of the outcome rather than the process by which the decision was made.

“Asked to reconstruct their former beliefs, people retrieve their current ones instead.” Daniel Khaneman, Thinking, fast and slow.

The danger here is that people get blamed for a decision that resulted in a negative outcome despite the unpredictability of the event. In corporate decision making this can result in people relying on bureaucratic solutions to avoid blame which leads to extreme risk aversion.

Image of Dilbert cartoon where boss explains decision making process

Image Source: Dilbert.com

It can also result in business people receiving unjustified rewards (e.g. bonuses) for being irresponsible with risk taking and just being lucky. This can be seen prior to the 2008 financial crisis where banks and other financial institutions were paying risk takers massive remuneration packages for activities that put the whole financial system at risk. Due to the complexity of some of the assets their AAA ratings proved to be illusory.

Kahneman asserts that any comparison of how successful or not companies have been is to a large extent a comparison between how lucky or not they have been. In every story of a successful company there will have been moments when the destiny of a firm could easily have turned in an instant.

So, is the analysis of the situation more important or is it the process that is they key? Research conducted by Dan Lovallo and Oliver Sibony studied 1,048 major business decisions over 5 years. They found that “process mattered more than analysis by a factor of 6”.

I have no use whatsoever for projections or forecasts. They create an illusion of apparent precision. The more meticulous they are, the more concerned you should be. We never look at projections … —Warren Buffett

This does not mean that analysis is unimportant and should not be undertaken. Rather it should be treated with caution as most economic forecasts are wrong. When making decisions it is essential that we explore uncertainties and encourage discussion of opinions that may contradict the views of senior stakeholders. This helps to prevent confirmation bias taking hold and gets people to acknowledge gaps in expertise. Behavioural economics challenges many assumptions about human decision making that have been unchallenged by conventional economic thinking.

 

Image of Dilbert cartoon where data with human biases is given to boss

Image Source: Dilbert.com

Intelligence and a high IQ are not normally associated with stupidity. But behavioural economics suggests that our propensity to make rash, foolish or irrational decisions is often not related to our IQ. No one is immune from making daft decisions and our reliance on IQ and educational qualifications as an indicator of competence can be a recipe for disaster. When the business culture gives too much reverence to people with certain qualifications and skills this can lead to rewarding decisions based mainly upon intuition rather than evidence.

CAVE Men! Colleagues Against Virtually Everything. People invest a lot of time and effort into their existing strategy or ideas. Dan Ariely calls this the not-invented here bias. People also have a tendency to value their own ideas significantly more than others’ ideas. This can result in obsessive focus on poor ideas and probably explains some of the less successful decisions that we come across in business.

Confirmation bias also means that we tend to ignore information that does not align with our existing beliefs. We subconsciously seek and are drawn to evidence that confirms our view of the world. People are very good at overlooking facts that undermine their opinions and will follow the crowd that most closely supports those beliefs.

Image of Dilbert cartoon with confirmation bias which behavioural economics identifies as a common reason for sub-optimal decisions

Image Source: Dilbert.com

So where does this leave the customer insight professional? It demonstrates the need for a comprehensive strategy for promoting the use of insight and collaboration to facilitate innovation and evidence based decision making.

  • Stakeholder management is essential not just to obtain the buy-in and support of senior management, but also to counteract many of the myths about how research and insight is undertaken.
  • Use storytelling to engage people at an emotional level. Our brains become more active when we are told a story, not only the language processing part of our brain, but also other areas we would normally use to experience the events of the story in real life. Some evidence suggests that our brains can synchronize with the brains of the person telling a story.
  • Spend time getting to understand your audience and their preconceptions. “What You See Is All There Is” tends to be strongly influenced by survey research when it comes to insight. Voice Of the Customer surveys are often given as an example of what research involves and yet these can be fundamentally flawed by relying on asking people direct questions.
  • Never underestimate the importance of how choices are presented and ensure you are fully prepared so that you avoid uncertainty about your recommended approach. Behavioural economics demonstrates that there is no such thing as a neutral choice architecture.
  •  Immerse yourself in the customer facing side of your business by meeting and observing how your organisation interacts with your customers. Don’t rely on third parties or management to identify the real challenges customer facing staff have to deal with.
  • Identify information gaps to highlight the need for research and insight. Our illusion of understanding sometimes needs reminding how little we really know about the world.
  • Challenge default methods of conducting research. Examine the potential for alternative approaches to insight, including experiments, observation and collaborative methods.
  • Encourage a culture of experimentation. For instance use A/B and multivariate testing on your website to understand what content most engages and motivates your existing and potential customers.
Image of Dilbert cartoon about AB testing and inability to predict winner of A/B test of optimiser

Image Source: Dilbert.com

 

  • To counter hindsight bias always ask key stakeholders before you commission a project what they expect the outcome/findings to be. You can then use these as hypothesis to prove or disprove their views.
  • Encourage all areas of the business to share insights and engage with the research process. It shouldn’t just be marketing and customer services that buy-in to customer insight. This helps avoid group think by bringing diversity into the decision making process.

Behavioural economics provides a valuable framework for developing strategies for behavioural change. It can also helps us to better understand some of the barriers in organisations to developing a culture of research and experimentation. This does have to come from the top of the organisation, but if you can get senior management on board then behavioural economics can be used throughout your organisation to assist in decision making and behavioural change.

 

Related posts: 

To access links to all my posts go to my archive.

Why are Voice Of the Customer surveys fundamentally flawed? A behavioural economics view of VOC surveys.

5 ways to get more valuable insights from your Voice Of the Customer programme. Strategies to ensure your VOC programme has impact and delivers genuine insights.

What does mountaineering tell us about human motivations? A look at how what people say is not consistent with what they do and what really motivates us humans.

Are most of our decisions the result of other’s behaviour and opinions? How social learning and copying drives much of our behaviour.

What do business people really think about market research? The misconceptions about market research from my personal experience on the client-side.

Is financial decision making more rational?A review of how rational people are when making purchasing decisions and how we use rules-of-thumb to reduce cognitive load assist decision making.

Thank you reading my post. If you found this useful please share with the social media icons on the page.

 

Further reading:

 

image

Thinking, Fast and Slow

image

Herd: How to Change Mass Behaviour by Harnessing Our True Nature

image

How to Get People to Do Stuff: Master the Art and Science of Persuasion and Motivation

image

 Decoded: The Science Behind Why We Buy

 

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as Deezer.com, Foxybingo.com, Very.co.uk, partypoker.com and Bgo.com. He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.