Does Usability Research Measure Reality?
Usability research is recognised as essential for checking whether a site or app is intuitive and easy to navigate to create a great customer experience. Usability research helps inform our decisions about the choice architecture. Remote usability research solutions or face-to-face user interviews are often used to identify the main usability problems, but do these methods of research reflect real behaviour?
How many usability research proposals acknowledge that the process of undertaking usability research can influence the behaviour we observe? We may have taken users out their natural environment and set them objectives that lead them to behave in a certain way.
Behavioural scientists have found that many of our decision are made automatically by our unconscious brain, and that the context and our underlying psychological goals heavily influence the choices we make. We also behave differently when we are aware that we are being observed.
Asking respondents direct questions is especially problematic as people over-think issues because they switch to their slow, rational brain when encountering a mentally demanding task. Unfortunately most of the time when we are browsing a website we rely on our fast, intuitive, unconscious brain to make decisions without really engaging our conscious thought processes. The implication here is that we cannot even access the rationale behind much of our behaviour when interacting with a website and so there is no point asking us to explain ourselves.
“People don’t have reliable insight into their mental processes, so there is no point asking them what they want.” Daniel Kahneman, Thinking, fast and slow.
Context is important:
Avoid taking people away from their natural environment if at all possible. Certainly don’t use focus groups as this is about far away of a normal browsing behaviour as you can get. How often do you search the web with a group of people you have never met and discuss your likes and dislikes of the site, with full knowledge that someone is observing you behind the mirror?
This is why remote user testing methods have an advantage over some face-to-face methods. Participants can be in their normal environment, with their normal distractions and so their behaviour is less likely to be influenced by the testing process. Don’t get me wrong, there will still be some bias as a result of the testing method, but it may be substantially less than techniques which take the user out of their normal browsing environment.
source: Remote user testing – UserZoom:
Observe and listen rather than ask:
You will get more meaningful insights from simply observing and listening to your users during a usability test as past behaviour is a more reliable indicator of future behaviour. Try to avoid verbal interventions as much as possible because people don’t like to admit when they do something wrong and you are likely to influence how they then behave in any future tasks. If you do want some verbal feedback, just ask your testers to say what they are doing as they go through the task.
But always keep in the back of your mind that usability testing is about informing your judgement, and not to prove or disprove someone’s opinions. It is also an iterative process that should begin early on in the development of a design and go through to after it has been implemented.
Implicit Research Methods:
As I have already mentioned, most of our daily choices are made by our fast, intuitive brain which means we don’t have time to rationalise why we are making those decisions. New implicit research techniques such as functional MRI, EEG, biometrics, eye tracking, facial decoding and implicit reaction time studies (IRTs) are allowing marketers to access the sub-conscious part of the brain to better understand how we respond to communications and designs.
Eye tracking research helps identify which specific elements of a page or message attract our attention, but also the communication hierarchy of messages. Heatmaps allows us to display this data to reveal the proportion of visitors who noticed each of the key elements on a page, plus the frequency and duration of gaze on each element.
Click and mouse movement heatmaps from visual analytics solutions such as Hotjar and Decibel Insights can provide similar insights for existing pages. For true eye tracking research though solutions from Affectiva and Sticky allow for you to evaluate both new and existing web page designs.
Source: Click Tale:
A/B Test Usability Testing Results:
In the final analysis the only way you will ever know for sure if a change identified through usability research improved agreed success metrics is to conduct an online controlled experiment in the form a A/B test. It is only when visitors are acting on their own impulses without any intervention from other parties and with their own money that you will see how they behave in reality on a website or app.
Prioritise the insights you get from usability testing to decide which are worthy of A/B testing and which should just be implemented as they are either no-brainers or unlikely to have a large impact on your success metrics. A/B testing will give you the evidence to show exactly how much difference your usability testing has had on your conversion success metrics.
If you found this post useful please share using the social media icons on this page.
You can view my full Digital Marketing and Optimization Toolbox here.
To browse links to all my posts on one page please click here.
- About the author: Neal provides digital marketing optimisation consultancy services and has worked for brands such as Deezer.com, Foxybingo.com, Very.co.uk, partypoker.com and Bgo.com. He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
- Neal has had articles published on website optimisation on CXL and Usabilla.com. As an ex-market research and insight manager he also had posts published on the GreenBook Blog research website. If you wish to contact us please send an email to firstname.lastname@example.org. You can follow us on Twitter @conversionupl, see his LinkedIn profile or connect on Facebook.