Category Archives: Website Optimization

Why Are Most A/B Test Results A Lie?

Are most A/B test results illusory?

In a report on A/B testing Martin Goodson at Qubit suggests that “most A/B winning test results are illusory”. Andre Morys at Web Arts goes even further and argues that “90% of test results are a lie.” If their estimates are correct then a lot of decisions are being made based upon invalid experiments and this might help explain why many non-CRO managers are sceptical about the sustainability of A/B test results.

So, why are some A/B test results invalid and what can we do about it?

1. Confirmation Bias:

Andre Morys suggests that confirmation bias results in many false positives. This is because optimisers naturally base most of their test hypothesis and designs on their own attitudes and opinions and ignore information that contradicts these ideas. As a result they become too emotionally attached to their design and stop experiments as soon as the testing software indicates they have a winner.

Stopping a test as soon as statistical confidence is achieved can be highly misleading because it does not allow for the length of the business cycle and it is important to also consider source of traffic and current marketing campaigns. You should run tests for a minimum two business cycles if possible so that you make allowance for day-to-day fluctuations and changing behaviour at weekends. Thus tests should run for at least 2 to 4 weeks depending upon your test design and the business cycle.

2. Survivorship Bias:

This refers to our propensity to focus on people who survive a journey (e.g. returning visitors or VIP customers), but ignore the fact that the process they have been through influences their characteristics and behaviour. Even returning visitors have survived in the sense that they weren’t put off by any negative aspects of the user experience. VIP customers may be your most profitable users but they are not a fixed pool of visitors and their level of intent will often be much higher than your normal user.

The danger is that by including returning users in for example an A/B test for a landing page experiment they will behave very differently from new users who have never seen the site before. For tests relating to existing users the process of excluding outliers can reduce problems with VIP customer influencing test results, but consider excluding VIP customers completely from you’re A/B tests as they do not reflect normal users.

For more on survivorship bias see my post: Don’t let this bias destroy your optimisation strategy!

3. Statistical Power:

Statistical power refers to the probability of identifying a genuine difference between two experiences in an experiment. To achieve a high level of statistical power though we have to build up a sufficiently large sample size and generate a reasonable uplift. However, when working in a commercial organisation there is often a lot of pressure to achieve quick results and move on to the next idea. Unfortunately this can sometimes undermine the testing process.

Image of confidence interval at 95% level

Before you begin a test you should estimate the sample size required to achieve a high level of statistical power, normally around 90%. This means that you should identify 9 out of 10 genuine test differences. Due to taking a sample of observations and the natural random variation this causes we know that tests will also sometimes generate a false positive. By convention this is normally set at 5%.

According to an analysis of 1700 A/B tests by only around 10% of tests achieve a statistically significant uplift. This means that if we run 100 tests we might expect 10 of those tests to generate a genuine uplift. However, given current traffic levels for each site we estimate that we would need to run each test for 2 months to achieve 80% power.  This means we should in theory identify 90% of uplifts or 9 tests. Using the p-value cut off of 5% we would also anticipate 5 false positives. Thus our tests generate 9 + 5 = 14 winning tests.

The danger here is that people are too impatient to allow a test showing an uplift to run for a full two months and so they decide to stop the test after just two weeks. The problem with this is that the much smaller sample size decreases the power of the test from 90% to perhaps as low as 30%.   Under this scenario we would now expect to achieve 3 genuine uplifts and still get 5 false positives. This means that 63% of your winning tests are not genuine uplifts.

Before running a test always use a sample size calculator  and estimate the length of time needed achieve your required statistical power. This allows you to consider the implications on the power of the test if you do decide to cut it short. Assume a much greater risk of a false positive if you do stop tests early.

4. Simpson’s Paradox:

Once you begin a test avoid altering the settings, the designs of the variant or the control and don’t change the traffic allocated to the variants during the experiment. Adjusting the traffic split for a variant during a test will potentially undermine the test result because of a phenomenon known as Simpson’s Paradox. This occurs when a trend in different groups of data vanishes when data from both groups is combined.

Experimenters at Microsoft experienced this issue when they allocated just 1% of traffic to a test variant on Friday, but increased this to 50% on Saturday. The site received one million daily visitors and although the variant had a higher conversion rate than the Control on both days, when the data was aggregated the variant appeared to have a lower overall conversion rate.

Image of A/B test from Microsoft which suffers from Simpson's Paradox
Image Source: Microsoft

This occurs because we are dealing with weighted averages. Saturday had a much lower conversion rate and as the variant had 50 times more traffic that day than it did on Friday it had a much greater impact on the overall result.

Simpson’s Paradox occurs when sampling is not uniform and so avoid making decisions on sub-groups (e.g. different sources of traffic or type of device) using aggregate data. This demonstrates the benefit of targeted tests for example where you are only looking at a single traffic source or type of device.

When you do run tests for multiple sources of traffic or user segments it is best to avoid using aggregate data and instead treat each source/page as a separate test variant. You can then arrange to run the test for each variant until you achieve the desired statistically significant result.

Altering traffic allocation during a test will also bias your results because it changes the sampling of your returning visitors. Because traffic allocation only affects new users a change in the share of traffic won’t adjust for any difference in returning visitor numbers that the initial traffic split generated.

5. Don’t validate you’re A/B testing software:

Sometimes companies begin using A/B testing software without proper validation that it is accurately measuring all key metrics for all user journeys. It is not uncommon for testing software not to be universally integrated because different teams are often responsible for platform, registration and check-out.

During the process of integration be careful to check that all different user journeys have been included as people have a tendency to prioritise what they perceive to be the most important paths. However, users rarely, if ever, follow the preferred “happy path”.

Once integration is complete it is then necessary to validate this through either running A/A tests or using web analytics to confirm metrics are being measured correctly. It is also advisable to check that both testing software and web analytics align with your data warehouse as if there is any discrepancy it is better to know before you run a test rather than when you present results to senior management.

6. Regression To The Mean:

There is a real danger of looking at test results in the first few days that you see a large uplift (or fall) and you mention it to your boss or other members of the team.  Everyone then gets excited and there is pressure on you to end the test early to benefit from the uplift or to limit the damage. Often though, this large difference in performance gradually evaporates over a number of days or weeks.

Image of A/B test showing signs of regression to the mean


Never fall into this trap as what you are seeing here is regression to the mean. This is a phenomenon that if a metric is extreme the first time it is measured it will tend to move towards the mean on subsequent observations. Small sample sizes of course tend to generate extreme results and so be careful not to read anything into your conversion rate when a test first starts to generate numbers.

7. The Fallacy Of Session Based Metrics:

Most A/B testing software uses standard statistical tests to determine whether the performance of a variant is likely to be significantly different from the Control. This is based upon the assumption that each observation is independent.

However, if you use session level metrics such as conversion per session you have a problem. A/B testing software allocates users into either group A or B to prevent the same visitor seeing both variants and to ensure a consistent user experience. Sessions are therefore not independent as a user can have multiple sessions.

Analysis by Skyscanner has shown that a visitor is more likely to have converted if they have had multiple sessions. On the other hand an individual session is less likely to have converted if made by a user with many sessions.

This lack of independence is a concern as Skyscanner simulated how this affects their conversion rate estimates. They discovered that when they randomly selected users rather than sessions, as occurs in an A/B test, the variance is greater than assumed in significance calculations.

Image of estimated variance for session randomised and user randomised demonstrates the fallacy of session based metrics
Source Image: Skyscanner

Skyscanner found that the effect is greater with longer experiments due to the average number of sessions being higher. What this means is that month-long tests based on session conversion rate (i.e. users randomised) would have three times as many false positives as normally expected. However, when the test was based on users (i.e. randomised sessions irrespective of user) the variance conformed to that predicted by significance calculations.

Further, this problem also occurs whenever you use a rate metric that is not defined by what you randomised on. So per-page view, per-click or click-through rate metrics will also be subject to the same problem if you are randomising on users. The team at Skyscanner suggest three ways of avoiding being misled by this statistical phenomenon.

  1. Keep to user-level metrics when randomising on users and you will normally avoid an increased rate of false positives.
  2. When it is necessary to use a metric that will be subject to the increased propensity of false positives there are methods of estimating the true variance and calculating accurate p-values. Check this paper from the Microsoft team and also this one.
  3. As calculating the true variance and accurate p-values can be computationally complex and time consuming you can just accept a higher false positive rate. Use AA tests to estimate how much the metric variance is inflated by the statistical phenomenon.


When trying to avoid these pitfalls of A/B testing the key is to have a strong process for developing and running your experiments. A good framework for testing ensures that hypotheses are based upon evidence rather than intuition and that you agree the test parameters upfront. Make sure you calculate the sample size required and how long your test will need to run to achieve the statistical power you want to reach.

There is an opportunity cost to running tests to their full length and so sometimes you may want to end tests early. This can be fine if you allow for the lower level of statistical confidence and the increased risk of a false positive. Indeed, there is evidence to suggest that provided you are continuously running tests this can largely compensate for any increase in the rate of false positives.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as,,, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.


How To Undertake a Heuristic Evaluation of a Website

How does a heuristic evaluation improve conversion?

A heuristic evaluation is an expert based analysis that applies experience-based techniques to identify and discover solutions to problems with a user experience. It is a structured approach to assessing an existing user design and though it offers no guarantees about being right, it is an important initial step in a systematic approach to optimisation.

A heuristic evaluation is best undertaken as a group of 3 or more experts and if possible some customers to allow for an open discussion and detailed analysis. The more experience you have with online experiments the better as they provide real evidence to support your heuristic evaluation. As optimisation is a data-informed process it is important to always look for evidence to support and refute any hypothesis generated by a heuristic evaluation.

Why should you use heuristic evaluation?

As we become more familiar with our websites or apps we become more comfortable with the user experience and often can’t see even obvious problems with aspects of the design. We also tend to become defensive when we receive criticism and less innovative.

A heuristic evaluation allows you to challenge this status quo with a structured framework and experts who have a fresh perspective of the user experience.

Recruiting customers for the evaluation team ensures user feedback is also included in the review process.

A framework for heuristic evaluation:

At Conversion Uplift we base our analysis on the Widerfunnel Lift Model because this provides an excellent framework to review a digital user experience. The model highlights the importance of the value proposition which reminds us that you are going to be handicapped if you have a weak or inferior product/service.

The lift model is a framework we use to help optimise the digital experience


So, if your product/service is deficient in some way make sure you fix that first before trying to optimise your user experience. Sometimes people think that they can use persuasive techniques to convince visitors to purchase a poor product or service. That’s not a sustainable strategy in the long-run as you won’t build up a loyal customer base.

Also define your target audience before you begin the process. It is important you have a good understanding of the goals and possible use cases of your customers or prospects. If you have developed buyer personas these are useful when considering such factors as relevancy, anxiety and segmentation.

1. Value proposition:

Begin by taking a look at what your site tells the visitor about your value proposition. Assume your visitors don’t know anything about your brand, what does your homepage or landing page communicate about your proposition? You should also ask:

  • Why should users come to your site rather than a competitor’s?
  • What’s unique about your brand?
  • What tangible features does your brand communicate?
  • What incentives and offers are there to attract new customers?

Psychological goals:

As an addition to the model we have added psychological goals. These are those implicit motivations, such as belonging, power and self-development that drive our attention and behaviour.

We use the Beyond Reason implicit goal model which is based upon the latest research from the fields of psychology and neuroscience. Try and identify important psychological motivations by listening to how users feel about your product or service or by conducting market research (see implicit association test).

Psychological motivations drive attention and much our behaviour.
This motivation model is the intellectual property of BEYOND REASON.


Intangible benefits:

Next consider what evidence you have on your site to demonstrate intangible benefits such as credibility (e.g. professional reviews & awards) and social proof (e.g. testimonials and customer ratings). These can be powerful drivers of behaviour, especially if your user’s decision style is characterised by copying experts or their peers (see Decision Styles).

Image of testimonials from and Google Analytics

You should also review how your tangible costs are communicated and consider how you use packages as anchors (e.g. whether you show the most expensive first). Do you use exclusive pricing, are delivery costs clearly visible and could you use the power of ‘free’ to improve how costs are communicated?

2. Relevance:

Understanding who your visitors are and where they come from is key to relevance.  Check source relevance, the relevance to ads they may have seen, keywords that they may have used and search mode patterns are easily recognisable. Relevance can be improved through:

  • Targeting – Direct ads to most relevant landing page.
  • Dynamic customisation – Insert dynamic headlines or text snippets to improve the scent trail.
  • Standardization – Aggregate target segments into larger, similar groups to allow testing the overall value proposition.
  • Target audience relevance – Does the message meet prospects needs and  expectations and does it appeal to them?
  • Segmentation – Are you using customer value or behavioural targeting?
  • Call-to-action relevance – Are visitors ready to buy/register – what allowance is there if they are not – is relevant information available?
  • Tone relevance – Users respond differently to words, images and design according to their demographic characteristics.
  • Navigation relevance – Is it intuitive and can visitors find what they are looking for?
  • Competitor relevance – How does competitor’s value proposition and messages influence your conversion rate?
Image of homepage with different CTAs for separate user goals
Image Source:

3. Clarity:

Clarity helps minimise mental processing by reducing the cognitive load for visitors. Begin with evaluating how clear your value proposition is communicated and the prominence of your call-to-actions. When evaluating clarity consider the following elements:

  • Information hierarchy – How well is content organised on the page and via navigation?
  • Page-level hierarchy – Does eye-flow and content hierarchy aid decision making? Are all key messages above the fold?
  • Design clarity – Does it reinforce the value proposition, facilitate communication, promote content, legibility and readability?
  • Eye-flow clarity – Does it guide visitor’s eye or block its flow?
  • Image and colour clarity – What do they communicate?
  • Visual consistency – Are you following major web conventions?
  • Call-to-action clarity – Do you lose early and often?
  • Copy writing clarity – Have you tested headlines and support claims with evidence? A lack of evidence can lead to friction in the user journey.

4. Anxiety:

Unless you try to deal with anxiety your site will suffer from friction as anxiety can undermine trust and lead to visitors delaying decisions. To minimise anxiety it is necessary to consider the following points:

  • Privacy – Do you keep information requests to a minimum, justify the information you request and reassure visitors of your trustworthiness?
  • Forms – Have you moved optional fields to a thank you page?
  • Policy – Summarised privacy policy using plain English?
  • Usability – Allow for user mistakes & errors?
  • Error handling – Serve error messages adjacent to field & provide prompt feedback?
  • CAPTCHA – Use software to prevent bots rather than CAPTCHA?
  • Browser compatibility – Test on all major browsers & operating systems?
  • Technical problems – Check & solve technical glitches as these lead to a loss of credibility?
  • Page not found – Minimise 404 errors and display user friendly 404?
  • Effort anxiety – Are offers easy to redeem/use?
  • Fulfilment anxiety – Do your messages reassure users?
  • Security – Do you display familiar security seals when appropriate?
  • Fine print – Legal asterisk losses credibility & small print can ring alarm bells.
  • Brand reputation – Do you display third-party credibility indicators to reduce anxiety?
  • Delivery promise – Do you set clear expectations & clear fulfilment promise?
  • Guarantees & returns – Do you offer guarantees (e.g. money back) to reduce fulfilment anxiety?

5. Distraction:

A busy design with little white space and unnecessary content can be very distracting for users. It’s important that you communicate a compelling idea or key message to get your user’s attention. First impressions count and this is why the visual layout above the fold is a primary determinant of new visitor perceptions of your site. When considering distractions look at:

  • Find the most common user screen size – Do you design pages based upon user’s monitor size and not your designer’s screen?
  • Background – Do you use background textures, asymmetrical designs and complex graphics in backgrounds?
  • Where now – Is the starting point of the user’s journey obvious?
  • CTAs  – Does the design create a focused eye flow towards a single, prominent CTA?
  • Messages – Does your design focus on a single, clear message to minimise mental processing?
  • Irrelevant content – Do you have ads or existing customer content that is irrelevant to some users?
  • Navigation bars – Have you removed unnecessary navigation on landing pages or is your navigation sticky?

6. Urgency:

This is about considering if there is anything that will help motivate users to make an immediate decision. There are two types of urgency, internal and external urgency.

Internal urgency is dependent upon how the user feels and their needs or situation. This is more difficult to  influence, but we should make allowance for it if we can. Amazon for example uses the “Buy Now” CTA to allow users who are impatient to shorten the user journey and accept the default settings (e.g. one day delivery and related charges).

Image of product page with Buy Now CTA
Image Source:

External urgency refers to how marketing can influence or persuade users to make a decision now rather than later. This could be the use of a limited offer period for a promotion to create scarcity and use loss aversion to motivate visitors to take action now. Points to consider include:

  • Create internal urgency – Use an emotional appeal using descriptive copy and evocative imagery or create an aspirational story that resonates with visitor’s desires.
  • Scarcity – An explicit offer period creates an urgency to act, limited access offers or special editions also create scarcity. The fear of loss is also important motivator.
  • Respond with urgency – Treat customer enquires as time sensitive. Research indicates that contacting a lead within 5 minutes has 100 times greater success rate compared to if they are contacted after 30 minutes.

Ryanair uses scarcity with a flash sale to create urgency. The email clearly states that the sale ends on a specific date to encourage users to act now.

Image of flash sale from
Image Source:

What next?

Once you have completed your heuristic evaluation you should list out all the areas of interest (AOI) you identified for each page in a spreadsheet. There is a good chance that you will have more AOI’s than you can deal with at once. Whether this is the case or not you should go through each page and priorities the AOI’s so that you can focus on the most important aspects to improve your conversion rate.

You can use the prioritisation process that I outline here to determine how you treat each AOI. The important thing here is that you present your findings and recommendations to key stakeholders to get their buy–in and agreement on your priorities.


Conducting a heuristic evaluation is a valuable step in the optimisation process. It should be used to bring in experts from outside web team and if possible include customers or prospects.

Using the Widerfunnel Lift Model provides a great framework and gives everyone permission to be as critical as they can be of the existing user experience. This can help challenge some of the sacred cows, such as brand guidelines, that can prevent the business moving forward.

A heuristic evaluation can be especially useful when your website does not have sufficient traffic to run A/B and multivariate tests. But don’t use it in isolation. Always follow a systematic approach to optimisation as I outlined in my post on the 8 steps of conversion rate optimisation.

Here is a link to a cheat sheet for your own evaluation.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as,,, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.



What is the most effective strategy for innovation?

Are there many original ideas?

Have you noticed the obsession with originality and the desire to be first to market? Yet most of  the innovations that have shaped our world today are based on ideas or technology that were nothing new at the time.

For instance, Apple didn’t invent the touch-screen smart phone. IBM created the first smart phone (known as the Simon Personal Communicator), fifteen years before Apple launched the iPhone and the touch-screen was invented as far back as 1965. Apple took existing technology and made it their own through exceptional design and a determination to provide a great user experience.

Neither was MacDonald’s the first fast-food chain. It copied the idea from the White Castle restaurant in Wichita, Kansas. Facebook wasn’t the first social network and Henry Ford didn’t invent the car. So, innovation does not have to be about invention, it’s more likely to be about copying.

Why copy?

In his book Copy,Copy,Copy, Mark Earls suggests that copying (but not replicating) of ideas and experiences is something we naturally do and is the most efficient method of innovation. Copying is easy and is a great hack for making fast progress and not looking at problems in isolation. You get to benefit from the mistakes and the success of others that you have never even met. This may explain why being first to market with a new innovation is rarely an advantage.

What is poor copying?

Simply replicating an idea or product and the practice of benchmarking reduces value because it creates increasingly homogenised offers and there is no allowance for context or thought about fixing what’s broken.  Indeed, benchmarking damages profitability because of increased price-point competition and less differentiation.

‘Looser copying’ or from afar (e.g. from a different sector or culture) can add value because it creates variation through the changes or mistakes we make. This is essentially the concept of fixing what’s broken and keeping what works.

Why don’t we copy more?

In Western cultures we are often led to believe that each problem or challenge is unique and so requires a unique solution. This is almost always misguided because very few problems are new and it  gives the impression that it will be extremely difficult to solve. This often results in the rejection of many good ideas simply because of their similarity to solutions used elsewhere.

This is partly because of the individualistic culture we have in the West which values originality and discourages copying. This is not the case in China and other parts of Asia where copying is perceived in a much more positive light. The danger is that this attitude towards copying puts Western companies at a disadvantage. Why reinvent the wheel or if it’s not broken, why fix it?

“The truth is that most things are like other things – they certainly seem that way: the human mind’s ability to see similarities and connections between things is unbounded.” – Mark Earls, Copy,Copy,Copy

Are most innovations the product of a single individual? Nope, it’s a myth that innovations are the creation of a single person. In most cases many people are involved in innovation and they copy from different sources to create something new and exciting. What this suggests is that when done in the right way copying can produce novel and highly effective solutions.

Copying is what we do naturally:

Mark points out that because of our brain’s limited processing power we are forced to rely heavily on default heuristics – what did we do last time or what is everyone else doing? We copy ideas and behaviours all the time but often are not even aware of it. Copying allows us to outsource many aspects of cognitive thinking and provides us with a knowledge bank that further reduces the cognitive effort we may need to solve a problem.

Marketers needs to understand copying:

When we see brand shares following the Pareto 80/20 rule diffusion science tells us that people are not acting independently. Copying makes the most popular brands even more popular than they would be if everyone acted independently.

This produces a long-tail distribution of popularity (see right-hand curve) which is a strong indication of social influence and copying.  Otherwise you are likely to be looking at independent choice (as shown by the short-tail distribution).

Image of popularity distribution for independent and social decision making
Image Source: Copy,Copy,Copy – Mark Earls


The data shows that buyer choices in many markets and services is mainly driven by either directed copying (e.g. following experts or authority figures) or undirected copying (following the most popular variant). Just look at the power of social proof and we can see how important what other people are doing is in influencing sales.

“The paradox of conformity, of course, is that we are all conformists in some way, and yet we do not all do the same thing – far from it. In fact, we find the greatest diversity of behaviors in places where people are most densely packed together, such as New York, London, or Istanbul.” – Mark Earls – I’ll Have What She’s Having

In some markets we are now faced with so many choices that it is practically impossible to identify an ideal product or service to meet your needs. Just imagine trying to evaluate every TV or smart phone. In such instances we can either make an educated guess  or follow the behaviour of others.

Image of decision map showing informed and uninformed choice
Image Source: Copy,Copy,Copy – Mark Earls


By bringing these insights together Mark Earls and his co-authors in I’ll Have What She’s Having created a map of four decision making styles for marketers to use. In his latest book Mark Earls has used this map to create a pattern book of over 50 marketing strategies to encourage marketers to innovate using approaches that have been successful in other sectors. This is a valuable resource as it allows you to focus on execution rather than creating strategies from scratch.

Image of decision styles map with 4 quadrants
Source: Copy,Copy,Copy – Mark Earls

Implications for website optimisation:

  • Stop replicating ideas and designs from your competitors:

Far too many sites are obsessed with what their competitors’ are doing and copy exactly what they see them doing. This results in websites looking almost identical and reduces differentiation. Instead look outside your sector to see what is new and innovative. Copy loosely and then test to see if it adds value before rolling it out on your whole site.

  • Think about design trends:

Have you noticed how many websites have blindly followed certain design trends? Whether it’s over-sized hero images, auto-play video backgrounds, auto-sliders, flat design, parallax scrolling or the hamburger button on mobile designs. Online experiments have shown all of these trends damage conversion and sometimes frustrate and annoy visitors.

Instead of simply replicating new trends, why not consider what prompted the trend and the truths it embodies. You can then copy the elements or ideas that appear to be most beneficial and test them on your site to measure what impact they have on visitor behaviour. This way you can maintain your site’s unique character and still benefit from new developments or trends.

  • Be agile:

I recently participated in a 5 day Google design sprint to develop a new online brand’s proposition and website.  We spent a lot of time looking outside the sector of interest to get inspiration. We copied the elements of the market that we thought worked, but wanted to fix what was broken. We achieved this through taking ideas from many different fields and rapid iteration of designs.

Within the 5 days we created and tested a prototype design and helped give the project the momentum it needed. The whole agile process is based upon copying as otherwise it would require too much time and effort to make progress.

  • Identify your market’s decision style:

Assess which quadrant of the decision style map your market is most likely to sit in. You can then focus on the marketing strategies that are most likely to align with this decision style. As part of this process it also useful to identify other markets that have the same decision style to your own. This will allow you investigate how they apply appropriate marketing strategies and you may find novel ideas that you can incorporate in your own strategies.

It is possible that your market could fall between two quadrants and if this is the case it is worth to considering strategies from more than one quadrant. For example a decision about the preferred method of delivery of a service (e.g. offline or online) might be in the North West quadrant (i.e. considered choice), but the brand choice may be in the North East quadrant (i.e. copy experts).


Originality is overrated as most things are similar to something that already exists. This means copying allows us to benefit from other peoples’ mistakes and create novel solutions to difficult problems. However, just replicating what we see elsewhere can reduce value and lead to a lack of differentiation.

For marketers copying is often an important market dynamic that can heavily influence a customer’s decision style. Understanding the most common decision style in your market helps us identify appropriate marketing strategies to apply. Copying relevant marketing strategies from other sectors or cultures allows us to concentrate on the execution and learn from how others to create novel solutions.

For more details about how to copy effectively and how to use the decision styles map to execute relevant marketing strategies please get  Copy,Copy,Copy by Mark Earls.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as,, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @conversionupl, see his LinkedIn profile or connect on Facebook.


7 Strategies For B2B Marketing Personalisation

How to use personalisation in B2B digital marketing!

Personalisation and marketing automation are now proven strategies in digital B2C marketing for increasing conversion rates and revenues. For example eConsultancy  estimate that the suggest feature on generates an additional 10 to 30 per cent in revenues. Given the huge sales on Amazon this generates many millions of dollars for the company.

Amazon’s “Customers who bought this item also bought”

Image of Amazon recommendation results for customers who bought this
Image Source:


B2B websites have been significantly slower to adopt personalisation. However, according to research by Forrester B2B buyers are increasing their use of digital channels to research and complete transactions while still using more conventional channels during the customer journey.

Further, their experiences with B2C online sites has raised expectations of B2B sites so that buyers now demand much higher levels of service and personalisation during the user entire journey.

How do B2B sites up their game and begin to deliver on personalisation? In this post I will cover the following seven topics regarding B2B personalisation:

  • Personalisation vs customization – How are they different?
  • Enable B2B users with personalisation – Why it’s not all about selling in B2B?
  • Getting data for B2B personalisation – What and how to get it?
  • Choose a personalisation platform – What are your options?
  • Segmenting your B2B customers – What strategies should you use?
  • Implementing personalisation – Strategies for creating a personalised experience!
  • When personalisation goes wrong – Why does personalisation sometimes go wrong?

1. Personalisation or customisation?

These two terms are often confused with each other. Customisation refers to where prospects are offered a distinct choice, perhaps to indicate their sector or occupation. This allows the website to respond to this selection by making the experience more relevant to the user’s needs.

Customisation allows users to remain in control and select exactly what they want. The downside is that visitors don’t always know what they need or how a solution could improve the efficiency of their business. But customisation definitely has a place for many B2B websites, especially for first time visitors where you don’t know anything about their needs or the nature of their organisation.

B2B E-commerce site using pop-up to enable customisation

Image of customisation approach on
Image Source:


Personalization on the other hand refers to automatically serving content that is relevant to the individual visitor based upon data captured on the user. The advantage of personalization is that it does need any additional effort from the user because your automation platform does the hard work.

However, this does mean you are reliant on the data and analytics of your marketing platform to identify or infer each user’s needs. In some cases this can feel a bit creepy as visitors find that the website is too good at anticipating their preferences. We’ll deal with this issue later after we examine how to personalise the B2B customer experience.

2. A complex user journey:

B2B marketers have to target high-level decision-makers or a group of individuals and face a significantly longer business cycle than most B2C marketers. The B2B user journey is often has many more touch points and a complexity that appears to make personalisation more difficult to achieve.

Strategies for B2B Personalisation

B2B personalisation has the potential to significantly increase website revenues

The solution to this challenge is to create a user experience that enables visitors rather than sells to them. The B2B buyer will normally have to consult colleagues and is often heavily influenced by peers outside their organisation. To facilitate this process it’s important to create a user experience that automatically adapts to the behaviour and interests of the visitor rather than trying to push them towards a purchase.

3. Getting data for B2B personalisation:

Before you spend time and money on your data you should first identify what will be genuinely valued by your customer and what you need to know to achieve that. Start with the end in mind rather than the other way around.

The difficulty faced by B2B marketers is combining data about an individual visitor with information held about their organization in a meaningful and actionable way. Further, it’s important to combine all types of data (e.g. quantitative or qualitative) and channels (e.g. offline and online) to capture information on all touch points to build a picture of the prospect’s behaviour and interests.

Once you have the data you need you can begin to systematically generate inferences from your visitor and customer database to identify insights and create opportunities to personalise the customer experience.

Sometimes expertise is underrated and companies recruit junior people with little experience. Saving money on personnel is counterproductive.  A good data analyst or data scientist understands the capabilities of analytical and marketing automation tools. They know how to get the most value from your data solutions and can identify and develop new opportunities for personalisation without having to be given specific directions from marketing management.

However, obtaining data for personalization is often simpler than you think. Most web analytics software for instance captures a wealth of data on the characteristics and behaviour of your visitors. Google Analytics for example can provide you with data to personalize as follows:

  • New vs returning visitors – Someone who has visited your site more than once in the last 24 hours is likely to different motivations and concerns than someone who visits your site for the first time. Even a simple welcome back message can acknowledge that you recognize their high level of interest in your product or service.
  • Landing page type – Use your knowledge of the type of page a visitor lands on when they arrive on your site to personalize their experience by ensuring consistent messaging and content is displayed throughout the user journey. Don’t waste money on building dedicated landing pages if you are not going to use that knowledge to personalize subsequent screens with relevant content.
  • Day of week / Time of day – Does a visitor who browses your site on a Friday afternoon have different intentions and motivations than someone who is browsing your site on Monday morning? What about weekend traffic – some businesses don’t close on a Friday afternoon – shouldn’t your content reflect this demand at the weekend?
  • Conversion funnels – Where do visitors drop out of your funnel most often? Why not examine how you can personalize these stages in the process to improve engagement and reduce drop-out rates.
  • Page load time – Why not acknowledge when your site is taking longer than normal to load by personalizing messaging or content to win visitors back. People like it when companies say sorry.
  • IP address – Most medium to large companies have a unique IP address and so you should be able to identify key organisations to target from their IP address.


4. Choosing a personalisation platform:

According to a study by Gleanster for Act-On some 83 per cent of B2B marketers believe fragmented marketing platforms and systems prevent the implementation of marketing automation. This suggests that many B2B marketers either don’t have personalisation software or have not been able to fully integrate it with existing systems.

Whatever your situation is begin by deciding what your goal is. Is it to increase the conversion rate of lead generation efforts, improve cross-sales, encourage repeat purchases or reduce cart abandonment? You can then set KPIs to monitor progress and begin considering personalization techniques.

You can then start to look at what software is needed. There are generally two types of software available, A/B & multivariate testing solutions that offer personalisation as a feature and dedicated marketing automation software.  The former include Adobe Marketing Cloud, Google Optimize 360, Optimizely, Oracle and Quibit. If you already have an A/B or multivariate testing tool check out its capabilities for personalisation as you may find this can meet some of your needs.

Top dedicated personalisation software companies include Act-On, Acquia, Baynote, BevyUp, Boomtown, Certona, Dynamic Yield, Edgeverve, Evergage, Flytxt, IgnitionOne, Magnetic, nectarOM, Peerius, Real-Time, Syntasa and Strands.

A key consideration here is finding a platform that integrates with your legacy systems and CRM solution in particular. Personalisation requires real-time access to data across all customer touch points and so it may be necessary to establish a network of technology partners who have the experience and knowledge to plug gaps where they exist in your internal capabilities.

5. Segmenting your B2B customers:

A popular strategy for creating a highly personalised customer experience is the buyer persona. This involves building a detailed profile of important user segments which includes demographic and firmographic data such as job title, function, management level, budgetary responsibility, and industry sector.

Buyer persona template online tools

Creating a buyer persona for each key customer segment using data and research allows organizations to improve their understanding of customers and prospects which enables them to construct a more personalized and targeted customer experience. Other strategies for implementing B2B personalization include:

  1. Segment specific – Use industry vertical or customer segment criteria
  2. Stage specific – Apply personalization according to stage in the buying process
  3. Account specific – Use details of the prospect organization to tailor the experience
  4. Lead specific – Tailor according to details of the individual lead

Examples of how you can personalize the user experience include:

  • IP address for large organisations to target individual companies.
  • Geographical data such as city, region, country or seasonal factors.
  • Behaviour on device (desktop, mobile and tablet).
  • Demographics such as gender, age or cultural background.
  • First party data – information that you have captured yourself and your customers are aware you hold.
  • Third party data – information from CRM or social media and other third-party sources that users may not be aware you hold about them.

The important point to consider here is to use experimentation to identify what works and what doesn’t for your prospects. Best practice is only a guide and should not be taken as gospel.

6. Implementing B2B personalisation:

Now that you have your detailed buyer personas you can identify key characteristics or behaviours that allow you to allocate a visitor to a specific persona. Use these criteria to segment your email list into, smaller, more targeted lists. This will allow you to deliver personalised email campaigns based upon important drivers of behaviour such as sector, job title and management level.

Image of personalised email from 47 Links

Using your buyer personas to classify web visitors should also allow you to deliver a highly personalised web customer experience to replace generic and static web content that can often be a turn-off for users. As Karl Wirth, CEO of Evergage, points out there are four core principles of user experience which marketers need to consider. These are remember me, understand me, help me and surprise / delight me.

  • Remember is about retaining and utilising information about the user’s behaviour or profile to deliver a personalised customer experience. This means acknowledging returning visitors and ensuring the user experience reflects past behaviour and previously captured on the prospect.
  • Understand me relates to recognising returning visitors and applying the knowledge the organisation holds on customers to deliver content based upon their known interests and needs.
  • Help me is about making it easy and enjoyable for visitors to achieve their goals. This involves monitoring user browsing behaviour and purchasing history to provide relevant recommendations and directing customers to useful information or recently viewed items. Don’t make it difficult for users to find what they are looking for.
  • Surprise and delight may seem difficult to achieve online, but in reality this is what personalization is about. Going that little extra to acknowledge good customers and inform visitors of relevant and available offers can go a long way to make users feel valued.

Buyer personas also make it easier for you to identify potential prospects in the social sphere. Listening to conversations people have on social media and having one-to-one dialogue with prospects can allow you to better understand their problems and needs so that you can tailor content accordingly.

Engaging with prospects on social media demonstrates to your audience that you value their input and helps create the ultimate personalised user experience.

7. When personalisation goes wrong:

Predicting behaviors is never easy and so it is inevitable that sometimes you will get it wrong. Poor personalization leads to a poor user experience. You can minimize this by doing extensive qualitative and quantitative research to better understand your audience and ensure your buyer personas are based upon real customer segments.

Personalization should only be used when it’s helpful and has a clear benefit for the user. It should be intuitive, useful and create a natural user experience. It should not feel “creepy” or like Big Brother is watching you.

It’s important to be open and transparent with customers about using data to personalise the user experience to manage expectations and reduce the chance that it becomes “creepy”. Personalisation can become problematic when organisations rely too heavily on inferred or purchased data that has not been freely given by users. When customers voluntarily give information or confirm the accuracy of data this reduces the likelihood that it will be perceived as “creepy” when it’s used to predict customer needs.

This example below of personalisation from Evergage allows the visitor to see why content has been recommended to them. This level of transparency helps to avoid personalisation becoming “creepy”.

Image of personalisation on
Image Source:

There also needs to be a value exchange for all concerned and so it is important to set out what personalisation means for different buyer personas or segments to fully understand the benefits for all parties. Holding user data is a privilege and so it is essential to set high standards for how it is stored and used.

The rise of big data has led to a culture of hoarding data that companies don’t use and visitors don’t know they have. Like any asset data deteriorates over time and so it is important to regularly review and cleanse data to ensure it still usable. To avoid storing up problems for the future you should ask two questions every time you capture a new item of data:

  1. What will the customer’s attitude be towards us holding this data?
  2. What do we want the customer to do?

Unless you can clearly answer these questions it is best not to use the information. In addition, ensure your data base architecture enables you to identify which data was given freely and which was bought from third-parties.

Don’t forget to build in self-service alternatives by designing your information architecture to enable users to easily locate what they are looking for without having to rely on personalized content. This helps to prevent problems occurring when your personalization goes wrong.


Personalisation should begin and end with what’s best for the customer. Set clear goals for what you want to achieve and invest in both qualitative and quantitative research to get a much deep understanding of what motivates your prospects. Create strong buyer personas based upon evidence to help guide your strategy.

Avoid hoarding data for the sake of it. Always have a clear view of your objective and seek out data that allows you to achieve that goal. Give priority to data that has been given freely by visitors and don’t be over-reliant on making inferences from third-party data.

The challenges of B2B personalization reinforce the need to have the right tools for the job. Even the best research and insight is of little value unless you have a platform that is capable of extremely fast delivery of content and is scalable.

There will always be instances when personalisation doesn’t work and so build in self-service options to enable visitors to find what they are looking for. Avoid over-reliance on third-party data and be open with your visitors about how data is used.

However, when used effectively B2B personalisation can be a powerful strategy for improving the customer experience and for increasing conversions. Just because the B2B decision making process is more complex this should not be a barrier to using personalisation to generate more revenues from your digital user experience.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @conversionupl, see his LinkedIn profile or connect on Facebook.

5 Free Digital Tools That Every Start-Up Should Have

Get free tools to increase Conversion!

When you establish a digital start-up budgets are often very tight and yet it is important to have visibility of how your business is performing in the digital space.  When I advise start-ups on how to optimise their website the first thing I do is set them up with five free tools to begin this process. You can’t expect to understand how to improve something unless you have feedback and if possible you can measure it.

1. Google Analytics:

This is an excellent web analytics solution that allows you to understand more about who your visitors are (e.g. demographics and device) and also what content is most popular on your site. If you have a blog this is essential as otherwise how will you know which posts are generating most interest from your visitors.

Some of the key metrics you should be checking on GA include:

  • Mix of new to returning visitors – is your content attracting users back to your site,
  • Bounce rate – are they viewing more than one page,
  • Average time on page – how engaged are they with your content,
  • Source of traffic – how well is your SEO, social media and paid marketing working,
  • Device – what proportion of visitors use a mobile, tablet and desktop device,
  • Demographics – age, gender and interests of visitors
Demographic data from Google Analytics
Image Source: Google Analytics


Even if start-ups do have GA I often find they haven’t configured it properly and are getting false readings. Make sure you set up filters to exclude internal traffic by using your IPs and  set a filter for language spam. You can also link the account to your Adwords and Adsense accounts. For more details see this post – I’ve installed Google Analytics – Now what?

There are many web analytics packages on the market, but Google Analytics is one of a few that offers free permanent access to such a comprehensive and robust analytics platform. See my article on web analytics solutions for more details of how to use GA.

2. Google Search Console 

Previously called Webmaster Tools, the Search Console is a comprehensive tool for understanding how your site is performing in Google Search, but also what might be holding you back in getting more organic traffic.

Image of Google Search Console Dashboard
Image Source: Google Search Console Dashboard

The Search Console includes a really useful dashboard (see above)  which shows a graph of how many clicks your site has received from Google in the last 28 days, plus the number of site errors, including page not founds (404s) and the number of pages that have been submitted and ranked by Google.

The console has five primary navigation elements that you should be checking on a regular basis.

  1. Search Appearance:
  • This is mainly more technical SEO data including structured data,  rich snippets, HTML  improvements and AMPs (Accelerated Mobile Pages).

2. Search Traffic:

  • This provides data on search results, including clicks, impressions, CTR and average position. In addition it also shows number of external links to your site and what they link to, internal links, security issues and mobile usability issues.

3. Google Index:

  • This shows the number of pages on your site that have been indexed, blocked resources and remove URLs.

4. Crawl:

  • This section provides data on errors detected by Googlebots and the fetch as Google function allows you to submit a URL for indexing. The sitemap tab allows you to submit a new sitemap for your site to Google.

5. Security:

  • Finally, the security tab tells you if have Google has identified any potential security problems with your site.

For more in-depth instructions see my post; How to use Google’s Search Console to improve conversions. 


3. Optimize:

In 2017 Google’s introduced Optimize its free version of its testing and personalisation solution Google Optimize 360. Optimize enables you to run up to 3 A/B tests (or multivariate tests) at a time and provides a includes a visual editor to set up simple experiments in a matter of minutes. It also allows you to run redirect experiments where you test a totally new page against the existing experience. As you would expect it integrates fully with Google Analytics.

Image of types of experiments in Google Optimize
Image Source: Google Optimize

This is a simple but effective testing tool for companies not intending to conduct more than three simultaneous experiments or personalisation campaigns. It’s easy to use and is an ideal solution for start-ups that wish to use a data driven approach to website development.

Read my post, How to get started with Google Optimize for more details of this free solution.

4. Hotjar :

Hotjar is a brilliant tool as it combines behavioural analytics, such as session recordings, with easy to use visitor feedback tools. Provided you have less than 2,000 page views per day you can sign up for their free basic plan. For sites up to 10,000 page views a day the start-up plan costs only €29 a month.

Image of Hotjar dashboard
Image Source: Hotjar Dashboard


The session recording and replay feature is something I love because it’s like watching an un-directed usability test and they can provide some great insights into the behaviour of users on your site. However, the free plan also allows you to create up to three click, scroll and movement heat maps, a six step conversion funnel and form analytics.

Image of Hotjar movement heat map
Image Source: Hotjar heat map

That’s not to mention online polls,  surveys to be sent to users, an incoming feedback tab and a recruiter feature for getting people to participate in user testing. This is an amazing free tool that no start-up should be without. Please read my review of Hotjar for more details.

5. MailChimp:

It’s important to remain in regular contact with your customers and prospects. Otherwise they can easily forget you even exist. Email marketing is one of the most effective and productive means of doing this. MailChimp provides a free professional email marketing service which allows you to send up to 12,000 emails a month to less than 2,000 accounts.

Image of MailChimp dashboard overview
Image Source: MailChimp Dashboard Overview

MailChimp has a very easy to use and informative user interface which allows you to quickly create professional looking emails. To design emails it offers 20 basic layouts and over 300 templates using  drag and drop technology.

There is also an email editor for HTML and plain text alternatives to design your own template. It’s easy to import your contact list from a CSV or TXT file, copy and paste from Excel or import from an email marketing platform.

MailChimp also offers comprehensive campaign performance metrics so that you can see how many recipients received, opened and clicked on your emails. It also advises you on how many email bounced and unsubscribed. Because it easily integrates with Google Analytics you can track how well your campaigns drive traffic to your website.

MailChimp has now added autoresponders or what they refer to as automation  to free accounts. These are automated emails triggered by an event such as registration, purchase or abandoned basket. This is a really useful feature that often you have to pay for to get access to. So, don’t delay, sign up to MailChimp today.


These five free tools are a must for every start-up. Don’t be reliant on SEO or marketing agencies to provide these resources, you will learn much more if you use them yourself. If you do need help in setting these tools up I can offer assistance as part of my marketing consultancy service, Otherwise, don’t delay and sign up now to these invaluable tools to assist you in growing your  online business.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @conversionupl, see his LinkedIn profile or connect on Facebook.

Why Should You Use A Google 5 Day Design Sprint?

What I learned from a Google 5 day design sprint

I was recently asked to use my expertise in user behaviour and conversion optimisation to contribute to a Google 5 day design sprint run by New Haircut. It was a very intensive, but enjoyable five days working on a new digital concept. It’s a great way of making rapid progress with a digital project and so I would like to share my thoughts on making the most of this process.

 What is a Google 5 day design sprint?

It’s basically a framework for rapid development and testing of a design prototype to challenge assumptions and create a user experience that meets agreed business objectives.

It achieves this by bringing together the business owner/decision maker with other key stakeholders and people with relevant expertise (e.g. product and conversion experts) and skills (e.g. UX designer). A suitable member of the CRO team should attend the workshops as they can provide valuable insights for both user motivations and what influences online behaviour. The process at a very high level comprises:

  1. Preparation:  Briefing and recruit 8 participants.
  2. Day 1: Develop a team-wide understanding of the challenge and visualise it using a simple user journey map
  3. Day 2: Team members individually sketch their solutions
  4. Day 3: Each member votes on the most compelling solution(s) and the UX designer visualises the whole user experience through a storyboard.
  5. Day 4: Create a realistic working prototype.
  6. Test the prototype with at least 5 prospects and record detailed reactions and feedback.

The framework for designing a user experience is very effective at making rapid progress with an idea. However, before proceeding you need to have agreed a clear vision and completed research to understand the market, users and the competition as otherwise the sprint is likely to lack direction. It’s important you have a specific problem that requires a solution. 

The Home Depot (THD) for example used a 5 day design sprint to generate and develop ideas for a new personalised homepage. The vision was to design a page that would dynamically deliver relevant content based upon targeted user segments. They defined their business goal as follows:

“Homepage will act as a personalized dashboard that serves relevant content to each customer and can be accessed from anywhere.” – Source: The Home Depot, 2017

OK, so you’ve got a high level appreciation of what happens during a a 5 day sprint. How is this implemented in reality? If you want more details of how a 5 day design sprint progresses  read on as I have broken the process into 7 stages:

 1. Preparation:

 The workshop preparation is very crucial for the design sprint’s success. This begins by selecting up to eight participants from different areas of the company including marketing, product, design, conversion and IT/platform to provide an understanding of technical limitations. You should also consider recruiting external expertise if necessary as it’s important to get different perspectives on the challenge and possible solutions.

A briefing document can then be circulated to all the participants to explain the nature of the challenge, the desired outcome of the sprint and the agenda. The logistics, such as location, resources available and food should also be confirmed. Personally, I also find it useful to do some preparatory research on related subjects such as looking at how other sectors deal with the challenge and refresh my knowledge of the conversion related topics that might be needed.

2. Understand:

On day one it’s good to re-iterate the vision and discuss the overall goal of the design sprint to ensure nothing significant has changed. During a warm up session each area has the opportunity to brief the team on their area of responsibility so that everyone understands the specific business, marketing, design and technological challenges faced by the sprint. This should include feedback on the customer and the competitive landscape.

Image of sprint questions to be answered on a flip chart
Image Source: New Haircut


As an external expert my briefing focussed on user motivations and how they are goal driven. I used the BJ Fogg model to outline a useful framework to consider when designing a user journey.

Image of BJ Fogg's behavioural change model
Image source: BJ Fogg


During these talks each member of the team wrote down “How might we” question on post-it notes to capture thoughts and ideas generated by the discussion. This kind of question encourages participants to challenge existing ways of thinking and find new solutions to the problems the group is faced with.

After all the briefing sessions have been completed it is time to vote on the “how might we” statements. Each person in the team gets three dots to vote with and the sprint owner is given six dots.

3. Define:

This stage involves defining the nature of the challenge in more detail such as creating buyer personas and identifying the individual steps of the customer journey. The user journey map is especially important as it allows the team to identify which step or steps they want to prototype and test with users.

Often the team will also generate a design principles list. This comprises adjectives that we would like customers to describe the product or service. At this point the decision maker should be ready to select three “how might we” statements as targets for the prototype.

4. Diverge:

This step is based upon the view that group brainstorms aren’t effective and so team members work individually to generate ideas for possible solutions. But as research suggests that our first idea is often not the best solution, this process uses the power of iteration to improve on our initial thoughts.

Each participant is asked to sketch a potential UI to get started. We were then given an additional seven minutes to sketch a further seven potential UI solutions. This activity helps generate lots of different potential solutions to the problem.

Image of outputs from the sketching exercise
Image Source: New Haircut

Participants are then given 30 minutes to sketch out all relevant UI screens for either mobile or desktop. This can be based upon any of their iterations they sketched in the 7 minutes. All user journeys are then displayed on a wall in the workshop room for everyone to review.

5. Decide:

Each member of the design sprint team now votes for their preferred solution(s) by placing dot stickers on the sketches on display. The project owner also votes but they have a different colour sticker to show their preferences.

Image of voting on UI sketches
Image Source: New Haircut


Each member of team is then asked to choose another person’s design to walk though and explain to the rest of the team. After each user journey sketch is discussed the person who designed it explains their thinking behind the design.

The project owner then decided which elements of the different customer journey sketches we would take forward. However, some sprint leaders use the “Risk vs Reward” scale to evaluate each popular solution. This shows what’s easy to implement and important to customers so that the team can decide which designs to prototype.

Image of storyboard in Google 5 day sprint
Image Source: New Haircut

We then create a storyboard of the chosen design iterations to be discussed with the team. This allows everyone to visualise the whole user journey that will be prototyped and tested.

6. Prototype:

Here the team was split into two groups. One group worked on the design of the prototype and the second group wrote the copy for the designs. It’s important to produce a prototype that is accurate and professionally designed to ensure it is clear and easy to test. Otherwise the danger is that feedback will be overly influenced by usability and design matters that will be resolved before the product is launched.

7. Validate:

The final day of the design sprint is spent watching around 5 to 8 usability tests undertaken using the working prototype. Although the number of participants may be small this kind of study can help validate whether your proposition and user experience are delivering what you set out to achieve. Usability testing is especially good at helping to refine the user journey flow, identify shortcomings and challenge assumptions.

During each session we wrote down thoughts and ideas  on post-it notes.  We then placed  these notes on a whiteboard which was divided up into the individual stages of the prototype user journey and other relevant categories (e.g. the proposition). This provided the backdrop for our final discussion about what we had learned from the user testing and how we might change the prototype and the proposition as a result.


As a conversion rate optimiser I have often been expected to help develop new concepts or product enhancements without the time or budget to validate the assumptions behind them. The 5 day design sprint provides a great opportunity to develop a concept using quick validations without using too much scarce resource.

Technology now allows us to create quick prototypes for minimal cost and by incorporating user testing into the sprint we can obtain timely feedback to assess the potential for the solution we’ve developed.

The process avoids early stage projects getting stuck in the bureaucracy of an organisation and breaks down silos by recruiting participants from across multiple departmental functions. From a conversion optimisation perspective it is also beneficial because it puts the customer needs at the centre of the process which is not always the case with new ideas and enhancements.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.

Dark UX or Persuasive Design?

What is Dark UX?

As a Conversion Specialist I find dark UX interesting because it essentially puts the needs of the company before that of the customer. It is the opposite of normal UX principles as dark UX employs design to trick people into taking a decision that may not be in their best interests and sometimes without them being fully aware of what decision they have made.

Dark UX has the potential to leave a bitter taste in your mouth when you realise what has happened. Know one likes being tricked or mislead and the user experience may lead to high levels of distrust and dissatisfaction among customers.

In some cases a design breaks a web convention or norm to nudge you in the opposite direction to what you expect. Alternative choice options  are sometimes hidden below the fold or reduced in prominence so that you may fail to notice them.

Depending upon your point of view you may see this as persuasive design or dark UX. How do we judge what is a great design as opposed to a manipulative design?  Here are some interesting examples that could be perceived as great design or dark ux.

Good design or dark UX?

Amazon is seen as the gold standard in e-commerce and so it’s not generally associated with dark UX. However, I came across this interstitial page for Amazon Prime after clicking on proceed to checkout.  The primary call-to-action (CTA) is clearly “Sign up and pay now”, but what if you just want to pay without signing up to Prime?

Image of Amazon Prime interstitial after proceed to checkout

When a user clicks on the “Proceed to checkout” CTA they expect to be served the checkout and so I was surprised and slightly confused to see this page the first time I  came across it.  I almost instinctively clicked on “Sign up and pay now” without thinking, but then realised the page was trying to get me to sign up to Prime.

Image of CTAs on Amazon Prime's Interstitial page

At first sight it’s not obvious that there is an alternative CTA to “Sign up and pay now”. After a number of seconds scanning the page I found a hyperlink  positioned  to the left of the primary CTA.  It uses loss aversion to good effect as it reads  “No thanks – I don’t want Unlimited One-Day Delivery”.

Is this just persuasive design or dark UX – it’s difficult to say. I can see how thoughtful the design is, but could it result in some users signing up to Prime without meaning to? Possibly.

Ryanair were the experts:

A more clear-cut example of dark UX was the old Ryanair travel Insurance section. This stated that “If you wish to purchase travel insurance please select your country of residence”. Below the list of passengers there was a further instruction; “If you do not wish to purchase travel insurance, please select ‘Don’t insure me’ in the drop-down box.”

Image of Ryanair Insurance - country of residence
Image Source:

The “Don’t insure me” was hidden between “Denmark” and “Finland”. You also had to select this separately for each passenger because it is under country of residence. This design was heavily criticised  by various commentators and may be as a result it has since been replaced by a much more user friendly user interface.


Image of 2017 Ryanair travel insurance user interface
Image Source:

Clever design?

I recently reviewed the  registration form  and was impressed with how it has been improved since I last looked at it. The design uses a very well implemented accordion style form that initially displays three fields and then shows another field when you complete those first fields.

Image of step 1 of registration form

When I completed the form I came across the registration successful page below.  The primary CTA – “Apply Now” is prominent and a benefit of being able to spread the cost is clearly communicated. However, what if you don’t want to spread the cost and apply for an account? Similar to the Amazon example the alternative action is not immediately obvious.

Image of registration successful page

That’s because the alternative payment method is outlined in low contrast text below the discreet horizontal line. The copy reads; “or if you’d like to pay by card just continue to shop and then choose the pay by credit or debit card option when your’e in the checkout”. I’ve highlighted the copy below to confirm where it is located.

Image of registration successful page with secondary CTA highlighed

From a conversion perspective this is clearly a winner and probably generates a substantial return on investment. The alternative option is visible when you study the page, but many visitors may not notice it before they click on the primary CTA and apply to open a credit account.

It  is a very well thought out design and is certainly persuasive. What they could test though is the use of loss aversion for the alternative CTA copy. Something like – “No thanks – I don’t want the ability to spread the cost – continue shopping.” This might help offset any drop in conversion if the CTA was made more prominent.

Who decides?

The final decision on whether a design meets an organisations’s standards and is consistent with its brand values rests with senior management. Inevitably this reflects the company culture and how customer-centric the organisation is. This explains why sometimes a design may be perceived as persuasive in some organisations and dark in another.

When you are working on a new design it can be difficult to take a step back and review it objectively. That’s why usability testing and setting suitable success metrics is so important. If after implementation you see an increase in returns/cancellations or a rise in customer complaints related to the page that is probably giving you the answer.

In the end the customer has the final say. If users feel they have been tricked or mislead by how a website is designed they are likely to either complain or switch to a competitor if and when they have the opportunity. Provided customers have a choice to go elsewhere  there will always be pressure to remove unsatisfactory design  practices.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.


Hotjar Analytics Review

Hotjar Behavioural Analytics:

If you want to improve your website conversion rate it’s essential that you understand the behaviour and motivations of your visitors. If like me you run a start-up and have a very limited budget you might think you can’t afford behavioural analytics and customer feedback tools to evaluate how well your website is performing.

Indeed, when I worked for a large online gaming company we spent tens of thousands of pounds a month on the behavioural analytics tool ClickTale. However, before the ClickTale contract was due for renewal I came across a new behavioural analytics and customer feedback solution called Hotjar.  By switching to Hotjar I saved the company over £100,000 a year. Further, if your website has less than 2,000 page views a day you can get Hotjar for free.

What is Hotjar?

Hotjar is a combined behavioural analytics and customer feedback tool. It offers most of the features you would expect from a behavioural analytics solution including browser recordings of users navigating your site, mouse movement/clicks heat maps and form analytics.

However, unlike most such tools Hotjar also offers customer feedback tools including on-site feedback polls, online surveys and research recruitment. It also has a very simple and intuitive user interface. It’s a very easy tool to start using and does not require hours of training to get the most out of it.

Image of Hotjar dashboard

The above dashboard  allows you easily navigate to all the features that Hotjar offers and there is a clear on-boarding process to get you stared.

How easy is it to add Hotjar to my site?

Hotjar works by adding a JavaScript snippet of code to your site just before the head close tag. It needs to be on every page where you wish to record visitors or gather feedback. It normally only takes a few seconds to add the code to your site. For WordPress sites I use the plugin Insert headers and footers.

Adding code to the header can be problematic for some click and drop website builders like The platform does not allow you to add JavaScript to the page header and this means you can’t install Hotjar on such sites until they address this issue.


The basic free plan is available for sites with up to 2,000 page views a day.  For start-ups with up to 10,000 page views a day you can get Hotjar for just €29 a month. However, for more established businesses there are a range of more comprehensive plans outlined below.

Image of Hotjar business pricing plans

Each plan offers the following suite of features:

Browser recordings:

Browser recordings are like an undirected usability test. The feature allows you to record and replay a visitor’s session so that you can view what they see in their browser as they navigate around your site. These recordings are a fantastic way to observe how your customers move around and navigate on your site.

Image of Hotjar browser recording

They can help you identify where usability problems exist as they show clicks, scrolls and movements.  You can also tag individual recordings giving you the ability to categorise them according to an observed behaviour or characteristic (e.g. added to basket or struggled with CAPTCHA).

Heat maps:

Click, scroll and mouse movement heat maps can be created in Hotjar by simply entering the page URL and specifying the limit of page views you wish to capture.

Image of Hotjar movement heat map

Heat maps give you a clear indication of where user attention is focused on your page and will show you how far down the page visitors are scrolling. Hotjar allows you to generate heat maps for desktop, tablet and mobile devices.

Conversion funnels:

The funnel feature of Hotjar allows you set up and monitor visitor completion rates for a user journey of up to six steps. If you have a transactional site this can be especially useful feature as it allows you to track users through the purchase process and identify the biggest drop off points in your funnel.

Image Hotjar conversion funnel graph

Form analytics:

If you have any kind of form, whether it is registration form or check out process, this feature allows you to track visitors as they proceed through the journey and identify when or if they drop out.

Image of Hotjar form analytics graph

For example, if you have a registration form for your site you may find that a username question causes a lot of drop offs and prevents visitors from signing up. This might be because many of the obvious usernames have already been taken.

The analytics from this feature include:

  • Conversion rate of the form
  • Seconds to fill out a field
  • Total form sessions
  • Field drop off chart
  • Failed and successful form submissions



This is such a great feature of Hotjar as it allows you to engage with visitors and ask a few simple questions about the user experience whilst they are on your site. The first time I used a poll I asked visitors to the homepage of gaming website “What is missing on this page?” We got lots of mobile visitors saying that they couldn’t see the login box or that they were unable to login.

On checking the homepage with a mobile device we discovered the login box had been hidden by Marcoms for mobile users as they wanted them to encourage them to download the app. Unfortunately these users were existing customers who already had the app and needed to login to their accounts to verify their details. Because we got the feedback from the poll we were able to rectify the problem immediately.

The feature allows you to target specific pages with your poll or give visitors on any page of your site the opportunity to participate in the poll. Apart from short text answers you can also use:

  • Long text answer
  • Radio buttons
  • Checkboxes
  • Net promoter score

Image of setting up poll in Hotjar

When setting up questions you can also specify where respondents should go next dependent upon their answer to a question. This ensures that respondents are only asked questions that are relevant to them.


This allows you to design longer questionnaires to email to your customers for more in-depth feedback. It’s a great way of engaging with your email list on a regular basis to gather their views on a variety of topics.

At the gaming company we surveyed new customers to find out what had most appealed to them about the site to see if our value proposition was aligned with customer perceptions. We also asked them if they played on any competitor sites to understand who we were competing against.

Image of Hotjar surveys feature

You have the same selection of question types to choose from as you have with the polls feature.

Incoming – in beta:

Hotjar recently added a new feature in beta – called incoming.  This is a “Feedback” tab that you can place on either a specific page or your whole site to allow visitors to rate the user experience.

Image of Hotar incoming quesiton

I’ve only just stared using this feature and so it’s too early to say how it has been received. However, I think it’s great to show you encourage and value visitor feedback as this can improve the perception of your site as a whole.


If you want to conduct usability testing Hotjar has a recruiter feature which allows you to collect profiling information and contact details in exchange for an offer of a gift to incentivise the process.  You can then arrange to conduct a usability test by sharing screens using Skype or another web meeting solution.

Image of Hotjar recruiters

This is a great feature of Hotjar as it’s important to keep in touch with your users and regularly review your website user experience. This is especially relevant when you re-design an area of your site or add new functionality and you want to understand how well visitors use the new pages or feature.

Areas for improvement:

For many small to medium sized businesses Hotjar offers an amazing solution for little or no cost. However, it does have its limitations and there are areas for improvement.

The free plan does limit you to just three heat maps and 1,000 page views per heat map.  If you up-grade to the Plus plan this rises to 2,000 page views and the Business plan gives you up to 10,000 page views.

Unlike more expensive solutions Hotjar does not allow you to filter your heat maps according to visitor type (i.e. new or returning visitors).  This can be very useful as the behaviour of new and returning visitors often reflects different user goals.

As Hotjar is a relatively new tool they have concentrated on improving functionality rather than integrating with related solutions. This does mean that Hotjar does not integrate with many testing, web analytics or CRM solutions. Some integrations are available, notably with Hubspot, but most are not available at present and this is a major drawback for any site conducting online experiments.

The conversion funnel also lacks a level of sophistication. For many sites users can’t be expected to follow a simple sequential funnel and so the ability to create multiple route alternatives would be a benefit.

However, Hotjar do have a full product roadmap to further enhance and improve the functionality of the solution. If they can also integrate fully with complementary tools this will be a big improvement.


For many organisations Hotjar is an ideal behavioural analytics and customer feedback tool. For start-ups like my business Hotjar offers exceptional value for money. I can’t afford a significant subscription and my traffic levels don’t justify it.

Hotjar meets an important need for online businesses. If you are setting up an online business then Hotjar is a no brainer and even if your site is established it is still extremely good value for money.

Related post:

Behavioural analytics – 15 free and paid visual analytics solutions to boost conversions

Customer feedbackHow to use online Voice of Customer tools to boost conversions.


Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @northresearch, see his LinkedIn profile or connect on Facebook.

5 Ways of Using Word Clouds to Enhance Digital Marketing

Word Clouds For Digital Marketing:

Word clouds are a great creative tool that many of us love to play around with. They generate an image composed of words from content that has been submitted in the tool. The size of individual words in the word cloud indicates its frequency or importance.

However, word clouds are also very useful tools for content creation, brand evaluation, competitor analysis, SEO keyword targeting and customer insight.

1. Marketing Content:

Finding images that are suitable for your blog or social media posts can be time consuming and problematic as many images are protected by copyright and have permissions which restrict usage. However, we know from research that blog posts with images have much higher engagement levels and are more likely to be shared. So, to save time and money copy and paste your new blog post into a word cloud to create a free and unique image for your new content.

2. Evaluate Your Value Proposition:

Word clouds are also useful to evaluate how well your value proposition messages are embedded in your content.  Check how well your brand messages are being communicated by plugging in your website address or social media URL.

Image of word cloud for homepage
Word cloud of using


World cloud tools like Tagxedo allow you to input your Twitter and social media feeds. This can highlight the value of the content you are sharing and allow you to identify the themes that are occurring most often. Do these correspond with how you want to be perceived by your social media followers?

Analyse your LinkedIn profile in a word cloud generator to ensure your profile communicates a professional tone and is using the best words to promote your expertise.

If you have a Yelp page, use a word cloud to assess how your customers describe your service.

3. SEO Keyword Audit:

Word clouds are a great SEO tool as they instantly indicate which words are most prominent in your marketing content and how Google or other search engines are likely to assess your content.

4. Competitor Research:

Just as you can evaluate your own marketing content and keywords using a word cloud, you can also use the same approach to undertake some competitor research.  Plug in your competitor’s URL into a word cloud to identify the keywords they are targeting and how consistent their value proposition is communicated.

Image of word cloud of
Wordcloud of using

5. Analysing Customer Feedback:

Customer feedback is not just collected through online polls and surveys. Customer conversations are also a great source of feedback, whether via telephone calls, live chat, product reviews or emails. You will probably be surprised at the number of sources potentially available to you and the volume of feedback.

Don’t let the volume of feedback put you off as word clouds are an excellent means for processing verbatim customer conversations to obtain quick and clear insights. Such conversations can be a great source of insights for developing hypothesis for A/B and multivariate tests.

However, before proceeding with using a word cloud to analyse customer conversations it is wise to do some preparatory work to clean up your transcripts or survey responses. Otherwise you may find that duplicate feedback or similar meaning terms reduce the effectiveness of your word cloud at communicating key insights.

Duplicate Responses:

It’s not uncommon for some visitors to answer a survey or leave feedback multiple times. Unless this is dealt with a single customer can skew your analysis, especially if they have repeated the same comments on multiple occasions.

This is often easy to spot if the respondent has to leave an email address or another unique identifier. Once you have identified the culprits go through and review their feedback and delete all but the first response. It’s better to be consistent with your method as otherwise you will be bringing in subjective bias into the analysis.

Combining Terms:

I once launched a poll on a homepage on a mobile responsive website by asking the open-ended question; “What is missing on this page?” We received lots of comments from mobile visitors about being unable to login, sign in or see the login box. All of these responses obviously related to the same issue and so it was sensible to combine the terms. This can easily be done by using the “Replace” function in a spreadsheet.

It’s also worth looking out for plurals and replacing such terms with the singular version of the word.  Acronyms can also be problematic if some respondents use the full phrase and so search for such inconsistencies to replace acronyms where necessary.

Weighting Results:

When presenting word clouds you sometimes want to give more weight to certain terms because of what you know about their impact on your business. For instance you may want to give more prominence to feedback on your most popular webpages or blogs in your word cloud.

Word cloud tool Wordle allows you to change the weighting of certain words by making adjustments in the advanced mode. For example you could weight words according page views to reflect the popularity of a page or blog they relate to. However, make sure you make this clear when you present your word cloud as otherwise this can create a misleading impression of the feedback received.


Word clouds are flexible and free tools that can save you time and money. Before you splash out money on buying competitor analysis, SEO keyword audits or text analytics tools try out word clouds first.

Below are nine of the best free word cloud solutions available.

The 9 Best Free Word Cloud Tools:

My recommended word cloud is as this is an easy to use but flexible tool with some great advanced settings. Visually it also looks superior to most other tools.

Image of word cloud generated by
Word cloud from blog post using


  1. ABCya!:A word cloud for kids that may be relevant if your website is of interest to children. Type or paste text into the box below and press the arrow button to view the word cloud generated.
  2. Jason Davies: A great tool if you want to generate a word cloud from a blog or website.
  3. Tagcrowd: Allows you to set specific criteria for your cloud such as language, maximum number of words and minimum frequency.  Allows you to create a word cloud from a webpage URL, Twitter ID and other social media feeds.
  4. Tagxedo: Create word clouds from a URL, Twitter ID, ID, news, search, RSS feed, uploading text or enter it yourself.
  5. WordArt (Formerly Tagul):  A word cloud generator with advanced features including words inside words, rich font choices, roll-over effects, custom shapes, colours and fonts and export in vector formats.
  6. Tricklar: This site claims to use high quality media sources from around the world to generate word clouds. I found it difficult to find words or phrases that it would generate a word cloud for and so maybe only useful for popular subjects.
  7. This is a free word cloud generator which can be used with most browsers. On the homepage got o “File” and upload a document or PDF, paste text (by File dropdown), input a URL or amend the word list (dropdown). In the word list you can even add links to individual words by entering the URL after the word.
  8. WorditOut: Advanced filters allows you to filter the text to display or remove words and change their importance. Select your own layout by choosing your own colours, fonts, and sizes or let WorditOut find a random look for you.
  9. Wordle: A simple word cloud generator which allows you to set the weights of words.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @northresearch, see his LinkedIn profile or connect on Facebook.

Why Are Silos Killing Your Growth Strategy?

Is a silo mentality killing CRO?

Conversion rate optimization (CRO) is on a roll. Companies are falling over themselves to recruit CRO specialists and managers as part of their growth strategy. They may not all have the same responsibilities but they are usually expected  to deliver sustainable growth through optimisation of user journeys. This often includes developing and coordinating  a roadmap for A/B and multivariate testing.

Often though CRO is implemented too narrowly without an effective plan. For example when I was employed in a CRO role I was asked to create a testing roadmap within a couple of weeks without any budget for research or usability testing. It was as if I was the oracle for CRO. However, CRO is a business strategy and not the sole responsibility of an individual department or business unit.

This misguided view of CRO  does not lead to sustainable business growth as it  soon runs out of steam once the obvious things to fix are exhausted. Further, cro requires collaboration across departments to draw in a diverse range of skills and expertise.

Unfortunately in many  organisations CRO suffers from the same silo mentality which results in slow, bureaucratic business decision making and a culture which encourages waste and strangles innovation.  Despite a desire to improve business efficiency the basic building blocks of most organisational structures are still the traditional functional silo.

Why does this matter for CRO?

For CRO to deliver business growth it needs to be embedded in the culture and requires a collaborative approach to business change. For example testing needs to integrate with the product roadmap, reflect business goals and inform marketing strategy. Companies which operate within rigid silos find CRO fails to deliver sustainable growth because the silo mentality discourages collaboration and cultural change.

“The great defect of scale, of course, which makes the game interesting—so that the big people don’t always win—is that as you get big, you get the bureaucracy. And with the bureaucracy comes the territoriality—which is again grounded in human nature.” – Charlie Munger – Berkshire Hathaway

Silos also prevent organisations fully exploiting the collective knowledge of the customer that is held across the organisation. Silos make it more difficult for customer data and knowledge to be effectively shared to inform decision making that creates the most value for the customer and organisation.

Image of Kodak logo

Kodak was organised into product silos and in the 1970s it controlled around 90% of the film market and 75% of the camera market. They invented the digital camera in 1975 but believed they were in the business of making film because their largest operational silo reinforced this perception to defend its own self-interests.

Kodak went bankrupt in 2012 after selling its digital camera business to save the film business.

Why do silos form?

Silos are a natural reaction to organisational growth and greater complexity. Companies create specialist functions in response to business expansion and increased work load. Silos facilitate the building of expertise and knowledge for specific business purposes.

Our herd instinct encourages people to become strongly attached to those within our immediate group. We form emotional bonds with people we have shared experiences with. This can result in people putting the needs or goals of their business unit ahead of the objectives of the organisation as a whole.

Sheep on the road image

As projects are often resourced and financed from within a function means that outcomes and insights are not always shared outside of the business unit concerned. This can lead to both physical and emotional obstacles to collaboration between business units. In the worse-case scenario this can result in conflict between departments as they aim to protect their own interests rather than those of the organisation as a whole.

Why do silos harm growth and innovation?

Silos create competing subcultures within a business. This limits ways of working and communication between business units, but it also reduces social interaction and trust between people working for the same business. Digital silos disrupt information and knowledge sharing which is essential for CRO.

This is why large organisations can be bureaucratic and slow to adapt to both internal and external demands for change. This often damages the customer experience because it becomes difficult to respond swiftly to changes in customer needs or preferences.

What problems do silos cause for CRO?

Fragmented vision:

Unless CRO is part of the culture of the organisation functional business units will continue to create their own goals and strategy for optimising the user experience or conversion rates. Apart from duplicating effort and resources, this results in the vision of the organisation becoming fragmented. This creates conflict and dilutes the effectiveness of customer focused strategies.

Inefficiency – silos create friction and mistrust

The conflict created by business units having their own vision for optimisation damages trust between functions and often causes inefficiencies as departments fail to share knowledge and duplicate tasks. I’ve seen this result in brands replicating successful A/B tests conducted by another business unit rather than acknowledging and learning from the test. In addition priorities are set according the business unit’s goals rather than what is important for the business as a whole.

With CRO this often results in experiments being delayed due to a lack of design or development resource. Unfortunately such delays can make tests less relevant or obsolete in some cases as product releases change the default experience.

Image of Sony HQ in Japan

Sony case study:

In 1994 Sony reorganised into eight stand-alone business units. Initially this led to cost savings and an improvement in profitability. But managers also began trying to protect their units, not just from competitors but also from other departments. Sony’s business units became less willing to share experimental ideas or rotate the brightest staff between departments.

Collaboration stopped and nobody wanted to take risks. In the late 1990s when the Internet began to disrupt the distribution of music each Sony department tried independently to experiment with new solutions. However, none collaborated with the Sony Music Entertainment Group (previously CBS records) because SME refused to cooperate with any department as its officials were terrified that digital music would undermine revenues from records and CDs.

Image of Sony Walkman

Apple on the other hand worked as one cohesive company with one profit and loss. Engineers at Apple developed a two-step solution to allow users to download music from the internet and then listen to them on a portable device.

Apple  also developed the iStore to provide music groups with some royalties and designed the platform so that it could be accessed by anybody using any platform. Sony’s digital music system on the other hand relied on proprietary technology that had to use their own platform. This limited Sony’s ability to compete with Apple in both the music download market and the portable music player market which Sony had practically created with the Walkman.

In 2001 Apple launched the iPod and has since dominated the portable music player market. Sony failed to take a significant share of a market it once led with the Sony Walkman.

False assumptions:

Silos are very good at perpetuating myths based upon inaccurate perceptions that circulate without being challenged in an individual business unit. This is because a silo mentality encourages people to become over-reliant on information circulated from within their own department.

I experienced this one time when I challenged why the term “CAPTCHA” was being used in content aimed at customers. The response that came back was they had checked with their team (all web developers) and they all felt their friends and families would understand the term. There was zero awareness that perhaps their team and even their close friends and family were not representative of a typical customer.

This level of overconfidence in the knowledge held within the silo can reduce willingness to interact with other business units and limit opportunities to share knowledge and expertise.  Employees may therefore become isolated and feel dissatisfied which can have serious consequences for staff morale and retention. This can damage efforts to improve the customer experience and often leads to complacency which can be disastrous for CRO.

Limited customer insight:

Image of online surveys

To understand user expectations and experiences it is critical that data and insights are combined and processed from sources throughout the organisational structure. If data is not freely shared and combined with insights from other units the opportunity cost can be huge. It significantly limits the value of customer data to the organisation and individual departments as connections and insights are not fully uncovered.

With CRO this is often characterised by one piece of market research or insight being used to push through an update based upon subjective opinions. This can create conflict between teams as one piece of information should never be used in isolation to support a change in the customer experience. Such a culture can seriously damage the customer experience and reduce conversion rates.

Blame culture:

A fragmented approach to organisational change programs such as CRO caused by the dominance of individual silos creates a blame culture and can result in complacency. When CRO related targets are not met the temptation for some departments is to blame other areas of the business rather than take responsibility for their own failings.

A blame culture discourages collaboration because such initiatives can threaten a person’s career or credibility within their own department. In CRO it is important to encourage people to prove themselves wrong by creating online experiments to challenge existing best practice.

This is often how innovation happens as if we don’t take risks we are unlikely to create anything new and transformational. However, to identify improvements in the customer experience it is essential that people from different areas of expertise work together in a collaborative and trusting environment.


When all think alike, then no one is thinking - Walter Lippman - The danger of groupthink

Small teams drawn from a single business unit have been found to be prone to make poor decisions due to their tendency to suffer from what psychologists call groupthink. Companies often believe that by recruiting the most talented and smartest people they can prevent poor decision making. The opposite is often true as small homogeneous groups suffer from a lack of diversity, insufficient independent thinking and a desire for conformity.

 “Diversity and independence are important because the best collective decisions are the product of disagreement and contest, not consensus or compromise.”  – James Surowiecki, The Wisdom of Crowds

People in such teams are also often over-respectful of senior people and so don’t sufficiently challenge ideas from these team members. Diversity is important because homogeneous groups become more cohesive than diverse teams and as a consequence they become more reliant on the group for ideas and support.

This insulates the group from external sources of information and makes group members more assured that the group consensus is the best solution to the problem.

 “Suggesting that the organization with the smartest people may not be the best organization is heretical, particularly in a business world caught up in a ceaseless “war for talent” and governed by the assumption that a few superstars can make the difference between an excellent and a mediocre company. Heretical or not, it’s the truth, the value of expertise is, in many contexts, overrated.” – James Surowiecki, The Wisdom of Crowds

Of course many large organisations insist on recruiting the brightest minds from the top universities. However, this restricts both racial and cognitive diversity which enables homogeneous groups to form.

How do you break down silos?

CRO needs to be an agile process to allow teams to respond to new customer knowledge and insights from the multitude of sources available to a customer focused organisation.

Collaboration is essential to enable knowledge and data to be delivered to where it is needed. It is vital to any organisation that seeks to improve profitability by improving the customer experience and conversion as no single team can optimise the user experience on their own.

To break down organisational silos to develop a strong CRO culture it is necessary to begin at the top of the business and change old working practices. The key strategies to create a more collaborative approach to CRO are:

CRO needs board level buy-in:

If you look at the companies that have most successfully implemented a CRO strategy such as Amazon, and, they have people right at the top of the organisation who are passionate about CRO and take ownership of developing a supportive culture. Skyscanner for instance recently promoted their chief experimenter to become Director of Experimentation.

Having a board level manager in charge of CRO gives the strategy the profile and support needed to build a culture of customer focus and experimentation. A junior manager would never have the clout to try and optimise the product as well as web content.

Agree a unified vision for CRO:


Image of Amazon signIt is important to have a single unified vision for the company to retain focus and ensure consistency of objectives within the leadership team. Amazon has a very strong and clear  vision as follows:

“Our vision is to be earth’s most customer-centric company; to build a place where people can come to find and discover anything they might want to buy online.” – Source: The

The focus on customer centricity gives clear direction to all employees so that everyone can relate the vision to their own job. This helps CRO because people can see the relevance of optimisation to the company’s vision and their own individual and departmental goals.

Set a common goal:

Silos often cause conflict because individual business units set tactical goals and objectives that can discourage collaborative working. To avoid this problem the leadership team need to set out a single goal for the company that they can all agree on and is their number one priority. For example Amazon’s Jeff Bezos said;

“It is the company’s goal to make it irresponsible to not be a Prime member.” – Jeff Bezos

This should be clearly communicated to all employees so they can understand how they can contribute individually to achieving the goal. This assists CRO because having a clear goal makes it easier to identify a relevant success metric for experiments to be evaluated by.

Agile product management:

To facilitate collaboration across individual business units it is necessary to form product management teams that draw expertise from a diverse range of areas of expertise. For CRO this means having teams which include developers, UX designers, marketing specialists, experimenters and product managers to develop hypothesis, build tests and implement successful ideas.

Identify why trust has broken down:

It’s important to understand the reasons for a lack of trust and get people to recognize the problems that silos create before implementing solutions.  Changing a culture of working within silos has to be planned to reduce resistance and obtain co-operation. For CRO this means frequent communication and workshops to engage people throughout the business.

Flexible communication solutions:

The growth of global teams can prevent knowledge sharing and timely decision-making. Further, the use of email, phone and real-time messaging services has created a virtual workplace that fragments communications and reduces face-to-face interaction.  Business process management and resource planning software can help break down silos walls and allow people to work together on shared goals and encourages the exchange of information and ideas.

Let go of CRO:

One way of killing collaboration in CRO is not being willing to let go and allow others in the organisation to have some autonomy. Sure, there is a need for oversight and co-ordination, but beyond that it is necessary to trust people to follow the agreed framework for CRO.

It is important to get people across the business trained up to add their input and expertise to the CRO program. Your UX designers should provide wireframes for your ideas, copywriters content, and your own developers should be best suited to building experiments. Sometimes it can help by bringing in a CRO agency to co-ordinate this process, but in the long-run it is perfectly possible to create all the necessary expertise within most organisations.


Finally, to understand how well CRO is being implemented ensure you build in feedback and monitoring systems to evaluate how well your strategy is working. CRO is a change management process and so adoption will take time and is rarely universally successful. However, as gaps or problems are identified you have the opportunity to review your approach and try new ideas.

Beware of vanity metrics, such as number of tests per month, which often don’t closely fit the desired outcome. This can result in  the cobra effect which can lead to unexpected and unwanted behaviour. Look out for these behaviours and adjust targets accordingly as some people do try to play the system.

Thank you for reading my post and I hope you found it useful. Please share using the social media icons below if you like this post.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital marketing optimisation consultancy services and has worked for  brands such as, and He uses a variety of techniques, including web analytics, personas, customer journey analysis and customer feedback to improve a website’s conversion rate.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @northresearch, see his LinkedIn profile or connect on Facebook.