Category Archives: Multivariate Testing

How is AI Disrupting Conversion Rate Optimisation?

Using Evolutionary Pressures To Optimise:

Digital marketing is a zero-sum game – it’s survival of the fittest. Brands have to respond to changing customer needs and pressure from competitors or they go out of business.

What if you could use these evolutionary pressures to automatically adapt and adjust your site according to what has the highest conversion rate? And if your audience changed, perhaps due to a TV campaign, wouldn’t it be great if your site responded by optimising your user experience for the new audience profile? But rather than only improving a single page, what if it could simultaneously optimise multiple pages in the user journey?

Well, with the advent of AI and evolutionary algorithms this time has arrived! Sentient, a company born out of the minds that developed the technology behind Apple’s Siri, has come to market with Ascend. Sentient have combined evolutionary computation (a form of AI which uses mechanisms inspired by biological evolution), and deep learning to create a market leading optimisation solution. What is unique and exciting about Ascend is that it is capable of autonomous decision-making to assist businesses improve their bottom line and enhance the customer experience at the same time.


What are the benefits of Ascend?

Sentient Ascend is the first testing and optimisation solution developed by integrating AI, evolutionary algorithms and deep learning technology. As a result it has the capability to revolutionise how testing and optimisation is carried out. The main benefits of Ascend are:

  • Massively complex multivariate tests that have over 1 million possible combinations can be completed with Ascend that would be impossible with traditional MVT solutions. Below is an example of the kind of test that is now feasible with Ascend.


Image of multivariate test with over 1 million possible combinations
Source: Sentient Ascend


  • Ascend requires lower traffic levels than traditional optimisation solutions because it uses what it discovers about the performance of a particular combination of elements to predict how that combination will influence the conversion rate in the future.
  • Testing is completed with greater speed and double digit uplifts in conversion rates are normally achieved within the first 2 months of employing Ascend. Recently completed tests have achieved between a 12% and 48% uplift in conversions.
  • It can optimise multiple pages simultaneously to improve conversion rates throughout a user journey.
  • Indeed, for underwear brand Cosabella, Ascend tested 15 different changes to the homepage header, category page, product page and shopping cart design. Using standard multivariate testing would have required 160 tests, instead of the automated process that Ascend manages for you. This improved conversions by 35% compared to the control experience.
  • Automates the testing program so that once all your ideas have been input into Ascend it employs all the power of AI to adapt and respond to user interactions to identify the best performing combination of changes to your site or web app.
  • It allows for tests to be paused and new ideas to be input into the testing program as and when required.
  • Automatically adapts to a change in the visitor audience profile without the need for any manual intervention.

How does evolutionary computation work?

To give the evolutionary algorithm a purpose it is first necessary to define a fitness measure. With conversion rate optimisation (CRO) the fitness measure should be the conversion metric that you wish to optimise for such as sales, revenues, average basket value, first time deposit or sales leads. It is important to take care in selecting your fitness metric because it needs to be a characteristic that makes one experience (or algorithm) better than another.

With an evolutionary algorithm each page (i.e. a selected combination of elements) is classed as a genome and it uses genetic operators (i.e. selection, mutation and crossover) to create and maintain genetic diversity. In the example below two high performing pages (see column on the left) have been identified through selection (i.e. survival of the fittest).

Image of how Sentient Ascend uses evolutionary algorithms to optimise designs
Source: Sentient Ascend

However, a further generation of solutions can then be created through crossover (i.e. recombining elements from the two high-performing genome) to create children; the middle solutions above. Mutation (i.e. randomly altering one element in the child’s chromosome) encourages diversity amongst solutions and seeks to prevent the algorithm converging to a local minimum by avoiding solutions becoming too similar to each other. This is shown in the right column above.

Although each operator individually seeks to improve the solutions generated by the algorithm, the operators work together with each other to create an optimal solution that would not be possible if they were used in isolation of each other.  In the first instance the algorithm simply evaluates each page (i.e. genome) to identify if it performs well enough to be a parent for the next generation. Otherwise it will be rejected.

Image of illustration of how an evolutionary algorithm works

This allows literally thousands (out of millions) of experiences to be tested in a short space of time. But as Ascend learns which combination of elements create the best performing designs it automatically adjusts experiences according to how visitors respond. Below is an example of changes that Ascend can evaluate as part of single multivariate test.

The advantage of this technology is that it can create page designs that convert better than those designed by people because it automatically searches for unexpected interactions between elements. It is also doesn’t suffer from human misconceptions or biases, which means that it can generate surprising ideas that we might never have thought of ourselves.

What’s the catch?

Like any optimisation software Sentient Ascend relies on the quality of ideas and designs to generate uplifts in conversion. It is therefore essential to invest in the people who will be using Ascend to ensure they have the required skills and support to get the most out of this amazing solution.

To generate a sufficient quantity of ideas and designs for testing will take some time and resource as you are essentially compressing twelve months or more of testing into a single month or two. This is an analytical and creative process and so it will require the input and approval from various stakeholders if it is to be a success.

To keep Ascend fed with additional ideas after the initial test will also require further planning and support to ensure you get value for money from the solution. There is certainly a danger that rather than focusing on quality hypothesis users might be tempted to throw every idea into the mix without proper evaluation and prioritisation. This would be a recipe for a sub-optimal result as with any model if you put garbage in you will get garbage out.

As with any multivariate test it is advisable to run an A/B test to validate the winning experience. However, no worries, Ascend can manage this for you or you can use your existing A/B testing solution to conduct the experiment.


Sentient Ascend makes most existing testing software obsolete because it offers an automated platform for massively multivariate conversion optimisation. This allows you to test an enormous number of ideas in a shorter time period than is possible with existing solutions. It is also more efficient at discovering new combinations of elements that result in uplifts in conversion due to the evolutionary nature of the algorithms.

Further, as you can add new ideas as you test you don’t need to wait for the test to end to respond to changes in campaign execution or strategy. You can just keep testing continuously if you have the ideas.

Note: Conversion Uplift is now an accredited partner for Sentient Ascend.

Thank you for reading my post and if you found it useful please share using the social media icons below.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @northresearch, see his LinkedIn profile or connect on Facebook.


How To Optimize Your Website’s Performance Using A/B Testing

 6 Experiments For Higher Conversions:

 6 types of tests to optimise a website page



When people talk about A/B testing they often refer to call to action button changes and landing page tests.  They also sometimes talk about only changing one element on a page at a time to ensure you can tell exactly what generated the difference between the two experiences. This last point of view can be quite misleading and could hold back your optimization progress.

You need to base your testing programme on a best practice and systematic process of discovery, evidence and prioritisation. But once you have that in place you also need to consider how to build a test plan for each of your key pages or journeys.

This brings us to the question of what are the main types of tests that you should be including in your testing roadmap. I’ve outlined below six testing approaches to consider and you should be employing all of them to optimize your site and improve conversions.

1. Innovation Tests:

Unless you happen to work for Google or some other mega website you have to change more than one element at a time if you are to make quick progress in your optimization journey.  An innovation or re-direct test allows you to experiment with something completely different. This gives you the opportunity to ensure the new page is more aligned to your business goals. The idea is that you can leave all the baggage of the existing page behind and design a radical new experience that will allow you to leapfrog to a much higher conversion rate.

Find an important web page, one with lots of traffic and a conversion rate that you believe can be significantly improved upon. You can then use a heuristic evaluation of the page to identify areas for improvement and use the other stages of the optimization process to gather further insights to help you construct your new innovative design.

As the design is radically different from your existing page you may want to manage the risk that it actually reduces conversion by starting it on a relatively low proportion of traffic. However,
after a week or so, if you are not seeing a big drop in conversion you can increase the proportion of traffic that sees the new page to reduce the time it will take for the test to complete.


2. Optimise and Multivariate Tests:

Once you have found a new innovative design that performs better than you existing page you should look to dissect it to understand how you can further enhance its effectiveness. Provided you have sufficient traffic multivariate testing (MVT)  can be used. Unlike A/B testing MVT allows you to change content within multiple sections of the same page and compare all the possible combinations against each other. For example if you wanted to test changing two sections on a page and have two variables for each section that would generate 8 combinations.

2 x 2 x 2 = 8

However, adding just one more variable in a single section
increases the test combinations from 8 to 12.

2 x 2 x 3  = 12

MVT’s have the advantage that they allow you to isolate many small page elements to understand their individual impact on conversion.  You can also evaluate interaction effects between multiple independent elements to find compound effects.  This can save you time as you don’t have to create and test many different variations for a page element that might not even have much impact upon your conversion rate.

On the downside MVTs require more traffic to achieve statistical confidence than an A/B test. If you don’t have the traffic to support a complex MVT limit the number of combinations  or conduct a series of A/B tests instead. With MVTs you need to  ensure that all variations within each section make sense together. Once the MVT has identified which page elements contribute most to
conversion you should validate the winning combination using an A/B test to check that they deliver the promised uplift.


Image of two web pages with different button contrast

Source: NickKolenda

3. Real Estate Tests:

Although you may now have a high performing page, how do you
know that all the elements on the page are in the best location? Some of the elements on the page could be poorly performing from a conversion perspective because they are in a sub-optimal location. Perhaps your main call to action is too far down the page or testimonials are taking prime real estate above the fold and they would be equally as effective further down the page, just above
the fold.

Image of two webpage click heatmaps


Never assume that elements are in the best locations.  Your analytical tools, such as click and mouse movement heatmaps  should provide evidence that certain elements are not getting the attention you might expect, but to confirm this you will need to work with your web designers to develop tests that challenge the existing location of key assets on the page.  Try moving elements to different locations on the page but ensure that the page flow still works as otherwise that could influence the test result.

4. Inclusion/Exclusion Test:

Is that auto-rotating carousel really improving conversion? This is the stage in your page optimization process where you start turning off elements on your page to identify the conversion influences. If you remove your carousel from your homepage and you see a positive impact on conversion this tells you that you either have a poorly designed carousel or that you could use that prime real-estate for other types of assets that might increase

This type of test is ideal for pages like your homepage that have any different elements on them and could benefit from being de-cluttered. Having unnecessary assets on a page can be distracting and reduce engagement at an important stage in the user journey.  If an element is removed and there is no impact on conversion this could also be considered for removal or it could be moved to a less important page or location.

When removing an asset has a negative impact on conversion you know to retain it as showing it clearly improves conversion. However, you should then do follow-up A/B tests on this element to determine the best design for this type of asset.  Be cautious about removing assets that when removed show a positive impact on conversion if the element relates to specific use cases or conversion goals. Maybe the element has been poorly designed or is difficult to understand. If you have any evidence that this might be the case
A/B test different variants before deciding to remove from the page.

5. Segment and Target Your Tests:


image of tangerine segments

If you treat all your visitors the same you can only expect to have an average conversion rate. By definition some of your test variations will better meet the needs of certain visitor segments and as a result they may convert significantly higher for that group, but less well for other types of visitors. To further improve your conversion rate you should evaluate how you can segment and target your tests to create experiences designed to better satisfy
the needs of individual customer groups.

This approach will also boost your conversion rate because it leads to a much more dynamic website that responds to the needs of different user segments.  Set up key visitor segments (e.g. new and returning customers) in your analytics that you want to analyse and target with different content. This allows you to analyse your test
results to identify customer segments that performed significantly better than your average conversion uplift. You can then serve your winning test experience to all those visitor types that are more responsive to your new content.

Content automation is increasingly encroaching into this space and although it is a great tool, it is not a silver bullet. You can only
automate the content you have and if this is not optimal and engaging automating it will be of limited value. You should use A/B testing first to help create relevant content and understand how individual visitor segments respond to different user experiences. This will improve your chances of producing content that benefits from automation and is responsive to customer needs.

6. Test Iteration:

To avoid random and ad-hoc testing you should always base your tests on insights gleaned from previous tests or test additional assets following-on from an initial test.  Testing is a continuous process that enables your website to evolve gradually to better satisfy your customer needs and provide new insights to enhance your content marketing. A test and learn process is a much more scientific approach to website improvement than completely
redesigning your website from scratch.

Image of Test and Learn Process of A/B testing


In Conclusion:

By using these strategies to create a systematic plan for optimising
key pages on your site you are more likely to deliver substantial and
sustainable uplifts in conversion.  Each type of test is designed to provide specific insights and allow you to further enhance your conversion rate.

Never assume you have come to the end of your journey as your competitors will look to respond to your optimization strategy and disruptive technologies may change customer behaviour.  You will need to continue the optimization
process if you want to respond to changing visitor needs.

Thank you for reading my post. If you found it of interest please share this post by clicking on the social media icons below.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

  • About the author:  Neal provides digital optimisation consultancy services and has worked for  brands such as, and  He identifies areas for improvement using a combination of approaches including web analytics, heuristic analysis, customer journey mapping, usability testing, and Voice of Customer feedback.  By  aligning each stage of the customer journey  with the organisation’s business goals this helps to improve conversion rates and revenues significantly as almost all websites benefit from a review of customer touch points and user journeys.
  • Neal has had articles published on website optimisation on  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to You can follow Neal on Twitter @northresearch and view his LinkedIn profile.