A/B testing can use a Bayesian approach

A/B Test:

Definition:

An A/B test is a controlled experiment where visitors are randomly selected to view one of two variations of the same web/app page. It is also known as split testing and each design is usually referred to as a variant.

As traffic is split randomly between variant A and B we can be confident that any statistically significant difference in the success metric (e.g. the conversion rate) between the two designs is likely to be due to the change in experience and not the result of other external factors (e.g. source of traffic, day of week etc). Most organisations use a specialist A/B testing tool to build, run and analyse their A/B tests.

An A/Bn test is where there are multiple variants of the same web/app page. For example an A/Bn test with 3 variants will mean that roughly 33% of visitors will see each variant. The more variants you test the smaller the proportion of visitors will see each variant and the longer the test will need to run to obtain a statistically significant result. Also see multivariate test (MVT).

Resources:

Conversion marketing – Glossary of Conversion Marketing.

Over 300 tools reviewed – Digital Marketing Toolbox.

A/B testing software – Which A/B testing tools should you choose?

Types of A/B tests – How to optimise your website’s performance using A/B testing.

Digital Marketing and Insight