The z-score is the statistical measure of the difference between two randomly drawn samples. It is expressed as the number of standard deviations an element is from the mean. The larger the z-score the higher the probability that the two means have not been drawn from the same population and the difference between the samples is statistically significant.
Sometimes known as a standard score the z-score can be positioned on a normal distribution curve (see below). The value of a z-score starts at -3 standard deviations (on the left of the normal distribution curve) and rises to +3 standard deviations (on the right side of the normal distribution curve). To use a z-score for a statistical test you will need the mean and the standard deviation of the population distribution.
The z-score allows you to compare the results of a test or survey to a “normal” population. To achieve a “normal” distribution your sample needs to be drawn randomly to ensure it is representative and it is a large sample (i.e. over 100 as an absolute minimum). The Z-score can then indicate how your test result compares to the average population’s mean for the success metric.
The z-score is used as the critical value in the calculation of the margin of error in an experiment. The margin of error is calculated using either of these equations.
Margin of error = Critical value x Standard deviation of the statistic
Margin of error = Critical value x Standard error of the statistic
When you know the standard deviation of the statistic you can use the first equation. Alternatively use the second equation. To identify the critical value as a z-score, look for the cumulative probability equal to the critical probability (p*).
Conversion marketing – Glossary of Conversion Marketing.
Over 300 tools reviewed – Digital Marketing Toolbox.
A/B testing software – Which A/B testing tools should you choose?
Types of A/B tests – How to optimise your website’s performance using A/B testing.