Interpreting your email campaign test results correctly

Free online sample size calculator tools to help you correctly understand your results. Are you sure you are correctly interpreting your email marketing A/B test campaigns results? Have you considered the statisical significance of your results? If your B test gives better results than your A sample, then always use approach B, right? Wrong. Using what you think is better may give you worse results.

If treatment B gives better results than control A then B is without doubt better. Right? Wrong!

Why can it be wrong? Let's say you test a call to action and find a click rate of 3% originally and 5% higher on your new version. A 5% increase on 3% is 0.15%. If your campaign was sent to less than 38,000 people then the 0.15% difference is probably statistical variation and not a repeatable result.

Your original control variant may still out perform in the long run and using the new call to action will give worse results. Our calculators let you work out what sample size to use and how significant your results are.

The new call to action will give worse results.

Some complex maths is needed to take the campaign size and response rates to determine the validity of the result. We have done all the hard maths for you in our free calculator tools. It takes just seconds; enter three numbers to validate your results and have confidence.

Find out more about how to calculate sample sizes.