A/B testing is a scientific way to check how a change to a website is going to impact behaviour.
By monitoring the way users who saw the change behave and comparing their behaviour to those who didn’t see the change, we can evaluate if our proposed change has positively impacted important metrics.
If we were to assess the effectiveness of website changes by comparing the website from one month to the next, we can never be sure that the results (good or bad) were down to a change in the environmental conditions in which visitors came to your website, such as a competitor closing, a change in regulation, market conditions, the political landscape, or even the weather.
We test hypotheses which have been formed through extensive user research. Some of the metrics we track include:
- User conversion rate
- Average order value
- Revenue per user
- Transactions or sign-ups