Website optimisation test – what options are available?

Now before we start, we’re going be covering the CRO side of what a website optimisation test is. Overall website optimisation can also incorporate SEO and website speed. With SEO it’s about how you can optimise your website for rankings in search engines, where you can also run SEO website optimisation tests. We’re not going to cover that in this post, we’ll leave that for the SEO experts. And with website speed, it’s about checking which pages are slow and what’s slowing them down, where tools like gtmetrix.com are good for analysing this.

Website optimisation tests in CRO

Website optimisation (in the context of CRO) is the process of improving a website by focusing on the following areas:

Source: Eisenberg’s Hierarchy of Optimisation

• Functional – Does it work and do what it’s supposed to? Are there mistakes in the content or broken links?
• Accessible – Can people use the website on all devices, over all connection speeds and in all environments?
• Usable – Do people struggle to accomplish tasks or can they complete tasks efficiently and easily?
• Intuitive – Does the journey match the thought process of the user? Does it enable users to progress without thinking?
• Persuasive – Does your website answer all questions, beat the competition, encourage decision making, offer enough proof and remove all doubts?

In order to get to the point of optimising these areas, first you must undergo a period of research and analysis in order to learn about your audience and how they interact with your website. This involves an analytics health-check, analytics data analysis, a heuristic evaluation, competitor analysis, and qualitative and quantitative research including polls, surveys, and user testing.

Once you have analysed this data, you’ll have a comprehensive list of insights to act on and hypotheses to test, then your next step will be to create the variation for the first test you want to run.

What is a website optimisation test?

A website optimisation test is the process of identifying how a change (which is based on insight from research and analysis) will impact one of the areas in the above triangle. The test will determine how the change will impact the amount of people converting on your website.

Why do you need to run a website optimisation test?

A test is another method of learning more about how differently your audience behaves when presented with an alternative version of an element/page/website/experience.

Not every change requires a test. Usually when you’re fixing anything on the ‘functional’ level, we’ll implement these right away since they are fixing issues. If we’re changing something as well as fixing it though, as a minimum it’s worth getting user feedback via user testing to reduce any risk in case there are better ways to solve the problem for your users which might result in an even higher conversion rate.

What types of website optimisation tests can you run to see the impact of changes?

Note: If you don’t have the traffic to run an A/B test (work it out for yourself here), then you’ll only have the first two options to work with.

#1: Implementing a change right away and measuring after a period of time

After the change has been implemented, it will be measured after around a month (to account for some seasonality e.g. weekends, payday) and then a decision will be made on whether to keep the change, iterate, or scrap it.

It can be useful in some situations but it’s hard to measure the results because you don’t know where you would have been, had you not made the change. Your conversion rate will change due to traffic, seasonality, the market and hundreds of other variables – so it might not always be clear and may be difficult to see any change in conversion rate.

If you’re implementing a change on a certain segment e.g. the mobile checkout, then you may be more likely to pinpoint the conversion rate change as some variables will be removed but it’s still hard to know for sure your change has increased conversions.

Pros: Fine for risk-free changes like content mistakes, bugs, and fixing site speed issues.

Cons / Limitations: Not fine for any type of test that carries a degree of risk. One thing you learn from years and years of A/B testing is that you can never be certain what type of change will work and won’t work, it can be quite surprising. If you have any doubt, go to step #2.

#2: User testing to gain feedback

Running your changes via a panel of users will help you to validate whether your changes could affect your website positively before you implement.

You can do this via user testing or using tools like usabilityhub. You can run your new page design via a panel and ask questions regarding clarity, or next steps. You can also use a quantitative approach to this and run your changes through a panel and measure the results to get to statistical significance. Once you have had users validate your new design, you can implement and carefully measure.

An example of a usabilityhub study we created for Aldermore Bank’s new mortgage deal journey

Pros: It’s quick feedback that can be both qualitative and quantitative and will give you a good idea whether your solution is heading in the right direction.

Cons / Limitations: Running user feedback requires the skill to ask the right questions to user panels and avoid any confirmation bias (e.g. not asking ‘do you think this new design is better?’). Also it’s still relatively difficult to track conversion rate differences properly via Google Analytics. You may want to look at micro-metrics and hard numbers of conversions.

#3: A/B testing / A/B/n testing / multivariate testing / split URL testing

If you have the traffic to run an A/B test, then you can test your changes via a control version of the website and a variation version. This allows you to measure one version against the other and determine the winning variation via some statistical analysis.

An A/B/n test is where you’ll run more than one variation against the control. A multivariate experiment tests multiple elements in tandem – if you have 2 alternative versions of 2 separate elements on the same page to test, then you’d be running 4 versions against the control, which requires a larger sample size (so you’ll need more traffic).

A/B testing requires some knowledge of statistics, but this can be learned. It also requires a tool to run the tests like Google Optimize (free but limited) or Convert Experiences / VWO.

An example of a winning A/B test in VWO

Pros: This type of testing is an accurate scientific approach to measuring changes on your website if you have the traffic.

Cons / limitations: Making changes can be slow if you can only run one test with small changes at once. Some experiments can produce false positives or false negatives. People can test improperly, not running tests for long enough, not pre-calculating sample sizes and misinterpreting results.

If you’d like to chat about user research, A/B testing, or how website tests can increase conversions, then you can email us on [email protected] or call us on 0161 236 1188.