A/B Testing Guide for Improving Websites

Want to improve your website performance, but not sure what improvements will work best?

You can use A/B testing to discover which variations will convert more visitors into sales or leads.

I have created this guide to reveal all you need to know about A/B testing and best practices for using it.

First the basics. What is A/B testing?

A/B testing is the comparison of two or more pages or variations on your website to find which performs better for increasing one of your major goals (e.g. increasing sales, leads or click-through rate).

For example, A/B testing one homepage (A) versus another homepage (B) to see which performs better for increasing click-through rate and sales.

As shown in the example below, homepage A would be the current version and homepage B would have proposed improvements, like better headline, imagery, benefits and call-to-action button.

a/b testing example

What are the benefits of A/B testing?

Ultimately A/B testing enables you to improve your website so that it converts a higher percentage of visitors into sales or leads (known as your conversion rate).

And the higher your conversion rate, the more revenue you make on your website, and the more profitable it is – everyone wants that, right?

Here are some of the other benefits of A/B testing:

A/B testing takes out the guesswork of not knowing what changes will increase your conversion rates, and sales or leads.
It allows you to measure the impact of changes on your website – without having to push new content live and hope it works.
It allows you to make changes much quicker and easier – without risking launching content that doesn’t increase sales or leads.

What results can you expect from A/B testing?

The results will vary depending on how good your A/B test ideas are, but here are some great examples of the results you can get:

I’ve helped many clients get great results – including a 20% increase in leads with just one A/B test.
Even just a small 10% increase in conversion rate can result in $1,000s in increased sales.
The sky is the limit with A/B testing results! For example, GetResponse.com increased sign-ups by 150% and Barrack Obama raised an additional $60 million.

How does A/B testing work?

A/B testing is performed on websites using an A/B testing tool to help you set up and run A/B tests.

This type of tool lets you test one or more new page or page element versions against the current version (called the control).

In the example above, the tool would show 50% of visitors homepage A, and it would show homepage B to the other 50% of visitors.

It would then calculate which version, if any, results in significantly higher percentage of leads or sales (called conversion rate), and declare a winner if one is found (called a statistically significant winner).

Using an A/B testing tool you can test multiple pages against the control (not just one) or test specific page elements like headlines, images or buttons.

And you can do A/B testing on any page you want – for example your product pages, checkout flow or landing pages.

How to get started with A/B testing on your website?

There are five main basic steps for getting started with using A/B testing:

First you need an A/B testing tool, like VWO or Google Optimize (check out my A/B testing tool comparison guide).
You then need some good A/B testing ideas (ideally gathered from your web analytics and visitor feedback).
Prioritize which A/B testing ideas are likely to have highest impact on your website goals (to help with this use my A/B testing prioritization tool in my free toolbox).
Design and launch the highest impact A/B test ideas on your website using your A/B testing tool. Simple changes can be made by using the visual editor in these tools, or use their code editor to make more complex changes.
Lastly, you need to analyze and learn from the results to create even better follow up A/B tests.

My A/B testing tool success guide covers these and other steps in more details, and will ensure you get even better results from using your A/B testing tool.

What are some of the best practices for A/B testing?

A/B testing can be hard to get good results, and you need to learn best practices to ensure you get better conversion rate increases (and more sales or leads). Here are some of them:

A/B test elements on your key pages first – like your homepage, top entry pages and your checkout or signup flow.
Don’t just guess at what needs improving or listen to what your boss thinks – this often leads to poor A/B test results.
Use your web analytics tool to find great insights and A/B testing ideas (e.g. poorly performing pages to improve first).
Get visitor feedback to discover issues and create higher-impact A/B test ideas (e.g. from surveying and usability sessions).
A/B test page elements that are above the page fold – these are seen more often so have a high impact on conversions.
Use revenue as a goal for your A/B tests in addition to sales or leads – it is essential to show senior management the revenue impact.
Always wait at least 7 days before declaring an A/B test winner. This allows for daily variances in traffic to level off.
To get even better results, get expert A/B test ideas and perspective from conversion rate optimization and A/B testing experts.

Do you need to A/B test everything you want to improve?

You don’t have to A/B test every improvement you want to make to your website. In fact quite often you won’t have enough traffic to A/B test everything. Here is a list of some key elements that can have big impact by A/B testing them, and what should just be launched without A/B testing.

A/B test it – website elements always worth A/B testing:

Any time that it is unclear which improvement version will perform better, particularly when it comes to elements regarding psychology and influence, these are definitely worth A/B testing to find the one with the highest conversion rate. Here are some examples:

Headlines (these have a huge impact on visitor engagement)
Website copy on key pages like the homepage and service/product pages
Hero images on the homepage
Call-to-action wording on buttons
Incentives for capturing email addresses (discounts versus guides etc)
Influence and persuasion elements mentioning scarcity or urgency

Launch it – website improvements to launch instead of A/B testing:

These elements are considered best practice and will improve any website, so should just be launched without needing A/B testing first. This frees up time to A/B test other elements worth A/B testing. Here are some examples of what to just launch:

Usability fixes and improvements (improving confusing or difficult navigation)
Prominent unique value proposition elements on key entry pages
Purchase risk reducers like guarantees, free shipping and free returns
Good filters and sort options on category pages
Increasing number and quality of product or service images
Social proof like reviews and ratings, ‘as featured in’ and third party ratings
Secure messaging and icons in the checkout or signup flow

You can certainly do follow up A/B testing to fine tune these or iterate on the exact location or style of them, but the key thing is to just launch them first because they are so important to have.

Looking for high-impact A/B test ideas?

Don’t just guess at what to A/B test or you will get limited results. To ensure you get high-impact A/B test ideas and the best results, consider my A/B testing services. Or to help you get started, grab my high-impact A/B test ideas guide below for free.
[ois skin=”Ribbon Footer Skin – High Impact”]