Frustrated because you want to do A/B testing to improve your website but don’t have enough traffic?
You are not alone. One of the questions I receive most from businesses is how to improve their website if their traffic is too low for A/B testing. So what can you do?
The good news is there are some things you can change when creating an A/B test that will mean you need less traffic.
And the best news news is that you don’t actually need it for improving your website. This is because A/B testing is only one part of successful conversion rate optimization.
First, check if your traffic is too low to run an A/B test
Before we discuss other ways to improve your low traffic website, it’s obviously important to first understand if you actually have enough traffic, and to learn what else is important.
How much traffic do you need to run an A/B test on a web page?
To put it simply, you need at least 5,000 visitors per week to the page you want to run an A/B test on. If you don’t have this much traffic, it will take a long time (often months or never) for your A/B testing tool to gather enough data to find a statistically significant result. Use this handy A/B test length calculator to give you an idea of how many days it will take you to get a result. Also, the more page variations you test, the more traffic you will need to get a result. But you need something else too…
Is it just about having enough website traffic to run an A/B test?
Even more importantly, you also need enough ‘conversions’ on your website to run an A/B test. This is because to run a test, you need to tell the A/B testing tool what determines success, and this is usually a major goal like a purchase, a sign-up or a form completion. And the less conversions your website gets per week, the longer it will take the testing tool to find a winning result. As a guideline, your website needs at least 500 conversions per week for a simple A/B test (250 per test version).
Try these techniques if you don’t have enough traffic
Don’t have enough traffic? Don’t give up hope about trying to improve your website! You can still use some techniques to increase the chances of you being able to run an A/B test, or use some alternative approaches to give you similar insights and website improvement results. Here are some great techniques to try:
a) Use an engagement A/B test success metric instead of orders or sign-ups.
As mentioned earlier, remember it’s not just about having enough traffic to run an A/B test – having enough conversions is even more important to be able to get a testing result. And you can define a conversion as something that happens more often – if you use an A/B test success metric that happens very frequently like a click to a specific page (like the next page in your checkout) instead of orders, this will count as many more conversions, and therefore much greater chance of being able to get a result from an A/B test.
This different type of success metrics works particularly well on the homepages for increasing engagement, using any click as the A/B test success metric. Other ideas for this are using ‘adds to cart’ instead of completed checkouts, or clicks to the sign-up page instead of completed sign-ups.
b) Increase the minimum detectable effect on conversion rate for your A/B test.
When you use an A/B test duration calculator there is one thing you can change that will mean you need less traffic for A/B testing. There is a minimum improvement in conversion rate that you want the A/B testing tool to detect (see image below). And the higher you make this number, the less traffic your tool will need to run the test. This is often about 20% by default, and you can increase this to 30% or more.
That is good right? Well there is a bad side. If you increase it to 30% for example, if one of your variations gets between a 1% and 29% increase in conversion rate, then your tool won’t be able to detect that winning variation. Therefore if you make this higher than 50%, you will need to create a really strong winning A/B test variation for the tool to detect it – which can be very hard!
To increase the chances of a getting bigger winning conversion rate increase (like over 30%) you will need to create A/B test variations that are radically different, for example changing multiple elements at the same time like headlines, imagery and call-to-action buttons.
c) Use Google Ads experiments to split test and find high-converting wording.
With this technique, instead of creating an A/B test in a testing tool, you use Google Ads experiments to find which ad variations get most clicks on wording relating to the page you want to improve. This is ideal for testing wording for headlines, benefits, call-to-actions and other important words on your website.
To do this simply create a Google Ads experiment using a few different ads that emphasize different headlines and descriptions, let it run and see which has the highest conversion rate (see below for an example of an experiments report). Then once you have found the winning ad, replicate that winning ad text on the page you wanted to improve, and watch your conversions and sales grow.
d) Use a landing page tool to test and find high-converting wording, imagery and call-to-actions.
You can also use a landing page creation tool to help you find best converting variations of key elements like your headlines, benefits, imagery and call-to-action buttons. You then use those learnings from those landing pages and use them on your regular website pages. You can do this because these tools have a limited version of A/B testing built-in to them.
To do this A/B testing with a landing page tool like Unbounce or Leadpages, create a few simple pages that have different wording, imagery and call-to-actions relating to the page you wanted to test. You will then need to make the conversion event to be a click through to the next page so you get a winning result faster. Once you have found out which versions of your landing page convert your visitors the most, you then take the winning variations and add that to the page on your website you wanted to improve.
e) Do user testing on proposed improvements by getting indepth feedback from visitors.
User testing helps you gain indepth ‘qualitative’ feedback from visitors about each of your proposed website improvements, in comparison to ‘quantitative’ testing that is based on just A/B testing result numbers. This type of user testing also helps you understand the reasons why visitors prefer particular variations of proposed improvements (something that A/B testing tools can’t actually tell you), and is ideal for creating even better follow-up improvements.
To do this type of testing, create another version of the page you want to test, and then using a tool like UsabilityHub or Usertesting.com show both versions to testers that match your target demographic (or use your own website visitors) and ask them questions based on which page version they prefer and engages them better. Then simply launch the page version that has the best feedback (or make additional improvements first and repeat the process). This technique gives such great feedback and insights you should always do this before launching proposed website improvements.
f) Launch your improvements and then monitor impact on key metrics.
This technique is where you simply launch the new improved page version for one week and check for the impact on key metrics like website conversion rate, sales or leads. Then determine if there is a good improvement in comparison to the previous version of the page (you will need to create a good benchmark of metrics to compare this to – at least a month ideally). Then if the new page performs better you keep it live and continue to monitor the impact. If it performs worse, then you roll back to the previous version of the page.
Although you don’t get statistical significance with this method, it’s better than just launching new pages and hoping for the best. To reduce the chance of issues, while you are monitoring new improvement launches try not to make any major change in traffic source or any other major website changes that might impact your key metrics.
When doing this type of website improvement launches, the quality of your improvement idea is really important for determining success. Rather than just guessing, to increase the chances of success I suggest you get website improvement recommendations from CRO experts like myself.
You can still do CRO if you don’t have enough traffic for A/B testing
You may think you can’t do conversion rate optimization (CRO) to improve your website if you don’t have enough traffic for A/B testing. Many people even think that A/B testing and CRO are the same thing. Fortunately these thoughts are both wrong. But why?
It is because A/B testing isn’t essential for improving websites and is only one of the 4 main elements of CRO. Conversion research, user experience (UX), and website persuasion are the other 3 essential elements. Learn about these other elements of CRO to help improve your website.
Ultimately you need to realize it’s not just about having enough traffic to do A/B testing, and that there are many other ways to find the best variations of your website.
Have you tried any of these techniques for running A/B tests on low traffic websites? Or which ones of these will you try first for your website?