One of the questions I receive most from online businesses is how to test improving their website if their traffic is too low for A/B testing. Which is very important to do because running A/B tests using a tool like Visual Website Optimizer is one of the quickest ways to find which versions of your website increase sales or leads the most.
So to help websites with low traffic (like small business websites), I’ve created a guide revealing how much traffic you need for A/B testing, and techniques to help you test and improve your website if you don’t have enough traffic. You will see that you don’t actually need an A/B testing tool to improve your website.
First, check if your traffic is too low to run an A/B test
Before we discuss others ways to improve your website if you don’t have enough traffic for A/B testing, it’s obviously important to first understand if you have enough traffic!
How much traffic do you need to run an A/B test on a web page?
To put it in very simple terms, you need at least 5,000 unique visitors per week to the page you want to run an A/B test on. If you don’t have this much traffic, it will take a long time (3 weeks or more, if ever) for your testing tool to gather enough data to find a statistically significant result. This handy A/B test length calculator gives you an idea of how many days it will take to get a result. Also, the more page variations you test, the more traffic you will need to get a result. But you need something else too…
Is it just about having enough website traffic to run an A/B test?
In a nutshell, no. Even more importantly, you also need enough ‘conversions’ on your website to run an A/B test. This is because to run a test, you need to tell the A/B testing tool what determines success, and this is usually a major goal like a purchase, a sign-up or a form completion. And the less conversions your website gets per week, the longer it will take the testing tool to find a winning result. As a guideline, your website needs at least 500 conversions per week for a simple A/B test (250 per test version).
What happens if I have enough traffic but not enough conversions?
If you have enough traffic to run an A/B test, but not enough conversions like sales or signups, then consider changing the conversion to be something simpler that happens more often, like a click to a specific next page in the purchase/signup flow to determine success. I will discuss ideas for this in more detail shortly.
How much traffic would I need to use a multivariate test (MVT)?
A multivariate test is an advanced A/B test, where your tool simultaneously tests multiple versions of different page elements (rather than simply testing one page or element versus another in an A/B test). This requires much greater levels of traffic, because the tool needs to show each combination of elements being tested to enough visitors to be able to build statistical significance. If you have low traffic, you simply won’t be able to run an MVT.
Try these techniques if you don’t have enough traffic
Don’t have enough traffic? Don’t give up hope about trying to improve your website! You can still use some techniques to increase the chances of you being able to run an A/B test, or use some alternative approaches to give you similar insights and website improvement results. Here are some great techniques to try:
a) Use an engagement test success metric instead of orders or sign-ups. As mentioned earlier, remember it’s not just about having enough traffic to run an A/B test – having enough conversions is even more important to be able to get a testing result. So if you use a success metric that happens very frequently like a click to to a specific page (like the next page in your checkout) instead of orders, you will get more conversions, and therefore much greater chance of getting a test result much quicker. You can even do this on homepages, using any click as the conversion metric. Other ideas for this are using ‘adds to cart’ instead of completed checkouts, or clicks to the sign-up page instead of completed sign-ups.
b) Buy extra traffic for a short while to the page you want to test. If you can afford it, the simplest technique is to just spend more on driving traffic to the page you want to to do an A/B test on. This way you hopefully get enough traffic to run an A/B test, and could be driven by using Google Adwords or Facebook Advertising (which I think is better and cheaper). If you have a a very large email subscriber base, you could also try sending an email campaign that drives traffic to the page you want to test, which may result in enough traffic for you to run an A/B test. Not ideal as it’s just a quick traffic spike, but it can help increase traffic along with other methods.
c) Use Google Adwords to test and find high-converting wording. With this technique, instead of creating an A/B test in a testing tool, you use Google Adwords split testing function to find which ad variations get most clicks of your headlines and wording that relate to the page you want to test. This is ideal for testing wording for benefits, call-to-actions and other important words on your website. Here are a few variations in Google Adwords to help illustrate this testing:
To do this, simply create a few different ads in Google Adwords that emphasize different headlines and descriptions, turn on the ad rotation function (for fairest results, use ‘rotate evenly’ option), and see which ad version gets the most clicks to your website. Then once you have found the winning ad, replicate that winning text on the page you wanted to test, and watch your conversions and sales grow! Here is a great guide that shows a company that used this with great results.
d) Use a landing page tool to test and find high-converting wording, imagery and call-to-actions. As an even better option than using Google Adwords to run tests to improve your website, you can use a landing page tool that offers basic built-in testing functionality (if you can afford it).
To do this with a landing page tool, create a few simple pages that have different wording, imagery and call-to-actions relating to the page you wanted to test, and find out which version engages your visitors the most. You then simply take the winning content and add that to the page you wanted to initially test. You can use a landing page tool like Unbounce or Leadpages to run this type of test (note, they don’t gather or show you statistical data, so the results won’t be as good or accurate as using an A/B testing tool).
e) Do user testing by getting indepth feedback from visitors. User testing is based on gaining indepth ‘quality’ feedback from visitors about each of your proposed improvement ideas, in comparison to ‘quantitative’ testing that is based on just A/B testing result numbers. This type of user testing also helps you understand the reasons why visitors prefer particular variations of proposed improvements (something that A/B testing tools can’t actually tell you), and is ideal for creating follow-up improvements.
To do this type of testing, create another version of the page you want to test, and then using a tool like Usertesting.com show both versions to testers (you can even use your own website visitors) and ask them questions based on which page version they prefer and engages them better. Then simply launch the page version that has the best results. This technique gives such great feedback and insights, that I suggest using this in addition to using an A/B testing tool.
f) Launch your improvements and then monitor impact on key metrics. This manual testing technique is where you simply launch the new page version you want to test for one week, check key metrics at the end of the week, then revert back to current page (the control page) for a week and gather data. After the 2 weeks, you see which version converted the most visitors for your major website goals. This has several drawbacks though – it does not give you statistical significance, and your visitors will see two different versions over the two weeks which can be confusing for them (with an A/B testing tool that wouldn’t happen because it cookies visitors to ensure consistent experience).
When doing this type of improvement launches, the quality of your improvement idea is really important for determining success. To increase the chances of these working well I suggest getting recommendations and test ideas from CRO experts like myself.
Ultimately you should realize that you don’t have to do A/B testing to be able to improve your website – it’s only one part of successful conversion rate optimization. While it’s certainly very useful to do A/B testing, using web analytics insights and gathering visitor feedback are just as important to find out what to improve on your website.
Now you know some techniques for improving your website if you have low traffic! Have you tried any of these techniques for running tests on low traffic websites? Or which ones of these will you try first?