How to Improve Your Website If Your Traffic is Too Low for A/B Testing

Last updated By | 34 Comments

low traffic A/B testing

Are you trying to improve your website to gain more sales or leads? Frustrated because you wanted to try A/B testing but it seems like you don’t have enough traffic?

You are not alone. One of the questions I receive most from online businesses is how to improve their website if their traffic is too low for A/B testing.  Which is important because running A/B tests using tools like Visual Website Optimizer is one of the quickest ways to find which versions of your website increase sales or leads the most. But what happens if you don’t have enough traffic?

To help improve websites with low traffic (like small business or startup websites), I’ve created a guide revealing some great techniques to test and improve them. You will also learn you don’t actually need an A/B testing to improve your website, and that it is only one part of successful conversion rate optimization.

First, check if your traffic is too low to run an A/B test

Before we discuss other ways to improve your low traffic website, it’s obviously important to first understand if you actually have enough traffic and to learn what else is important.

How much traffic do you need to run an A/B test on a web page?

To put it in very simple terms, you need at least 5,000 unique visitors per week to the page you want to run an A/B test on. If you don’t have this much traffic, it will take a long time (3 weeks or more, if ever) for your A/B testing tool to gather enough data to find a statistically significant result. You should use this very handy A/B test length calculator to give you an idea of how many days it will take you to get a result. Also, the more page variations you test, the more traffic you will need to get a result. But you need something else too…

Is it just about having enough website traffic to run an A/B test?

In a nutshell, no. Even more importantly, you also need enough ‘conversions’ on your website to run an A/B test. This is because to run a test, you need to tell the A/B testing tool what determines success, and this is usually a major goal like a purchase, a sign-up or a form completion. And the less conversions your website gets per week, the longer it will take the testing tool to find a winning result. As a guideline, your website needs at least 500 conversions per week for a simple A/B test (250 per test version).

What happens if I have enough traffic but not enough conversions?

If you have enough traffic to run an A/B test, but not enough conversions like sales or signups, then consider changing the conversion to be something simpler that happens more often, like a click to a specific next page in the purchase/signup flow to determine success. I will discuss ideas for this in more detail shortly.

How much traffic would I need to use a multivariate test (MVT)?

A multivariate test is an advanced A/B test, where your tool simultaneously tests multiple versions of different page elements (rather than simply testing one page or element versus another in an A/B test). This requires much greater levels of traffic, because the tool needs to show each combination of elements being tested to enough visitors to be able to build statistical significance. If you have low traffic, you simply won’t be able to run an MVT.

Try these techniques if you don’t have enough traffic

Don’t have enough traffic? Don’t give up hope about trying to improve your website! You can still use some techniques to increase the chances of you being able to run an A/B test, or use some alternative approaches to give you similar insights and website improvement results. Here are some great techniques to try:

a) Use an engagement A/B test success metric instead of orders or sign-ups.
As mentioned earlier, remember it’s not just about having enough traffic to run an A/B test – having enough conversions is even more important to be able to get a testing result. And you can define a conversion as something that happens more often – if you use an A/B test success metric that happens very frequently like a click to a specific page (like the next page in your checkout) instead of orders, this will count as many more conversions, and therefore much greater chance of being able to get a result from an A/B test.

This different type of success metrics works particularly well on the homepages for increasing engagement, using any click as the A/B test success metric. Other ideas for this are using ‘adds to cart’ instead of completed checkouts, or clicks to the sign-up page instead of completed sign-ups.  

b) Buy extra traffic for a short while to the page you want to A/B test.
If you can afford it, the simplest technique is to just spend more on driving traffic to the page you want to to do an A/B test on. This way you hopefully get enough traffic and conversions to run an A/B test, and could be driven by using Google Adwords or Facebook Advertising (which I think is better and cheaper).

If you have a a very large email subscriber base, you could also try sending an email campaign that drives traffic to the page you want to test, which may result in enough traffic for you to run an A/B test. Not ideal as it’s just a quick traffic spike, but it can help increase traffic along with other methods.

c) Use Google Adwords to split test and find high-converting wording.
With this technique, instead of creating an A/B test in a testing tool, you use Google Adwords split testing function to find which ad variations get most clicks of your headlines and wording that relate to the page you want to test. This is ideal for testing wording for benefits, call-to-actions and other important words on your website. Here are a few variations in Google Adwords to help illustrate this split testing:

adwords split testing

To do this, simply create a few different ads in Google Adwords that emphasize different headlines and descriptions, turn on the ad rotation function (for fairest results, use ‘rotate evenly’ option), and see which ad version gets the most clicks to your website. Then once you have found the winning ad, replicate that winning text on the page you wanted to test, and watch your conversions and sales grow! Here is a great guide that shows a company that used this with great results.

d) Use a landing page tool to test and find high-converting wording, imagery and call-to-actions.
You can also use a landing page creation tool to help you find best converting variations of key elements like your headlines, benefits, imagery and call-to-action buttons. You then use those learnings from those landing pages and use them on your regular website pages. You can do this because these tools have a limited version of A/B testing built-in to them.

To do this A/B testing with a landing page tool like Unbounce or Leadpages, create a few simple pages that have different wording, imagery and call-to-actions relating to the page you wanted to test. You will then need to make the conversion event to be a click through so you get a result faster. Once you have found out which versions convert your visitors the most, you then take the winning variations and add that to the page you wanted to initially test.

unbounce a/b testing

e) Do user testing on proposed improvements by getting indepth feedback from visitors.
User testing is based on gaining indepth ‘qualitative’ feedback from visitors about each of your proposed website improvements, in comparison to ‘quantitative’ testing that is based on just A/B testing result numbers. This type of user testing also helps you understand the reasons why visitors prefer particular variations of proposed improvements (something that A/B testing tools can’t actually tell you), and is ideal for creating even better follow-up improvements.

To do this type of testing, create another version of the page you want to test, and then using a tool like Usertesting.com show both versions to testers that match your target demographic (you can even use your own website visitors) and ask them questions based on which page version they prefer and engages them better. Then simply launch the page version that has the best feedback (or make additional improvements first and repeat the process). This technique gives such great feedback and insights, that I suggest always using this in addition to using an A/B testing tool.

f) Launch your improvements and then monitor impact on key metrics.
This technique is where you simply launch the new improved page version for one week, check key metrics at the end of the week, then revert back to current page (the control page) for a week and then check key metrics again. After the 2 weeks, determine which version converted the most visitors for your major website goals. If the new page version performs better then launch it. Although you don’t get statistical significance with this method, it’s better than just launching new content and hoping for the best.

When doing this type of website improvement launches, the quality of your improvement idea is really important for determining success. Rather than just guessing, to increase the chances of success I suggest you get website improvement recommendations from CRO experts like myself.

You can still do CRO if you don’t have enough traffic for A/B testing

You may think you can’t do conversion rate optimization (CRO) to improve your website if you don’t have enough traffic for A/B testing. Many people even think that A/B testing and CRO are the same thing. Fortunately these thoughts are both wrong. But why?

A/B testing isn’t essential for improving websites because it is only one of the main elements of CRO – web analytics, web usability, visitor feedback and persuasion are the other essential elements. Gaining many high impact insights from visitor feedback and web analytics will give you some great ideas in particular for improving your website with excellent results.

Wrapping Up

Ultimately you need to realize it’s not just about having enough traffic to do A/B testing, and that there are many other ways to find the best variations of your website.

Have you tried any of these techniques for running A/B tests on low traffic websites? Or which ones of these will you try first for your website?

Like this article? Please share it using these icons below... thanks!

122
SHARES
FREE CONVERSION TOOLBOX TO BOOST YOUR WEBSITE SALES
Get free access to my 7 expert guides and discounts to boost your website sales or leads
Also includes blog updates and we won't spam you!
  • http://www.titanium-jewelry.com/ J.R. YATES

    Good advice here Rich. Many of us smaller online businesses feel overwhelmed at a/b testing due to lower traffic. But we still need to be constantly testing and improving our site for increased conversions!

  • http://rich-page.com/ Rich Page

    Thanks Ron, glad you liked the advice, and hopefully you feel less overwhelmed soon!

  • http://metzmiranda.wordpress.com/ Metz

    Pretty helpful.

    This is a good step by step guide on how to improve your website. Identifying your traffic is necessary if it is low or not. The A/B test could help many marketers. A must share techniques. :)

    Your post has been shared on Kingged.com, IM social bookmarking site, enabling me to find this good piece.

  • Sara

    Nice article, though I thought a rule of thumb for tests and conversions was to have at least 100 conversions per variable?!

  • http://rich-page.com/ Rich Page

    Thanks Sara! I’ve heard that mentioned a few times – personally think 100 conversions per test variable is too high, and unachievable unless you have very high levels of traffic. That’s why these techniques are handy :)

  • http://rich-page.com/ Rich Page

    Thanks Metz! Glad you like my guide.

  • http://conversionscientist.com Brian Massey

    You can also split testing with less than 100 conversions. You just need really big wins. If you have a treatment with 20 conversions and another with 40 conversions, a 100% difference is something you can probably bank on, even with such small numbers. However, if one treatment got 20 conversions and the other got 30, that 50% increase is too close to the margin of error and shouldn’t be considered an improvement (even though it feels like a win).

  • http://rich-page.com/ Rich Page

    Yes thats a great point Brian! Good to have a scientist around here ;)

  • http://www.smartbusinessrevolution.com/ John Corcoran

    This is great advice, Rich. Thanks for such a thorough discussion.

  • http://rich-page.com/ Rich Page

    Thanks John – glad you liked it. Hopefully it can help you on some of your lower traffic pages…

  • Shiv Ettes

    Wow, lots of information packed in here. I don’t know much about A/B testing, but now I’m thinking it’s more important than I thought it was. Thanks for the warning about multi-armed bandit testing; definitely doesn’t sound ideal.

  • mony1
  • Bart Waldon

    I am going to have lest than 1,000 visitors to my site per month… Should I still ab test?

  • Excelanto

    Thanks for your valuable posting.I have collect more than information from your website. It is really wonderful blog. please added more than tips. i’m working in Erp in india.Here providing very low price and Quality ERP,Cloud ERP, CMS , responsive webdesign and ERP. you have any more than information kindly make me call this number 044-42127512 or send your mail info@excelanto.com….

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content as you, internet would be a much better
    place. Please keep it up! sutura silhouette

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content as you, the internet would be a much better
    place. Please keep it up! dermatologista florianopolis

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content you, the internet would be a much better
    place. Please keep it up! dermatologista florianopolis

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content as you, the internet would be a much better
    place. Please keep it up! Taxi Chur

  • Billy Simmons

    There is a new advertising and promotion engineering available since makes all the alternatives that are other appear to be the laugh they’re. You can start getting targeted prospects observing your offers in
    only a few momemts, also it merely requires a few mouse clicks to get going! Viral Dominator generate viral traffic and promotion instantly – your ads get viewed over 3,000 times per month with only 10 referrals!
    http://viraltrafficdominator.com/

  • bdctg

    Really great post, Thank you for sharing
    This knowledge. Excellently written article, if only all bloggers offered the
    same level of content as you, the internet would much better place. Please
    keep it up! contador florianopolis

  • raymond

    hae rich. im planning on starting a travel blog about africa but im worried i may get a low traffic on my website. please give me an advice on how i can make it a success. thanks

  • Mohammed Saimon

    Hey rich, just an excellent post. thanks for it.

    Regards

    GadgetGhor

  • ruhul amin

    Thank you very much, this is really a large amount of information content-rich articles, I am glad that the author could get out to share with you It is very useful. Try to keep on posting these kind of things. Thanks for sharing. aviation

  • NicoF

    Small remark regarding Google Experiments: yes, by default it runs the multi-armed bandit logic, but you can choose to select ‘rotate evenly’ instead (which is something I ALWAYS do)

  • http://boosta.ru/ Andrei Baklinau

    It’s very interesting point.

  • Nikolai

    Great article Rich! Few thoughts: I would be inclined to choose Adwords over Facebook for a cleaner test: on Adwords one can limit the audience by running “cleaner” traffic from 1 carefully chosen keyword with 1 ad, versus Facebook approach where it can be harder to choose a uniform audience, statistically valid for a small test, simply based on targeting options available.

    Having said that, I think one should run experiments on the same demo (and source of traffic) that the website if targeted to: Google and FB audiences vary a lot and more often than not, are in different browsing modes alltogether.

  • http://rich-page.com/ Rich Page

    Thanks for the comment and tips Nikolai! I totally agree that its best to experiment using similar audiences, as that can have a big impact on conversion rates.

  • http://rich-page.com/ Rich Page

    Thanks NicoF – I will have to check that out. I didn’t notice that option last time I used the tool.

  • Abrar Shahriar

    Hi Rich,

    I just go through this article. While reading it, I feel like I am reading a story. This is just great. I have learnt something new from this…
    5000 unique visitors in a week for A/B testing (small business).
    I wish I could have..

    But if conversion gets near to 100%, then number of visitors are not matter.

  • http://www.danielmelbye.com/ Daniel Melbye

    The question is where is the line drawn and how do you calculate this i.e. 100% is good, what about 95, 90? what it’s an 85% improvement

  • http://dotnetasansol.com/ Dot Net Asansol

    Thank you so much for your advice me http://gamesphp.com

  • http://www.alexanderhipp.com/ Alexander Hipp

    If you need more thoughts about the topic, please have a look here http://tinyurl.com/zxss8o3

  • https://plus.google.com/113259068312528852444 Steve Shaw

    While I get the point, the idea that 5,000 uniques to a page is a minimum before you can do any testing may put many people off split testing, when they could actually still achieve meaningful results. It’s more about the length of time.

    For example, using Google Analytics allows up to 90 days per Experiment; split tests don’t have to complete in a week to be useful. Even if an experiment takes a couple months for a result, it’s still worth doing, and may bring in significant extra revenue to be able to then increase traffic and speed up testing in future.

  • https://www.analytics-toolkit.com/ Georgi Georgiev

    Hi Rich,

    Some good advice here, but maybe it would have helped if you got in a bit deeper into the four variables that affect the required sample size for a test: the baseline, the certainty of the result (statistical signifiance) required, the sensitivity of the test (statistical power) and the minimum effect that would be interesting from a business standpoint. Moving any of these 4 gears can drastically change the required sample size, but it is always a trade-off between speed, efficiency and flexibility on one hand and accuracy, certainty, sensitivity and predictive value on the other. Properly cleaning the baseline so it includes only users with the prospect to convert is the only thing which is not a trade-off, but is sometimes overlooked.

    Another way to drastically reduce the number of users it takes to run a test is to use a sequential testing approach. I don’t know if you explored such approaches, but for example using AGILE A/B Testing you’d be able to reach statistically rigorous conclusions 20% to 80% faster than with fixed sample designs. There is a free white paper out, detailing the approach, while a simple tool to apply the method is available here: https://www.analytics-toolkit.com/ab-testing-calculator/ .

    P.S. “Buy extra traffic for a short while to the page you want to A/B test.” seems like a bad advice, since this traffic might very well not be representative of the traffic the tested page(s) would get after the test is over, thus the predictive value of the test may be negatively affected!