How to Improve Your Website If Your Traffic is Too Low for A/B Testing

Last updated By | 34 Comments

low traffic A/B testing

Are you trying to improve your website to gain more sales or leads? Frustrated because you wanted to try A/B testing but it seems like you don’t have enough traffic?

You are not alone. One of the questions I receive most from online businesses is how to improve their website if their traffic is too low for A/B testing. So what can you do?

The good news is that there are some things you can change when creating an A/B testing tool that will mean you need less traffic.

And the best news news is that you don’t actually need it for improving your website. This is because A/B testing is only one part of successful conversion rate optimization.

In this article you will learn great techniques for improving your website if you have a low amount of traffic.

First, check if your traffic is too low to run an A/B test

Before we discuss other ways to improve your low traffic website, it’s obviously important to first understand if you actually have enough traffic, and to learn what else is important.

How much traffic do you need to run an A/B test on a web page?

To put it in very simple terms, you need at least 5,000 visitors per week to the page you want to run an A/B test on. If you don’t have this much traffic, it will take a long time (often months or never) for your A/B testing tool to gather enough data to find a statistically significant result. You should use this very handy A/B test length calculator to give you an idea of how many days it will take you to get a result. Also, the more page variations you test, the more traffic you will need to get a result. But you need something else too…

Is it just about having enough website traffic to run an A/B test?

In a nutshell, no. Even more importantly, you also need enough ‘conversions’ on your website to run an A/B test. This is because to run a test, you need to tell the A/B testing tool what determines success, and this is usually a major goal like a purchase, a sign-up or a form completion. And the less conversions your website gets per week, the longer it will take the testing tool to find a winning result. As a guideline, your website needs at least 500 conversions per week for a simple A/B test (250 per test version).

What happens if I have enough traffic but not enough conversions?

If you have enough traffic to run an A/B test, but not enough conversions like sales or signups, then consider changing the conversion to be something simpler that happens more often, like a click to a specific next page in the purchase/signup flow to determine success. I will discuss ideas for this in more detail shortly.

Try these techniques if you don’t have enough traffic

Don’t have enough traffic? Don’t give up hope about trying to improve your website! You can still use some techniques to increase the chances of you being able to run an A/B test, or use some alternative approaches to give you similar insights and website improvement results. Here are some great techniques to try:

a) Use an engagement A/B test success metric instead of orders or sign-ups.
As mentioned earlier, remember it’s not just about having enough traffic to run an A/B test – having enough conversions is even more important to be able to get a testing result. And you can define a conversion as something that happens more often – if you use an A/B test success metric that happens very frequently like a click to a specific page (like the next page in your checkout) instead of orders, this will count as many more conversions, and therefore much greater chance of being able to get a result from an A/B test.

This different type of success metrics works particularly well on the homepages for increasing engagement, using any click as the A/B test success metric. Other ideas for this are using ‘adds to cart’ instead of completed checkouts, or clicks to the sign-up page instead of completed sign-ups.  

b) Increase the minimum detectable effect on conversion rate for your A/B test.
When you use an A/B test duration calculator there is one thing you can change that will mean you need less traffic for A/B testing. There is a minimum improvement in conversion rate that you want the A/B testing tool to detect (see image below). And the higher you make this number, the less traffic your tool will need to run the test. This is often about 20% by default, and you can increase this to 30% or more.

That is good right? Well there is a bad side. If you increase it to 30% for example, if one of your variations gets between a 1% and 29% increase in conversion rate, then your tool won’t be able to detect that winning variation. Therefore if you make this higher than 50%, you will need to create a really strong winning A/B test variation for the tool to detect it – which can be very hard!

To increase the chances of a getting bigger winning conversion rate increase (like over 30%) you will need to create A/B test variations that are radically different, for example changing multiple elements at the same time like headlines, imagery and call-to-action buttons.

c) Use Google Adwords experiments to split test and find high-converting wording.
With this technique, instead of creating an A/B test in a testing tool, you use Google Adwords experiments to find which ad variations get most clicks on wording relating to the page you want to improve. This is ideal for testing wording for headlines, benefits, call-to-actions and other important words on your website.

To do this simply create a Google Adwords experiment using a few different ads that emphasize different headlines and descriptions, let it run and see which has the highest conversion rate (see below for an example of an experiments report).  Then once you have found the winning ad, replicate that winning ad text on the page you wanted to improve, and watch your conversions and sales grow.

d) Use a landing page tool to test and find high-converting wording, imagery and call-to-actions.
You can also use a landing page creation tool to help you find best converting variations of key elements like your headlines, benefits, imagery and call-to-action buttons. You then use those learnings from those landing pages and use them on your regular website pages. You can do this because these tools have a limited version of A/B testing built-in to them.

To do this A/B testing with a landing page tool like Unbounce or Leadpages, create a few simple pages that have different wording, imagery and call-to-actions relating to the page you wanted to test. You will then need to make the conversion event to be a click through to the next page so you get a winning result faster. Once you have found out which versions of your landing page convert your visitors the most, you then take the winning variations and add that to the page on your website you wanted to improve.

e) Do user testing on proposed improvements by getting indepth feedback from visitors.
User testing helps you gain indepth ‘qualitative’ feedback from visitors about each of your proposed website improvements, in comparison to ‘quantitative’ testing that is based on just A/B testing result numbers. This type of user testing also helps you understand the reasons why visitors prefer particular variations of proposed improvements (something that A/B testing tools can’t actually tell you), and is ideal for creating even better follow-up improvements.

To do this type of testing, create another version of the page you want to test, and then using a tool like UsabilityHub or Usertesting.com show both versions to testers that match your target demographic (you can even use your own website visitors) and ask them questions based on which page version they prefer and engages them better. Then simply launch the page version that has the best feedback (or make additional improvements first and repeat the process). This technique gives such great feedback and insights that I suggest you always use this method for gaining feedback on proposed website improvements.

f) Launch your improvements and then monitor impact on key metrics.
This technique is where you simply launch the new improved page version for one week and check for the impact on key metrics like website conversion rate, sales or leads. Then determine if there is a good improvement in comparison to the previous version of the page (you will need to create a good benchmark of metrics to compare this to – at least a month ideally). Then if the new page performs better you keep it live and continue to monitor the impact. If it performs worse, then you roll back to the previous version of the page.

Although you don’t get statistical significance with this method, it’s better than just launching new pages and hoping for the best. To reduce the chance of issues, while you are monitoring new improvement launches try not to make any major change in traffic source or any other major website changes that might impact your key metrics.

When doing this type of website improvement launches, the quality of your improvement idea is really important for determining success. Rather than just guessing, to increase the chances of success I suggest you get website improvement recommendations from CRO experts like myself.

You can still do CRO if you don’t have enough traffic for A/B testing

You may think you can’t do conversion rate optimization (CRO) to improve your website if you don’t have enough traffic for A/B testing. Many people even think that A/B testing and CRO are the same thing. Fortunately these thoughts are both wrong. But why?

It is because A/B testing isn’t essential for improving websites and is only one of the main elements of CRO. Web analytics, user experience (UX), and persuasion are the other essential elements. You can learn all about these other elements of CRO to help improve your website.

Wrapping Up

Ultimately you need to realize it’s not just about having enough traffic to do A/B testing, and that there are many other ways to find the best variations of your website.

Have you tried any of these techniques for running A/B tests on low traffic websites? Or which ones of these will you try first for your website?

Found this article useful? Please share it on social media. Thanks!

  • Good advice here Rich. Many of us smaller online businesses feel overwhelmed at a/b testing due to lower traffic. But we still need to be constantly testing and improving our site for increased conversions!

  • Thanks Ron, glad you liked the advice, and hopefully you feel less overwhelmed soon!

  • Pretty helpful.

    This is a good step by step guide on how to improve your website. Identifying your traffic is necessary if it is low or not. The A/B test could help many marketers. A must share techniques. 🙂

    Your post has been shared on Kingged.com, IM social bookmarking site, enabling me to find this good piece.

  • Sara

    Nice article, though I thought a rule of thumb for tests and conversions was to have at least 100 conversions per variable?!

  • Thanks Sara! I’ve heard that mentioned a few times – personally think 100 conversions per test variable is too high, and unachievable unless you have very high levels of traffic. That’s why these techniques are handy 🙂

  • Thanks Metz! Glad you like my guide.

  • You can also split testing with less than 100 conversions. You just need really big wins. If you have a treatment with 20 conversions and another with 40 conversions, a 100% difference is something you can probably bank on, even with such small numbers. However, if one treatment got 20 conversions and the other got 30, that 50% increase is too close to the margin of error and shouldn’t be considered an improvement (even though it feels like a win).

  • Yes thats a great point Brian! Good to have a scientist around here 😉

  • This is great advice, Rich. Thanks for such a thorough discussion.

  • Thanks John – glad you liked it. Hopefully it can help you on some of your lower traffic pages…

  • Shiv Ettes

    Wow, lots of information packed in here. I don’t know much about A/B testing, but now I’m thinking it’s more important than I thought it was. Thanks for the warning about multi-armed bandit testing; definitely doesn’t sound ideal.

  • mony1
  • Bart Waldon

    I am going to have lest than 1,000 visitors to my site per month… Should I still ab test?

  • Excelanto

    Thanks for your valuable posting.I have collect more than information from your website. It is really wonderful blog. please added more than tips. i’m working in Erp in india.Here providing very low price and Quality ERP,Cloud ERP, CMS , responsive webdesign and ERP. you have any more than information kindly make me call this number 044-42127512 or send your mail info@excelanto.com….

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content as you, internet would be a much better
    place. Please keep it up! sutura silhouette

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content as you, the internet would be a much better
    place. Please keep it up! dermatologista florianopolis

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content you, the internet would be a much better
    place. Please keep it up! dermatologista florianopolis

  • Everything 5 Dollars

    Really great post, Thank you for
    sharing This knowledge. Excellently written article, if only all bloggers
    offered the same level of content as you, the internet would be a much better
    place. Please keep it up! Taxi Chur

  • Billy Simmons

    There is a new advertising and promotion engineering available since makes all the alternatives that are other appear to be the laugh they’re. You can start getting targeted prospects observing your offers in
    only a few momemts, also it merely requires a few mouse clicks to get going! Viral Dominator generate viral traffic and promotion instantly – your ads get viewed over 3,000 times per month with only 10 referrals!
    http://viraltrafficdominator.com/

  • bdctg

    Really great post, Thank you for sharing
    This knowledge. Excellently written article, if only all bloggers offered the
    same level of content as you, the internet would much better place. Please
    keep it up! contador florianopolis

  • raymond

    hae rich. im planning on starting a travel blog about africa but im worried i may get a low traffic on my website. please give me an advice on how i can make it a success. thanks

  • Mohammed Saimon

    Hey rich, just an excellent post. thanks for it.

    Regards

    GadgetGhor

  • ruhul amin

    Thank you very much, this is really a large amount of information content-rich articles, I am glad that the author could get out to share with you It is very useful. Try to keep on posting these kind of things. Thanks for sharing. aviation

  • NicoF

    Small remark regarding Google Experiments: yes, by default it runs the multi-armed bandit logic, but you can choose to select ‘rotate evenly’ instead (which is something I ALWAYS do)

  • It’s very interesting point.

  • Nikolai

    Great article Rich! Few thoughts: I would be inclined to choose Adwords over Facebook for a cleaner test: on Adwords one can limit the audience by running “cleaner” traffic from 1 carefully chosen keyword with 1 ad, versus Facebook approach where it can be harder to choose a uniform audience, statistically valid for a small test, simply based on targeting options available.

    Having said that, I think one should run experiments on the same demo (and source of traffic) that the website if targeted to: Google and FB audiences vary a lot and more often than not, are in different browsing modes alltogether.

  • Thanks for the comment and tips Nikolai! I totally agree that its best to experiment using similar audiences, as that can have a big impact on conversion rates.

  • Thanks NicoF – I will have to check that out. I didn’t notice that option last time I used the tool.

  • Abrar Shahriar

    Hi Rich,

    I just go through this article. While reading it, I feel like I am reading a story. This is just great. I have learnt something new from this…
    5000 unique visitors in a week for A/B testing (small business).
    I wish I could have..

    But if conversion gets near to 100%, then number of visitors are not matter.

  • The question is where is the line drawn and how do you calculate this i.e. 100% is good, what about 95, 90? what it’s an 85% improvement

  • Thank you so much for your advice me http://gamesphp.com

  • If you need more thoughts about the topic, please have a look here http://tinyurl.com/zxss8o3

  • While I get the point, the idea that 5,000 uniques to a page is a minimum before you can do any testing may put many people off split testing, when they could actually still achieve meaningful results. It’s more about the length of time.

    For example, using Google Analytics allows up to 90 days per Experiment; split tests don’t have to complete in a week to be useful. Even if an experiment takes a couple months for a result, it’s still worth doing, and may bring in significant extra revenue to be able to then increase traffic and speed up testing in future.

  • Hi Rich,

    Some good advice here, but maybe it would have helped if you got in a bit deeper into the four variables that affect the required sample size for a test: the baseline, the certainty of the result (statistical signifiance) required, the sensitivity of the test (statistical power) and the minimum effect that would be interesting from a business standpoint. Moving any of these 4 gears can drastically change the required sample size, but it is always a trade-off between speed, efficiency and flexibility on one hand and accuracy, certainty, sensitivity and predictive value on the other. Properly cleaning the baseline so it includes only users with the prospect to convert is the only thing which is not a trade-off, but is sometimes overlooked.

    Another way to drastically reduce the number of users it takes to run a test is to use a sequential testing approach. I don’t know if you explored such approaches, but for example using AGILE A/B Testing you’d be able to reach statistically rigorous conclusions 20% to 80% faster than with fixed sample designs. There is a free white paper out, detailing the approach, while a simple tool to apply the method is available here: https://www.analytics-toolkit.com/ab-testing-calculator/ .

    P.S. “Buy extra traffic for a short while to the page you want to A/B test.” seems like a bad advice, since this traffic might very well not be representative of the traffic the tested page(s) would get after the test is over, thus the predictive value of the test may be negatively affected!