May 24th, 2012 by Rich Page
“Website testing is easy”. I hear that all the time. Unfortunately it isn’t. There are many ways to slip up and make common mistakes when testing. To effectively test and optimize your website so that it better engages and converts your visitors, that are many pitfalls that you need avoid, and many online businesses are making these, often without knowing it.
Based on my experiences and working with clients over the years, I have put together a list of common mistakes that you need to avoid when you are running tests to optimize your website and its conversion rates. Many of these are mentioned in my new book in greater detail, if you haven’t picked up yourself a copy yet (and it’s on sale right now too!)
1: You test random things, with no insight or hypothesis used
If you test random things based with no logic behind them, this is a mistake because your test results won’t be very focused and will rarely yield very high conversion lifts. Instead, you need to make sure you are creating a hypothesis and insight for each test idea – for example, using web analytics or visitor feedback tools to come up with test ideas to improve your website. This is a key first step to make sure you do for any of your tests.
2: You usually end up testing what your HiPPO wants you to test
This next one is all too common – letting your HiPPO (highest paid person’s opinion) decide on what to test improving on your website. This is not good because often they have preconceived notions of what to test and what is best for your visitors, and are quite often wrong. And their test ideas will very often not end up with very good results. Worst still, they often squash test results that they don’t agree with or like. Influence your HiPPOs by educating them, and use hypothesis and insight for your test ideas instead, as just discussed.
3: You use a web analyst to test instead of a dedicated test resource
Testing is a full time job! While using a web analyst to do your testing is okay while starting out on your website optimization journey, you really need a dedicated person (eventually a team) to create, run and analyze your tests. If you don’t have this dedicated resource you will quickly find that you your web analyst is too busy to set up a very many tests per month, and will end up rushing the ones they do. This means you won’t be able to increase your websites conversion rate very quickly, and risk making other mistakes when rushing.
4: You don’t iterate and do follow up testing on your test results
This mistake is where once you have finished testing something on your website, you move on to test something different without learning about what you just did. This is a mistake because you are missing out on improving your conversion rates even further for the elements you just tested. For example, don’t just test the color or wording of a call-to-action button, test the location of it. You should always look to get more juice out of the test and do follow up iteration testing – it’s a critical part of an effective long term optimization strategy.
5: You presume you know what your visitors are doing on your site
This can be a fatal flaw when you are testing your website – thinking you already know what your visitors are clicking on and doing on your website, because you may end up testing page elements that your visitors aren’t using much or don’t care about. The truth is, visitors quite often do things differently than you might think. This is why its critical to make use of good web analytics data to learn what they are doing and come up with test ideas, including visually analyze what they are doing on your website.
6: Your test variations aren’t sufficiently different
When you are creating different variations to test on your website, you need to make sure they look different enough for your visitors. If you are creating ones that are barely any different, with very tiny changes (particularly on banners or images) you are going to be limiting your converison rates because your visitors won’t notice the difference, and won’t convert any differently. T get better results, go for a few bold changes for each test (even call-to-action buttons and headlines) – ones that your visitors will better notice (and educate your design team on importance of doing this!)
7: You create crazy complicated MVTs
You may think its ‘cool’ to test many things per once per page, with many different variations per thing being tested. The truth is, this isn’t a very good effective use of an MVT, as it takes a long time to create these, and you may end up testing elements that have no impact on conversion, often resulting in pretty average conversion lifts. Instead, go for creating a simpler conversion influence MVT, where you create only one variation for each element to find which ones have an impact on conversion, then do follow up A/B tests on those to increase your conversion rates. Much more focused and requires much less traffic to get results!
8: You decide on a winner or end a test to soon
Sometimes you may be so happy with a test result, you may launch the winner too soon, or rush and not want to wait very long for test results – this is problematic when it turns out to be not the best performing version, as you would be missing out on higher conversion rates from the actual winning version. Before you can feel confident that you have found the right winner, you need to have enough statistical confidence for the result (usually at least 80%) and you also need to wait for result fluctations in your test winners to die down.
9: You have preconceived notions of what tests are going to win
This one I hear of all time, and is a frustrating mistake that is easily avoidable. No matter how much of an expert you are, don’t presume you know what test versions are going to win, or only test things that you think matter – often test results can be the opposite of what you expect, and even opposite of what works on another website or common web conventions. Test everything, and trust the test results – don’t discard the test because you don’t agree with the result, or launch a version as soon as it starts winning just because you want it to win.
10: You are not targeting visitor groups for your tests
This last one is big way to kill high conversion rates, and is one that many online businesses slip up with. Segmenting and targeting your content to different visitor groups has amazing results. This is because it makes your website much more engaging and personalized for your visitors, and as a result converts them at a much higher rate. For example, to better engage and convert them, you should be targeting new visitors with different messaging (welcome guides, coupons etc) than what your repeat visitors see. This is also a key reason to pick a better testing tool than Google Website Optimizer, because you can’t do any segmenting and targeting in this tool. Get targeting!
So there we have it. Don’t feel bad if you are making some of these testing and optimization mistakes – you aren’t alone! You can read all about how to fix these in my new book. And feel free to post your most common test frustrations that limit your conversion rates – let’s discuss!