Learn a Better Website Testing Strategy and Approach

In my new role working at Adobe as a conversion specialist and while working on my upcoming book, its becoming increasing obvious that many companies lack an effective yet simple approach and strategy for their website testing efforts. Therefore I thought I would do a quick summary post about this to help give your website testing efforts a kick-start.

Therefore, I have come up with a simple and effective 5 step testing approach for you to understand that will help improve your testing results:

1. Create Insights for Testing from Analytics & Feedback Tools. Rather than simply testing random things on your website , or listening to what your HiPPO wants to test, and limiting your chances of good conversion lifts, instead you should ensure that all of your tests are based on insights you have discovered about your website. This will help improve the quality of your tests and increase the chances of you seeing great conversion lifts. These insights should ideally come from findings in your web analytics tool and from feedback you have gathered from your visitors using great tools like UserTesting.com, Loop11.com, UserVoice.com and KissInsights.com.
2. Create a Test Hypothesis. After you have found some kind of insight from your analytics tools and visitor feedback tools, you should then come up with a good test hypothesis based off the insight. This should come in the form of a statement about your website that you then try and prove true or false by testing. This hypothesis would then be used to create your test plans, and after priotization occurs, an actual test. Here are a few examples of test hypothesis:
   a) By adding in website security logos on the shopping cart checkout page, this will reduce visitor purchase anxiety, resulting in higher order completion rate.
   b) By reducing the number of choices on the homepage, you will reduce confusion for what visitors should click on, therefore reducing bounce rate.
3. Create, Implement and Run Test. Next you need to actually create the test and implement it on your website and make sure it works correctly. While creating the test in your website testing tool, in particular you should know which success metrics you will use to help you measure success, and you should also set up visitor segments for your test for you to later analyze. Once you have implemented the test on your website (ideally with the help of a web developer) you should then QA the test to make sure its not breaking anything on your website, and check that the results are coming in to your tool as expected.
4. Analyze Test Results & Launch Winners. This next part is often where sub-par test results get pushed live without proper analysis. When analyzing your results you should in particular make sure you gather enough data (at least 2 weeks of results, and at least 100 conversions for each version being tested) and also make sure the predicted winner has enough statistical confidence from the tool to declare it a winner (at least 90% confidence). If you don’t do this you will risk pushing a version live that isn’t actually the best version, thus reducing your potential converison uplift. You should also not just push one winner live to all your visitors; instead you should segment your test results to find higher conversion lifts for different visitor segments (for example your paid search or first time visitors may convert higher to a particular test version) and push winning versions to each of these visitor segments.
5. Run Test Iterations to Maximize Conversion Lift Impact. This part is sadly often neglected by many online marketers who are testing and optimizing their website. Rather than simply moving on and testing something different, you should also iterate on your last test by learning from the results (even if the test didn’t work well you should be able to draw conclusions to learn from). Doing this test iteration usually results in much higher conversion lifts than your initial tests generate. Many companies don’t do a good job of this so it will be a great source of competitive advantage if you do it well. Here are a few examples of test iterations:
   a) After testing and optimizing your registration flow, test your key entry pages leading to your first registration page.
   b) After testing the color and size of your call to action buttons, you should then test the wording on them too.

This simple approach really can be a game changer for your testing and optimization efforts, particularly if you currently only do random testing or adhoc testing. So before you run your next website test you should try this approach instead, and you should hopefully be pleasantly surprised by the improved results.

Lastly, this topic of testing and optimization strategy is going to be a key part of my upcoming book, where you will learn more detail about this approach and many other website optimization best practices. This book should hopefully be out by spring 2012!