Using an A/B testing tool is one of the quickest and easiest ways to boost your website sales and leads (without having to buy more traffic!)
To help you understand the different tool options available to you, I have created a guide that compares the three leading tool choices: Visual Website Optimizer, Optimizely, and Google Content Experiments (previously called Google Website Optimizer).
All of them have their advantages and disadvantages (particularly Google Content Experiments), so I have created a grid comparing Visual Website Optimizer (VWO) with its major rivals, including pricing and functionality. I’ve included some ratings for each feature, and a total score too.
Let’s get started with the ratings (which I officially updated May 20th 2015):
|Google Content Experiments|
|Cost||Free.||Plans start at $9 per month for ‘personal’ level (2,000 visitors).||Free for starter level (50,000 visitors but one project only).|
|Test creation options||5/10 Basic wizard setup, no visual or code editor.||9/10 Advanced test creation options, accessible for all levels.||8/10 Advanced creation, but multi-page and multi-channel test only for enterprise.|
|Ease to add test code||7/10 Simple, but several page tags needed.||9/10 Very simple – only single code line needed.||9/10 Very simple – only single code line needed.|
|Multivariate test options||0/10 None.||8/10 Many options available at all levels.||9/10 Available for both free and enterprise level.|
|Targeting ability for tests (new visitors etc)
||0/10 None.||8/10 Good targeting behavioral options for all levels, with basic ability to build custom ones.||8/10 Advanced behavioral targeting options, but only standard ones in free level.|
|Conversion goals & success metrics||9/10 Very good – can use Google Analytics goals as conversion goals.||8/10 Great – good selection of goals and metrics, including revenue.||8/10 Great – good selection of goals and metrics, including revenue.|
|Test reporting and segmenting||7/10 Average reporting, but ability to analyze Google Analytics segments.||8/10 Newly improved. Better reporting and added ability to segment results.||7/10 Basic reporting with free level, advanced reports included in enterprise level.|
|Overall comments||Promising, but step back from Google Website Optimizer (new features coming soon).||Great tool & includes good usability basic tool to gain test ideas from your visitors & testing experts.||Definitely more of the ‘expert’ tool, but is more expensive for the premium features.|
|Overall rating||5/10 Very lacking, needs key features.||9/10 Has slight edge over Optimizely for beginner A/B testers.||9/10 Better features for testing pros – although it charges more for them.|
|Free trial?||Try Content Experiments – free||Get a 30-day free trial of VWO||Try the free version of Optimizely|
As you can see, Google Content Experiments is seriously lacking in functionality, so I’m now going to focus on the differences between Visual Website Optimizer and Optimizely.
Visual Website Optimizer Overview
In May 2014 they performed a major overhaul on their interface and launched new features like test report segmentation and support for app testing (and check out this great guide on the improvements). Here are some pros and cons that should be noted:
- Includes other testing related basic tools like heat maps and visitor surveying.
- Intuitive new user interface with great test design wizard to help you.
- Excellent ‘idea factory’ helps you create better test ideas (by type/element etc).
- All features included in monthly cost – no adding fees for greater functionality.
- No built-in multi-page testing functionality (across navigation/check-out etc).
- Offer fewer real-time support options in comparison to Optimizely.
- Has no free limited version, unlike Optimizely’s recent change in pricing.
Try the tool for yourself: Get a free 30 day trial of Visual Website Optimizer
Optimizely has also grown leaps and bounds recently as an outstanding A/B testing tool. Here are the notable pros and cons of the tool:
- More of the ‘premium’ tool in comparison to Visual Website Optimizer.
- Competes well with Adobe Target and other enterprise level testing tools.
- Offers more options for running behavioral targeting tests (enteprise level only).
- Better integrated options for experts, including options for multi-page testing.
- Offers extensive community and academy for getting the most from the tool.
- Has the cheaper basic service level, but charges more for functionality like MVT.
- The free version comes with little support – just through their community and FAQ.
- Set-up test wizard a bit too simple – doesn’t highlight key test functionality well.
Try it for yourself though: Get a free 30 day trial of Optimizely
Which is better? Visual Website Optimizer or Optimizely?
Really it’s kind of like a Pepsi Vs Coke thing – many experts will only use their favorite,with many casual users not really noting the key differences.
To help you make up your mind, here are my expert two-cents about both tools:
- Visual Website is better for beginner and casual testers, with extra built-in (but basic) tools like heatmaps and visitor surveying.
- Optimizely offers better options for advanced testers, but charges a premium for features like MVT testing. Also offers a free basic level.
Overall, Visual Website Optimizer has the slight edge for casual A/B testers, with extras like heatmaps, but its more expensive than the free version of Optimizely. For serious A/B testers, I definitely recommend the Optimizely enterprise level – more comprehensive offering, even though you have to pay much more for it.
Ultimately I suggest you try a free trial for both of the tools and see which one you are most comfortable using and meets your needs better.
Extra reading: If you want to get better results from using either of these testing tools, don’t forget to read my essential user guide for testing tool success!
Bottom Line for Google Content Experiments Users
It’s time to start using a better testing tool! Even though it was a great idea by Google to integrate it into Google Analytics, it’s unfortunately taken one step forward but two steps back. While it offers improved conversion goal reporting, you can’t even do multivariate testing in it yet.
So unless they greatly beef it up soon (which there is already talk of), the free version of Optimizely is much better if you are looking to do A/B testing with no cost initially.
That’s my expert two cents. Now over to you the readers – what is your favorite A/B testing tool, and why? Please comment below!