A/B Testing Tools: VWO Compared to Google Optimize and Optimizely

Last updated By | 28 Comments

visual website optimizer screenshot
Using an A/B testing tool like VWO is one of the easiest ways to discover which variations of your website increase your website sales and leads the most.

To help you understand the different tool options available to you, I have created a guide that compares three leading tool choices: VWO, Optimizely, and the newly launched Google Optimize.

All of them have their advantages and disadvantages, so I have created a grid comparing VWO (Visual Website Optimizer) with its major rivals, including pricing and functionality. Below the table I’ve also included an overview along with the pros and cons for each of the tools.

Let’s get started with the ratings:

Google Optimize VWO logo Optimizely
Cost rating
10/10 Free (with premium 360 version also available). 9/10 Plans start at $49 per month for ‘startup’ level (10,000 visitors). 5/10 Costly custom pricing plans, aimed at the enterprise market.
Test creation options and ease of use
8/10 Good visual editing options but only using Chrome browser plugin. 9/10 Great easy to use visual editing options, plus CSS and code editing options. 9/10 The best visual editing options, plus CSS and code editing options.
Ease to add test code 9/10 Very easy. Just one extra line of Google Analytics code needed. 9/10 Very simple – only one tag needed. 9/10 Very simple – only one tag needed.
Testing types available
8/10 A/B tests, MVT and redirect tests included. Multi-page and mobile tests only in premium version. 8/10 A/B tests, MVT and redirect tests included. Multi-page and mobile tests only in premium version. 9/10 A/B tests, MVT redirect, multi-page and mobile tests included in all plans.
Targeting ability for tests (new visitors etc)
8/10 Google Analytics segments used for targeting, and very simple to use. 8/10 Good targeting behavioral options for all levels, with basic ability to build custom target segments. 9/10 Great targeting options in the standard level, with advanced targeting at higher account levels.
Conversion goals & success metrics 7/10 Google Analytics goals are used only, no customized goals. 9/10 Great – good selection of goals and metrics, including revenue. 9/10 Great – good selection of goals and metrics, including revenue.
Test reporting
6/10 Fairly good reporting but no dashboard overview. 8/10 Great reporting options with a useful dashboard overview. 9/10 Excellent results reporting and dashboard overview.
Customer service
1/10 No support offered. Only included in premium version. 8/10 Email support included in basic plan, 24/7 phone support in enterprise plan. 8/10 Good support via online ticket system. Client onboarding included for all plans.
Overall rating 7/10 Good for beginners but lacks advanced features and support options. 9/10 Highly recommended for all types of users, with reasonable costs. 8/10 Better features for A/B testing pros – although much more expensive.
Free trial? Try Google Optimize – free Get a 30-day free trial of VWO Get a 30-day free trial of Optimizely

 

Now let’s get more detailed, with an overview of each of these tools, including their pros and cons.

Google Optimize overview

Google Optimize was only just launched in 2017 and is a major improvement over their previous A/B testing tool which was built into Google Analytics (called Google Content Experiments). It has limitations but is improving. Here are the notable pros and cons of the tool:

Pros:

  • Very good tool for free, much better than Google Content Experiments.
  • Excellent integration with Google Analytics makes A/B testing very easy.
  • Easy to analyze reports in greater depth using Google Analytics.
  • Can be upgraded to their premium 360 version to unlock more features.

Cons:

  • Only 5 tests can be run at the same time (unlimited when upgrading to 360 version).
  • Multi-page and mobile testing not included (only in 360 version).
  • No support included, only through forums or paid 3rd party consultants or agencies.

Rating on G2Crowd: 4.4/5 (as of September 2018)

Try it for yourself: Sign up to Google Optimize for free

VWO overview

There are now two versions of VWO. One is their classic A/B testing tool, and the other is their new premium CRO tool which offers A/B testing plus other essential tools like visitor recordings, survey tools and form analysis. Here are the pros and cons of the classic A/B testing version:

Pros:

  • Includes other A/B testing related tools like visitor click heat maps.
  • Intuitive new user interface with great A/B test design wizard to help you.
  • Excellent ‘idea factory’ helps you create better test ideas (by type/element etc).
  • Can be upgraded to an excellent set of CRO tools like visitor recordings and surveys.

Cons:

  • No built-in multi-page testing functionality (across navigation/check-out etc).
  • Sometimes it’s hard to create tests across pages with complicated URLs.
  • Offer fewer real-time support options in comparison to Optimizely.

Rating on G2Crowd: 4.2/5 (as of September 2018)

Try the tool for yourself: Get a free 30 day trial of VWO

Optimizely overview

Optimizely has evolved from being one of the most popular tools a few years ago to recently becoming one that focuses on enterprise sized businesses (while removing their free starter level plan). Here are the notable pros and cons of the tool:

Pros:

  • Has the best A/B testing options for experts and pros.
  • Competes well with Adobe Target and other enterprise level A/B testing tools.
  • New Optimizely X features ability to optimize mobile apps and other platforms.
  • Offers extensive community and academy for getting the most from the tool.

Cons:

  • Very expensive, which limits this to mainly companies with big budgets.
  • Sometimes it is difficult to preview A/B test variations easily and quickly.

Rating on G2Crowd: 4.4/5 (as of September 2018)

Try it for yourself: Get a free 30 day trial of Optimizely

Which is better? VWO, Google Optimize, or Optimizely? 

To help you make up your mind, here are my expert two-cents about these tools:

  • Google Optimize is the best choice for beginners or those with no budget for A/B testing yet, but has limitations.
  • VWO is a great tool for those looking for more A/B testing features, with affordable plans to suit different budgets.
  • Optimizely is best for advanced users with excellent features, but high costs limit this to companies with very big budgets.

Ultimately I suggest you sign up for Google Optimize to see if that meets your needs, and also try a free trial of VWO to see the benefits of their more advanced A/B testing features. If your company can afford Optimizely and you want expert features, then use that tool instead.

Extra reading: If you want to get much better results from using these A/B testing tools, then don’t forget to read my essential user guide for A/B testing tool success!

Other A/B testing tools worth considering

I have reviewed 3 of the most popular A/B testing tools, but there are certainly some other newer tools. Here are some of the best newer ones for you to also consider, each with their own strengths:

  • Omniconvert – a multi-purpose tool that also offers personalization, surveys, and website overlays.
  • Convert – growing fast and in particular has great data privacy and excellent support.
  • A/B Tasty – a good tool that also includes useful options for visitor recordings and heatmaps.

There are also other good tools purely for the enterprise market like Adobe Target, Qubit, and Monetate.

Wrapping up

That’s my expert two cents. Now over to you the readers – what is your favorite A/B testing tool, and why? Please comment below!

Found this article useful? Please share it on social media. Thanks!

  • Why would Google make a worse product than they previously had available to users?  In other words, why would they discontinue Website Optimizer and roll it into Google Analytics with fewer features making it less useful?  Can someone please address this?

  • Thanks for the comparision – I agree in your thoughts about VWO and Optimizely but haven’t checked out Content Experiments yet. 

    But do you have any insight into how the different testing tools affect load speed? A few times we’ve experienced troubles in this area with Optimizely.   

  • Good post Rich. 

    Also recommend this resource to people:
    http://www.whichmvt.com/

    It’s a site created by the team at Conversion Rate Experts which covers pretty much every testing tool there is!

  • Good post Rich. 

    Also recommend this resource to people:
    http://www.whichmvt.com/

    It’s a site created by the team at Conversion Rate Experts which covers pretty much every testing tool there is!

  • Good question! First of all, it wasn’t making them any money directly, and didn’t have good ties with Adwords like Analytics does to make it worthwhile. In terms of lack of features, I know this is just the first release – they just wanted to get the concept right, and will add key features soon, hopefully MVT. Keep your eyes peeled…

  • I have to say that I love Optimizely. It’s just so easy to use, and they are really helpful when you’re not good with code. (I should start charging them for all my recommendations.)

  • Hi Rich, 

    You missed ZenTester. Super easy to use, has most of the features of the others and can be used for free too (definitely low cost ;)). 

    Regards, Tim

  • Pingback: Web Analytics Tools Not Named Google | Teach to Fish Digital()

  • Hedvig

    thanks Blair for sharing this extended test, helped me to decide

  • Mark Hall

    Thanks for the comparo, Rich – nice quick synopsis for a newer CRO guy like me. Now I know which tools to explore in more detail.

  • martijn

    Hi, my name is Martijn and I work at Optimizely. Check out this article: http://blog.optimizely.com/2013/12/11/why-cdn-balancing/

  • Rich, this comparison is not accurate in many aspects. Optimizely built-in segmentation rules. Its targeting options are phenomenal. Optimizely has multi-page experiments that VWO doesn’t. You don’t mention support – Optimizely has phone support in the US and Europe, cover a wide range of timezones. VWO support is only India time, over email.

    And – try comparing VWO heatmaps to any dedicated heatmap tool such as Clicktale, Crazyegg etc – numbers don’t match, at all.

  • Hi Peep – thanks for your comments. At the time of initially writing this over a year ago, the targeting aspects of Optimizely weren’t very apparent or as good as they are now. I will certainly be investigating their targeting options in more depth, and clarifying that in the matrix.

    In terms of support, yes, that is very valid (even though some testers rarely use support depending on what they are testing). I will add a support section on the matrix to help readers understand those differences.

    And totally agree with your point about VWOs built-in heatmaps and visitors feedback tools, they certainly don’t compare to using more robust specific tools like you mentioned.

    I am going to be adding more sections to this comparison very soon to make it even more comprehensive and up to date. Please feel free to email me or reply with other feedback about these tools. Thanks!

  • Hi Peep,

    Thanks for your comment. Regarding support, you’re not accurate. We’re giving 24×7 support and our support is phenomenal. I suggest you check out tweets and testimonials such as: http://screencast.com/t/X9zqXIs7nbB and https://twitter.com/dalbke/status/355693689902862337

    Just because we’re based out of India doesn’t mean we don’t provide support. Moreover, we do provide phone support if an issue isn’t able to get resolved over email.

    Moreover, VWO has 15 segmentation and targeting options including geo, referral, browser and many other options. Here’s the full list: http://visualwebsiteoptimizer.com/knowledge/what-are-different-types-of-segment-options-available/

    Yes, VWO heatmap options doesn’t have all the bells and whistles that are in dedicated tools but the tight integration of heatmaps with A/B tests more than compensates for it. Moreover, VWO has options such as “consolidated heatmaps” that even dedicated tools don’t have. Consolidated heatmaps allows you to generate heatmap by aggregating clicks over multiple pages that share same layout. It’s very useful for eCommerce sites that have same layout for product pages.

  • What I meant by segmentation was analyzing test results across segments. How did the test perform for one segment or another.

    I had poor customer experience with you myself, what can be more accurate than first-hand experience? Especially with very slow response times. I complained several times and I only heard back defensive rhetoric (“other people like our support!”) like now. You can’t invalidate my own personal experience saying “no our support was good”. It was not.

    I’m glad that your support is now 24/7, that will be of great help.

  • Yep, agreed, you can’t invalidate personal experience. You did have a poor experience but I recommend you to try once again. We’ve been constantly improving and support is a big priority for us.

  • I think you’re wrong, Google content experiment allows multivariate test. It’s just not accessible in analytics, you have to add code on the page.

  • Wesley van der Hoop

    Sadly, I have the same experience. Of the three e-mail conversations I had, two were not supportive at all. Long response time, the answers not solution oriented etc.

    I asked them not to bill me, they did it anyway (thank god the transfer was blocked). Then I reminded them I did not want to get billed. No response. Two days later they send me a reminder that my bill is still open!

    I honestly don’t know why I am still using VWO.

  • Taruna

    Hi there,
    I am Taruna, marketer at Visual Website Optimizer. I am really sorry about the bad experience you had with our support and billing. Please let me know your VWO Account Id at taruna.manchanda@wingify.com so that I can get this sorted at the earliest.

  • Darren DeMatas

    Good stuff, Rich! Thanks for updating the comparison chart.

  • vladmalik

    I would say the opposite. We’ve used both and we found Optimizely to be the more basic tool. VWO is more customizable, reports are more detailed, and you can download all data. We’ve found Optimizely a bit limiting.

  • Interesting to hear your thoughts – VWO has certainly improved a lot recently.

  • No problem Darren! Glad you liked it.

  • Yes, unfortunately they don’t make it obvious to the average user though – and the reporting doesn’t work well for it.

  • Pete Storey

    From a pretty low level of usage/investigation of either, I’d say Optimizely is slightly more user friendly (just preferred the interface), but for us at 100k visitors a month, it’s TEN TIMES as expensive as VWO. Yikes. It’s not that good.

  • Paul Peltokangas

    While VWO and Optimizely are both good products there is a fundamental restriction to what can be optimised on your website. Once you have chosen your Java Script Main Frame both with have to adhere to the logic with in it. If you want to go pro and start modifying your website without any limitations try out Frosmo!

  • MaximusPhotography

    This is a fake compare. It’s been used to promote optemizely. Crap

  • Mccarthy Carlos

    WELCOME TO THE GREAT ?ILLUMINATI CHURCH, Are you a business man ,politicians,musician,musician,student and you want to be very rich in life,powerful and be famous in the world join the Illuminati online today and get instant sum of 5 Million Dollars with a free home any where you choose to live in the world and also get 200,000 Dollars monthly as a salary ,join us today to become one of our official member today , you can achieve your dreams by been a member of the great Illuminati ,no sacrifice, or human life needed, Illuminati brotherhood brings along wealth and famous in life, you have a full access to eradicate poverty away from your life now. it only a member who is been initiated into the church of Illuminati have the authority to bring any member to the church, so before you contact any body you must be link by who is already a member, Join us today and realize your dreams. we also help out our member in protection of drugs pushing, once you become a member you will be rich and famous for the rest of your life, with this is your dream and heart desire can be accomplish,if you really want to be a member of the great Illuminati ,then you can contact us now on out direct Email : illuminatiworldchurch@hotmail.com

    Email: illuminatiworldchurch@hotmail.com
    Telephone Number call or text :+2348070940793

    Thank you.