Google Website Optimizer Sucks?

Last updated By | 8 Comments

If you are regular reader of my blog (and hopefully you are), you will have probably noticed by now that I am a huge fan of testing to build websites that truly meet visitors needs. And one of the free tools you can use to test ideas to help improve your website is Google Website Optimizer (many more tools to help are in my new free website optimization ebook by the way). And for the most part, this is an amazing tool. But, I have recently discovering more and more things that well, kind of suck about it. So just what sucks about it? Read on to find out…

1: Introduce non-traditional based conversion events.  Google Website Optimizer is great for testing websites that have easily definable goals, like newsletter signups, ebook downloads, or product purchases. This is because you can easily tag the thanks page with the conversion code. But what about for other types of websites that don’t have traditional goals, like media websites? Currently the only non page-based event you can use is time spent on page as a conversion, but its not easily done in the tool. What about tracking conversions on link clicks? And what about tracking conversions on ajax related dynamic page elements that don’t necessarily result in a new page being created after clicking on it (so can’t add the conversion code to it). Surely this is going to be even more problematic in the future given the amount of websites now using dynamic elements on websites that don’t refresh or change the page.

2: Adding multiple conversion success pages for one experiment. Currenly you can only add one conversion page per test. Adding this would enable you to do one test that includes several links all on the same page that go to many different pages. And by this, I mean it would be great to see if changing one thing on the page influenced the clickthrough rate of the page. For example, testing the usage of underlined links, or changing the color of links or headers.

3: Introduce pageviews per visit as a conversion event. This is really a continuation of my first point. Many media/content rich websites don’t have regular goals – often the goal is simply to increase page views per visit (for ad revenue). Allowing for a way to test which versions contribute to the most pageviews per visit would be an excellent thing to be able to report on, and is also a great indicator of how engaged a visitor is.

4: Allow segmentation of visitors who convert. This is more advanced functionality, like Test and Target offers. I know its probably a tough one to integrate, but it would be VERY useful to gather real insight into what happens before and after visitors have converted. For example, segmenting for referral sources (someone might convert differently if they come from paid search in comparison to organic search) and for what the visitor then usually does  after converting (much like active segments would reveal). For example, one test may win, but may result in high bounce rates after they convert. Without this segmentation, there is no way of knowing what happens after the conversion.

5: Allow email notifications when test is successfully completed. Come on Google, surely this is surely a fairly easy one to implement. There is no way of knowing when a test is competed, and its pretty annoying to have to keep logging in and checking everyday. And this could take days if you are testing a low traffic website or pages!

6: Integrate Google Website Optimizer with Google Analytics. Wouldn’t it be great to see test results in Google Analytics and vice versa? Surely this should be fairly easy to do in practice and would allow much greater analysis and insight for tests and the impact they have. There are probably advanced hacks out there that let you do this, but why not offer this, much like Google Adwords integrates with Google Analytics?

So there you have it. Hopefully Google is listening? Does anyone else have pet peeves with Google Website Optimizer? How about Brian Eisenberg in particular? (Brian just wrote a great book all about Google Website Optimizer by the way). Post your responses below, and who knows, maybe they might get implemented sooner rather than later!

Also, a quick shout out to the many many cool fellow bloggers that I met at the Blogworld Expo last weekend in Vegas! You know who you are :)

Like this article? Please share it using these icons below... thanks!

FREE CONVERSION TOOLBOX TO BOOST YOUR WEBSITE SALES
Get free access to my 7 expert guides and discounts to boost your website sales or leads
Also includes blog updates and we won't spam you!
  • http://www.ecommerceblog.nl Bertjan

    Clear message! Google Weboptimizer is great, but these suggested improvements would bring testing, without spending extraordinary amounts of money, to a higher level.

  • Gustav

    actually, i’ve done onclick conversion events using WSO before. i learned about it on the user forum. i also have it integrated with google analytics using the roi revolution method.

    i absolutely agree on the email notification idea.

  • Jeff

    Good points here. Tagging events like Ajax and Flash is needed to really get a clear picture what is happening.

  • http://www.google.com Tom Leung

    Hi Rich:

    Thanks for the feedback! I can’t say I loved the title but it’s great to hear the kind of insight in the post itself

    While I agree we have lots of opportunities to make GWO even better, you may be interested in checking out some of the info here which talks about some of the current non-traditional goal support. http://www.google.com/support/websiteoptimizer/bin/topic.py?topic=14314

    Also, if you’re ever interested in participating in a user research study for GWO (which also earns you some AdWords credits), feel free to sign up here. https://spreadsheets.google.com/viewform?key=pAGNlrH_Fajh6oSNWEQcDWA

    In terms of staying on top of our latest improvements, whenever we ship new features (like our recently launched pruning feature) or have interesting announcements about testing, we blog about it at websiteoptimizer.blogspot.com

    Lastly, if you have other feedback or would like to talk more about your experience with GWO some time, email me and we can set up some time to chat on the phone.

    Thanks again for the great ideas!

    Tom Leung,
    Product Manager,
    Google Website Optimizer

  • http://testingblog.widemile.com Billy Shih

    Hi Rich,

    I’m curious when you said that it “take days if you are testing a low traffic website or pages!” Days is a long period of time for you? With Google Website Optimizer all but the smallest of tests will take a very long time to run since GWO uses full factorial test design requiring hundreds or thousands of conversions to get a reasonable confidence intervals that point to a statistically sound winning experiment.

    Even using Widemile’s platform, which uses fractional factorial, I typically allow my multivariate tests to run 2 weeks to counter any anomalies in traffic and include both weekdays and weekends. At the very least, I would strongly recommend against running a multivariate test less than a week even if you get a lot of traffic. A/B split tests, on the other hand, can be run for a few days if there is enough traffic and the difference in conversion rates is large.

    I talk about full factorial versus fractional factorial testing in more depth here:
    http://testingblog.widemile.com/2008/07/24/primer-full-and-fractional-factorial-test-design/

    Having said that, you are listing some great problems with GWO. Not to say that GWO is a bad product at all, more so that the maturity level of online marketers with testing and optimization is rising. I hope to hear from more marketers like you so that the testing industry can continue to grow and learn from its users.

    -Billy Shih
    Optimization Analyst, Widemile

  • http://www.rich-page.com Rich Page

    Billy, I couldn’t agree more with you. When I meant days, I meant that in the sarcastic english sense (i’m from england originally). You are very right – it can take weeks and weeks, sometimes months to get results back on a test that has low traffic. In fact I set up a test over a month ago on a low trafficked page, and there is still not enough data to give an accurate test result.

    Also, I very much like your blog, and will spend some time reading it. Great to hear from a fellow website optimizer!

  • http://blog.avangate.com Claudiu Murariu

    Totally agree with you. Website Optimizer Sucks. it runs great with “normal” websites, but add a couple of sub domains, secure conversion pages and you work the hell out of it to make it work. We’ve quit on it and designed our own ab testing platform.

  • Shane

    I completely agree with you on #1 – we were having a hard time so we contacted google and received quite a few tips from them on how to trigger the conversion scripts without a conversion page. We summarized what we discovered in our blog – http://www.conversionvoodoo.com/blog/2010/05/how-to-track-conversions-without-a-conversion-page-in-google-optimizer/