A/B Testing Basics

If you’ve spent much time in the online media world, you’re probably familiar with A/B testing. If the term is new to you, it’s pretty simple – you show different versions of a web page to site visitors and measure which one is better at getting visitors to convert. Your definition of “conversion” could be having the person register on your site or sign up for your newsletter or invite a friend or purchase a product. In any case, you have a specific goal and want to find the most cost effective way to achieve it … you should try A/B testing.

There are a number of commercial tools for A/B testing available ranging from free to quite expensive. Generally, the free tools, like Google Website Optimizer, require you to create your own page variations and will require some amount of developer involvement, while the commercial tools provide more user-friendly options for creating different versions of your landing page. If you have access to a developer or even moderately technical person, I recommend using Google Website Optimizer – it’s free, flexible and easy to set up.

Define Your Conversion Path

The purpose of A/B testing is to find the best way to get site visitors to do something that is valuable to your business. We’re going to drive site traffic to a specific page, usually a specially designed landing page, where we want the visitor to perform some action. If they complete that action, they’ll be taken to another page on the site, we call that the conversion page. For example, if we bring visitors to a landing page and want them to purchase a product, the conversion page would be the “order confirmation” page that they see once their purchase is complete.

Create Landing Page Variations to Test

Once you’ve defined what you want to test, you need to create at least 2 variations of the first page that people see when they enter your site. Instead of spending hours or days agonizing over what you think might be the most powerful copy or images to connect with visitors, create a few different versions and test and measure!  The most common page components to test are:

  • Page headline
  • Placement and wording of the call to action message
  • Placement and size of images
  • Page copy

Using Google Website Optimizer

  1. google website optimizer ab testingCreate a new A/B experiment in Google Website Optimizer and give it a name.
  2. Enter the URLs for each different landing page variation you want to test (at least 2, no more than 127)
  3. Enter the URL for the conversion page for this experiment.
  4. Google will generate some tags that you need to add to each of your landing pages and the conversion page – this is where a technical person comes in handy.
  5. Once those tags have been added to your pages, proceed to the next step and Google will attempt to validate that it sees all of the tags.  Once they’ve all been validated, click the button to run your experiment!

Running the Test

In order to get meaningful results in a reasonable amount of time, you need to drive traffic to these pages, generally with a campaign of some sort.  Use the “original page URL” in all of your campaigns – when someone visits that URL, Google takes over and will randomly send the visitor to one of your test pages. Over time, you’ll see the conversion rate of each page variant and the probability that any one of the variations will perform better than the original. If no clear winner is emerging, it could be a sign that there isn’t a big enough difference between the variations you’re testing.  You can pause or stop a test at any time.

ab testing with google website optimizer

Multivariate Testing

Another, slightly more complicated version of scenario testing is called Multivariate Testing. Here, instead of creating multiple versions of the page you want to test, you create one page with specially marked sections. This way, if you want to test 3 different page headings, 2 different images and 3 different sets of copy, you don’t have to create individual pages to cover each possible combination – you let Google Website Optimizer do the work. Identify the different sections you want to test, create content variations and add Google script tags (the wizard will walk you through the steps). Once your multivariate experiment is active, Google will generate random combinations of the sections you’re testing and present them to page visitors.

Create additional follow-up experiments and continue to improve the conversion rate of your landing pages.