The Simple Guide to Split Testing

At first, the terminology used to describe split testing can be confusing.  A/B split testing?  Multivariate analysis?  But don’t worry – you don’t need a PhD in Statistics to understand how to use this powerful tool to improve your website!  Instead, follow the instructions below to start making data-driven decisions about improvements to your site.

But first of all, what is split testing?  Put simply, split testing is the process of serving two or more versions of a page to visitors at random and tracking which version results in the most conversions.  Conducting split testing over time can help you to make your site more effective, leading to more sales, more leads and more conversions.

Let’s look at an example.  Say you have a simple long-form sales letter as a website.  It’s a single page full of text, and you’d like to be sure that the content you’ve written and the design elements you’ve chosen are resulting in the maximum possible number of sales.  Instead of just assuming that your site is perfect (a major trap that many site owners fall into), you decide to implement a split testing program.

To get started, you create two versions of your sales letter, changing one or more variables on each page.  Because you’re just getting started, you choose the simpler A/B split testing process, which tests a single variable at a time (compared to the more advanced multivariate testing, which allows you to compare multiple changes at once).  You create two different versions of your page headline, add the two pages to a split testing program and sit back and wait for data to be collected that will show you if one headline is more effective than the other.

Let’s look at each of these steps more detail…

Step #1 – Choose Between A/B Split Testing and Multivariate Testing

The first step in starting a split testing program is to decide between A/B split testing and multivariate testing.  As mentioned previously, A/B split testing compares changes in a single site variable (like a headline, image, call to action or page color).  It’s a great place for beginners to start, as it will be easier to understand exactly which changes lead to the biggest improvements in conversions, which can help inform future changes or split tests.

The downside to A/B split testing is that it can be time-consuming.  There are an endless number of variables on a website that can be tested, as well as an infinite number of possible versions for each variable.  When you’re comparing a single variable at a time, it can take much longer to see significant improvements in your conversion rate.

However, multivariate testing can be more difficult to understand and use correctly.  Testing multiple variables at a time will allow you to hone in on winning combinations more quickly, but also prevents you from generating data on which specific changes make the biggest difference on your website.  Because this information is often vital to new website owners, it’s a good idea to start with A/B split testing and advance to multivariate testing when you have a better understanding of the process.

Step #2 – Create Multiple Versions of Your Page

Once you’ve chosen the specific type of split testing you’ll carry out, you’ll need to create pages containing the individual versions of each different element that will be tested.  For example, if you plan to conduct an A/B split test on the headline of your website, you’ll need to create two home page files to be entered into your split testing script.

Keep in mind that there are a number of different elements that you can test, as every element of your site plays a role in creating a “buying” atmosphere.  However, there are some elements that tend to result in bigger conversion rate improvements than others, so consider starting with any of the following variables for your first split tests:

• Headline – Since the average website visitor spends less than seven seconds deciding whether to stay on a page or hit the back button, the headline you use to capture this visitor’s attention is vitally important.  Making changes here can dramatically improve your bounce rate.
• Site colors – Although it sounds “new-agey”, color theory actually plays a big role in how people respond to a website.  So if you aren’t seeing as many sales as you’d like, consider testing out a new color scheme to see if it improves your conversion rate.
• Bonus offer – If you use a bonus offer on your site to encourage email newsletter signups or any other action, consider testing the specifics of your offer.  Some industries will respond better to a coupon code versus a free ebook, for example, and you won’t know which one performs best for your audience until you test it.
• Form locations and buttons – Any time you have a form on your website, whether it’s an email opt-in form, a “Request more information” form or any other data collection system, consider testing both the placement of the form and the text and colors used on your form buttons.  Using heatmaps of your site can help you determine the best test locations for these items.
• Calls to action – The text you use when asking a visitor to complete a defined action (whether it’s purchasing your product, filling out a lead form or performing some other task) is incredibly powerful.  So don’t just assume that the first call to action you put together will be the most effective – instead, test different versions to identify the option that’s most successful.

Step #3 – Add Your Page Versions to a Split Testing Script

Once your page versions are complete, you’ll need to add them to a split-testing script that will handle serving up each option randomly and tracking the number of conversions that result from each page.  Although there are plenty of different paid scripts out there that can handle this task, the cheapest and easiest to use option is Google’s Website Optimizer program.

To get started with this free program, navigate to the site above and set up a new experiment.  You’ll need to enter the URLs of each page option, as well as the URL of the page that indicates a conversion has occurred (typically your “Thank You” page).  The program will then return a small block of code to install on each page – once this is in place, your split test will start automatically.

Step #4 – Analyze Your Data and Launch New Split Tests

If you use the Google Website Optimizer program, you’ll be able to track the results of your split test and determine when a statistically significant winner has been found.  If you’re using any other program, don’t let an early conversion cause you to prematurely end your test.  Instead, be sure to wait until your split test has received enough views and conversions to prove conclusively that one page is better than the other.

Once you know which version of your page performs best, simply eliminate the losing version and start a new split test with another version of your test variable.  Over time, you’ll find that making these small changes to your website will add up to big changes in your overall conversion rate.

Image: Daniel Waisberg