3 Things to do Before A/B Testing – Research is Everything

It's very easy to just jump in and start replicating the A/B tests you see in countless case studies – someone changed their button color to orange and got 33 percent uptick in conversion; someone else got a 50 percent increase by adding videos to the product pages.

However, the reality is that what works for others will not necessarily work for you. Jumping straight into testing will eat up your budgets, time and buy in without producing the results you expect.

This is why research and analysis is crucial before starting the A/B testing program. Finding the weak spots of your website, identifying your audience and understanding the actual issues will give you a very good starting point to start optimizing.

Keys to success

  • Conduct research.

    Set up surveys to get the qualitative research input. Try user testing. Getting even a couple of outsiders to use your product in front of you will open up your eyes to insight you might not come up with yourself. There are plenty of tools (HotJar, Sumome, VWO) which offer fairly cheap, but powerful services that help track user activity on your pages with surveys, heatmaps and mouse tracking.

    Keyword search, SEO
    Photo courtesy of keys4up.com

    The most useful question you can ask your visitors in a survey is, 'What is your primary reason to be on this page?' Follow up with 'Did you succeed in your task?' Give your visitors the option to input their thoughts. You might find out that the visitors might not be using your website the way you intend them to. This is not a problem though; this is an opportunity to build upon the feedback you already have and make the pages more relevant.

  • Dive deep into data.

    Google analytics
    Photo courtesy of Business News Daily

    Google Analytics provides incredible insight if you know what to look for. For example, you can findout which high traffic pages have huge bounce rates. These pages are losing you money. Find out why this is happening by segmenting by traffic source, device,browser and user type tosee if you can detect any patterns.

    Finding out where users go after landing on your website is also useful.This knowledge can help you plan multistep tests, where you optimize the full, or at least a part of the user journey. To see the pages right after the landing page, you need to use a landing page report and choose ‘entrance paths’ to show the following pages. This is the same thing as the navigation summary report in Google Analytics.

  • Build hypotheses

    A/B Testing Hypotheses
    Photo courtesy of Visual Website Optimizer

    A successful A/B test is impossible without a good hypothesis. A good hypothesis is impossible without research. Many tests fail because the hypothesis is flawed or nonexistent. By understanding the data provided, you will be able to pinpoint the pain points of your users, define the causes and create actions to solve for them.

    A good hypothesis follows this formula from Medium: Because we saw (data/feedback), we expect that (change) will cause (impact). We’ll measure this using (data metric).

    Good hypotheses give your testing program direction and increase your learning, even if the tests fail.

After going through these steps, you should be able to start building your tests professionally. A/B testing enables you to make decisions by eliminating guesswork. Eliminating guesswork in the actions that lead up to the actual testing will provide you with a strong base to get the most out of your conversion optimization activities.

Gediminas Brencius is iBus Media's CRO specialist.