Prioritise A/B Testing to Improve Conversions

A/B testing, also known as split testing, is an invaluable part of any digital marketing strategy. Marketers are often tempted to use intuition to predict the success of their assets including landing page designs, email copy or call to action buttons. Instead of relying on assumptions to make marketing decisions, it is better to run A/B tests.

A/B testing enables you to compare 2 variations of something to learn which is more effective. Thus, you can try things out to discover which content enables you to reach your conversion goals faster whilst providing the best customer experience. 

What is A/B Testing in Digital Marketing? 

A/B testing eliminates assumptions out of conversion optimization allowing digital marketers to make data informed decisions. 

The tests work by presenting different content to different user groups and using the results of their reactions to inform product and marketing strategies.

Tests are conducted for a set period with a defined audience.

The version that results in the desired uplift is known as the ‘winner.’ Implementing this winning version on the tested element can improve your CRO and increase business ROI. 

Metrics to measure include:

• conversion rate
•  click-through-rate
•  page impressions
•  web page bounce and exit rates
•  revenue lift.

How do you Perform an A/B Test?      

It is important to be thorough in the planning, execution and post period of your testing to gain the full benefits of split testing.

Prior to A/B testing:

1. Select one independent variable to test.
2. Identify a primary metric to focus on.
3. Create the ‘control’ – unaltered version of what you are testing, and the ‘challenger’ – the version you will test against your control.
4. Split your sample groups equally and randomly.
5. Determine your sample size.
6. Decide the statistical significance you need to achieve.
7. Ensure you are only running one test at a time.

During A/B testing:

1. Use an A/B testing tool.
2. Test both variants simultaneously.
3. Run gtests long enough to obtain a substantial sample size.
4. Collect qualitative feedback from users to support quantitative data.

After A/B testing:

1. Focus on your goal metric when doing analysis.
2. Measure the significance of your results using an A/B testing calculator.
3. Take action based on your results.
4. Plan your next A/B testing experiment.

Why A/B Testing is Important                             

When done consistently, A/B testing can improve your business performance substantially. 

Benefits of regular bucket testing include: 

• Improved user engagement
• Improved content
• Reduced bounce rates
• Increased conversion rates
• Higher conversion values
• Ease of analysis
• Quick results
• Everything is testable
• Reduced risks
• Reduced cart abandonment
• Increased sales

Optimization not only results in a sales boost, but optimized changes can also result in better user experiences which, in turn, build trust and loyalty in the brand, creating repeat customers and, therefore, increased revenue.

What Can you Test with A/B Testing?       

To use your time effectively, devote your A/B testing efforts to the most impactful elements of your digital marketing. The ten most effective elements to A/B test are:

1. Headlines and copywriting
2. Call to actions (CTAs)
3. Images, audio and video
4. Subject lines
5. Content depth
6. Product descriptions
7. Social proof
8. Email marketing
9. Media mentions
10. Landing pages

A/B Test Example    

WallMonkeys Case Study

WallMonkeys, a wall decals brand, ran A/B tests on their homepage.

Objective

WallMonkeys set out to optimize their homepage for conversions.

The control/original testing variable

The original homepage featured a headline overlay and a stock image.

WallMonkeys used Crazy Egg Heatmaps to see user behaviour on the homepage. 

Heatmaps allow you to see the clicking activity on your website to determine which areas on the site users are drawn to. 

Users concentrated their activity on the headline, CTA, logo, and search and navigation bar.

The variation/new version of the original testing variable

WallMonkeys then exchanged the stock-style image with a more playful version that would show visitors the benefits they could enjoy with their products.

Result

Conversion rates for the new design were 27 percent higher than the original design.

WallMonkeys continued the testing. For the second test, the company replaced its slider with a prominent search bar on the premise that users would be inclined to search for items which they were interested in.

Result

This A/B testing resulted in a 550 percent increase in the conversion rate. 

The incredible results from the test show not only the impact split testing can have on a business, but also the importance of continuous testing.

How Long Should You Run Your A/B Test?                                 

False results are a common mistake when running A/B testing. They result from stopping a test too early or stopping it in the middle of a business cycle. Doing so can produce results that are off, sometimes by a small margin, sometimes by an order of magnitude. This means you will make decisions on the wrong data, which is considerably worse than making decisions with no data at all.

To avoid this, ensure you stick to the following principles each time:

1. Run your test until you have reached the minimum sample size required for your results to be statistically valid.
2. Run your test  for at least one complete business cycle.
3. When running tests for more than one business cycle, do not stop in the middle of the additional cycles (do not stop after two and half business cycles for example).

Analysing A/B Testing Results 

Gathering data is perhaps the easier part of hypothesis testing. Drawing insights from that information and converting those insights into actions is what leads to successful results.

Whatever the outcome of your A/B test, positive, negative, or inconclusive, you must gather insights from the results. 

When you are analyzing A/B test results:

• Ensure you are looking for the correct metric. If multiple metrics are involved, you need to analyze all of them individually. 
• Perform segmentation of your A/B tests and analyze them separately to get a clearer picture of what is happening.
• Monitor visitor behavior analysis tools such as Heatmaps, Scrollmaps, and Visitor Recordings to gather further insights into A/B test results.

The Top Recommended Tools for A/B Testing in 2021

It can be difficult to identify the right A/B testing tool. The “right” tool will vary depending on the business. 

The most valuable tools are the ones that are reasonably priced giving companies room to spend more of their money on developing an effective testing process. 

The following are the top tools CRO experts recommend as listed in order of the frequency with which the experts mention them.

Tools for running A/B testing

1. Optimizely
2. VWO
3. Convert Experiences
4. SiteSpect
5. AB Tasty
6. Evolv
7. Google Optimize
8. Qubit
9. Adobe Target

Tools For Gathering Data

1. UsabilityHub
2. Google Analytics
3. Crazy Egg
4. UserTesting.com
5. UserZoom
6. ClickTale
7. HotJar
8. Mouseflow
9. Inspectlet
10. SessionCam
11. Lucky Orange

By following the steps in this A/B testing guide, you can confidently plan your own optimization strategy. When executed consistently and effectively, A/B testing can reduce a lot of risk of undertaking an optimization project. If you need an external team to assist with your testing, consider scheduling a strategy session with a digital marketing agency with CRO services. Marketers like to joke that A/B testing stands for “Always Be Testing.” So keep testing and learning!