What Is Split Testing? (And How to Do It Right in 2024)

By the end of 2023, the average conversion rate for an ecommerce store is 4.29%. For every 100 visitors you get, around 95 of them leave without making a purchase.

And that’s if you’re average. If you’re below average or in another industry, you might even lose up to 99 of every 100 visitors.

If you’ve already invested big in driving paid or organic traffic, you want to maximize your return on investment (ROI) by closing more of those visitors.

That’s where split testing comes into play. It’s when you create a new version of a page on your site, an email, or an ad — a version you think might convert better — and put it to the test with real visitors.

This guide will explain exactly what split testing is, cover its benefits, and walk you through how to do it yourself.

Split testing is when you set up a controlled experiment with two variants of a product, landing page, or other item and use the result to decide which version to use going forward.

It’s a common strategy used in digital marketing and product research, but it can help you make informed decisions throughout every area of your business.

A “variant” means an almost identical version of the page with a few (or even a single) key differences. For example, you may change the headline or the featured image in the variant.

Split testing is all about making small, isolated changes to see what really makes a difference in driving more sales or traffic.

You can then take what you learn from these tests and apply it to your future campaigns and other pages and products.

If you don’t split test before making final design changes or other important decisions involving your marketing campaign, you’re relying on guesswork.

When you split test, you can start making accurate, data-based decisions about your campaigns.

Split testing vs. A/B testing

Split testing is named because you are “splitting” the traffic to a new variant on a different URL, like “yoursite.com/landing-page-a” and “yoursite-com/landing-page-b” whereas A/B testing is the name for any user experience test where you compare two different variants, even if you don’t split the traffic through using an URL.

The main difference is that A/B testing is used outside the realm of digital marketing, while split testing is an industry-specific term.

A successful split test leads to more sales from your existing traffic or more clicks from the same number of impressions on a banner or video ad.

For example, Zalora — an online fashion retailer in Asia — boosted its ecommerce checkout rate by 12.3% by making a few key copy changes to its product pages.

But that’s not even close to the only benefit. You also get:

• A better understanding of your potential customers: Knowing what headlines, images, or benefits they prefer can help you gain new insights into what drives your target audience.
• Better customer engagement: Optimizing the product and your onboarding process with split tests can create a more loyal customer base.
• Reduced bounce rates: By split testing your pages to find the right match, you can also reduce bounce rates and increase the dwell time on your pages.
• Reduced risk: You can test a new design before jumping all the way in.
• Better content and design: Your creative teams will start to naturally create text and visual content that better aligns with your target audience.

The actual approach to setting up a split test depends on the platform you’re on and what you’re trying to test. Also, just randomly deciding to test a page or ad is a bad idea. Instead, you should start by finding clear opportunities and existing weak points.

Let's break it down.

Identifying opportunities and weak points

Most companies and websites don’t have millions of daily active users. That means you can’t afford to just randomly set up split tests on every single aspect of your website.

With a limited number of visitors, you need to strategically choose to test things with a lot of potential upsides.

For example, you’re much better off split testing an underperforming landing page than trying to add a new color button to your homepage for no reason at all.

Identifying these opportunities and weak points in experience comes down to analyzing the data.

Most websites use a website analysis tool, like Adobe or Google Analytics. Monitor user behavior reports, like the landing pages view, to see which pages are underperforming in driving conversions.

You should also look at metrics like bounce rate and the average session duration. If any landing pages stand out with poor results, like a high bounce rate and no conversions, you need to change something.

To find opportunities, focus on pages with a high volume of visitors or conversions. That’s where even a small change, like a button in a new color, can make a significant impact.

If you’re looking to split test ads or marketing emails, you should also examine historical data before creating a test. Look for types of images, keywords, and promotions that have performed extra well or poorly in the past.

Forming the test and hypothesis

Now, it’s time to form your hypothesis.

What are you going to change, and why do you think it’ll improve the results of the page or ad?

For example, if the bounce rate of a landing page is higher than normal, you might hypothesize that it’s because the headline and featured image don’t grab the visitor’s attention.

As a result, you may rewrite the headline to include power words — terms that evoke emotions in the reader, like exclusive, and guilt-free — or rephrase it to better highlight the benefits of your product.

You can also feature an image where someone actively uses your product, rather than just a static product shot, to make it more engaging.

There are four elements to forming a split-testing hypothesis correctly:

• Finding an underperforming page or ad.
• Correctly identifying and defining the problem.
• Designing a potential solution based on your understanding of the problem.
• Setting criteria for what you’ll consider a success.

To an experienced split tester, the success criteria are almost always a statistically significant improvement in conversions or engagement.

That means that a test isn’t necessarily over after 1,000 page views or 100 sales. It’s over when you have a clear-cut result.

You can use one of the many freely available testing significance calculators available online — just Google it.

But you should also be careful about trusting these results of statistical significance too early. For example, if in 100 visits variant A gets 10 sales while B gets 1, the calculators will proudly state 100% statistical significance.

But this isn’t a controlled, double-blind study — you have very little control over who you send to each page.

So there may just have been nine more people who were ready to buy in the first 100 visitors to variant A.

So the best approach is to set criteria where you need to see a statistically significant improvement along with a large sample size, such as 200 conversions.

Split testing an email marketing campaign

The first step is to use an email automation tool that offers split testing as a feature. With ActiveCampaign, it's built into our platform, and you can set it up in a few clicks.

Create a new campaign and select split testing

Instead of creating a standard or autoresponder campaign, choose the “split testing”campaign type to create your test.

Then, you need to select whether you want to just test subject lines and from information or also email contents.

The next step is to create as many variants of the email as you want and decide how to split the audience between each variant.

You can easily switch between variants in our drag-and-drop email builder.

You can also go from split testing to multivariate testing by clicking the “+” button to add more test emails. We only recommend this option if you have a significant audience size.

Here are some ideas for changes to test in your emails:

• Personalizing subject lines or email headlines
• Testing different header images
• Using emojis in subject lines
• Using different templates
• Testing HTML vs. plain text

Split testing your emails is crucial to increasing your open rate and conversion rate.

With ActiveCampaign, you can also split test different marketing automation flows. Over time, you can create the optimal experience for every person who visits any page on your website.

Split testing a landing page

Create a meaningful variant

Using a tool, such as Google Optimize, you should create a new variant of the landing page.

When creating a variant, Google Optimize will duplicate the original page. It'll superimpose any changes you make to the original when the variant is shown to your audience.

You can change HTML content, images, text, and the design and layout of the page with CSS.

For a variant to be meaningful, it needs to be in line with your original hypothesis:

• If you think the header and featured image are weak because of a high bounce rate, change those in the variant.
• If the conversion rate is low, try changing up the calls to action or highlighting the purchase buttons with white space.

Setting the weight and starting the test

The weight of a variant is how much traffic a split test variant will receive. If the original and variant are both at 50%, they’ll share the traffic evenly.

With a tool like Google Optimize, you can also set up advanced audience targeting rules based on geography or traffic source. You can even connect it to your Google Analytics account to target or exclude specific remarketing audiences.

These settings guarantee the test happens on a more even playing field with meaningful results.

Repeat this process to learn more about your audience and how to sell to them effectively.

Split tests aren’t just relevant for landing pages and emails. You can also test an ad set in Facebook ads or set up an SEO split test for your blog content. You can split test just about anything in your digital marketing campaign.

Split tests offer unique insights about different segments of your audience.

For example, you may discover a pattern — like visitors from social media preferring a short-form video landing page vs. organic traffic preferring long-form text-based landing pages.

You should then make sure each segment of your audience sees the version that resonates the most with them.

93% of B2B marketers attribute their revenue growth to personalization. And it’s also a powerful tool in B2C markets: 36% of online shoppers want more personalization, not less.

With ActiveCampaign landing pages, for instance, you can easily tailor the web experience of each visitor based on their interactions with your brand.

This type of web personalization is the next-level approach to dive into after a few successful split tests.

Instead of trying to show one optimized page to all kinds of different people, you can show the individual the page that resonates the most with their unique needs.

With this technology, you can give every visitor exactly what they want and expect from your site.

Segmenting your email audience

With an advanced email marketing tool, personalization and segmentation are part of the package.

For example, with ActiveCampaign, you can segment your contact list based on:

• Engagement (opens, link clicks)
• Site activity or purchases
• Signup location
• Profile
• Geographical location
• Deals or promotions

You can also use our marketing automation tool to build personalized experiences for different kinds of visitors and leads.

For example, you can send a wedding lead into a funnel that includes free wedding invitation templates, a guide to the best wedding caterers in their city, and finally, your recommended venues.

Then for a business lead, you can create a funnel that includes tips for arranging business conferences, tips on getting sign-ups to their event, and finally, a coupon for a discount on renting one of your conference venues.

Split testing is an effective method to not just get more sales but also learn crucial insights about your audience.

Use it as a tool to create a more segmented and personalized experience for every visitor.

If you’re not sure where to start, ActiveCampaign can help you start experimenting with split testing to improve your email marketing results.

Try ActiveCampaign’s free trial today and see how our split testing and advanced personalization tools can transform the impact of your marketing campaigns.