Email Marketing A/B Testing Made Simple

It’s time to remove the mystery around A/B testing and show you how easy it is to get the most out of your email marketing.

Marketing is a mix of art and science. Before the Internet, marketing was mostly seen as an art. Have you seen Mad Men? Don Draper meditates on the beach and somehow comes up with an amazing idea that entices millions of people to buy Coke.

How did he know his idea would work? The answer is that he had no idea. He got lucky.

Circumstances have changed now that science has emerged to test the art of marketing.

Today, you can measure and test almost EVERYTHING in email marketing. This is one of the reasons why email is so effective. You can evaluate your ideas using an A/B test to compare two email ideas and determine which elements work best.

Have you tried testing your email campaigns? It’s easier than you might think. This article will give you the tools to get started.


A/B split testing: what it is and why you shouldn’t fear it

Most people skip testing because they don’t know how or what to test. If this is you, you are missing out on a huge opportunity to improve your campaigns.

A/B split testing is a way of evaluating and comparing two things. In email marketing, you simply set up two emails that are exactly the same except for one variable, such as a different subject line. You then send the two emails to a small sample of your subscribers to see which email is more effective.

Half of your test group receives Email A and the other half get Email B. The winner is determined by what you are trying to measure. For instance, if you want more people to open your emails, you use open rate as your success metric.

You can then send the winning email to the rest of your subscriber list with the confidence that it is your best option.

MailerLite offers an A/B split test feature that guides you step-by-step through the entire process. We keep it simple. 


How to approach A/B testing for accurate results

The most important A/B testing rule is to only evaluate one thing at a time. This way you can measure the results and know exactly what is working or not working in your emails.

For example, if you are wondering if your opens will improve by changing the subject line versus the preheader, you would run two separate tests.

  1. First, test two different subject lines with the same preheader.
  2. Conduct a second test using the winning subject line with two different preheader texts.
  3. Once you’ve tested both variables on their own, you can combine the winning subject line and preheader for optimal results.

Since you can literally test any variable within your email campaign, it can seem overwhelming figuring out where to start. We’ve identified the five best things for you to test. Let’s take a look.


Five Campaign Elements You Should Test


1. Test subject line

An effective subject line can make all the difference in a successful campaign. Testing subject lines is a good way to start optimizing your emails. MailerLite makes it easy to test subject lines with a simple click.

When you are on the A/B test page, select Email Subject and type in the 2 subject lines you want to test.

ab testing mailerlite subject line

Not sure what subject lines work best?  Here are some approaches that you can A/B test.

Ask a question in your subject line

Most tests show that asking questions can have a very positive effect on open rates. Questions feel incomplete on their own. Using a question will inspire readers to open your email in search of an answer.

Questions starting with ‘Did you know…’ and ‘Do you want…’ are great ways to catch your readers’ attention and keep them reading. See if it works for your audience.

Ask a question

Rather than writing:

3 ways to boost your productivity

Try:

Do you want to boost your productivity?

Personalize your subject line

Using your reader’s name or location in the subject line adds a feeling of connection, especially when it’s a name. Bonus: it increases click-through rates. Test it out to see if it works for your audience.

Personalized subject line

Rather than writing:

20% off sale!

Try:

Janet, we’re giving you 20% off this week

Personalization is an effective tactic, but don’t use it every time. It’s all about the surprise factor. If they see their name on every subject line, it loses its sparkle.

Use symbols or numbers

Using non-standard characters like ☞ or statistics in the subject line is an easy way to stop a reader’s wandering eye and capture attention. You can also try using emojis.

Just make sure you are using these non-standard characters in a relevant way. The subject line should always relate to the content within your email.

Create a sense of urgency

Fear of loss is a stronger motivator than the desire to gain. We all hate missing out. Craft your offer so it feels like your audience will lose something if they do not open the email.

Subject lines that create a sense of urgency or exclusivity can result in 22% higher open rates. Using phrases like “don’t miss out”, “today only” or “24-hour giveaway” in your subject lines will encourage your readers to act immediately.


2. Test sender’s name

sender name ab testing

In our experience, the ‘From’ field is even more important than your subject line. If you have a personal connection with your audience, they will open your email regardless of the subject line.

Case in point: you don’t care about the subject line when you get an email from your mum. We usually open and read emails that are sent by someone we trust and know.

Here are some ways you can change the ‘From’ field and A/B test it.

A real person vs. company name

If your campaign comes from a company, experiment with using your company name vs. the name of your marketing person vs. the name of your CEO/Founder.

Usually, when customers send emails including their own name combined with the company name, they get higher open rates.

Add sender's name

Rather than writing:

MailerLite

Try:

Ilma from MailerLite

Your full name vs. the first name only

Seth Godin likely gets higher open rates using ‘Seth Godin’ opposed to just ‘Seth’.

Different email addresses

If you’re sending emails from info at yourcompany.com, you may want to test this against first name at yourcompany.com.

Male vs. female

Depending on your audience, you may want to experiment with having a male versus a female sender. Just make sure it is a real person. Remember, it’s all about building trust.


3. Test email content

test email content

Testing the content of your newsletter can be tricky because if you’re simply changing the text, it’s hard to identify the one variable that causes a conversion. One aspect of your content that can be reliably tested is the CTA, or call-to-action.

The CTA is the most important part of securing clicks. It’s the final gateway before a reader converts.

Here are some aspects of CTAs you can change and test:

Repeated CTAs: Including too many links is overwhelming – but having just two or even three links pointing to the same ultimate goal generally leads to a lift in conversions rather than a drop-off. CTAs are usually best placed on a clickable button. Try to repeat your CTA in your signature or postscript (P.S.) – you’ll be surprised by the results.

Text on buttons: Try shorter and longer text versions on buttons. Test out a typical CTA versus a creative one. Play around with text to see which word or phrase converts more visitors.

CTA text ideas

Rather than writing:

  • Buy Now
  • Purchase
  • Add to Cart

Try:

  • I’m in!
  • Let’s do this!
  • Gimme this

Test different sizes, fonts and even ALL CAPS: For some reason, all caps performs the best. See if it works for you.

Different colors for buttons: Use contrasting colors. Some marketers say that red increases click-through rate.

Try varying the location of your CTA button, making some CTAs more prominent than others.

Consider using arrows and other visual elements to guide the reader → Some of the most successful CTAs out there have arrows pointing at them. It creates a sense of direction and guides the visitor to an important element on the page. This is a way of prioritizing information and creating a flow. Test it.


4. Test images (or no images)

ab test image

Images are powerful tools to convince your customers to act. Experimenting with images is a fun way to take the pulse of your readers and see what they respond to.

Here are some ideas of what you can test:

  • Image of people versus a product.
  • One versus multiple images.
  • Text on image versus no text.
  • Screenshot of a video.
  • Animated GIF versus static image.
  • Serious straightforward image versus a goofy one.
  • Colorful visuals versus black and white visuals.
  • A stock image versus an image of your employees or customers in action.

5. Test your preheader text

preheader testing

When customers receive your email, the subject and preheader text will be the key elements they use to determine if opening and interacting with your email is worth their time.

Preheader text is like a continuation of a subject line, so you can test it in the same ways as a subject line: ask a question, create a sense of urgency and so on.


Why you should start A/B testing your emails today

There is nothing mysterious or complicated about A/B testing. Email marketing is much harder without A/B testing. Your campaigns will never improve without learning what works and what doesn’t.

MailerLite makes it easy to set up an A/B test. Start small and try testing your subject line to improve your open rates. Once you get the hang of it, you can go deeper.

To set up an A/B test on MailerLite, check out our tutorial video to get started.