Create an A/B Test campaign

💡 Good to know
The A/B test is available for Business and Enterprise plans.

When it comes to contact engagement, even small variations to your newsletters can have a big impact. A/B testing campaigns help you learn what works best for your contacts (and what doesn't).

Brevo allows you to run A/B Test campaigns on your email subject lines or content.

❓What is an A/B Test campaign?

An A/B Test campaign (or split test campaign) allows you to test two different variations of an email campaign (A or B) with a portion of your recipient list. The "winning" or most successful variation of your campaign will be sent to the rest of your audience to help improve your statistics.

When creating an A/B Test campaign, you will:

  • Choose one element to test, such as your email subject line or content.
  • Select the sample size of recipients for the test (each variation will be sent to exactly half of the sample).
  • Choose the method for determining the winner: best open rate or best click rate.
  • Set the duration for when the test should run.

Each recipient in your sample will receive only one of the two variations: version A or version B. After the test period, the winning variation will be selected based on your criteria (open or click rate) and will be immediately sent to the rest of your campaign's recipient list. 

This process ensures the best version of your campaign is sent to the largest group of recipients.

🧪 What elements can you test in A/B Test campaigns?

Subject Lines Email Content
Email subject lines are the very first thing your recipients will see when receiving your campaign since subject lines are visible to them before they even open your campaign. 
Research shows that 68% of people open an email based on the subject line alone.
campaigns_subject-line-view_EN-US.png

It is important to use an effective subject line for your campaigns depending on your audience. You can run A/B Test campaigns on email subject lines to determine for example if it is more effective to: 

  • Use short or long subject lines.
  • Use punctuation and/or emojis. 
  • Include personalization (like your contact's first name), a creative teaser, or mention an incentive, etc.

📝 Create your A/B Test campaign

A/B Test campaigns are created in the same way as regular email campaigns but include additional steps to set the test criteria.

Before you start

  • A/B Test campaigns can only be sent to recipients who have received a campaign from you through Brevo in the past. It means that you have to send at least one regular email campaign before sending an A/B Test campaign. 
  • For statistically relevant test results, you should send your A/B Test campaign to at least 5,000 recipients.
  • It is impossible to send A/B Test campaigns with our Send time optimization tool (Send at the best time
  • Make sure you imported your contacts into Brevo and have your lists of contacts or segments ready to be used. 
  • Starting in February 2024, Gmail and Yahoo plan to enforce new sender requirements. Ensure you comply with these new requirements to keep reaching your recipients' inboxes. To learn more, check our dedicated article Prepare for Gmail and Yahoo's new sender requirements.
  • If you wish to use your own email template, make sure it is ready to be used or learn how to create an email template before creating your campaign. 

⚙️ Configure your A/B Test campaign

To create an A/B Test campaign:

  1. Go to Campaigns > Email.
  2. Click Create an email campaign.
  3. Click on the A/B Test tab and fill in the relevant information:
    1. Choose which element you would like to test between Subject lines and Email content.
    2. Write a Campaign name that will help you find it easily in the list of your campaigns.
      This name will only be visible to you.
  4. Click Create A/B campaign
    2022-09-30_15-34-19__1_.gif
  5. Choose a sender and recipients from the From and To sections respectively and set Advanced settings if needed.
    To learn more, check our dedicated article on how to Configure your campaign.
Subject Lines Email Content
💡 Good to know
Use artifical intelligence (AI) to generate a catchy subject line for you in a few seconds with our integrated AI, the Brevo AI assistant. To learn more, check our dedicated article Auto-generate engaging marketing content with the Brevo AI assistant.

If you decided to test two different subject lines in your campaign, you will see in the Subject section that there are two Subject Line fields: A and B.

Click Add subjects and enter the two different subject lines you want to test in the A and B fields respectively:

campaigns_ab-test-subject_EN-US.gif

💡 Good to know
You can insert personalization into your subject lines and preview text using the 👤  icon. To learn more, check our dedicated section on Subject in Creating an email campaign.

✂️ Define Split Rules

When creating an A/B Test campaign the Split Rules section is added to your campaign creation flow. These rules will determine the size of the test groups, how the winning version will be selected, and the amount of time for which the test will run. Click Add split rules to start configuring them. 

Select the size of your test groups

❗️ Important
You must first select your recipients to define test groups. 

The default test group size is 50% of the total recipient list with versions A and B each being sent to 25% of the recipient list. The remaining 50% of the recipients would receive the winning version after the test ends. The maximum size of each test group is 50% and each recipient in the test groups will receive only one version of your campaign. To change the test group size, simply move the slider right to increase the size or left to decrease it. 

campaigns_ab-test-slider_EN-US.gif

How will the winning version be chosen?

Select the method for determining the winning version: choose whether the version with the highest Open Rate or the highest Click Rate wins. campaigns_ab-test-open-click-rate_EN-US.png

How long should we run the test? 

Select how long the test will run. After the test duration completes, the winning version will be identified and sent to the remaining recipients. The duration may be defined in hours or in days with a maximum test duration of 23 hours or 7 days. 

campaigns_ab-test-duration_EN-US.gif

🚀 Finalize your campaign

After completing each section of the email campaign creation flow, you will find in the top-right corner of your screen three options: Save & QuitPreview & Test, and Schedule.

❗️ Important
The Best time and Send in batches options cannot be used to schedule the sending of A/B Test campaigns. You can only choose between the Send now or Schedule options to send your A/B Test campaign. 

The test versions will be sent to the respective test groups depending on the schedule option you choose and the winning version will be sent automatically to the rest of the recipients as soon as the test duration ends.

➡️  For example
If the test duration is 4 hours and the campaign is scheduled to be sent at 1:00 pm on a certain date, the winning version will be sent out to the remaining recipients at 5:00 pm on the same day.

You will be able to edit and see your A/B Test campaign in your campaign list. To help you identify an A/B Test campaign from a regular campaign in your campaign list, an A/B Test tag is added to your campaign name.

campaigns_tag-ab-test_EN-US.png

📊 Access and understand your A/B Test campaign reports

Once your campaign is sent to all your recipients, you can access the report and see which version of your A/B Test campaign won. To access your report, click on the name of your A/B Test campaign from your campaign list once it has been sent to all your recipients. You will have access to all the statistics on your A/B Test campaign. Click the A/B Test results tab to see which version of your A/B Test  campaign won. 

campaigns_ab-test-report_EN-US.png

🤔 Have a question?

If you have a question, feel free to contact our support team by creating a ticket from your account. If you don't have an account yet, you can contact us here.

💬 Was this article helpful?

22 out of 38 found this helpful