Run an A/B test

Follow these steps to set up and send an A/B test campaign.

What you'll need

• An installed Email or WhatsApp, Push or SMS channel.

• At least two (and up to a maximum of eight) message templates with different content.

• A list or segment of opted-in contacts.

Step one: Create your A/B test campaign and add content

1. Follow the instructions to send a campaign.

2. Once you've configured the channel you're sending from, and selected the first message template that you want to test in the 'Campaign content' section, scroll down to "More Settings" and expand the dropdown.

1. Switch on the toggle to create A/B test. This will duplicate the selected content as both 'Variation A', and 'Variation B'. (Scroll back to the top of the page to see these variants created for you, ready for editing)

1. Click 'Variation B', and click edit email. You will be taken to the email editor where you can make edit your email template, configure the template language, and any variables. Click save once you're happy with your changes and you'll be returned to the campaign editor.

1. If you want to add more variations, click the Clone button on one of the templates in your A/B test, then repeat the process of selecting and configuring the new content.

Step two: Configure your A/B test settings

Now that you've created your A/B test campaign and added the templates that you want to test, it's time to configure the rest of your settings.

1. Under 'Winning metric', select either Click through rate, Read rate (WhatsApp only) or Open rate (Email only).

1. Under 'Test size', drag and drop the bar between the test versions and the 'winner' to determine the percentage of recipients who will receive a test version of the campaign.

2. Under 'Test duration', enter the number of hours you want to run your test for before the winning version is sent.

1. Under 'Winner fallback', either select Manually select the fallback winner, then choose the variation you want to send in event of an unclear test result, or select Send all test variations equally to remainder recipients.

Step three: Preview and test your A/B test campaign

The final step before you can start testing your campaign on your audience is to preview your campaign content, and send test variations to yourself or your team.

1. In the Campaign builder, go to the 'Email preview" panel on the right-hand side.

2. From the Variation list, select the template you want to preview and test.

1. To send a test email, click Send a test email in the top right-hand corner. Select the variant you want to receive from the dropdown. From the Simulated contact drop-down, select the contact you want to send a test campaign message to. If you're sending an email, pick the contact's email address. If you're sending a WhatsApp, pick the contact's number.

1. Click Send a test message.

2. The recipient will receive a test email. Check that all the templates look correct, that any variables have been replaced, and that any media and buttons are displaying correctly, and give it a final review for spelling and grammar.

Step four: Send your A/B test campaign

Ready to send your A/B test campaign? In the Campaign builder, go to section 3: Schedule. Then choose either:

• Immediate to immediately send your test campaign variations to a percentage of your recipients.

• Fixed send time to wait until a set date and time to send your test campaign variations to a percentage of your recipients.

• Optimal send time to deliver messages at the best time for each recipient.

Sending phases

• Once your A/B test campaign has started to send, the status will change to In progress.

• Once the A/B test is complete, the status will change to Waiting while the results are analyzed.