Run an A/B test
Follow these steps to set up and send an A/B test campaign.
What you'll need
At least two (and up to a maximum of eight) message templates with different content.
Step one: Create your A/B test campaign and add content
Follow the instructions to send a campaign.
Once you've configured the channel you're sending from, and selected the first message template that you want to test in the 'Campaign content' section, scroll down and click Create A/B test. This will duplicate the selected content as both 'Variation A', and 'Variation B'.
Click 'Variation B', and choose a new template from the dropdown. You can see a preview of the selected content on the right-hand side of your screen. Finish configuring the template language, and any variables.
If you want to add more variations, click the Clone button on one of the templates in your A/B test, then repeat the process of selecting and configuring the new content.
Step two: Configure your A/B test settings
Now that you've created your A/B test campaign and added the templates that you want to test, it's time to configure the rest of your settings.
Under 'Winning metric', select either Click through rate, Read rate (WhatsApp only) or Open rate (Email only).
Under 'Test size', drag and drop the bar between the test versions and the 'winner' to determine the percentage of recipients who will receive a test version of the campaign.
Under 'Test duration', enter the number of hours you want to run your test for before the winning version is sent.
Under 'Winner fallback', either select Manually select the fallback winner, then choose the variation you want to send in event of an unclear test result, or select Send all test variations equally to remainder recipients.
Click Continue to preview.
Step three: Preview and test your A/B test campaign
The final step before you can start testing your campaign on your audience is to preview your campaign content, and send test variations to yourself or your team.
From the 'Preview' screen, go to the 'Preview and test' panel on the right-hand side.
From the Variation drop-down, select the template you want to preview and test.
From the Simulated contact drop-down, select the contact you want to send a test campaign message to. If you're sending an email, pick the contact's email address. If you're sending a WhatsApp, pick the contact's number.
Click Send a test message.
The recipient will receive a test email. Check that all the templates look correct, that any variables have been replaced, and that any media and buttons are displaying correctly, and give it a final review for spelling and grammar.
Step four: Send your A/B test campaign
Ready to send your A/B test campaign? Click Send campaign in the top right-hand corner of your screen, then choose either:
Send now to immediately send your test campaign variations to a percentage of your recipients.
Schedule campaign to wait until a set date and time to send your test campaign variations to a percentage of your recipients.
Sending phases
Once your A/B test campaign has started to send, the status will change to In progress.
Once the A/B test is complete, the status will change to Waiting while the results are analyzed.
You've just scheduled an A/B test! Time to sit back, relax, and watch the results roll in.
Last updated
