A/B Testing
Overview
This article outlines how to set up A/B tests using . makes this process very easy. You can test different subject lines, senders, or entirely different emails. Your winner will be chosen automatically after you launch.
Although there are best practices for getting opens and clicks, there is no exact solution for getting the most out of your message. The only way to be sure you're taking the best approach is to put it to the test!
We do not recommend using A/B test emails within date-based automations.
Creating an A/B Test
When you add a new email to an automation, you'll see the option to add an A/B test.
If you've already created an email and want to add an A/B test:
- Click on the three dots in the top right of the card.
- Select Convert to A/B Test.
A/B Test Options
- Email A: This is the first email you'd like to test. Create this email like any other in the system, or work on an existing email you've converted to A/B test.
- Email B: This is the second email you'd like to test. Create it just like any other in the system. You won't be able to convert an existing email for Email B.
- A great option is to copy Email A and modify it. Unless you are testing a completely different variation such as Plain Text vs HTML email, this method will make sure only the details you want to change are different.
- How many contacts would you like to test?: Select how many emails should be sent before a winner is chosen.
- What metric should determine the winner?: Depending on what you are testing, you may choose different options to determine the winner. If you are testing subject lines, the winner is probably the highest open rate. If you are testing design, messaging or buttons, it is probably the highest click or job view rate.
How the Test is Conducted
Once you've selected your options, you will see A/B test is now in progress. When a group of contacts goes into an A/B test, they will alternate between receiving email A and email B.
View Stats
Although the test runs automatically, you can always check back to view the stats and see how many people have received each message. Clicking on View Stats allows you to see:
- How many messages have been sent.
- Rates for all the actions.
- Which one is "winning".
Test Completion
Once everyone in the test group has received the messages, will pause everyone else for 24 hours to give the recipients time to receive and act upon the emails. After the 24 hour period, will pick the winning variation and send that message to the rest of the audience.