A/B Testing

Optimize your automation campaigns with A/B Testing in Bullhorn Automation. This feature allows you to test multiple variations of a workflow, such as different email content or messaging strategies, to determine which performs best. By analyzing results like open rates and engagement metrics, you can make data-driven decisions to enhance your outreach and improve overall effectiveness. This guide provides step-by-step instructions for setting up A/B tests, monitoring performance, and applying insights to future campaigns, ensuring your automation strategies deliver maximum impact.

Creating an A/B Test

When you add a new email to an automation, you'll see the option to add an A/B test. 

 If you've already created an email and want to add an A/B test:

1. Click on the three dots in the top right of the card.
2. Select Convert to A/B Test A method of comparing two versions to determine which one performs better, such as which subject line, sender, or content performs better..
   
   

A/B Test Options

• Email A: This is the first email you'd like to test. Create this email like any other in the system, or work on an existing email you've converted to A/B test.
• Email B: This is the second email you'd like to test. Create it just like any other in the system. You won't be able to convert an existing email for Email B.
  • A great option is to copy Email A and modify it. Unless you are testing a completely different variation such as Plain Text vs HTML email, this method will make sure only the details you want to change are different. 
• How many contacts would you like to test?: Select how many emails should be sent before a winner is chosen. 
• What metric should determine the winner?: Depending on what you are testing, you may choose different options to determine the winner. If you are testing subject lines, the winner is probably the highest open rate. If you are testing design, messaging or buttons, it is probably the highest click or job view rate.

How the Test is Conducted

Once you've selected your options, you will see A/B test is now in progress. When a group of contacts goes into an A/B test, they will alternate between receiving email A and email B.

View Stats

Although the test runs automatically, you can always check back to view the stats and see how many people have received each message. Clicking on View Stats allows you to see:

• How many messages have been sent.
• Rates for all the actions.
• Which one is "winning".

Test Completion

Once everyone in the test group has received the messages, Bullhorn Automation will pause everyone else for 24 hours to give the recipients time to receive and act upon the emails. After the 24 hour period, Bullhorn Automation will pick the winning variation and send that message to the rest of the audience.