A/B Testing

A/B Tests can be set up and run between any number of creatives within a campaign. Based on your goals, you can choose to optimize on overall conversion, CAC, or ROAS. Common test variables include:

• Size or Format: 4 x 6 vs. 6 x 9 Postcard
• Promotional Offer: "Free Shipping" vs "$10 OFF" vs "20% OFF"
• Call to Action: Website URL vs. QR Code

... and so much more!

Test Setup

To setup your A/B test, you'll first want to upload all your creatives to the same campaign - the platform will automatically randomize between all active creatives as they move to production. If you upload 2 creatives to test, each creative will have a 50% chance in the rotation and if you upload 3 creatives, each one will will have a 33.33% chance, and so on.

One Time Send Launch

If you are launching via One Time Send, you'll want to initiate a new send and select the creatives you would like to use for the A/B test:

Continue through the flow and you'll see confirmation of the A/B test using multiple creatives on the last step before launch. If one of your creatives is set as the Default, that tag will be ignored when selecting the Multiple Creatives option via One Time Send.

When you process a One Time Send that uses the Multiple Creatives A/B testing feature, you are billed according to your highest-priced, mailable creative.

• This will first draw from your credit balance
• If your credit balance is insufficient, an additional charge will be made to the card on file to make up the difference

At the time you process the mailing, we don't yet know the final distribution of addresses, because the system must first append or cleanse addresses and check your suppression settings before resulting in the final set of mailable addresses and randomizing creative assignment.

Once the creative assignment has finished, and final A/B testing ratios are confirmed, if the actual cost was lower than what was billed, the difference will return to your account's credit balance.

Trigger Launch

When setting up a Poplar Mail trigger via platform integration, you have the option to specify a creative_id along with campaign_id in the webhook payload. If creative_id is NOT specified in your webhook trigger, the platform will automatically randomize all active creatives under the campaign - this is how to A/B test when activating your trigger.

If creative_id isn't specified and one of your creatives is set as the Default, then only the default creative will trigger.

Analyzing Attribution Results

If you've shared Transactional Data with Poplar for in-platform reporting, you'll see the A/B test results broken down by creative at the bottom on the campaign's Overview page:

Additionally, if you download the raw match data there will be a creative_id column that specifies which creative was matched or attributed.