Send the Winning Version to the Rest of Your Audience

An experiment ends with a decision, not a send. Once you've recorded a winner, Deploy winner turns the winning arm into an ordinary email for the rest of your audience. It leaves out everyone who took part in the test, and sends nothing until you do.

Keeping the two apart is deliberate. The decision is a record of what the evidence showed. Sending the winner is a marketing choice you make afterwards, with its own audience, timing and offer.

This guide follows Retro Revival, a fictional clothing shop, with made-up numbers. Its experiment tested 10% off (arm A, the control) against free shipping (arm B), each hidden under a scratch-off card, on 4,000 of the 6,420 subscribers who like rock and the 1990s. Test which reward sells more with a scratch-off A/B test runs that experiment from the start.

Start from a recorded winner

Deploy winner appears on the experiment's results page once a winner is recorded, while the experiment is Decided or Closed. Retro Revival's purchase conversion came out 5.0% for free shipping against 3.7% for 10% off, a significant difference. The team recorded free shipping as the winner and closed the experiment.

Deploying stays off the experiment's main line. It doesn't change the experiment, its results or its decision, and you can deploy after closing. Nothing is deployed for an "inconclusive" or "no difference" decision, and a holdout can't be deployed, since it has no email.

What Deploy winner creates

Deploy winner opens a short confirmation, then Create the email makes one new email, named after the experiment and the arm. It's what the winning arm's recipients got:

• The arm's content and offer. The email with that arm's changes applied, or the arm's own email if it sent different content.
• Its subject and sender. The arm's subject line, and the sender the arm was mailed from.
• The experiment's contact list as its audience.

Then it opens the new email's Send page. A notice there names the experiment and the arm, and the subject and the experiment's audience filter are filled in for you. Nothing is sent until you send it from that page. Clicking Deploy winner again opens the same email instead of making a second one.

Who gets it, and who doesn't

Every send of the new email leaves out everyone the experiment assigned, in any arm and for good. They already got a version of this email, and their results belong to the test. Each send reports them as skipped, with the reason "In an experiment on this email".

Retro Revival's filter, Rock–Alternative AND 1990s, still targets 6,420 contacts. The 4,000 who were in the experiment are left out, so 2,420 subscribers can get the winning email.

You can change the audience before sending. The filter on the Send page is an ordinary one, so you can widen it to the whole list or narrow it to one group. Contacts who unsubscribed, or are otherwise not eligible, are skipped as on any send.

The offer comes with it

If the winning arm had a store reward, the new email keeps it. Every send mints a single-use code for each recipient on your connected store, valid for the same number of days the arm's codes were. Retro Revival's free-shipping codes last 7 days, counted from each send.

Before you send, check that the offer still makes sense. A free-shipping or discount offer costs money on every order it brings in. The experiment's revenue numbers aren't a profit calculation: they exclude taxes and shipping, and they already reflect discounts.

Sending to more than 200 contacts

A send from the Send page reaches at most 200 recipients at a time. Retro Revival's 2,420 need more than one send. Two ways work:

• Several one-time sends. Pick up to 200 contacts each time. The send history tags who already got the email.
• A recurring send. Under Recurring send, add the same filter rules and create a daily or weekly schedule. Each run sends to up to 200 matching contacts who haven't had this email yet, until everyone has. The experiment's contacts are checked and skipped once each along the way, so the first runs send fewer.

Recurring sends are included from the Lite plan up.

Or pick the rest's version before launch

If you'd rather not wait for the whole window, decide how the rest of the audience gets its email before you launch. Send the result to the rest, in the experiment's Schedule & window step, tests on a sample first, then sends one arm to everyone else the audience targets. It sends in batches with no 200-contact limit. It needs a random sample, so there's a rest to send to.

The policy you choose decides what the rest gets. Say Retro Revival had tested on a slice of 2,000 contacts and decided after 48 hours. At that point, 49 free-shipping recipients had bought against 36 with 10% off: 5.0% against 3.7%, but not yet significant (p = 0.15).

• Send a significant winner would have sent the control, 10% off, to the other 4,420 contacts. No arm had won yet.
• Send the leader (profit policy) would have sent free shipping, because it looked best. That's a reasonable bet when arms are close, but it isn't evidence that free shipping is better.

Either way, this choice only decides what the rest receives. The experiment still runs its whole window, and you still record its decision on the full result.

Getting started

Record a winner on the experiment's results page and close the experiment. Then click Deploy winner, check what the new email carries, and Create the email. On its Send page, check the subject, the audience filter and the offer, and send in batches of up to 200 or set up a recurring send. Reading experiment results covers recording the decision first.