Two ads run side by side. One says 20% off the starter set with a sunrise video. The other says free shipping on the bestseller with a founder talking to camera. One wins. What did you learn? Nothing you can use, because the product, the mechanic, the promise and the creative all changed at once.

Hold one thing constant

• In a creative test, hold the offer constant. Same product, same mechanic, same promise; different hooks, formats and angles.
• In an offer test, vary one of the five decisions. Same product and two mechanics, or two products and one mechanic. Change two at once and the winner can’t tell you why it won.
• Behind each offer, run many creatives, not one. This is the counterintuitive rule. If offer A gets one ad and offer B gets one ad, you’re testing two ads. Give each offer a spread of angles and formats, enough that no single creative can carry or sink it, and judge the offers on the whole group of ads behind each one.

One ad per offer is a creative test wearing an offer test’s name.

Where to run it

Run offer tests in a separate lane from your scaling campaign, with its own budget per offer, so the platform can’t pour the whole budget into whichever offer got lucky in the first two days. Run them on the audiences you plan to scale into. An offer that wins with past visitors and existing customers tells you little about strangers, and existing customers shouldn’t see new-customer offers at all (chapter 19).

On-site, the cleanest test is a holdout: a random share of new visitors sees the new offer, the rest see the old one, and you compare contribution per visitor. Randomization does what a platform comparison can’t: it makes the two groups alike before the offer touches them.

How much evidence is enough

Ad platforms don’t split traffic randomly; they chase early winners. Treat platform comparisons as structured evidence, not experiments. Before calling a winner, want something like 30 to 50 new customers per offer at the very least, and a gap that holds up across a full week. Below that, the honest verdicts are “extend,” “stop for business reasons,” or “decide on judgment and say so.” Judgment is fine. Judgment dressed up as a significant result isn’t.

Write the bet before you run it

Every offer test gets five lines, written before launch: what changes, which of the five decisions it is, what it has to beat, what you expect and why, and what you’ll do in each outcome. If a result wouldn’t change your plan, don’t run the test. Appendix B has the template.

Do this

• Look at your last offer test. Name every one of the five decisions that differed between the offers. If more than one did, the result is a lead, not a lesson.
• For the next one, fund each offer with at least five creatives and write the five-line bet first.