Part six · Chapter 22

COUNT THE LOOP, NOT THE LINK

A referral dashboard counts codes. Some of them were used by people who were coming anyway. Judge the loop on total new customers, against a group that never saw the ask.

Your referral app will tell you how many customers came through a link or a code. It can't tell you how many would have bought anyway. The friend who was already on your email list, the customer's partner who shares the same card, the shopper who found a code in a coupon forum: all counted as the loop's work.

Look again at how the numbers in part two were stated. Houston didn't say the referral program drove 35% of signups; he said it permanently increased signups by 60%, measured on all signups. PayPal's S-1 counted accounts, not bonuses paid. The honest measure of a loop is total inflow.

A referral code is a receipt, not proof.

Hold out the ask, not the product

The Second Order introduced the Holdout Digit: a random number written once to every profile, used to keep a fixed group out of marketing. For a loop, use it to withhold the prompt.

  1. Pick the groupA random tenth of new customers, by digit, never sees a referral ask: no post-purchase block, no in-box card, no referral email.
  2. Leave the door openThe referral page stays reachable for everyone. You're measuring what the ask adds, not whether sharing is possible.
  3. Count referred first orders per 100 customersIn both groups, over the same weeks, completed and unrefunded. The gap is what the prompt produced.
  4. Fix the read date firstWrite the verdict date before launch. On a small file, the holdout can confirm a big effect, not measure a modest one.

OVRLND's plan wrote a random digit from 0 to 9 to every profile at ingestion and opened every rebuilt flow with an exit for digit 0: a 10% universal holdout, read over the launch month plus 30 days so both groups rode the same cycle.

Check who the referred customers were

A cheaper test runs on the data you already have. Take last quarter's referred customers and check how many were already known to you before the referral: on your email list, with a prior visit, a past order under another address. That share is a floor on "would have come anyway." It's often a third or more, and it goes straight into the calculator in Pick the Currency.

Always show the denominator

One rule from OVRLND's scorecard applies everywhere: no channel revenue number is shown without total store revenue beside it. In mid-2026, flows earned about 4% of a store doing about $136,000 a month, and the target was 8% to 10%. A referral program reporting "$40K referred revenue" means one thing next to $400K of store sales and another next to $4M.

Read the holdout

Run your numbers

What the referral ask really added

Example numbers from a 90-day read with a 10% holdout. Replace with yours.
extra referred customers per 100 who saw the ask
of the referrals credited to the ask that it actually caused
reward cost per referred customer the ask added
The range is a 95% interval on the difference between the two groups, using the normal approximation. Read it on the date you fixed before the test started.
Wrong for you if

Run the prompt holdout for one quarter. If referred first orders per 100 customers in the treated group, minus the holdout's, match what your referral dashboard credited, attribution isn't overstating your loop, and you can manage it from the dashboard.

Do this

This is one chapter of Close the Loop, which is free and readable in full on a single page with no form in front of it.