Before anyone explains a number, ask how much it moves when nothing has changed. The answer is usually more than the move you’re explaining.
Every weekly number is the sum of thousands of small causes: who happened to be paid that Friday, the weather in Texas, what the ad auction did on Tuesday, which day the email went out, a creator’s post you never saw. Those causes don’t stop. They make every number wobble, every week, with no story behind the wobble.
Start with the smallest source of wobble: pure chance. Say a brand ships 10,000 orders a month, about 2,300 a week. Even if nothing about the store, the traffic or the customers changed, the weekly count would bounce around 2,300 the way coin flips bounce around half heads. The 95% range from chance alone is about 94 orders either way, or 4%. Two weeks compared with each other can differ by almost 6% by chance alone Derived.
That’s the floor, and it’s the best case: a big count and a simple metric. Smaller denominators wobble far more.
| Number | Typical week | Chance alone, either way | As a share of the number |
|---|---|---|---|
| Weekly orders | 2,300 orders | ±94 orders | ±4% |
| Site conversion rate | 2.5% of 90,000 sessions | ±0.10 pts | ±4% |
| Email click rate | 1.5% of 20,000 recipients | ±0.17 pts | ±11% |
| Repeat rate, one month’s cohort | 25% of 400 customers | ±4.2 pts | ±17% |
| One SKU’s sales | 40 units | ±12 units | ±31% |
| A campaign’s order rate | 0.15% of 12,000 recipients | ±0.07 pts | ±46% |
Derived95% ranges: 1.96 × √(p(1 − p) / n) for rates and 1.96 × √n for counts. This is sampling chance only. Real weeks vary more.
Read the last two rows again. A campaign to 12,000 people that normally produces 18 orders can produce 26 or 10 with nothing different about it. A new SKU selling 40 units a week can sell 28 or 52. Those are the numbers most teams make their fastest decisions on.
The smaller the denominator, the louder the noise. The loudest numbers get the fastest decisions.
The table understates it, because the thousands of small causes add their own variation on top of sampling chance. Walter Shewhart of Bell Laboratories, who invented the control chart, called them chance causes, and the rare big ones assignable causes; W. Edwards Deming later called the second kind special causes Reported. A good week and a bad week from the same steady system are both made of chance causes. Nobody can find “the reason” for one of them, because there isn’t one.
You can’t compute that total wobble from a formula. You measure it from your own history, which is what the chart in the next chapter does. How wide it is depends on the business. In the example in the next chapter, the limits sit about 20% either side of average weekly revenue, and that store has nothing wrong with it.
Open rate is the noisiest number in the building, and not only because of chance. Apple’s Mail Privacy Protection downloads remote content “in the background by default,” whether or not the reader engages with the email, and loading that content is what makes an open register. Apple says it stops senders from learning when or how many times a message was opened Reported. So a change in opens can come from how many of your recipients use Apple Mail on a given send. Judge subject lines and sends on clicks, orders or revenue per recipient. For deliverability, which does use opens as one input, see The Whole Machine.
Campaign results carry the small-denominator problem too. Most campaigns produce a few dozen orders, and a few dozen orders move by a third on chance alone. One campaign doesn’t tell you whether an offer, a subject line or a day of the week works. Twenty campaigns might.
A monthly cohort of 400 first-time buyers can show a 30-day repeat rate of 21% or 29% from the same underlying behavior. Read repeat rate by cohort and over at least three cohorts before you credit a new post-purchase flow or blame a shipping delay. The Second Order covers how to build the cohort view.
There are two mistakes, and fixing one makes the other more likely. You can treat noise as a signal: explain it, act on it, change something that wasn’t broken. Or you can miss a real signal because every week already has a story and the real one looks like the rest. Shewhart’s chart was designed to balance the two, so that you react rarely enough to avoid chasing noise and often enough to catch real change. Gut feel guards reliably against neither.
This is one chapter of The Noise Floor, which is free and readable in full on a single page with no form in front of it.