Part two · When you can’t test · Chapter 7

RESEARCH THAT FINDS THE LEAK

Five customer sessions, one survey question, your funnel by device and a hard look at three competitors. Most stores find more in a week of this than in a year of tests.

Testing tells you whether a change worked. Research tells you what to change. Small stores need research more than big ones, because when you can only test a few things a year, the quality of your ideas decides everything.

The funnel, by device

Start with the numbers you already have. For the last four weeks, split by phone and desktop: sessions, product page views, add to cart, reached checkout, entered shipping, entered payment, ordered. Find the biggest single drop on each device. That step is where you watch people in the next section.

Watch five people

In 1993 Jakob Nielsen and Thomas Landauer published a model of how many usability problems a test finds as you add participants. Nielsen’s popular summary is that five users uncover about 85% of the problems, and that three rounds of five, fixing between rounds, beat one round of fifteen Published. It’s a model, not a guarantee: when Laura Faulkner drew random groups of five from 60 participants, some groups found 99% of the problems and others 55% Published. But the point stands. You don’t need many people to find the big problems.

Recruit five people who fit your customer and have never used your site. Give them a phone and a task, like “buy something for your sister’s birthday under $50,” and ask them to think out loud. Don’t help. Watch where they hesitate, what they tap that isn’t a link, what they look for and can’t find. Record it if they agree. The first session usually finds something the whole team missed.

You don’t need many people to find the big problems. You need to watch, and not help.

Ask the people who just bought

A one-question survey on the order confirmation page, or in the first email after purchase, finds the objections that almost cost you the sale. The conversion research firm CXL suggests surveying recent first-time buyers within days of purchase, aiming for at least 100 responses, and asking questions like “What made you almost not buy from us?” Reported. Read every answer and group them. The biggest groups are your product page’s missing answers.

Four more sources

A heuristic review, where people who know usability judge your pages against a checklist, is useful too, and Nielsen Norman Group recommends three to five reviewers working separately. It also warns that such reviews “cannot replace user research” Reported. Use it to prepare for the five sessions, not instead of them.

What comes out

One ranked list of problems, each with its evidence: how many of the five sessions hit it, how many survey answers mention it, where it sits in the funnel. Problems that are plainly broken go to chapter 8. Problems with an uncertain fix go to the ladder in chapter 6.

Do this

This is one chapter of The Honest Test, which is free and readable in full on a single page with no form in front of it.