Not your customers. A small, self-selected slice of them: the delighted, the furious, and a few people who review everything.
Your average rating looks like a survey result. It isn’t. A survey asks everyone; reviews come from whoever chose to write.
Jakob Nielsen called it participation inequality in 2006: in most online communities, about 90% of users only read, 9% contribute occasionally, and 1% contribute most of the content. His example from Amazon: one reviewer alone had written 12,423 book reviews Reported. Clay Shirky’s Here Comes Everybody (2008) makes the same point at book length: contributions follow a steep power law, and the average contributor doesn’t exist.
Plot most products’ reviews by star and you get a J: a tall bar at five stars, a smaller bump at one, and little in between. Nan Hu, Paul Pavlou and Jennifer Zhang found that 78% of book ratings, 73% of DVD ratings and 72% of video ratings on Amazon were four stars or higher Published.
Then they ran the test that settles it. In a controlled experiment, everyone rated the same music CD. Their ratings formed an ordinary hump in the middle. The same CD’s ratings on Amazon formed a J Published. Same product, different shape, because on Amazon nobody was required to rate it.
Two filters explain it. Purchasing bias: people who expect to dislike a product don’t buy it, so never review it. Under-reporting bias: the delighted and the furious “brag or moan”; the mildly satisfied stay quiet. So the average is a poor proxy for quality; look at the spread and the two peaks too Published.
The middle of your customer base is the part that doesn’t write.
Leif Brandes, David Godes and Dina Mayzlin (2022) added a third filter: attrition. Reviewers with moderate experiences are more likely to stop reviewing over time than those with extreme ones. In a large field experiment with an online travel platform, plain review-request emails, with no incentive, made the reviews that came in less extreme Published. Asking everyone brings back the middle. That’s the strongest research case for a systematic ask, and chapter 8 is about how.
Eric Anderson and Duncan Simester found that about 5% of reviews at a large private-label retailer came from customers with no record of buying the product, and those reviews were significantly more negative. Many came from the firm’s most loyal customers Published. Some are gift recipients, not fakes. But it’s a reason to label verified buyers. Spiegel’s analysis of PowerReviews data found a verified-buyer badge raised the odds of purchase by 15% Reported (vendor data).
This is one chapter of The Proof File, which is free and readable in full on a single page with no form in front of it.