Part two · How reviews work · Chapter 7

WHY RATINGS DRIFT DOWN

As reviews pile up, each new one tends to come in a little lower. Usually that isn’t the product getting worse. It’s the review system working as designed.

A product launches at 4.8 and a year later sits at 4.5. Often nothing went wrong with the product. The later buyers are different people, reading a longer and messier set of reviews, and they rate differently.

The pattern

David Godes and José Silva (2012) separated two clocks: how long a product has been on sale, and how many reviews came before a given one. Once they controlled for calendar time, the time pattern actually rose. But the sequence pattern fell: each successive rating tended to be lower than the one before, controlling for the product and the reviewer Published.

Their explanation: as reviews pile up, it gets harder to tell which apply to you, so more people buy something that doesn’t suit them, and rate it lower Published. Two other findings point the same way. Early buyers have unusual tastes, so the first reviews can mislead later buyers (Li and Hitt). And frequent reviewers tend to differentiate themselves from the crowd, pulling ratings down over time, while occasional reviewers follow it (Wendy Moe and David Schweidel) Published.

Drift is mostly mismatch. The fix is helping the wrong buyer notice before they buy.

Compare like with like

Slow the slide

Do this

This is one chapter of The Proof File, which is free and readable in full on a single page with no form in front of it.