Three features sold with big conversion numbers. Each can help. Most of the headline numbers don’t measure what the feature caused.
Quizzes, pop-ups and site search share a statistical trap. The people who use them were already different from the people who don’t. Compare users with non-users and the feature gets credit for the difference.
Quiz vendors report that quiz takers convert several times better than other visitors. The quiz vendor Octane AI’s site claims “4x higher conversion,” with no sample, period or comparison group disclosed Reported. Shoppers who take a quiz are engaged by definition. The only way to learn what a quiz adds is to show its entry points to a random half of visitors and compare the halves.
A quiz’s lasting value is often the answers, not the conversion. Store each shopper’s current answers on their profile in your email platform, so segments and emails can use them, and store each completed quiz as a dated event, so you can see answers change and trigger flows from them. Version the quiz, so a changed question doesn’t silently break every segment built on it. And drop any question you can’t name a use for. Every extra question costs completions. I’ve written the full data model up as a note: Most brands that have quiz data cannot build one segment from it.
Omnisend, an email platform that sells pop-ups, reports that across 1.24 billion pop-up views in 2025, the average sign-up rate was 2.1%: 2.0% for a standard form, 2.3% for a multi-step one and 3.5% for a spin-to-win wheel Reported. A pop-up grows the list, which is worth a lot. It can also cost orders by covering the product on arrival, and Google discourages full-screen pop-ups on mobile (chapter 10).
So on phones, prefer a small banner, or a pop-up that waits until a second page view or real scrolling. Judge the pop-up on both sides of the trade: sign-ups gained, and conversion on the pages where it shows, against a random slice of visitors who never see it.
Users of a feature were different before they used it. Compare random halves, not users with non-users.
“Shoppers who search convert two to three times as often” traces to blog posts from 2013 and 2015 that don’t say how it was measured Reported, and it measures intent: people who search already know what they want. The better reason to care about search is that it’s often broken. Baymard’s 2026 benchmark of more than 170 sites found mediocre-or-worse search on 46% of desktop and 58% of mobile sites. 66% failed to handle searches for things other than products, like “returns” or “shipping,” and 54% failed on abbreviations or symbols Reported.
The fix is a weekly habit. Pull the searches that returned no results, add synonyms and redirects for the common ones, and make sure the search box is visible on mobile, not hidden behind a menu.
This is one chapter of The Honest Test, which is free and readable in full on a single page with no form in front of it.