• Trustworthy Online Controlled ExperimentsRon Kohavi, Diane Tang and Ya Xu, 2020The manual. Everything in part one is here in more depth, from people who ran testing at Microsoft, Google and LinkedIn.
• Experimentation WorksStefan Thomke, 2020How companies like Booking.com built a culture of testing, and what it asks of leaders.
• Rocket Surgery Made EasySteve Krug, 2009A short, practical guide to running your own usability sessions, the method in chapter 7.
• Don’t Make Me Think, RevisitedSteve Krug, third edition, 2014The classic on why people get stuck on websites, and still the fastest way to see your own site as a stranger does.
• UncontrolledJim Manzi, 2012On why controlled experiments beat intuition in business, and how rarely ideas work.
• Statistical Rules of ThumbGerald van Belle, second edition 2008Where the sample-size rule in chapter 3 lives, with dozens more like it.

And the research: Kohavi, Deng and Vermeer (2022) on false winners; Fabijan and colleagues (2019) on broken splits; Johari and colleagues (2022) on peeking; Deng, Xu, Kohavi and Walker (2013) on variance reduction; Nielsen and Landauer (1993) and Faulkner (2003) on how many users to test; Lewis, Singh and Fay (2006) and Shehu, Papies and Neslin (2020) on free shipping; Iyengar and Lepper (2000), Scheibehenne and colleagues (2010) and Chernev and colleagues (2015) on choice. Full references are in Appendix C.