The A/B Testing mindset at a company evolves through four stages. Intuition: In the beginning, intuition drives decisions. The company acknowledges the user, but does not solicit feedback. Data is sparse. Data driven: Decision makers use data to supplement their intuition in cases when they are less confident. The data lacks richness and methods for processing data are crude. Causal statements are drawn from situations that do not warrant them. Data is not respected. A/B Testing: The company unearths the practice of A/B Testing and embarks on the well-trodden path of successful companies. A/B Testing gains followers, however infrastructure is nascent and statistical methods are questionable. The goal is to get a number, not necessarily a correct number. The thought is that a number from an A/B Test must be trustworthy because, well, it's from an A/B Test! Sound A/B Testing: The company is educated on the fundamentals of A/B Testing. The company adopts sounds practices, produces trustworthy numbers, and makes informed go/no-go decisions. Regardless where your company is on the journey, this book will guide you to the last stage.
