Evidence and reasoning
A/B test
- Evidence and reasoning
Definition
An experiment that compares two versions of something to see which performs better.
Example
A product team randomly assigns visitors to two checkout versions, then compares the same purchase-completion measure.
See how it works
- Decide the hypothesis and success measure
- Randomly assign participants
- Collect comparable outcomes
- Assess effect, uncertainty and possible harms
Check your understanding
Common mistake: The variant with the higher observed rate has not necessarily demonstrated a reliable improvement.
Why randomly assign people to variants?
Show explanation
It helps reduce systematic differences between the groups so the outcome comparison is fairer.
For your role
Choose your role to see what this term means in your work.
Why it matters
It shows which option actually works better, instead of relying on opinion.
Where it is used
Website and app experiments, marketing campaigns and product launches.