Numbers and measures
Statistical significance
- Numbers and measures
Definition
A way of judging whether a result is likely to be real instead of due to chance.
Example
A new checkout has a slightly higher observed conversion rate. A statistical test helps assess whether the difference is compatible with random variation under a specified null hypothesis.
See how it works
A p-value is assessed against an agreed testing plan; effect size and practical importance need separate consideration.
Check your understanding
Common mistake: Statistical significance does not mean the change is important, nor that there is a 95% chance the hypothesis is true.
Can a tiny change be statistically significant?
Show explanation
Yes. With enough observations, a very small effect can be statistically detectable but not worthwhile.
For your role
Choose your role to see what this term means in your work.
Why it matters
It stops you acting on results that may be down to chance.
Where it is used
Experiment results and research reports.