A/B test significance calculator
Paste your visitors and conversions to see the lift, the chance version B beats A, the p-value and a confidence interval. Free, no signup.
A: 3.00% converted. B: 3.60% converted.
Version B is significantly better at 95%. Check it ran its planned length before you ship it.
How it works
Enter visitors and conversions for each version. You get two views of the same data:
- Chance B beats A (Bayesian): the probability that version B's true conversion rate is higher than A's, given everything you've seen. "92%" means you'd be wrong about 8% of the time if you shipped B.
- p-value (frequentist, two-sided z-test): how surprising your result would be if there were really no difference. Below 0.05 is the usual bar for "statistically significant".
The confidence interval shows the range of lifts that fit your data. If it crosses zero, you can't rule out that B is no better, or worse.
A result is only trustworthy if you decided the sample size before you started and ran at least one full week. Checking every day and stopping at the first good number inflates false wins; see the peeking problem guide.
Questions people ask
How do I know if my A/B test is statistically significant?+
Enter each version's visitors and conversions. If the p-value is below 0.05 (or the chance to beat the original is above your bar, commonly 90 to 95%) and the test ran for its planned length, the result is significant.
What's the difference between chance to beat and p-value?+
The chance to beat is the probability B is better, given the data. The p-value is the probability of seeing a difference this big if there were none. They usually agree; the chance to beat is easier to explain to a team.
Is 95% significance always required?+
No. It's a convention. Many teams ship at 90% when a wrong call is cheap and reversible, and hold out for 99% for risky changes like pricing.
Learn more
More free tools
Want the tests run for you?
AI agents read your analytics and payments, find where you lose the most money, build the fix and test it. Every test is judged on revenue, not clicks. Plans from $29 a month.