An AB test sample size calculator tells you how many visitors each variant needs before you launch. Enter your baseline conversion rate, the smallest lift worth detecting, a significance level (usually 95%) and a power level (usually 80%), and it returns visitors per variant. A 3.0% baseline and a 15% relative lift need about 24,200 per variant.
That number decides whether a test is worth running at all and how many weeks it will take. This guide is part of the A/B testing hub. It covers the inputs, the formula with a worked example, test duration and what to do when traffic is too low. Reading the result once the test ends is a separate step, covered in the guide to calculating statistical significance.
How many visitors per variant does an A/B test need?
An A/B test needs as many visitors per variant as the sample size formula returns for its baseline conversion rate and minimum detectable effect. Low baselines and small lifts push the number into the tens of thousands, while high baselines and large lifts can need under 2,000 visitors per variant at 95% significance and 80% power.
Sample size is the number of visitors (or users, if you randomize by user) each variant must receive before you read the result. It is fixed before launch, because the significance math assumes one look at a planned sample.
The spread between pages is large. In hypothetical examples computed with the two-proportion formula, a page converting at 10% that tests for a 30% relative lift needs about 1,800 visitors per variant. A page converting at 2% that tests for a 10% relative lift needs about 81,000. Same tool, same settings, a 45 times difference.
Every variant needs the full sample. A test with a control and three variants needs four times the per-variant number, which is why the number of variants is one of the first things to cut on a low traffic site.
Which inputs does an AB test sample size calculator need?
An AB test sample size calculator needs four inputs: the baseline conversion rate of the control, the minimum detectable effect, the significance level and the statistical power. The first two describe your page and the change you hope to see. The last two set how much risk of a false winner and a missed winner you accept.
- Baseline conversion rate. The current conversion rate of the page or step you test, taken from recent analytics data for the same audience the test will see.
- Minimum detectable effect (MDE). Optimizely defines it as "the smallest improvement you are willing to detect". A relative MDE is a percentage of the baseline (15% of 3.0% is 0.45 points). An absolute MDE is in percentage points.
- Significance level (alpha). The false positive rate you accept. The NIST/SEMATECH e-Handbook describes alpha as "the risk of rejecting a true hypothesis." A 95% significance setting means alpha is 0.05.
- Statistical power. GrowthBook's docs define power as "the probability of observing a statistically significant result" when the change has a real effect. GrowthBook uses 80% by default as of October 2026.
The relative versus absolute setting causes the most input errors. Evan Miller's calculator offers both, with Absolute selected by default as of October 2026.
What is the AB test sample size formula, step by step?
The AB test sample size formula for two conversion rates is n = (z for alpha + z for power)² × (p1(1 - p1) + p2(1 - p2)) / (p2 - p1)², where p1 is the baseline rate, p2 is the baseline plus the minimum detectable effect, and n is the sample per variant. At 95% significance and 80% power the z values are 1.96 and 0.84.
This is the z test approximation for two independent proportions, as written by Hae-Young Kim in Restorative Dentistry & Endodontics (2016). A z value is the number of standard deviations on a normal curve that matches a probability. Hypothetical worked example: a 3.0% baseline and a 15% relative MDE.
- Set p1 and p2. p1 = 0.030. A 15% relative lift gives p2 = 0.030 × 1.15 = 0.0345.
- Add the z values and square them. (1.96 + 0.84)² = 2.80² = 7.84.
- Compute the variance term. 0.030 × 0.970 + 0.0345 × 0.9655 = 0.0291 + 0.0333 = 0.0624.
- Square the difference. (0.0345 - 0.030)² = 0.0045² = 0.00002025.
- Divide and round up. 7.84 × 0.0624 / 0.00002025 ≈ 24,163 visitors per variant, or 48,326 for a two-arm test.
With the same rates, 90% power (z = 1.28) raises the sample to about 32,350 per variant, and a 90% significance level (z = 1.645) lowers it to about 19,030. For very small samples, Kim points to Fisher's exact test instead.
How do baseline rate and minimum detectable effect change the sample size?
Baseline rate and minimum detectable effect change the sample size in opposite directions. A higher baseline lowers the required sample, and a smaller minimum detectable effect raises it sharply: because the difference is squared in the formula, halving the MDE roughly quadruples the visitors each variant needs.
The table applies the two-proportion sample size formula with 95% significance (two-sided), 80% power and an equal traffic split. It is a hypothetical lookup, not benchmark data, and the numbers are visitors per variant.
| Baseline conversion rate | 10% relative MDE | 15% relative MDE | 20% relative MDE | 30% relative MDE |
|---|---|---|---|---|
| 2% | 80,588 | 36,649 | 21,082 | 9,784 |
| 3% | 53,148 | 24,163 | 13,896 | 6,445 |
| 5% | 31,196 | 14,174 | 8,146 | 3,773 |
| 10% | 14,732 | 6,683 | 3,834 | 1,770 |
Two decisions follow from the table. Pick the MDE from business value first (the smallest lift that pays for building and maintaining the change), then check whether your traffic can reach it. Optimizely makes the same trade-off explicit: a "low-MDE experiment would require a larger amount of traffic".
How do you turn sample size into test duration?
You turn sample size into test duration by multiplying the per-variant sample by the number of variants, dividing by the weekly traffic that enters the test, and rounding up to whole weeks. Whole weeks keep weekday and weekend behavior equally represented.
In a hypothetical test with a 3.0% baseline and a 15% relative MDE, 24,163 visitors per variant across two variants is 48,326 visitors. A page with 15,000 eligible visitors a week reaches that in 3.2 weeks, so the test runs four full weeks. Eligible visitors are those who enter the test after targeting.
The Statsig power analysis docs say its results "show the minimum number of weeks needed to reach this MDE for different allocation percentages." Optimizely's MDE article (as of October 2026) works through four variations at about 8,000 visitors each and 10,000 visitors a week: about 3.2 weeks.
What can you do when traffic is too low for an A/B test?
When traffic is too low for an A/B test, change the test design before launch: test a bolder change with a larger minimum detectable effect, move the test to a higher traffic page, cut the number of variants, or measure a primary metric closer to the change. Stopping a test early is not one of the options.
GrowthBook's power docs give the short version for underpowered tests: "Increase your number of weekly users or reduce your number of treatment variations."
- Raise the MDE. At a hypothetical 3% baseline, moving from a 15% to a 30% relative MDE cuts the sample from 24,163 to 6,445 per variant, if the change is bold enough to deliver it.
- Pick a metric closer to the change. Clicks on the tested element convert at a higher rate than purchases, and a higher baseline lowers the sample.
- Use a higher traffic page. A sitewide element or a product page template collects visitors faster than one landing page.
- Cut variants. A control plus one variant needs half the traffic of a control plus three variants.
For a hypothetical site with 3,000 eligible visitors a week and a 3% baseline, a 15% MDE needs 17 full weeks and a 30% MDE needs 5. The conversion rate optimization guide covers research methods for changes too small to test.
How do calculator inputs differ between A/B testing tools?
Calculator inputs differ between A/B testing tools in three ways: whether the minimum detectable effect is relative or absolute, whether power is shown, and whether the math is fixed-horizon or sequential. The same test can produce different sample sizes in different tools, so compare settings before numbers.
| Calculator | Inputs asked | MDE type | Power | Output |
|---|---|---|---|---|
| Evan Miller sample size calculator | Baseline conversion rate, Minimum Detectable Effect, Statistical power, Significance level | Absolute or relative, Absolute by default | Shown as an input | Sample size per variation |
| Optimizely MDE estimate | Baseline conversion rate, statistical significance, MDE, traffic | Relative lift in the docs example | Not stated in the MDE article | Visitors per variation and run time |
| GrowthBook power calculator | Population, up to 5 goal metrics, expected effect size per metric | Effect size per metric | 80% default | MDE and run time from weekly users |
| Statsig Power Analysis Calculator | Metric's historical mean, variance and traffic volume | MDE from metric history | Typically 0.8 | Minimum weeks per allocation |
Optimizely's MDE article lists about 8,000 visitors per variation for a 15% baseline, a 10% lift and 95% significance (as of October 2026). The fixed-horizon formula with 80% power gives 9,244 for the same rates. Optimizely's Stats Engine uses sequential experimentation, not fixed-horizon tests, so the two are not meant to match. For choosing a platform, see the guide to conversion rate optimization tools. Ad platform tests such as Performance Max asset testing and Google Ads Conversion Lift follow the same power logic.
Common mistakes with A/B test sample size calculation
The common mistakes with A/B test sample size calculation come from wrong inputs rather than wrong arithmetic. A mixed-up MDE type, a site-wide baseline or a forgotten variant all return a confident number for a test that cannot detect what it was built to find.
- Entering a relative MDE in an absolute field. Typing 15 as absolute on a 3% baseline asks for a jump to 18%, so the sample comes out far too small.
- Using a site-wide conversion rate. The baseline must match the page and audience in the test. A checkout step and a blog post do not share a rate.
- Forgetting that each variant needs the full sample. Multiply by the number of arms before you estimate run time.
- Choosing the MDE from traffic instead of value. A 40% MDE picked to fit traffic only catches changes few designs deliver. Start from what a lift is worth, then redesign the test if traffic falls short.
- Stopping as soon as the result looks significant. Evan Miller's How Not To Run an A/B Test (2010) puts it plainly: "Decide on a sample size in advance and wait until the experiment is over."
- Counting sessions when users are randomized. Sample size, baseline and results must use the same unit the test assigns to variants.
