Conversion

How to A/B Test a Landing Page With Low Traffic

Most SaaS landing pages don’t get enough visitors for a classic A/B test. How many visitors you really need, and what to test instead when traffic is low.

How to A/B Test a Landing Page With Low Traffic

Key takeaways

  • Detecting a small lift takes tens of thousands of visitors per variant. Most B2B pages never get there.
  • With low traffic, test big changes (offer, headline, page structure) instead of button colours.
  • Choose one primary metric and a stopping rule before the test starts.
  • When you can’t test, ship the change, watch the trend and talk to users.
On this page
  1. How much traffic an A/B test really needs
  2. Test bigger changes
  3. Use a metric closer to the page
  4. Run tests longer, but not forever
  5. When to skip the test

A/B testing advice is usually written for sites with millions of visitors. A B2B SaaS landing page might get 3,000 a month. Run the same playbook on that and you’ll wait months for a result you can’t trust.

The short answer: with low traffic, test fewer and bigger changes, measure something closer to the page, and accept that some decisions are better made without a test.

How much traffic an A/B test really needs

The number of visitors you need depends on your current conversion rate and the size of the change you want to detect. Here is what it looks like for a page converting at 3%, at the usual 95% confidence and 80% power:

Lift you want to detect Conversion rate Visitors per variant
10% relative 3.0% → 3.3% about 53,000
20% relative 3.0% → 3.6% about 14,000
50% relative 3.0% → 4.5% about 2,500

At 3,000 visitors a month split across two variants, the first row takes about three years. The last row takes under two months. That is the whole argument of this article in one table.

Test bigger changes

Small tweaks produce small lifts, and small lifts need huge samples. With low traffic, only test changes big enough to move the number by 30% or more:

  • A different offer: demo vs. free trial vs. free check.
  • A new headline that makes a different promise.
  • A shorter page with one call to action instead of three.
  • Social proof above the fold vs. none.

Leave button colours and microcopy to sites with the traffic to measure them.

Use a metric closer to the page

If your goal is paid customers but only 20 sign up each month, you’ll never see a difference. Measure the step the page controls instead: clicks on the main call to action, form starts or demo requests. Those happen more often, so you reach a result sooner.

Run tests longer, but not forever

Run every test for at least one full week so each weekday is included, and set the sample size before you start. Stop when you reach it, not when the graph looks good. Checking every day and stopping at the first green number is the most common way to fool yourself.

When to skip the test

Some changes don’t need a test. If a pricing page is missing the answers buyers ask for, add them. Our pricing page FAQ guide lists twelve. Ship clear improvements, watch the trend for a few weeks and talk to five customers. Save testing for the decisions that are close calls.

Centipage’s A/B Tester sets up experiments like these on code-based sites and reports results with a confidence score, so you know when a result is real. See how it works.

Frequently asked questions

What is the minimum traffic for an A/B test?

It depends on your conversion rate and the lift you want to detect. At a 3% conversion rate, detecting a 20% relative lift needs roughly 14,000 visitors per variant, while a 50% lift needs about 2,500.

How long should an A/B test run?

At least one full week, so every weekday is included, and until you reach the sample size you planned. Two to four weeks is typical for B2B pages.

Can I trust a test that reached significance after two days?

Usually not. Early results swing a lot, and checking repeatedly until something looks significant inflates false positives. Decide the sample size first and judge the test when you reach it.

Nik Zechner

Founder of Centipage. For 8 years he has designed and shipped websites for startups like Digimarc, Almetra AI and MarvelX. Why he’s building Centipage.

Your website, 1% better every day.

Centipage’s agents find what to improve on your site, make the change and wait for your approval before it goes live.