A/B testing for SaaS

SaaS is one of the easiest businesses to split test well, because most of your revenue passes through a handful of pages you control: the homepage, the pricing page, the signup form and the cancel flow. The catch is that money arrives late. A trial that starts today might pay in 14 days or never, so the tests that look best on signups are often not the ones that make more money.

Start with the pricing page and the trial offer, judge every test on revenue per visitor, and only test changes big enough for your traffic to measure.

Why SaaS testing is different

Three things set SaaS apart from a store or a course business.

Revenue is delayed and recurring. A visitor who starts a trial has earned you nothing yet. If you call a test after a week, you are judging trial starts, not payments. Your test window needs to cover the trial length plus the time people take to decide.

Signups mislead more often than they help. Removing the card requirement from a trial, cutting a form field or adding a free plan all tend to raise signups. Whether they raise paying customers is a separate question, and sometimes the answer is no. This is why revenue per visitor is the number to judge on.

Cancellations are a funnel too. For a subscription business, a saved cancellation is worth as much as a new customer. The cancel flow is easy to forget, and every subscription it saves is revenue you had already won.

The tests to run first

Run these roughly in order. Each one changes something a buyer notices, which matters when your traffic can only detect big differences.

1. Show annual billing first

Set the billing toggle to annual by default and show the annual price as a monthly figure, with the yearly total underneath. Measure revenue per visitor over the test and the share of new customers on annual plans. For the setup, see how to A/B test pricing.

2. Trial versus no trial

Compare your current trial against paying on day one, or a 14-day trial against a 7-day one. Measure revenue per visitor, and keep counting until the last trial in the test has ended. See free trial vs no trial for how to read the result.

3. Card upfront versus no card

Ask for a card at trial start in version B. Expect fewer trials. Judge it on paid customers per visitor and revenue per visitor, never on trial starts.

4. Move the recommended plan

If the "Most popular" badge sits on your cheapest plan, move it to the middle one. Measure revenue per visitor and the plan mix. A lower paid rate with a higher average plan can still win.

5. Add a pause option to the cancel flow

Offer a one to three month pause before the final cancel button. Measure revenue retained per customer who starts the cancel flow, over at least 60 days. A pause keeps the price intact, which makes it a safer first test than a discount. More in the cancel flow guide.

6. Rewrite the homepage headline around the outcome

Replace a feature headline with one that names who it is for and what changes for them. Measure revenue per visitor, not clicks on the signup button. More headline tests are in the landing page guide.

7. Cut the onboarding to one first task

Change the first screens after signup so they point at a single task that shows the product working. Measure trial-to-paid revenue per new signup.

How much traffic you need

Here is a made-up example to show the maths. Your pricing page gets 30,000 visitors a month, about 1,000 a day, and 3% of them become paying customers.

A standard sample size calculation (95% confidence, 80% power) says:

So a test can confirm a 20% change in about four weeks. A 10% change takes over three months, which is longer than most tests should run. At this traffic, test changes that could plausibly move revenue by 20% or more, such as price, plan structure and trial terms. A new button color will not register.

If your trial is 14 days, add those 14 days on top, because revenue from the last visitors in the test arrives after their trial ends. Plug your own numbers into the sample size calculator or the test duration calculator.

Common mistakes

• Calling a winner on trial starts or signups. Wait for payments.
• Stopping the moment version B looks ahead. Early leads flip often. Check the result in the significance calculator only once the planned sample is in.
• Letting a visitor see one price on Monday and another on Thursday. Keep each visitor on the same version.
• Testing prices on logged-in customers. Test on new visitors and honor every existing price.
• Running three tests on the pricing page at once with 1,000 visitors a day. Each one gets a third of the traffic and none finishes.
• Ignoring a broken split. If version B gets noticeably more or fewer visitors than planned, run the numbers through the SRM checker before you trust the result.

How Outtest fits

Outtest connects read-only to your billing tool (Stripe, Paddle, Chargebee, Polar, Lemon Squeezy and others) and your analytics, finds where the funnel loses the most money, then designs, launches and monitors the tests above. It judges every website and pricing test on revenue per visitor from your billing data, and shows signups only for context.

A version wins only after at least 7 days, a 90% chance of beating the original and a lift of at least 10%, all adjustable in Settings. A test that hasn't cleared all three by day 42 ends as a draw. Changes to pricing, checkout and cancel flows always wait for your approval, and bigger changes like trial length arrive as a GitHub pull request you merge yourself. After a win, 5% of visitors keep seeing the original, so you can see what the winner actually earned.