Checkout A/B testing for e-commerce and SaaS

Checkout A/B testing should start from why people leave. In most stores the biggest reasons are costs that appear late, being forced to create an account, a long form, a missing payment method and not trusting the page with a card. Test changes that remove those, split visitors at random, and judge each test on revenue per visitor, with average order value and refunds alongside, not on checkout completion alone.

Checkout is also the riskiest page to test, because a bug there loses real orders within minutes. Test every payment method and device before launch, start with a small share of traffic, and check daily that orders, payment failures and the traffic split look normal. The payoff can be large. Baymard Institute estimates that the average large e-commerce site could raise its conversion rate by 35.26% through better checkout design.

Why checkout tests are different

• They sit next to the money. A broken address field or a payment button that fails on one browser loses orders within minutes.
• They change order value as well as conversion. Shipping thresholds, upsells and plan choices move both, so conversion rate alone can point the wrong way. Revenue per visitor catches both.
• Fewer visitors reach checkout than land on your pages, so tests there take longer to reach an answer.
• Hosted checkouts limit what you can change. On Shopify or Stripe Checkout, many tests happen on the cart page, in the steps before checkout, or through settings. See A/B testing on Shopify and A/B testing with Stripe.

Start from why people abandon

Baymard's abandonment research puts the average documented cart abandonment rate at 70.22% across 50 studies. It also asked US online shoppers why they abandoned. After leaving out the 42% who were just browsing, the top reasons map to tests:

Baymard also found the average US checkout shows 23.48 form elements by default, and that an ideal flow can use 12 to 14. Counting your own fields is a quick way to find a test.

A worked example

This is a made-up example. A store tests its free shipping threshold at $50 (A) against $75 (B), with 30,000 visitors per version over four weeks. Shipping costs the store $7 per order it ships free, and gross margin on products is 50%.

Here is the maths. A's revenue is 840 × $62 = $52,080 and B's is 780 × $71 = $55,380. B converts 7% fewer visitors and earns 6.3% more per visitor, because people add items to reach the higher threshold. Gross profit is 50% of revenue minus free shipping: $26,040 − 588 × $7 = $21,924 for A and $27,690 − 351 × $7 = $25,233 for B, 15% more.

Is the revenue difference real? Suppose order values vary with a standard deviation of about $40 in A and $45 in B. The variance of revenue per visitor is then about 149 in A and 180 in B. The standard error of the difference is the square root of (149 ÷ 30,000 + 180 ÷ 30,000), about $0.105. The gap is $0.11, which gives B roughly an 85% chance of earning more per visitor. That is encouraging but not decisive, so the test should keep running. The revenue per visitor calculator runs this check for you.

A test judged on conversion rate would have picked A and left profit on the table.

Show the full price

Hiding costs until the last step feeds the top reason shoppers give for leaving. In the UK it can also break the rules. The UK regulator's price transparency guidance says the total price should normally include any mandatory charges, and that mandatory delivery charges must be included in the invitation to purchase. This is general information, not legal advice, but it means a test of "fees revealed at the end" is not a test worth running in the UK.

The same goes for discount claims at checkout in the EU, where a "was" price must be the lowest price in at least the previous 30 days. The pricing guide covers the rules for price tests.

E-commerce checkout tests to try

1. Shipping cost shown on product pages vs only at checkout. Measure revenue per visitor.
2. Free shipping threshold at two levels. Measure revenue per visitor, average order value and gross profit per visitor.
3. Guest checkout as the default vs account creation first. Measure checkout completion and revenue per visitor.
4. Express wallet buttons at the top of checkout vs at the payment step. Measure revenue per visitor on mobile.
5. Delivery date ("Arrives Thursday") vs delivery range ("3 to 5 days"). Measure revenue per visitor.
6. Order bump at checkout (a small add-on) vs none. Measure revenue per visitor and checkout completion.
7. One-page checkout vs multi-step. Measure checkout completion and revenue per visitor.
8. Discount code field visible vs behind a link. Measure revenue per visitor, since a visible field can send people off to search for codes.

SaaS checkout tests to try

1. A plan summary that says what happens next ("You'll be charged $49 today, then monthly") vs the default. Measure revenue per visitor and refunds.
2. Annual upsell on the checkout page vs none. Measure revenue per visitor at 30 and 90 days. See annual vs monthly pricing.
3. Tax shown before checkout vs added at checkout. Measure revenue per visitor.
4. Invoice payment offered for higher plans vs card only. Measure revenue per visitor on those plans.
5. Seats pre-set to 1 vs the number the buyer entered earlier. Measure revenue per visitor and seats per purchase.
6. Coupon field shown vs hidden. Measure revenue per visitor and average discount given.

How to run a checkout test safely

1. Test every payment method, browser and device in both versions before launch, including a real purchase and refund.
2. Start with 10 to 20% of traffic for the first day or two, then move to an even split once orders look normal.
3. Check the traffic split with the sample ratio mismatch checker. A version that errors for some visitors will quietly lose them.
4. Watch payment failures, error logs, refunds and support tickets every day.
5. Don't start a checkout test just before a big sale, and don't stop one early because it looks good on day three. The peeking problem guide explains why early stops mislead.

Outtest treats checkout as a protected area. Changes to checkout always wait for the owner's approval before going live, even when everything else runs on autopilot, and every test is judged on revenue per visitor from your payment tool.