How to A/B test CTA buttons (copy, placement and color)

When you A/B test a call to action button, test what the button says and what happens after the click first, then where the button sits, and color last. Copy and the offer behind the button change what visitors think they're agreeing to. Placement changes how many people see the button at the right moment. Color changes how much the button stands out, which matters only until it clearly stands out.

Be honest with yourself about color. The well-known color wins came from Google and Bing, which can detect changes of a fraction of a percent because they test on millions of users. On a site with thousands of visitors a month, a color test will almost always end in a draw. Check that your button has strong contrast with the page, then spend your testing traffic elsewhere.

What tends to move results

The sizes are rough judgments, not measured benchmarks. The point is the order. The closer a change is to what the visitor gets, the more it can move.

The honest answer on button color

The case for color testing rests mostly on very large companies:

• Douglas Bowman, a designer who left Google in 2009, wrote that a team there couldn't decide between two blues and tested 41 shades between them.
• In Kohavi and colleagues' rules of thumb paper, Bing changed a few font colors on its results page. The change improved revenue by over $10 million a year, and Bing replicated the result with 32 million users. Another Bing experiment, with over 10 million users, changed the ad background color and cut revenue by 12%.

These are real effects, measured on tens of millions of people, on a page where the colors mark ads and results. They don't transfer to a single button on a landing page.

The same paper is blunt about the popular "red beats green" button case study. The authors wrote that they don't see many sites with red call to action buttons and believe it is not a general result that replicates well.

What does matter is contrast. The WCAG accessibility standard asks for text contrast of at least 4.5:1 for normal text and 3:1 for the visual boundaries of interface components such as buttons. If your button fails those, fix it without a test. If it passes, its exact hue is unlikely to be your biggest problem.

A worked example on traffic

This is a made-up example. A page gets 20,000 visitors a month and 4% of visitors buy. A standard shortcut for sample size, at 80% power and 5% significance, is 16 × p × (1 − p) ÷ d² per version, where d is the difference in conversion rate you want to detect.

A color change might plausibly lift conversion by 2%, from 4.00% to 4.08%:

• d = 0.04 × 0.02 = 0.0008
• 16 × 0.04 × 0.96 = 0.6144
• 0.6144 ÷ 0.0008² = 960,000 visitors per version

At 10,000 visitors per version per month, that is eight years.

A copy change that tells people exactly what happens next might plausibly lift conversion by 15%, from 4.0% to 4.6%:

• d = 0.04 × 0.15 = 0.006
• 0.6144 ÷ 0.006² = about 17,000 visitors per version

That is about seven weeks. The copy test can give you an answer this quarter. The color test can't give you one this decade. The minimum detectable effect calculator shows the smallest lift your own traffic can detect in a set number of weeks.

Copy tests

Good button copy tells people what they get and what happens next. Things to vary:

• Specific vs generic: "Start my 14-day trial" vs "Get started".
• What they get vs what they do: "Get my quote" vs "Submit".
• First person vs second person: "Start my trial" vs "Start your trial". This is a small change, so expect a small effect.
• Commitment level: "See pricing" vs "Buy now". A softer ask may get more clicks, and the test tells you whether it gets more buyers.
• Price or time on the button: "Start for $12" or "Set up in 5 minutes", if true.

Put risk reducers next to the button only when they're true: "No card required", "Cancel any time", "Free returns". A false reassurance can win a test and then cost you in refunds and complaints.

Placement tests

• Button on the first screen vs after the proof section. Visitors who need convincing may not be ready at the top.
• A sticky button on mobile that stays at the bottom of the screen vs none.
• The main button repeated after each major section vs once at the end.
• One main action vs two, such as "Start trial" and "Book a demo" side by side.

Outtest tests button placement like any other page change and judges it on revenue per visitor from your payment tool.

Measure revenue, not clicks

Button tests are where click metrics mislead most, because the button is the thing being clicked. In the Kohavi paper, Microsoft Office Online tested a redesign with a strong call to action that also showed the price. Clicks per user fell 64%. The team had assumed clicks would convert at a steady rate, but the people who still clicked were better qualified and bought at a much higher rate. Judging on clicks would have picked the wrong page.

So judge on revenue per visitor, and use clicks only to explain the result. If a version wins clicks and loses revenue, the button is promising something the next page doesn't deliver.

Tests to try, and what to measure

1. Specific copy ("Start my 14-day trial") vs generic ("Get started"). Measure revenue per visitor.
2. "See pricing" vs "Start trial" as the main action. Measure revenue per visitor and trial-to-paid rate.
3. A true risk reducer under the button vs none. Measure revenue per visitor.
4. Sticky mobile call to action vs none. Measure revenue per visitor on mobile.
5. Button on the first screen vs only after the proof section. Measure revenue per visitor.
6. One main action vs two competing ones. Measure revenue per visitor.
7. A low-contrast ghost button vs a solid high-contrast one, if your current button fails contrast. Measure revenue per visitor, or just fix it.
8. Color with equal contrast, only if you have hundreds of thousands of visitors a month. Measure revenue per visitor and expect a draw.

If your traffic is small, A/B testing with low traffic explains how to choose changes big enough to detect. For where buttons fit among everything else on a page, see landing page A/B testing and what to A/B test first.