🧪 I just built a mini A/B testing system today to improve the conversion of my sign up modal, here's how I made it and what I learnt: 👇
AI agents that automaticallyincrease MRR, reduce churn, increase CTR, lift conversions, raise trial-to-paid, grow order value
Outtest reads your analytics and payment data, finds where you lose customers, then designs, launches and monitors split tests on your site, pricing and ads. It keeps the winners.
Read-only access to your data. Cancel any time.
Trusted by
Mockingbird
https://yourwebsite.com
PostHog
Analytics
Google Analytics
Analytics
Stripe
Payments
Shopify
Payments
Read-only access. Nothing on your accounts changes.
Not enough traffic? It tests your ads and your search rankings. Visitors don't sign up? It tests the page. Trials don't pay? It tests pricing and the trial itself. Site, pricing and ad tests are all judged on revenue from your payment tool.
Landing page
Judged on revenue per visitor, not signups
Pricing copy
Judged on revenue per visitor
Button placement
Judged on revenue per visitor, not clicks
Trial vs no trial
Judged on revenue per visitor once trials end
Cancel flow
Judged on revenue still coming in after 30 days
Onboarding copy
Judged on revenue per signup
Page layout
Judged on revenue per visitor
Meta ad copy
Judged on revenue per $ spent
Google SEO
Judged on Google clicks, against pages left unchanged
AI SEO
Judged on how often ChatGPT, Claude and Perplexity cite and recommend you
Judged on revenue, not clicks
Most testing tools stop counting at the signup. Outtest follows every visitor through to payment, so a new version only wins if it brings in more money.
Example: a louder headline, per 10,000 visitors
OriginalVersion BChange
Signups310380+70
Paying customers9486-8
Revenue$3,666$3,354-$312
Version B got 70 more signups, but 8 fewer people paid. At $39 a customer that's $312 less. Outtest keeps the original.
7 days
Every test runs at least a full week, and until each version has enough visitors to trust.
90%
The new version needs a 90% chance of beating the original.
+10%
And a lift of at least 10%. Smaller wins aren't worth the change.
If a test hasn't cleared all three after six weeks, Outtest calls it a draw, keeps your original and moves on to the next idea. Nothing ships because of one lucky week. These are the defaults. Outtest recommends settings that fit your traffic, and you can move all three in Settings.
Agents with good manners
Letting software change your live site is a big ask. These are the rules Outtest works by, on every plan.
Read-only data
Outtest can read your payments and analytics. It can't move money, issue refunds or edit your account.
You choose how hands-on
Approve every change before it goes live, or switch on autopilot and read the weekly email.
No made-up claims
Every new version is checked before launch. Agents can't invent a discount, a guarantee or a review.
Lift in money, not just percent
A small share of visitors always sees your original site, so Outtest can show what the winners earned.
10,000+
experiments a year at each of Microsoft, Amazon, Booking.com, Facebook and Google.
Harvard Business Review, 20171,000+
experiments running at the same time at Booking.com, every day.
Booking.com engineering paper, 20178 in 10
experiments at Google and Bing don't win. Even the best teams can't guess, so they test.
Harvard Business Review, 2017+12%
revenue from one Bing ad headline test that had been marked low priority. Over $100M a year in the US.
Harvard Business Review, 2017Neil Patel has a simple A/B test significance calculator, you add the numbers in there and it'll tell you if it's actually not random what you're seeing: neilpatel.com/ab-testing-cal… In this case, it's statistically significant that the modal with the testimonial converts better 🥳
Seems like it, here’s more results. Now that I can actually A/B test thumbnails I don’t have to guess and I just test and see what people want. My mouth would have closed years ago if I had this tool 😭
Many think that "41 shades of blue" was an example of analysis-paralysis, including @tomfishburne recently at marketoonist.com/2019/10/testin… The reality is that color scheme optimization was a huge win at both Google and Microsoft/Bing. See my Quora answer at quora.com/How-much-did-G…
At @duolingo, we ran 100s of A/B tests on our notifications. We learned a ton from these experiments. Here are our top 5 tips for writing copy that converts to DAUs 🧵
My friend @nevmed wrote a tweet about how I “split tested” my way to the color of the cover for Million Dollar Weekend. Here's a deep dive into how we used A/B testing and data to iterate our way to the current cover (and how you can apply it for your business). 1- Finish Show more
LOVE your help on which SUBTITLE for my book? 1- The surprisingly simple way to launch a 7-figure business in 48 hrs 2- Overcome your fear and launch a 7-figure business in 48 hrs 3- Awaken Your Creator’s Courage, Launch Your Dream Business & Create a Life You Love in 48 Hrs
I have a confession: Part of my intention promoting this paywall last year was to normalize weekly subscriptions Sorry, not sorry. I noticed the trend early in the year and tested it out for myself The result: Users preferred weekly subscriptions (better conversions) and it Show more
working on the same update for @SkipCalls thanks @adamlyttleapps for inspiration!
Public posts about split testing in general. Showing them doesn't mean these people use or endorse Outtest.
Pick how many tests you want each month
Every plan gets every feature. The only difference is how many tests go live each month.
Every plan includes
- One site
- Every agent
- Website, pricing, onboarding, cancel flow and ad tests
- Google SEO and AI SEO tracking
- Results judged on revenue
- Weekly email report
- Cancel any time
Questions
What do I need to connect?+
Your website address is enough to start. Connecting analytics (PostHog, Google Analytics, Plausible and others) shows the agents where visitors drop off. Connecting payments (Stripe, Polar, Whop, Shopify and others) lets them judge tests on paying customers. Each takes about a minute and is read-only.
How do changes reach my site?+
One small script, added with one click through Google Tag Manager, our Shopify, Webflow, Wix or WordPress apps, or by pasting one line. Bigger changes, like a new trial length, go through a GitHub pull request you approve.
How much traffic do I need?+
Less than you think, if the tests are bold. With 2,000 visitors a week and a 5% signup rate, a test can detect a 30% lift in about a month. Smaller sites get fewer, bigger tests, plus ad tests where traffic is cheap.
Will it slow my site down?+
The script is small and loads from a global edge network. It picks the version a visitor sees in the browser, with no extra trip to a server.
How do Google SEO and AI SEO tests work?+
Nobody can show Google or ChatGPT two versions of the same page at once. Outtest changes a group of pages and leaves similar pages alone. It then compares Google clicks and mentions in ChatGPT, Claude and Perplexity between the two groups.