How to A/B Test Landing Pages: A Step-by-Step Guide for Higher Conversions

Most A/B tests fail because people skip the boring parts. They swap a button color, wait three days, declare a winner, and ship it. Then they wonder why their conversion rate hasn't moved in six months.

A real test takes planning. Here's the full process, from picking what to test to deciding what to do with the results.

Step 1: Confirm you have enough traffic to test

Before you touch anything, check your math.

A/B testing needs volume. If your landing page gets 200 visitors a week and converts at 3%, you're looking at six conversions per week per variant. You'll need months to reach statistical significance, and by then your traffic mix will have shifted enough to make the test meaningless.

A rough rule: you want at least 1,000 conversions per variant for a clean test on small effects (10% lift or less). For bigger swings (30%+ lift), a few hundred conversions per variant can work.

If you don't have that traffic, A/B testing is the wrong tool. Use heatmaps and session recordings instead. Qualitative data scales down. Statistics don't.

Step 2: Find something worth testing

The biggest mistake in CRO is testing trivial stuff. Button color. Headline font. Whether the form is on the left or right.

These tests rarely produce meaningful lift, and even when they do, the impact is too small to detect without massive traffic.

Test things that change how visitors think:

• The offer itself: free trial vs free demo vs free audit
• The headline's promise: outcome-focused vs feature-focused
• The hero structure: long-form value prop vs short tagline
• Form length: 3 fields vs 7 fields
• Social proof type: logos vs testimonials vs numbers
• Pricing presentation: monthly vs annual default, with or without anchoring

If you're not sure what to test first, start with your hero section. It's the highest-leverage real estate on the page.

Step 3: Form a hypothesis, not a guess

A hypothesis has three parts:

1. What you're changing
2. Why you think it'll work (based on evidence)
3. What metric you expect to move

Bad: "I think a green button will convert better."

Good: "Session recordings show 40% of visitors scroll past the CTA without clicking. Replacing the generic 'Sign Up' button with 'Start My Free Audit' should increase CTA clicks because it tells visitors what they get instead of what they do."

The "why" matters. If you can't explain why a change should work, you're guessing. Guessing is fine for brainstorming, but it produces noise when you scale it across dozens of tests.

Step 4: Pick one primary metric

Every test needs one number that decides the winner. Not two. Not "we'll look at a few things." One.

For most landing pages, that's the conversion rate on the primary CTA. Signups. Demo requests. Purchases. Whatever the page exists to drive.

Secondary metrics (bounce rate, time on page, scroll depth) are useful for diagnosis after the test ends, but they don't pick the winner. If you let yourself optimize for whichever metric happens to move, you'll convince yourself every test is a win.

Step 5: Calculate the sample size before you start

This is the step everyone skips. Don't skip it.

Use a sample size calculator (Optimizely and VWO both publish free ones). Plug in:

• Your current conversion rate
• The minimum lift you care about detecting (usually 10-20%)
• Statistical significance threshold (95% is standard)
• Statistical power (80% is standard)

The calculator tells you how many visitors per variant you need. Now divide that by your daily traffic to estimate test duration.

If the math says you need 8 weeks, run it for 8 weeks. If you stop at week 3 because the variant is "clearly winning," you'll get burned by false positives. Early results lie constantly.

Step 6: Build the variant cleanly

A few rules:

Change one thing at a time. If you change the headline AND the CTA AND the hero image, you won't know which one moved the needle. The exception: redesign tests, where you're testing two corely different approaches against each other on purpose.

Match everything else exactly. Same load speed. Same fonts. Same tracking. If the variant is slower because you added a new image, you're testing image vs speed, not headline A vs headline B.

QA on mobile and desktop. Test the test before you launch it. Half of broken A/B tests have a layout bug on mobile that nobody caught.

Step 7: Split traffic evenly and randomly

50/50 splits are the default for two reasons: they reach significance fastest, and they're easiest to interpret.

Your testing tool should handle the randomization. What you need to verify:

• Each visitor sees the same variant on every visit (cookie-based assignment)
• Traffic sources are split evenly across variants (no accidental bias from a campaign hitting only one version)
• Both variants are live at the same time (never run them sequentially, traffic patterns change)

Step 8: Let the test run its full duration

Once it's live, don't peek. Or rather, peek but don't act.

Running a test for the full calculated duration matters because:

• Weekend traffic behaves differently from weekday traffic
• New visitors behave differently from returning ones
• Campaign traffic spikes can skew early results

Minimum duration for any test: two full business cycles, usually two weeks. Even if your sample size hits significance on day 4, keep going. Early significance is often noise.

Step 9: Read the results properly

When the test ends, look at:

Statistical significance: Is the result notable at 95% or higher? If not, you don't have a winner. You have a tie.

Confidence interval: This tells you the range of plausible true lifts. A "12% lift with a confidence interval of -2% to +26%" is not a win. The true effect might be negative.

Segment breakdown: Did mobile and desktop behave the same way? Did the variant win across all traffic sources? A variant that wins overall but loses badly on mobile is a problem, not a victory.

If the result is flat (no notable difference), that's still useful information. It means the thing you changed doesn't matter for conversions. Move on to the next hypothesis.

Step 10: Ship the winner, document everything

When you have a real winner, roll it out to 100% of traffic. Then write down:

• The hypothesis
• The change you made
• The result (with numbers)
• What you learned

This document compounds. After 20 tests, you'll have a playbook of what works for your audience that no consultant can sell you.

Common mistakes to avoid

Testing during a campaign launch. New traffic sources skew results. Test on stable traffic.

Running multiple tests on the same page simultaneously. They interact in ways that break the math.

Calling a winner based on a 5% lift with 200 visitors. That's noise, not signal.

Ignoring negative tests. Losing variants teach you what doesn't work. Document them.

Re-running a "close" test hoping for a different result. This is p-hacking. If your test was inconclusive, the answer is "it doesn't matter much," not "let's try again."

What to do when you can't run real A/B tests

Most landing pages don't have enough traffic for proper testing. If that's you, here's the better workflow:

1. Use heatmaps and recordings to find UX problems
2. Apply known best practices for improving conversions
3. Make several changes at once based on evidence
4. Compare before/after over a longer window

It's not as clean as A/B testing, but for low-traffic pages it produces faster results than waiting six months for one statistically notable test.

---

Want to know what to test before you set up the experiment? PagePulse analyzes your landing page and identifies the highest-impact issues to fix, so you spend your testing budget on changes that actually matter. Drop in your URL and see what's costing you conversions.