Two versions, split traffic, and reading it honestly
How a split test runs, and the part most guides skip: what the numbers mean when a hundred people saw the page rather than a hundred thousand.
Watch
What it does
How a test runs
Traffic splits between two versions
Visitors are shown one or the other, roughly evenly. Nobody sees both, so what they do is attributable to the version they got.
You change one thing
One headline, one button, one image. Change two and a result tells you something worked without telling you which.
It counts the action, not the visit
The measure is how many people did the thing the page exists for, which is the only number worth acting on.
How it works
Running one
- Pick the single element you think matters most. Usually the headline.
- Build the second version with only that changed.
- Run it until each version has had a few hundred visitors, not a few dozen.
- Keep the winner only if the gap is large. Otherwise call it a draw and move on.
Worth knowing
Worth knowing
- At small traffic this mostly measures noise. Eleven against eight is not a result, however much it looks like one.
- Most local businesses do not have the visitors for a fair test. Better copy usually beats better testing.
- A test left running forever is a decision never made. Give it an end date before you start it.
Related
Related
Website and funnel builder
Building the pages you are testing.
Building a funnel
The sequence a tested landing page usually sits in.
Forms and surveys
The form whose submissions you are counting.
Test one headline
Log in, duplicate your busiest page, change only the headline, and give the test an end date.