A/B testing, also called split testing, means showing two versions of something to different visitors at the same time and measuring which one gets better results. Version A is usually what you have now; version B changes one thing, such as a headline, button or price display.
How a fair A/B test is set up
- Pick one goal. For example, contact form submissions, which is your conversion rate.
- Write a hypothesis. "Showing prices on the page will increase enquiries because people stop wondering if we are too expensive."
- Split traffic randomly. A testing tool assigns each visitor to A or B and keeps showing them the same version on return.
- Run it long enough. At least one or two full weeks, so weekdays and weekends are both included, and until each version has enough conversions to compare.
- Check significance. A statistics calculator tells you whether the gap is likely real or just chance.
Running both at the same time matters. Comparing this month's page with last month's mixes in seasons, holidays and ad changes.
Examples that suit a small business
Email is the easiest place to start: Mailchimp, Klaviyo and similar tools can send two subject lines to a slice of your list and send the winner to the rest automatically. On a website, you might test a "Get a quote" button against "See prices", or a short form against a long one. Ad platforms also let you run experiments between two versions of an ad.
When A/B testing is worth it, and when it is not
Honest answer: a site with a few hundred visitors a month and a handful of conversions will rarely reach a trustworthy result on a page test. There, it is better to fix obvious problems directly and watch the numbers. Testing becomes useful when you have steady traffic, often on a high-traffic landing page or a large email list.
Mistakes that ruin tests
- Stopping when it looks good. Early leads often vanish; decide the sample size before starting.
- Changing several things at once. If B wins, you will not know which change did it.
- Testing trivia. Button shade rarely matters as much as the offer, the price or the headline.
- Hiding content from search engines. Use temporary redirects and canonical tags as Google advises, and do not show crawlers a different page than people see.
Related terms
- Conversion rate: the usual thing being tested.
- Call to action: a frequent test subject.
- Email marketing: subject line tests built into most tools.
- Growth hacking: rapid experimentation that leans on tests.