What is A/B Testing? | Glossary

Glossary

What is A/B Testing?

A/B testing shows two versions of a page, email or ad to similar groups of people at the same time to see which performs better. How it works and how to run one fairly.

A/B testing, also called split testing, means showing two versions of something to different visitors at the same time and measuring which one gets better results. Version A is usually what you have now; version B changes one thing, such as a headline, button or price display.

How a fair A/B test is set up

  1. Pick one goal. For example, contact form submissions, which is your conversion rate.
  2. Write a hypothesis. "Showing prices on the page will increase enquiries because people stop wondering if we are too expensive."
  3. Split traffic randomly. A testing tool assigns each visitor to A or B and keeps showing them the same version on return.
  4. Run it long enough. At least one or two full weeks, so weekdays and weekends are both included, and until each version has enough conversions to compare.
  5. Check significance. A statistics calculator tells you whether the gap is likely real or just chance.

Running both at the same time matters. Comparing this month's page with last month's mixes in seasons, holidays and ad changes.

Examples that suit a small business

Email is the easiest place to start: Mailchimp, Klaviyo and similar tools can send two subject lines to a slice of your list and send the winner to the rest automatically. On a website, you might test a "Get a quote" button against "See prices", or a short form against a long one. Ad platforms also let you run experiments between two versions of an ad.

When A/B testing is worth it, and when it is not

Honest answer: a site with a few hundred visitors a month and a handful of conversions will rarely reach a trustworthy result on a page test. There, it is better to fix obvious problems directly and watch the numbers. Testing becomes useful when you have steady traffic, often on a high-traffic landing page or a large email list.

Mistakes that ruin tests

  • Stopping when it looks good. Early leads often vanish; decide the sample size before starting.
  • Changing several things at once. If B wins, you will not know which change did it.
  • Testing trivia. Button shade rarely matters as much as the offer, the price or the headline.
  • Hiding content from search engines. Use temporary redirects and canonical tags as Google advises, and do not show crawlers a different page than people see.

Related terms

Ready to put this into practice?

We.Inc lets you describe a site in chat and watch AI build it live, then refine it visually or in code.

Start Building

Free to build, no card needed

Describe your business. Get a website ready to publish.

Ready to simplify everything?

Join 15,000+ businesses that replaced their entire marketing stack with one platform.

Free to build, no card needed · Talk to sales

Product

Who It's For

Features

Resources

Company

View Sitemap