Writing a good test hypothesis

Before and after: a size chart below the buy button of the linen duvet cover
A test hypothesis describes in advance which problem you see, what you change and which measurable effect you expect. With a hypothesis, every test teaches you something about your visitors, even if the variant loses. Without a hypothesis, you do not know why a winning variant won, and a losing variant teaches you nothing. Writing a hypothesis takes about five minutes.

The four components

A hypothesis answers four questions:
  • Problem: what do you see happening? Preferably something you can point to, in your web statistics, in questions to customer service or in reviews.

  • Change: what do you adjust to solve that problem?

  • Expected effect: what will the visitor do differently as a result?

  • Measurable goal: how can you tell? This is a goal you have set up under Goals.

Formulating a hypothesis

  • Put the four components in one sentence using this template:

    Because we see that …, we expect that … will lead to …, which we measure by …

  • In the row of your A/B test, click ⋮ and then Settings.

  • Enter the hypothesis on the Basic tab in the Description field.

This way, everyone in your team can still see later why the test was run.
Example from the demo shop:
Because we see that customers often ask customer service whether the linen duvet cover fits their duvet, we expect that a size chart directly below the buy button will lead to more orders of that duvet cover, which we measure by the goal Order completed.

Examples

  • Too vague: The homepage needs to look nicer. The problem, the change and a way to measure nicer are missing.

  • Only a solution, no problem: We are making the button green. The reason for green is missing. Better version: Because the button View the collection does not stand out against the photo of the bedding, we expect that a green button will lead to more clicks to the collection and more orders, which we measure by the goal Order completed.

  • Too much at once: We are changing the heading, the photo and the button on the homepage. If the variant wins, you do not know which of the three had the effect.

  • Good: Because visitors often ask in reviews how heavy the wool throw is, we expect that adding the weight to the product title will lead to less doubt and more orders, which we measure by the goal Order completed.

One change per variant

Test one change per variant. Then you know what made the difference, and you can apply that lesson to other pages. To compare several ideas, create more variants in the same test: each variant tests one idea.
An A/B test with several variants

Note: The more variants, the smaller each group of visitors, and the longer it takes before you can draw a conclusion.

Prioritising ideas

Give each idea three scores from 1 to 10:
  • Impact: how many visitors see it, and how close is it to the purchase? A button on the product page scores higher than a text in the footer.

  • Confidence: how sure are you that it works? A problem that customers mention themselves scores higher than a gut feeling.

  • Ease: how quickly can you build it? A text change in the editor scores higher than a new page.

Add up the scores and start at the top of the list. This way you spend your testing time on ideas that can deliver the most and go live quickly.

After the test

Write down the outcome next to your hypothesis: confirmed, rejected or no clear difference. A rejected hypothesis also gives you knowledge about your visitors, and makes your next hypothesis better.

See also


Copyright © 2026 Ternair.