Writing a good test hypothesis
The four components
Problem: what do you see happening? Preferably something you can point to, in your web statistics, in questions to customer service or in reviews.
Change: what do you adjust to solve that problem?
Expected effect: what will the visitor do differently as a result?
Measurable goal: how can you tell? This is a goal you have set up under Goals.
Formulating a hypothesis
Put the four components in one sentence using this template:
Because we see that …, we expect that … will lead to …, which we measure by …
In the row of your A/B test, click ⋮ and then Settings.
Enter the hypothesis on the Basic tab in the Description field.
Examples
Too vague: The homepage needs to look nicer. The problem, the change and a way to measure nicer are missing.
Only a solution, no problem: We are making the button green. The reason for green is missing. Better version: Because the button View the collection does not stand out against the photo of the bedding, we expect that a green button will lead to more clicks to the collection and more orders, which we measure by the goal Order completed.
Too much at once: We are changing the heading, the photo and the button on the homepage. If the variant wins, you do not know which of the three had the effect.
Good: Because visitors often ask in reviews how heavy the wool throw is, we expect that adding the weight to the product title will lead to less doubt and more orders, which we measure by the goal Order completed.
One change per variant
Note: The more variants, the smaller each group of visitors, and the longer it takes before you can draw a conclusion.
Prioritising ideas
Impact: how many visitors see it, and how close is it to the purchase? A button on the product page scores higher than a text in the footer.
Confidence: how sure are you that it works? A problem that customers mention themselves scores higher than a gut feeling.
Ease: how quickly can you build it? A text change in the editor scores higher than a new page.