What are they?
A Hypothesis is a scientific statement, used to theorise and explain the relationship between two or more variables. It is used to interpret specific events and help explain these events, behaviours or provide further investigation. In the digital world, we use them to predict user behaviour based on changing one or more variables.
Scientists often refer to them as educated guesses based on the analysis of data and research. Most are tested to some degree through different means to help validate them or further the research. However, it does raise one question. If so much research and data analysis is undertaken, why bother with testing at all?
Scientific rigour dictates it is not the hypothesis that is tested but the null hypothesis. This is a statement that contradicts the original hypothesis. Essentially, we are trying to prove ourselves wrong.
In this way the experimenter bias is removed from the equation. This ensures validity and reliability in the result, which increases the accuracy of your testing.
How are they used in digital?
They are often used informally in many different areas of digital marketing. For example, people undertaking pay-per-click advertising (PPC) use them when testing ad copy. Furthermore, programmatic display marketers use them for creatives and SEO professionals use them to try and identify key aspects of Google’s algorithm.
One area where they have established themselves as a key part of the process is conversion rate optimisation. Experts in this field have been utilising an academic style of methodology since the creation of the field.
One question to ask is are you doing enough to prove your theory wrong?!
How to establish a hypothesis
A well-established hypothesis or null hypothesis will state not only the change and expected outcome, but also the metrics being measured, the limits of the data and the amount of time required.
If you do not have the right metrics, you could create a test that will run forever as you will not know what success is. Below are factors you should know when making an hypothesis:
- Data: what lead you to this theory? what information did you find or analyse to make this assumption? E.g. Because we know users who view the delivery information page convert 10 times greater…
- Change: this is the thing you want to change. E.g. we believe adding the delivery date information to the basket page will…
- Metric: the unit of measurement you will use decide if the change has had the desired effect, this could be anything that indicate better performance and should reflect on the campaigns main KPI. E.g. will improve conversion rate of users…
- Audience: who are the people you are wanting to target and who will be affected by this change? E.g. for all users
- Measurement: what is success? This is the level of improvement needed for us to know the change has had a statistical impact on performance. E.g. we will know this when we see an 11% increase or decrease in conversion rate.
- Timeframe: the length of time the test will run before you can make a decision on the outcome. E.g. and we obtain 4 weeks of data.
We believe our hypothesis template is one of best ways to ensure a test has been planned thoroughly and you have every chance of getting tangible results from.
Once complete your hypothesis should resemble something like:
“Because we saw users who viewed the delivery page convert 10 times greater thank those who did not, we believe adding in the delivery date information to the basket page for all users will significantly improve conversion rate. We will know this when we see an increase of 11% in conversion rate and obtain 4 weeks of data.”
As mentioned before a good scientist will always try to prove their statement wrong and should consider using a Null Hypothesis, for this use the below template.
“Despite seeing users who viewed the delivery page convert 10 times greater thank those who did not, we believe adding in the delivery date information to the basket page for all users will not significantly improve conversion rate. We will know this when conversion rate does not increase or decrease by more than 11% and obtain 4 weeks of data.”
To find out how to find out the minimal detectable difference and sample size required for your test visit our sampling page.