The final step in the process of running a trial is working out what to do with the results.
An effect size (for example, a two percentage point increase in take-up) isn’t always the easiest thing to interpret or communicate to others. What we really want to know is the impact in meaningful, real-world terms.
Translate your results into meaningful impacts
Let’s imagine that you’ve tested a new two-year-old offer communication and observed a three percentage point increase in take-up. What does this mean in practice?
Go through these three steps to translate your results into a meaningful long-term estimate of impact:
- Develop some assumptions
- Estimate the overall impact
- Calculate the costs and benefits
The very final stage of the process is deciding whether or not to implement your intervention in the future or not. Whether or not you have a statistically significant result will play a big role here, but it is not the only consideration.
Some other questions to consider are:
- What was the effect size? Is it meaningful in practice?
- Are the benefits of the intervention worth the costs?
- What feedback did we get from parents and staff about the new process?
- Can we make any further tweaks to make it more effective?
- Do we need to test it further to make a decision?
The very final stage of the process is deciding whether or not to implement your intervention in the future or not. Whether or not you have a statistically significant result will play a big role here, but it is not the only consideration.
Some other questions to consider are:
- What was the effect size? Is it meaningful in practice?
- Are the benefits of the intervention worth the costs?
- What feedback did we get from parents and staff about the new process?
- Can we make any further tweaks to make it more effective?
- Do we need to test it further to make a decision?
