The programme works with children struggling in their education and those that are primarily (but not exclusively) eligible for Free School Meals or facing such issues as family breakdown, bereavement, poverty, bullying, being in the care system or other difficult circumstances.

It enables trained volunteers called “coaches” to work on a one to one basis with children, with the overall aim of improving a child’s behaviour and in turn raising their levels of learning.

What the CSAIF Funded: TLG were awarded £327,830 to establish Early Intervention Centres in 68 new areas, which would create the capacity to help around 600 children each year. Read the full impact evaluation.

About the evaluation

Level on Standards: Level 1 - they can describe what they do and why it matters, logically, coherently and convincingly.

Evaluator: Institute for Effective Education, University of York

Aim: TLG’s evaluation aimed to profile the level of emotional and behavioural difficulties experienced by children on the Early Intervention (EI) programme, and identify whether the programme reduced these difficulties.

Key findings:

1. Data suggests that participating children have fewer functioning difficulties and experience a reduction in symptoms of emotional and behavioural difficulties.
2. An electronic survey provided insight into patterns of delivery and implementation of the intervention, and provided feedback on TLG’s new evaluation tools. Coaches report delivering the core components of the intervention as intended, though there is some individual variation.

Methodology: A retrospective study of casefiles held by TLG on children who have completed the intervention, in order to unpack potential impact and distance travelled. An online survey to gather the views of coaches with regards to evaluation as well as the impact and implementation of EI.

311 casefiles were available for the retrospective analysis, with TLG selecting 98 cases for analysis. Overall sample sizes were very small for the various surveys (e.g. 66 for the SDQ), with attrition rates high across all measures (e.g. 44 at baseline, 7 at post-intervention, with only 4 filling in both). The report does acknowledge this problem, but it limits the confidence the reader can have in TLG’s positive impact and significant findings.

Why this Level: The evaluation does show a positive change in outcome using pre-post validated tools. Unfortunately, the response rate was low (particularly at follow-up) which resulted in a small sample size (e.g. just 4 young people filling in both baseline and follow-up versions of one tool). Because there is a strong chance that the data might therefore not be representative of all young people on the programme, the evaluation could not quite achieve Level 2.

About the evidence journey

Progress: Though this evaluation has not quite progressed TLG to a higher Level on the Standards of Evidence, it has put in place the processes to generate robust pre-post data that could give a strong indication of positive impact if the response rate can be significantly increased.

Next steps and Lessons learned: This has been a big journey for TLG over the past year and they have now developed a key person for leading on impact across the organisation. Some of the key lessons learnt and implications moving forward for Early Intervention are:

• Due to the dependency on their volunteer coaches putting in and keeping up-to date with their coaching evaluations and logging in their sessions, they have begun exploring the costs and viability of an App for coaches to simplify access and improve the quantity of recording.
• They are already assessing the training and how we bring more emphasis, comprehension and simplicity to the use of data for their coaches.
• Insist that schools give academic information if they want to take part in the programme.