High-quality teaching is critical to improving pupil outcomes and closing the inequality-related attainment gap. However, it isn’t always clear which classroom teaching practices make the biggest difference to teaching quality and pupil outcomes, and how to support teachers to make use of these practices.

In this project, we aimed to use innovative statistical modelling to understand how far it is possible to identify effective teaching across schools. We also wanted to understand if this analysis of effectiveness could be compared to data on classroom teaching observations to understand more about the practices associated with more and less effective teaching.

We worked with a national multi-academy trust (MAT) to access and analyse data on pupil outcomes across multiple secondary schools. The MAT provided Nesta with a range of data on pupil assessments and demographics, teaching groups and teacher information, as well as data from classroom observations. The latter consisted of ratings for a range of teaching practices.

Nesta used this information to test out the viability of using teacher value-add modelling (VAM) to estimate the range of teaching effectiveness across the MAT. These models attempt to estimate the impact of teachers on pupil outcomes whilst accounting for a wide variety of factors (eg, pupil prior attainment and demographics). VAM is widely used in the US where standardised tests and data on teacher effectiveness is more commonly available and used than in the UK (for instance, Kane et al, 2010). In England the use of VAM is less common although a recent study used a similar approach using Year 11 Maths and English results and teacher observation data (Burgess et al, 2022).

In this project we use year groups outside of Year 11. In Year 11 children typically receive a lot of additional support ahead of GCSE exams, so the effect of the teacher may be diluted.

Some researchers are wary of using this approach for estimating teacher quality, questioning how far other variables (such as pupils’ prior ability) can be controlled for (Baker et al, 2010) and warn against its use for high-stakes teacher performance management.

However, we were interested in VAM’s potential to help schools identify effective teaching at scale and in turn inform CPD priorities, rather than be used for teacher evaluation. Therefore, despite mixed views on VAM’s reliability, we wanted to explore its potential through this project.

The analysis was completed between November 2022 and February 2023 and included frequent consultation with leading academic experts: Professor Rob Coe and Dr Sam Sims.

We found encouraging evidence that teacher VAM can be utilised to understand patterns of teaching effectiveness in the context of English MATs. We also produced tentative findings on the association between teaching effectiveness and classroom observations, with some encouraging signals of consistency between the two (eg, in maths more effective teaching was associated with higher teacher observation scores, but this was not consistent across all subjects).

Our findings must be viewed as preliminary and in the context of some data limitations. For instance, we could only use two academic years of data on 150 teachers across 20 schools which resulted in a somewhat large uncertainty around the results of the VAM analysis.

There are also a number of important factors relating to pupil progress which we could not factor into our analysis (e.g. the quality of the home learning environment or impact of additional interventions such as tutoring), although this is a common problem for researchers rather than specific to this work.

We provided a report to the MAT which will be used to inform further development of data collection and analysis processes and systems, and as a springboard for further work exploring teaching effectiveness and classroom practices.

This work has demonstrated that MAT datasets offer new opportunities to explore pupil outcomes and teaching practices, and the correlation between these. The consistency and quality of these datasets will likely continue to improve, and researchers and MATs should work together to unlock the rich insights they hold. Where MATs collect robust data on classroom observations, these offer a particularly unique insight into the practices of teachers across schools, and when combined with pupil outcome data can help us understand more about what works in classrooms.

This project has also illustrated the high potential of teacher value-added models to help us understand more about pupil outcomes and the relationship with teaching practices. Utilising multiple academic years of data across more teachers would help bring further rigour to the models and allow more confident conclusions to be drawn.

Since the start of this project, the fairer start mission has increased its focus on the early years, and is not currently pursuing further projects in the secondary schools sector.

If you would like to discuss this work further with us, please email us.