How early does the disadvantage gap start when we’re looking at children’s outcomes? Does gender also impact on a child’s developmental assessment results?

These are crucial questions that at Nesta we need answers to in order to help close the disadvantage gap. We analysed Stockport’s early years data to find out if different characteristics affect how likely it is that you meet age-related expectations in a communication and language assessment.

We looked at gender, the deprivation of their local area and whether they were pre-term. Our analysis shows that in Stockport, boys who live in more deprived areas are more likely to not meet expectations in their developmental assessments. And once they have that poor result they are likely to continue to have a poor result rather than move to meeting expectations for their age.

This work builds on previous research Nesta did in Stockport where we discovered that as many as 17% of children who were meeting age-related expectations in their first developmental assessment then often didn’t meet them in their next developmental assessment. The analysis presented in our work was conducted using the Ages and Stages (ASQ) Socio-Emotional questionnaire but the same trends were seen in one of the other assessments that Stockport also used to measure children’s development in the early years - the WellComm.

The WellComm is a paid-for communication and language toolkit which includes an assessment and an associated intervention called the “Big Book of Ideas”. It is aimed at children between 6 and 59 months and there is a simple traffic light system for recording results. Red means the child is not meeting age-related expectations, amber means they are borderline and green means they are meeting age-related expectations.

The number of children that were seemingly developing on track for their age in the WellComm and then not did not meet the developmental assessment criteria for their age in later assessments was a concern for Stockport so we conducted some novel statistical analysis to delve deeper into these results. We delivered this project between autumn 2021 and autumn 2022 using a dataset of children’s assessment data from March 2019 to April 2022.

Our approach was to use a multi-state model. A multi-state model estimates the probabilities of moving between separate “states”. We can then change certain aspects of the model to see how those probabilities change. Are you more or less likely to move from one state to another if you change the characteristics of the model?

Figure 1(below) is a pictorial example of our multi-state model with the “states” that we’re considering, in our case red (not meeting age-related expectations), amber (borderline) or green (meeting age-related expectations) results, and the potential transitions between them.

For example, a child may have a green result on their first assessment and on their next assessment they have one of three possible outcomes: they stay as green, they move from green to amber or they move from green to red. These are known as transitions. For our analysis, we were interested in knowing what affected the probability of these transitions. For example, if they started off red, were they more likely to stay red, transition to amber or transition to green? Do these probabilities change if you are a boy or a girl? Using Stockport’s data we were able to investigate the probabilities of children transitioning between the red, amber and green results in their WellComm assessment broken down by gender, level of deprivation and whether they were preterm or not.

Our results from the multi-state model showed that if you’re a child growing up in Stockport, you are better off in the WellComm assessment and more likely you meet age-related expectations if you are a girl living in the least deprived areas. If you are a girl and live in the most deprived area, the probability of you getting a green assessment decreases by at least 30 percentage points, no matter when you do the follow-up to the first assessment. If you are a boy, no matter where you live, you are least likely to meet age-related expectations. Finally, being preterm seems to have a slight positive impact on those in the most deprived areas, with virtually no effect on those in the least deprived areas.

We emphasise that these results depict the state of WellComm assessment living in Stockport and cannot be generalised to the rest of the UK. However, it is what is happening in Stockport and importantly reflects on the ground knowledge.

From Nesta’s results, Michelle Morris, the Greater Manchester Clinical Lead for Speech, Language and Communication, shares how these learnings are being interpreted in the Stockport and GMCA context, and the changes they’re going to make.

“First of all, it is clear that children’s speech, language and communication status isn’t static, and we need to be able to identify any downwards trajectory early. Currently there are very few follow-up WellComm assessments if you’re meeting age-related expectations, therefore Stockport are going to implement more WellComm assessments into their pathways. Secondly, disadvantage has an effect on their probability of reaching a green in the WellComm. As such, in areas of Stockport where we know there are high levels of disadvantage, we’re planning to deliver language interventions as part of an augmented curriculum. Finally, boys are performing worse than girls, so more work is needed to fully understand why this is the case and to consider what we can do to support boys in Stockport.”

In addition to these clear actions as a result of Nesta’s work, Michelle also mentioned some further work happening in Stockport to supplement the findings.

“It is really important for us to understand families' thinking and their readiness for interventions, and consider that the dosage of the intervention also matters. When we see children’s speech, language and communication skills not increasing in line with age-related expectations, and that this can happen within a couple of months, we need to ensure that any interventions delivered have a high enough dosage to prevent this. However, we also know that it’s not just about the families - practitioner confidence and skills matter. As such, we will also be doing work to support our practitioners in delivering the WellComm and the associated interventions.”

Nesta's involvement in the WellComm project has increased our understanding of the local context in Stockport and highlighted key areas for impactful interventions. The analysis supports on-the-ground practitioners' experiences and has empowered them with sufficient evidence to facilitate and implement changes, something that would have been more challenging without Nesta’s analysis.

Methodologically, the innovative and novel statistical analysis of the WellComm data holds promise for extension to other local authorities with comparable high-quality assessment data. If consistent findings emerge, valuable insights from Stockport's implementation of changes in the WellComm delivery can be applied to benefit additional areas. However, limitations in the dataset, such as incomplete ethnicity data, underscore the need for enhanced and more complete data collection practices. Stockport's acknowledgment of this limitation and commitment to improve their own data collection will make our future work and evaluations more robust which will contribute significantly to our ongoing efforts in the area.