Schools need far more clarity and reassurance over how they will be judged in the absence of Progress 8 data, experts are warning.

The call comes as survey data shared with Tes reveals the huge array of different approaches being taken by secondaries to measure progress over the next two years, when P8 will be unavailable.

The findings also show that the majority of leaders say it will be difficult to track student progress without the national scores.

Ofsted and the Department for Education are, therefore, being urged to set out how they will hold schools to account, amid fears that they will rely on exam results without knowing students’ starting points.

‘A lot of people are really worried’

Tim Leunig, who originally created P8, said: “The DfE and Ofsted should provide reassurance about how they will look at outcomes as a lot of people are really worried. They should be really clear that this is not a return to the days of A* to C.”

The context behind GCSE performance is seen as particularly important for the next two GCSE cohorts - those currently in Years 10 and 11 - which had their transition to secondary school heavily disrupted by lockdowns caused by the Covid-19 pandemic.

These students were in Years 5 and 6 in 2020, meaning they did not sit key stage 2 Sats, on which P8 scores are based.

Successive research reports have highlighted the impact of the pandemic on these cohorts. For example, maths attainment for KS2 students was found by the National Foundation for Educational Research (NFER) to have been particularly hit by the lockdown in 2021.

• Background: What does the gap in Progress 8 mean for schools?
• Attainment concerns: P8 pause could leave schools facing more ‘challenging’ situation
• Read more: How does Progress 8 work?

The Education Policy Institute also found that pupils who were in Year 5 during lockdown - currently Year 10 - were one of only two year groups in primary school during the pandemic that had not caught up in reading by 2023.

However, the educational impact of Covid did not land equally, as shown by the widening attainment gap in education after that period.

Tom Middlehurst, curriculum, assessment and inspection specialist at the Association of School and College Leaders, said: “Covid has had a disproportionate impact on disadvantaged pupils and nowhere near enough resources have been committed to education recovery.

“It’s understandable that school leaders serving more disadvantaged communities are particularly concerned about the absence of progress data.”

In agreement, Carl Cullinane, director of research and policy at the Sutton Trust, said: “Ofsted and others will need to be even more careful than usual when looking at a school’s context when making any judgements based on attainment data.”

Michael McCluskie, headteacher at the St Lawrence Academy in Scunthorpe, said: “These cohorts didn’t really have full time education in person until Year 9, and we’ve found a lot of students just don’t have the same ability to self-study at home.

“There’s a lot of nuanced problems from the pandemic that haven’t been recognised and are still ongoing.”

Harris Federation CEO Dan Moynihan said not having P8 “will be tough on schools with disadvantaged intakes”. He suggested grouping schools with similar intakes would be slightly better than comparing them on raw attainment.

National data a ‘starting point’ in inspections

P8, along with Attainment 8 and the percentage of grades 4+ and 5+ achieved across certain subjects, usually forms part of the inspection data summary report that Ofsted inspectors use when preparing to go into a secondary school.

The data is used as a “starting point in understanding a school’s context”, Ofsted says.

Currently, when Ofsted inspects quality of education, more weight is given to curriculum than exam results.

However, it describes an ”inadequate” school as one where disadvantaged students make progress that shows little or no improvement, or is below that of others nationally.

So, how will inspectors judge schools without P8?

They “will not look at internal data that the school produces”, an Ofsted spokesperson confirmed. “This doesn’t mean that schools cannot use their own data to tell us what the school is like, but inspectors will not review the data itself.”

Some heads disagree with this approach.

Concerns over how Ofsted will view attainment data

Mr McCluskie said his school is expecting Ofsted before the end of the year and is concerned about how inspectors will view attainment data for the school, where two-fifths of students are eligible for free school meals.

“We want to be able to show our in-year progress data to Ofsted inspectors when they come and I think it’s a mistake that they won’t review internal data,” he said.

Andrew O’Neill, head of All Saints Catholic College in London, said Ofsted should say which datasets it considers suitable for benchmarking, “then work with schools on it in the inspection process...For instance, are they happy for Cognitive Abilities Tests (CAT) to be robust enough?”

A DfE spokesperson said other options it looked at for calculating progress were not found to be “sufficiently credible or robust”.

Jonny Uttley, CEO of the Education Alliance and visiting fellow at the Centre for Young Lives, told Tes: “It is a concern to schools that with Progress 8 disappearing for two years, there is no clear idea of how schools will be measured or how Ofsted might use performance data. However, this just illustrates the need for fundamental reform of school accountability.”

How are schools measuring progress?

Responding to a Teacher Tapp survey, 61 per cent of heads and senior leaders said the absence of KS2 data has made it more difficult to measure the progress of their students.

So, how are they attempting to fill the gap?

CAT scores are the most common way schools plan to measure progress, according to the survey. But a huge range of other methods were mentioned, including internal benchmarking tests, NFER assessments, the New Group Reading Test (NGRT), the Progress in Understanding Mathematics Assessment, and more.

And 32 per cent of senior leaders who responded to the survey of 830 senior leaders and heads said their school is using external assessment data like CAT4 to track progress.

Some 19 per cent said they would rely on internal assessments and 38 per cent said their schools were using a combination.

A minority - 6 per cent - said their school currently is not tracking progress for the current Year 11s.

Filling the gap of Progress 8

Mr Moynihan said Harris set targets for affected cohorts using CAT and the NGRT as a baseline.

“We found as we got to KS4 that some of the targets were distracting from raising standards for our students, so at this point looked for any significant changes to cohorts and found it to be broadly the same,” he added.

As a result, Harris focused on securing grades at 4+, 5+, 7+ and 9+ for students.

The Fischer Family Trust (FFT) is providing progress scores using CAT4 results, which many schools and trusts have indicated they intend to use.

Astrea Academy Trust’s director of secondary education Richard Tutt said it plans to use this approach as a proxy P8 measure.

“We will use these measures as a candle by which to judge whether more or less progress might have been made across our schools and within different subject areas,” he said.

Meanwhile, the director of education at Lift Schools, Phil Humphreys, explained the trust conducted a Sats assessment across its schools in Year 7 for students who did not take KS2 assessments. This has been supplemented with data from GL Assessment, who run CAT4 tests.

FFT education head of product strategy, Craig Whitlam, said schools had approached FFT during Covid to ask if they could use CAT to create estimates.

In the absence of P8, FFT will use CAT data from around 1,900 schools to provide analysis of progress and attainment data at headline and subject level for schools.

Mr Whitlam said there is a “strong correlation” between P8 outcomes based on CAT and KS2 assessments.

Future of the accountability measure

The P8 accountability measure will not be published for the next two years, and there will be no replacement in the meantime. ASCL has previously called this ”the least bad option”.

Its future beyond next summer is uncertain: the measure is one of the accountability measures being considered by the ongoing curriculum review, which will report back with recommendations before P8 is able to be published again in 2027.

Confederation of School Trusts deputy chief executive Steve Rollett said while P8 “has its limitations”, a “progress measure should be re-established as soon as it is possible to do so”.

“In the meantime, it will be important for those holding schools and trusts to account to recognise this blind spot exists and to be appropriately cautious in the inferences they draw about school performance,” he added.

Progress measures ‘favour advantaged areas’

Some, like Mr Cullinane, hope the P8 pause will lead to a revised progress measure that takes additional factors into consideration, like disadvantage.

“The truth is that although progress measures are fairer than attainment measures, they still favour schools in more advantaged areas,” agreed Mr Middlehurst.

In the meantime, several trusts are deriving their own ways to interrogate and improve progress.

For example, King’s Group Academies has developed what it calls “KS3 readiness knowledge” that it will assess all students on when they join in Year 7. The trust intends to spend the autumn term bringing all new students up to speed on any missing essential knowledge.

It also plans to use this to compare progress, and to assess whether students’ needs have been met over their time in secondary.

Consilium Academies said it is now increasingly using formative tracking methods to measure progress over time, with a particular emphasis on reading ability.

Responding to the call for further clarity over what happens in the next two years, a DfE spokesperson said: “We are committed to driving high and rising standards in every school, and so whilst schools are able to utilise appropriate internal metrics to track their own pupils’ progress, we will continue to publish remaining headline measures and return to publishing time-series.”

For the latest education news and analysis delivered every weekday morning, sign up for the Tes Daily newsletter