Progress 8 is now an established feature of the education system. No accountability metric will ever be perfect, but I personally think it has been a huge improvement on the previous system of five A*-Cs because it rewards those schools that genuinely add value to their students.

However, schools misuse it when they use it as a measure of individual class progress. 

At first sight, it might seem to make a lot of sense to use the Progress 8 methodology to measure individual class progress. With the old five A*-C measure, it would obviously be unfair to compare the results of a top set with a bottom set. But Progress 8 is measuring progress, not attainment, and pupils in the bottom set can still make progress.

And, indeed, Progress 8 would work as a measure of individual class progress, but only if individual classes were either not set at all, or if they were rigidly set using the KS2 baseline. In practice, I think very few schools set this way at GCSE. It’s more common for schools to create sets based on recent attainment and work ethic in the subject concerned.

But if you do set like this, it means you can’t use Progress 8, or any other value-added measure, to assess each class’s progress. Why? Because when you start creating sets like this, you are effectively creating a top set full of pupils who are making progress, and a bottom set full of those who aren’t.

The limitations of Progress 8

Imagine a hypothetical school which has just two maths sets:

The pupils in the top set are predicted to get grades 6-8 based on their KS2 baseline. The bottom set are predicted to get grades 3-5. Over time, the teachers in the maths department realise that a couple of pupils in the top set are struggling. Perhaps they got lucky on their Sats; perhaps they aren’t working as hard in maths as they could be. The department decides to move these two pupils down a set, and promote up two pupils with the opposite profile. If this process happens once a year, then before long the top set will consist not of pupils who had high prior attainment in maths, but of pupils who are making above average progress in maths. The bottom set will consist not of pupils with low prior attainment in maths, but pupils who are making below average progress. You’ve created sets based on progress, so of course, your top set will end up with a high average progress score and your bottom set won’t. That has nothing to do with the teaching in those two sets, and everything to do with the way they’ve been selected.

This effect was noted before the creation of Progress 8. The Centre for Evaluation and Monitoring at Durham University wrote about it in 2002. In 2013, AQA’s then head of research, Dr Chris Wheadon, highlighted the same issue. (Dr Wheadon is now the director at No More Marking, where I work). The introduction of Progress 8 makes this effect even more important. Progress 8 may be the best way we have come up with to assess the performance of a year group, but that doesn’t mean you can use it to assess the classes within that year.

Daisy Christodoulou is director of education at No More Marking and the author of Making Good Progress? and Seven Myths About Education. She tweets @daisychristo