Every once in a while you get presented with an argument that challenges a pre-existing belief that you’ve held for some time.

In such scenarios, you basically have two options: you can take what appears to be the approach of most politicians - ie, stick your fingers firmly in your ears and redouble your efforts to convince yourself that you were right all along (see the schools minister’s recent response to Programme for International Student Assessment findings on the impact of academic selection). Alternatively, you can accept that there is a chance you might have been wrong - or at least not entirely right.

I had just this experience recently when, as part of the assessment review group being led by the heads’ union the NAHT, we discussed the thorny issue of measuring progress in primary school. I have long been a strong advocate of the need for a contextual value added measure (CVA) when it comes to holding schools to account fairly.

It is worth recalling that we did have such a measure for quite a number of years. This took into account factors such as free school meal eligibility, special educational needs and disability, mobility and gender.

In fact, after a quick internet search, I came across a very interesting Department for Education (DfE) document from 2010 claiming that: “CVA gives a much fairer statistical measure of the effectiveness of a school and provides a solid basis for comparisons.”

However, within a matter of months, and with the arrival of a new government, that view was no longer in vogue and CVA was scrapped on the basis that the measure “entrenched low education aspirations in disadvantaged pupils’ groups”.

Barriers to good progress

This is the “soft bigotry of low expectations” argument. In reality, I don’t believe that teachers or schools did lower their expectations of pupils just because a school’s context was factored into their performance data after the event.

Having worked in a variety of schools with very different intakes, I have felt for a long time that while, on the surface, it seems intuitively right that all children with the same starting point should make the same amount of progress, in reality it isn’t quite as simple as that.

My experience is that the more challenging the circumstances, the more barriers you are likely to face in securing good progress.

This is not to say that teachers shouldn’t do absolutely everything in their power to overcome these obstacles and our expectations shouldn’t be just as high for every single pupil. However, it is a clear challenge and one that I have always thought should be taken into account, at least when it comes to retrospectively judging school effectiveness.

However, I have recently been forced to concede that CVA might not be quite the holy grail I have previously argued it to be when it comes to holding schools to account.

As it was put to me, any attempt to come up with a reliable CVA measure will inevitably be flawed. It will never be able to completely capture the full picture of a school’s context and instead will be based on a series of proxy measures.

What about all those contextual factors that don’t fit neatly into a government database or computer algorithm? I remember well many colleagues arguing passionately that the CVA measure didn’t really capture the true nature of their most recent cohort.

However, surely even a flawed CVA measure is better than no context-based measure at all?

The much bigger issue is that by striving to create such a measure we might just be playing right into the hands of those who would seek to hold schools and teachers to account based on data alone.

As Michael Tidd argued in his recent TES article: “Every step we take to try and make the data fairer, we end up giving the data undeserved credibility.”

Rather than trying to find the perfect measurement, we need to create a system where pupil performance data is seen as only one individual part of a much larger picture. Put simply, we need to put statutory assessment back into its box.

The creation of a CVA measure could have the unintended consequence of undermining that goal.

If a CVA measure has already taken context into account, then maybe there is no need for further investigation and discussion? It could be a very real case of being hoisted by our own petards.

I’m still not prepared to entirely give up my belief in CVA. However, I can see that when it comes to the reintroduction of such a progress measure, perhaps we need to be careful what we wish for.

James Bowen is director of middle leaders’ union NAHT Edge. He tweets at @JamesJkbowen 

For more columns by James, visit his back catalogue

Want to keep up with the latest education news and opinion? Follow TES on Twitter and like TES on Facebook