“England’s primary schools continue to improve, but secondary schools still remain a problem in large parts of our country,” Sir Michael Wilshaw said last week as he presented Ofsted’s 2015 annual report. Sir Michael’s concerns were taken further in his report itself, which warned: “Concerted action is now needed to address the continuing weakness in our education system after the age of 11. Unless that action is taken the nation will continue to fail thousands of children and young people.”

Ofsted’s criticism of the standards achieved in English secondary schools is based on two misconceptions. The first is Ofsted’s notion of linear pupil progression (See: ‘How many perfectly good teaching careers are in ruins because we treat pupils as data not individuals?’); the second is that secondary schools can raise their GCSE results in the same way available to primary schools - that is to raise standards of pupil performance year-on-year. This seems a reasonable expectation until you realise that it is impossible for secondary schools to do so, because schools in England are cursed by the yoke of comparative outcomes.

This is complicated, so let me explain.

Comparative outcomes is the name given to a system designed to address the problem of year-on-year grade inflation of GCSE and A-level results. Its effects are not as widely known as they should be. Essentially, it means that the levels a pupil cohort achieves nationally at key stage 2 in any one year will determine the grades available to that same pupil cohort five years later at GCSE.

The result of comparative outcomes for secondary schools is a zero-sum game. If one secondary school manages to develop more effective classroom strategies that raise pupil performance, another school’s GCSE results must drop, even if there has been no decline in the standards achieved by that school.

Comparative outcomes means system-wide improvements in teaching; learning and assessment cannot be reflected in rising GCSE grades, which could lead to more schools being counted as coasting or failing. Comparative outcomes means improvements in pupil performance (as teachers become more familiar with the new GCSE qualifications) count for nothing, because the national distribution of GCSE grades for year 11 pupils is pre-determined by their key stage 2 test results. And, of course, underlying the rationale of comparative outcomes is the belief in the absolute reliability of key stage 2 results…!

Ofsted does not appear to know about nor understand this conundrum. Ofsted berates secondary schools for low standards when it ought to be taking this into account: there is an artificial ceiling placed on secondary school standards however much they attempt to improve GCSE outcomes.

It is certainly the case that Ofqual, the exam’s regulator, is attempting to find an answer to the absolute limit comparative outcomes puts on secondary school improvement. Ofqual is developing a national reference test with similar test questions, to be taken each year, by a national sample of pupils in year 11. This test is a new measure to give more recent (than key stage 2) information on the ability of each year’s cohort of pupils. If the reference test works, it will provide more current and relevant information on which to base the distribution of grades nationally at GCSE and should allow secondary schools to demonstrate increased rates of progress not tied to key stage 2 Sats results from five years earlier.

But the jury is out on the national reference test and there are serious reservations about its potential validity and reliability - particularly when triangulated against the very different measures of key stage 2 Sats and GCSE outcomes. We will need to look closely at how the preliminary tests work in March. Ofqual estimates that it will not be ready for use as a benchmarker for the distribution of GCSE grades until 2019.

So there we have it. Primary schools can improve their key stage 2 Sats results because they are not limited by what their pupils achieved at key stage 1. Secondary schools have an artificial cap put on their ability to demonstrate the progress they have achieved with their pupils in key stages 3 and 4 because their GCSE outcomes are tied to key stage 2 test results.

Is Ofsted aware of this constraint on secondary schools’ ability to achieve higher standards of pupil attainment and better rates of progression from key stage 2?  There is no mention of comparative outcomes in the chief inspector’s report and, when I tackled Sir Michael on this issue at the launch of his annual report, he immediately passed the question to a colleague.

I am not confident Ofsted understands the limiting effects of comparative outcomes on secondary school performance, nor have I any evidence that Ofsted’s critique of the quality of English secondary schools takes this issue into account.

Dr Mary Bousted is general secretary of the Association of Teachers and Lecturers. She tweets at @maryboustedATL