Primary school “A” has a reading progress score of +2.48. In the performance tables, the school receives a nice green box containing the words “above average”. It is obviously very happy about this. Primary school “B” also has a reading progress score of +2.48, but it only gets a yellow box and is deemed to be “average”.

So how can two schools with the same scores have such seemingly different outcomes? The answer: confidence intervals.

A confidence interval is a range of scores constructed around a school’s progress score that will usually (in 95 per cent of cases) contain the national average of zero. In those cases where it doesn’t, the progress score is deemed to be unusual, or statistically significant.

The confidence interval takes account of the number of pupils in the cohort - the more there are, the narrower the interval; the fewer pupils, the wider the interval. This means a bigger school’s data does not have to shift as far to be statistically significant, which is fair because one pupil has less of an effect on overall outcomes.

Significant progress

School “A” has 25 pupils in the cohort and its confidence interval is +0.04 to +4.92. Because this confidence interval is entirely above zero, progress is deemed to be significantly “above average”. School “B”, on the other hand, has 24 pupils and its confidence interval is -0.01 to +4.97. Here, the confidence interval includes zero (just!) and progress is therefore classified as “average”, which is why school “B” gets the yellow box while school “A” gets a lovely shade of green. If school “B” had the same number of pupils as school “A” , it would have the same confidence interval and the same green box. Sadly, one high-flying pupil left the school in March and this caused the confidence interval to widen to include zero.

You will also note that the lower part of the school “B” confidence interval is -0.01. Increasing the progress score by just 0.02 (one fiftieth of a scale point) would be enough to make the data significantly above average, which, in this case, requires just one pupil to get one extra mark on the test. Unfortunately, on the day of the test, that pupil was distracted by a squirrel outside the window.

It is easy to infer cause from these colourful indicators, and we commonly use them as proxies for school standards, but all they really point to is a deviation from the national average that probably didn’t happen by chance. Is it right to suggest school “A” is that much better than school “B”? Or is it simply that one school had a squirrel and the other one didn’t?

James Pembroke founded Sig+, a school data consultancy, after 10 years working with the Learning and Skills Council and local authorities. www.sigplus.co.uk