The method used for statutory assessments of Year 6 primary writing may be delivering one in three “wrong” grades, according to research funded by the charity NESTA.

In the research, a writing task completed independently by Year 6 pupils was assessed by a panel of 30 teachers and a senior moderator.

A total of 349 Year 6 pupils from nine different schools in England took part. Every piece of writing was graded by the senior moderator and at least eight other teachers.

Testing: Sats and all primary tests will go ahead in 2021

Coronavirus: Teachers tell PM to ‘get a grip’ on Covid school safety

Phonics: Outrage as teacher pleas to scrap phonics test ignored

When using the statutory marking rubric, the Teacher Assessment Framework, the chance that a piece of writing would be given the same grade by a teacher and the senior moderator was 70 per cent.

The ‘benefits’ of comparative judgement

However, when using another method - comparative judgement - that percentage reached 88 per cent.

Using comparative judgement, teachers read two pieces of writing and decided which of the two was better. All these professional judgements were then aggregated to form a measurement scale in which every piece of writing was then fitted. 

One of the study’s authors, Daisy Christodoulou, said: “This research shows that there is wisdom in crowds of teachers. Aggregating the judgements of many teachers is better than relying on just one. Traditionally, the barrier to this kind of aggregation has been time: it takes too long to quadruple-mark every essay.

“But comparative judgement is much quicker, making it possible to combine many different judgements efficiently.” 

The study was carried out by No More Marking, an online provider of comparative judgement assessments, and Anne Pinot de Moira, an independent analyst and fellow of the University of Oxford.