A new pilot from Ofqual trialling an alternative to the way GCSEs and A levels are graded has had “promising” results, according to a report published by the regulator today.

The pilot investigated comparative judgement as a means of standardising exam grading between different cohorts and years of pupils.

Related: Ark’s head of assessment leaving to work on comparative judgement

Feature: How shall I compare thee…? Exploring comparative judgements

News: ‘Weak’ link found between teacher assessments and comparative judgement results

Currently, the system of comparable outcomes involves matching pupils’ grades in one cohort with those of the year before - assuming that “cohorts of similar ability should have similar pass rates in different examination sessions”, according to the report.

While it states that the rationale for this is to protect pupils sitting exams in the first year of a new qualification, it notes that in “steady-state” qualifications, this becomes less important, whereas noting “genuine changes in performance” becomes more so.

The pilot of comparative judgement looked at teachers and examiners taking a selection of pupils’ scripts across different years of assessment and ranking them on quality.

This allows exam scripts to be assessed in terms of the skills they demonstrate. The more scripts available, the greater the accuracy - the report notes, “The goal is usually to maximise the number of comparisons of available scripts across available judges and time”.

Ofqual’s report described the results of the pilot in assuring quality as “very promising” in capturing an expert judgement - that of a teacher or examiner - and using this to maintain standards over time.

However, a spokesperson for Ofqual said that more exploratory research is needed before it considers adopting comparative judgements as a means of assessing GCSEs or A levels, and that, currently, there were no plans to use this as a “magic bullet” to dispel doubts over comparable outcomes.

The report also showed different levels of accuracy depending on subject, with variable results in the grading of English literature AS - one of the subjects used in the pilot - where examiners’ assessment of the quality of scripts did not correlate strongly with the grades given, whereas they did for GCSE English language, for example.

However, an Ofqual spokesperson said this could be due to “statistical noise” and issues in the AS English literature mark scheme.

Sally Collier, chief regulator of Ofqual, said: “The research we have published demonstrates our strong commitment to innovation to make marking and awarding of grades as good as it possibly can be for students and all other users of these qualifications.”