It might sound like something that the head of resistant materials might warn you about, but the “sawtooth effect” has nothing to do with woodwork.

It is actually a student exam performance pattern that can be seen when assessments, such as GCSEs and A levels, are reformed. Ofqual has recently published new research about it. So, what do teachers need to know?

The sawtooth effect refers to the decrease that is seen in exam performance when a new specification is introduced, and then the improvement in exam performance in subsequent years. The name “sawtooth” comes from the jagged shape this creates when plotted on a graph.

Ofqual wants to ensure that there are what it calls “comparable outcomes” for candidates. This means that if the national cohort for a subject is similar (in terms of past performance) to the previous year, results should also be similar at a national level in that subject. 

Measures are taken to prevent grade inflation, as part of the “awarding” process that takes place once the marking period has ended. The awarding bodies set their grade boundaries in a process overseen by Ofqual, with each awarding body setting separate grade boundaries to reflect the small variations in challenge between each paper. 

The awarding process uses data already held on the cohort, as well as examples of work from previous years, to ensure standards are comparable from one year to the next; this avoids grades being awarded that are at odds with the grades that similar students received in previous years. 

GCSEs: The mechanics of the sawtooth effect

So, how does the sawtooth effect come into this process? There are two distinct parts to the sawtooth effect: a “disruption” effect; and an “enhancement” effect. 

The disruption effect is the decrease in exam performance when a specification first changes. This is attributed to the inexperience of the teachers delivering the content and the lack of teaching resources for the new specification.

The enhancement effect refers to the increase in performance when students are taught by teachers who are more familiar with the style of questions, and have access to more past papers, exemplar scripts, and general teaching resources.

How and why does the sawtooth effect happen?

Paul Newton, research chair at Ofqual, has written a report on the sawtooth effect. He explains that Ofqual accounts for this effect as it is working to ensure comparable outcomes.

“For the first year of teaching a new specification, we make a statistical adjustment for the initial drop in performance, applying the comparable outcomes principle,” he explains. “And then we do the same thing in the second year, but not quite as much as the first as there will be an element of recovery, as teachers become used to the exams, and then by the third year we assume we’re back to business as usual.”

Newton explains that Ofqual assumes that teachers recover in the first couple of years, and then get back up to par. As time passes and teachers have more information about the way exam questions are posed, and marks awarded, further improvement in grades is to be expected.

“The enhancement effect relies on [teachers] coming to see patterns in the assessment format, as teachers see more and more of the previous exam papers and are able to make accurate predictions regarding question patterns, or the likelihood of topics appearing.”

The sawtooth effect across the curriculum

Of course, not all subjects are assessed in the same way. Art, drama, and PE, for example, all have significant practical elements to their assessment. So does the sawtooth effect impact all subjects in the same way?

“It’s quite hard to tell, as it is so hard to investigate,” says Newton. “We also don’t ever really see the sawtooth effect, we just infer that it’s happening. It’s not the kind of thing you can see by looking at any particular exam, but when you average together lots of exam papers, you can see something important is going on.”

Do teachers need to worry about the sawtooth effect?

So, is the sawtooth effect something that teachers should be concerned about? Not really, says Newton. The effect isn’t a bad thing; it’s just a natural consequence of any testing system.

“We expect all teachers to do everything they can do to help their students, and this is desirable; we want teachers to get up to speed with the new specifications,” explains Newton.

Although the latest research on the sawtooth effect may help teachers to understand the complicated awarding process, Newton is clear that no one is expecting teachers to start adjusting what they do in the classroom.

“The consequences from the [sawtooth effect] report aren’t for teachers to worry about,” reassures Newton. “It is for exam boards, and examiners, and people like me who have to find ways to accommodate that reality.”

What does this mean for exams in 2021 and 2022?

We already know that, due to the impact that the coronavirus pandemic has had upon education, the Department for Education has decided to use 2020 grades as a point of comparison for students in 2021 as part of a package of interventions aimed at creating a “fairer” examination series.

Ofqual will use its knowledge of the sawtooth effect to achieve comparable outcomes in future years, once the disruption of the pandemic has subsided. However, we don’t yet know how this will impact grading in 2022 and beyond. 

Tes asked Ofqual, and was told it was too soon to say: “Ofqual has not made any decision about 2022, and that extends to any decision on comparable outcomes,” the regulator said.