In the wake of the publication of the first national external Scottish Qualifications Authority (SQA) exam results since 2019 on Tuesday, education secretary Shirley-Anne Somerville got a grilling from the BBC’s James Cook over progress in closing the attainment gap between rich and poor students. There had been “hardly any progress, according to many experts in the field”, he put it to her.
The thing is, the figures show that Scotland had made huge progress when it came to the attainment gap - if we look at 2020 and 2021, the Covid years in which national exams were cancelled.
At Higher, the difference in the pass rate between students living in the most and least deprived areas was 16.9 percentage points in 2019, the last time external exams were held. But in 2020, when teachers were responsible (eventually) for grading students, the gap had narrowed to 6.6 percentage points. In 2021, when we essentially had exams in all but name, the gap widened a little on the previous year but still remained significantly lower than 2019, at 7.8 percentage points.
Now, we could have a debate about whether we are comparing apples with pears - given the very different ways in which students have been assessed in recent years - but, equally, it doesn’t seem wise to simply attempt to erase what happened in 2020 and in 2021.
It should also be noted that, although external exams returned this year, the assessment arrangements were not the same as they were in 2019. Covid modifications to courses and assessments were still in place in a bid to compensate for the continued disruption to education caused by the pandemic - remember the sky-high staff and pupil absence rates earlier in 2022?
This meant that in some subjects coursework was removed, while in others teachers and students were told certain topics would not be in the exam. Marking was also more generous.
So we are still yet to have a “normal” exam year since Covid came to Scotland - and we won’t have one next year either.
SQA exam results 2022: tackling the attainment gap
Ultimately, the fact is that many students who would have passed last year - or the year before - didn’t this year. And we should at the very least be curious about why that happened, especially as many people are making a case for giving teachers more responsibility for assessing students. (Although it should be pointed out that not everyone is.)
Many teachers and headteachers - but clearly not all - have welcomed the return of the exams because they believe the teacher judgement system used in the past two years was unreliable and not fit for purpose, and that exams are fairer.
One headteacher told me that the controversial unofficial league tables that appear in some newspapers showed that a secondary school could climb more than 200 places between 2020 and 2021. The head questioned whether the quality of the teaching in that school had really increased so quickly, or whether the tables showed that there were question marks over how teacher judgement had been applied during Covid.
But is overly generous grading the only explanation for the improved performance?
The rapid review of the 2020 exams fiasco by the University of Stirling’s Professor Mark Priestley and Dr Marina Shapira suggested that teacher judgement could be “a more accurate assessment of achievement than exams (which are said to disadvantage some learners)”.
Priestley expanded on this point when he and Shapira gave evidence to the Scottish Parliament’s Education and Skills Committee in November 2020. He raised the possibility that “teacher estimates measure something different” to exams, adding: “The issue is not so much about how teacher estimates can predict an exam performance - it is about whether they provide a more valid measure of student performance over time.”
End-of-year exams for which students gather en masse in gym and dining halls are also notoriously stressful - is it possible that getting rid of that anxiety-inducing, nerve-wracking experience could, in itself, improve attainment?
In 2021 the pass rate was higher and the attainment gap far narrower than in 2019 - the last year of national external SQA exams before Covid - but teachers said that students essentially sat exams “in all but name”.
Covid restrictions, though, meant that students often sat these pseudo-exams in the familiar surroundings of their classrooms - not in exam halls - and papers were also often broken down into smaller chunks, so that students could sit them during normal timetabled classes. That meant they were examined for shorter bursts of time, with sometimes several days in between assessments.
In an article for Tes Scotland this week, Robin Macpherson, head of Robert Gordon’s College in Aberdeen, looked at what a more neurodiverse-friendly exam system might look like. Exams split over a few days to reduce stress and the need to focus for long periods was one potential improvement suggested by a student.
Is more teacher assessment the answer?
So, overly generous grading undoubtedly was a factor in 2020 and 2021 - but what else might have been driving the improvement in pass rates and the progress made in closing the attainment gap?
The trouble is that we don’t really know and, as assessment expert Professor Louise Hayward put it, speaking after the 2020 exam results, “we will continue to guess unless we go in and investigate”. Given we are talking about putting more emphasis on teacher assessment - and less on exams - surely we need to find out?
School Leaders Scotland (SLS) became the latest body to call for more teacher-led assessment in Tes Scotland this week.
SLS general secretary Jim Thewliss said that high-stakes end-of-year exams had been “long discredited” and that, under the current system, too much was predicated on what happened in an assembly hall over four or five weeks from the end of April.
Last week Melvyn Roffe, the principal of George Watson’s College in Edinburgh, criticised the current qualifications system for being “based on short-term recall of limited subject knowledge”, when “the real world prizes open, innovative thinking”.
A long line of other key figures and organisations have also expressed their dissatisfaction with the way Scottish students are assessed - the president of the Association of Directors of Education in Scotland and Glasgow education director Douglas Hutchison, in an interview with Tes Scotland in March, said that the exam system was “ripe for reform”. There were “multiple ways” for students to demonstrate what they had learned “other than an exam for two hours on a wet Wednesday morning”, he said.
And new EIS union general secretary Andrea Bradley wants a move away from “annual high-stakes qualifications” towards “exit qualifications”.
Meanwhile, Hayward is now, of course, leading the ongoing independent review of assessment and qualifications, which is due to make its recommendations in March.
In the meantime, the disconnect between teacher predictions and SQA results is likely to continue to be an issue.
Not just because, until we can explain why the divergence is so great, it will be tough for people to have faith in any new system that relies more heavily on teacher judgement. But also because any student this year whose estimate was higher than their final result can appeal - and this year, appeals are free.
Heads are therefore predicting a huge rise in the number of appeals that the SQA receives.
This means that teacher estimates could still influence students’ results this year, once appeals have been processed.
The SQA will, however, be keen to protect what it describes as the integrity of the qualifications system, and is unlikely to see a big leap in the pass rate following the appeals process as being commensurate with that.
So, once again, students and their teachers will be left wondering what went wrong. Next week we will feature one of those teachers, who is “fundamentally questioning his purpose as a teacher” after his Higher class’s attainment was “far lower than expected”, with students “predicted to get an A...failing after consistently succeeding all year round”.
That, if it is a widely shared view, is not a strong position from which to launch a new assessment and qualifications system.
Covid shook the foundations of the assessment system in Scotland - but instead of trying to draw a veil over the past two years, why don’t we try to learn from them?
Emma Seith is a senior reporter at Tes Scotland. She tweets @Emma_Seith
