Imagine what would happen if there were big problems at the heart of students’ final exams - problems that had seemingly gone unnoticed since new qualifications were introduced a few years ago. And imagine if key standards were slipping, without making headlines. Or that the level playing field students should be able to expect each year was found to be sloping to one side.

Well, according to the Scottish Qualifications Authority’s own data, such a scenario may already be reality.

While analysing my Higher and National 5 English results last year, I decided to go a little deeper than usual into the SQA figures, digging into published information not just on pass rates but also on national averages in different components. I wanted to compare trends over recent years. Even though my students had done very well, before long a worrying picture began to emerge.

The most obvious issue, at least to me, is that the Higher English pass rate - regarded as a key metric for overall student performance in Scotland - has dropped every single year since the introduction of the new qualifications, from 81 per cent in 2015 to 76 per cent now. It is a relatively small decline, but it is also a consistent one. And it is a pattern that should concern us.

Of course, pass rates for the new Higher have always been above those for the previous incarnation (which, although concerning, shouldn’t come as much of a surprise to anyone who has taught both). But even so, we should be prepared to ask difficult questions: why is the pass rate on such an obvious downward slope and how far are we going to let it slide before we confront the problem? Things aren’t exactly perfect on the surface. But go deeper and the worrying currents get even stronger.

Of particular concern is the N5 “reading for understanding, analysis and evaluation” paper (or close reading to you, me and almost everyone else). In 2018, the “national mean score” for this part of the exam sat at a miserable 47 per cent, meaning that the average result across the country on that paper was effectively a fail (even though the actual pass rate was around 85 per cent).

In a country where concerns about literacy levels remain a core educational and political issue, this should certainly be cause for alarm, especially when that figure represents a big drop in performance compared with 2014 - back then, the average close-reading score at N5 was a far more respectable 58 per cent.

Literary criticism

The poor reading performance is all the more striking when compared with the average result on the “critical reading” paper (in which students analyse and write about literature This stood at a remarkable 71 per cent, meaning there is a difference of nearly 25 percentage points between the two papers. That gap is significantly greater than 2014 (when it was 15 points) and is also much more severe at N5 than Higher (3 percentage points in 2018).

So there seems to be a fundamental - and increasing - imbalance at the heart of this vital qualification, with students being far more likely, for whatever reason, to do well in one paper than the other.

This naturally raises issues about the fairness of the overall exam. Why should it be so much easier to do well in one paper than the other? Are some students being disadvantaged by this pattern? But to answer those questions we might need to ask others, especially about the source of the problem. Is the SQA to blame or are we seeing the results of deficiencies in our pedagogical approaches during recent years? Or maybe the road to hell is paved with false dichotomies.

Skewed results

This sort of debate takes on even more importance when we look beyond English and cast our eye across the full suite of N5 and Higher data. In 2018, N5 English wasn’t the only course with an exam in which the average score was a fail - the same is also true of N5 art and design (38 per cent), N5 media (49 per cent), Higher computing (43 per cent), Higher drama (43 per cent), Higher human biology (48 per cent) and Higher PE (38 per cent).

There are other courses with significant gaps between two exam papers, and the data also makes it clear that students tend to do much, much better on “assignment-type” submissions than they do in traditional exams - another factor that skews overall performance.

All of this occurs against a backdrop of ongoing concerns among many teachers about the new exams, many of which have been extended as part of the SQA’s face-saving exercise following John Swinney’s decree in 2016 that unit assessments were to be scrapped. N5 modern studies papers have been stretched from 1 hour 45 minutes to 2 hours 20 minutes. Higher physics exams now go on for a total of three hours, with Higher maths taking 15 minutes longer than that. For students entitled to additional time, final assessments are even more arduous (for more on this, see bit.ly/SpecialArrangements).

Scottish education has enough issues to contend with right now, but we simply cannot turn a blind eye to problems with exams that - rightly or wrongly - are the passport to college, university and employment. The SQA’s recent extension of numerous papers has already caused issues, but the available data suggests that the problems might be much more deep-rooted.

Nobody - least of all our students - will benefit from a refusal to confront difficult questions head on. In fact, we owe it to them to find the courage to do so.

James McEnaney is a journalist, FE lecturer and former schoolteacher