Yesterday saw the long-awaited publication of Gordon Stobart’s comparative review of international qualifications systems - see the report here.

This report, written to complement the main OECD (Organisation for Economic Cooperation and Development) review of the Scottish curriculum, published in June, seeks to develop a broader understanding of issues relevant to qualifications reform, both examining systems that are similar to Scotland (some grounded in the same British tradition of awarding qualifications via terminal examinations) and alternative approaches. The report provides a comparative analysis of nine systems, four situated within the British tradition, with an emphasis on terminal examinations, and five taking alternative approaches.

Stobart’s report makes a number of recommendations related to reconfiguring the current system, including increasing its “resilience” through a greater recourse to continuous school-based assessment and technology-based approaches to assessment, supported by effective national moderation systems. It explores the “decluttering” of the current three-tier approach with its two-term dashes and overloaded diet of examinations, which reduce learning time. I would suggest that this is dependent on Scotland becoming a system that both places high trust in its teachers and provides the system resources needed to support this way of working. Stobart also calls for greater levels of consultation with young people, and an increase in the range of vocational qualifications, affording greater parity of esteem with their academic counterparts.

Also today: Qualifications were ‘fully aligned’ with CfE, says SQA

News: Could OECD qualifications review spell the end for S4 exams

Background: OECD review paves way for qualifications overhaul

Related: SQA to be replaced, education secretary reveals

The report points to the social rather than scientific origins of public qualifications systems and their associated assessment methodologies. An important observation to make is that Scotland’s current system is not a natural, God-given phenomenon, nor indeed even the best approach. Stobart suggests that Scotland’s approaches, with their social grounding in the British (Victorian) tradition of the high-stakes exams and multiple tiers/exit points are out of step with the rest of the world in many respects, and implies that Scotland could learn much from the comparative OECD study.

The report repeatedly talks about the notion of the resilient qualifications system, making the point that the UK systems, with their emphasis on terminal examinations, have coped comparatively badly with the pandemic. Covid-19 has thus provided impetus for reform, with one lesson being that systems reliant on continuous, school-based assessment have been more resilient in a time of crisis.

A key question relates to how the senior phase might be aligned better with Curriculum for Excellence, which, in line with curricular reform worldwide, has placed an emphasis on new skills and domains of knowledge (including digital technology) that are not always readily assessed by exams.

The report also highlights considerable challenges in moving to new systems. These include issues relating to cultural change (often opposition from within the system), the reliability of different forms of assessment and workload. For example, Stobart points to evidence from the 2021 “alternative certification model” (ACM), which suggests that a move to a particular teacher-based assessment approach increased workload, as teachers and young people embarked on a treadmill of mini exams, marked internally. This illustrates, as Stobart points out, that assessment systems are essentially a compromise between issues of validity, reliability and manageability.

“A dependable assessment is one that can reliably give a trustworthy estimate of students’ capabilities. It involves an optimal trade-off between construct validity, reliability, and manageability. A dependable assessment is one that can reliably give a trustworthy estimate of students’ capabilities. It involves an optimal trade-off between construct validity, reliability, and manageability.” (p8)

The above suggests that school-based assessment will need two developments:

1. Work to enhance teachers’ assessment literacy, thus broadening the assessment methods used, and in particular embedding them more in the day-to-day learning that takes place in classrooms.

2. The development of rigorous but manageable moderation systems.

The Stobart report also refers to the increasing use of national qualifications results for school accountability purposes, which has contributed to their lack of alignment with CfE and to tendencies such as formulaic teaching to the test, backwash into the earlier broad general education (BGE) phase (which becomes shaped by the demands of future qualifications) and curriculum narrowing. This is a serious issue that will need to be addressed as part of any programme of qualifications reform; Scotland will need to develop data collection related to attainment that does not exert significant backwash effects on the system, thus encouraging performativity, for example, surveys of achievement that sample representative populations.

It seems to me that this report raises three board implications for reform of Scotland’s qualifications system. First, there are implications related to the structure of the qualifications phase. For example, this raises questions about whether a series of steps on a ladder, as in the three-tier Scottish system, is tenable, or whether Scotland should develop a more holistic senior phase allowing longer and/or modular courses and fewer points of assessment. A related issue here is whether there should be a single leaving certificate (even a baccalaureate), signifying completion of secondary schooling, or single subject certification, as is currently the case across the UK. According to Stobart:

“This raises questions about the nature and purpose of national examinations at age 16 and the message they send. If they are intended to certificate the successful completion of the curriculum in the first five years of secondary school education, a curriculum which now involves a wider range of skills, are there more valid ways of assessing educational progress? Are traditional single-subject examinations outdated at this stage?” (p17)

National testing at 16 is rare internationally, and where it occurs it is generally much more limited in scope. In New Zealand, a system formerly very similar to Scotland’s three-tier approach, the School Certificate taken at 16 has been abolished. Stobart also points to systems where both vocational and academic qualifications can contribute to a school leaving certificate, addressing issues of parity of esteem between different pathways.

A second implication relates to assessment methodology, for example, whether a qualification is based on examinations and formal tests or assessed through a wider range of methods, including portfolios, orals, continuous assessment of course work, and whether qualifications are externally or internally assessed. Of course, these are not simple either/or dichotomies. I do not advocate the total replacement of exams by other forms of assessment. But we need to remember that an exam is simply a means of assessment, not an end in its own right.

I do, however, support developing a more eclectic approach that is based upon the principle of fitness-for-purpose - and this will vary from subject to subject. Developments in technology open up new possibilities here. Stobart makes the point - contrary to a much-repeated trope that exams are fairer for disadvantaged children - that varied methodologies are more equitable in diverse populations: “The use of more varied formats, for example school-based assessments and practical work in vocational qualifications, represent ways of making qualifications more fit-for-purpose for a more diverse candidature” (p18). He also reiterates that, in many systems, the bulk of marks constituting a qualification derive from teacher assessment (eg, Ontario, 70 per cent; Norway, 80 per cent).

A third implication relates to the underpinning model for the qualification. In the case of Scotland, qualifications reform raises the possibility of questioning Scotland’s adherence to the competency-based education and training (CBET) model, originally designed for vocational assessment. This may seem like an academic point, but CBET comes with consequences. For example, its emphasis on mastery has led to the notion that all content needs to be tested, leading to the procession of unit tests that have taken up teaching and learning time in national qualifications. Demonstrating mastery may well be important when learning to wire a plug, but is arguably less necessary when discussing a topic in history. Indeed, the use of the competency-based model for assessing academic subjects has been the source of much critique over the years.

A strong message from the report is that wholesale reforms of systems need to account for the views of teachers, as opposition to reforms imposed on the profession (no matter how well thought through) is likely to contribute to their failure. Full participation of the profession in qualifications reform would seem to be an essential prerequisite. Winning the hearts and minds of students and parents is also necessary - although existing evidence cited by Stobart suggests already strong support amongst these groups for a shift away from exams. As the Stobart report states:

“Public confidence is vital if an assessment system is to be effective, while a loss of faith in a system will undermine the status and value of qualifications. If they are perceived as unfair, a validity and reliability issue, or unmanageable (by students and teachers), results will not be trusted.” (p36)

In summary, the Stobart report provides much food for thought. It seeks to dispel a number of assumptions about the existing Scottish system, through comparison with other systems. And it offers a tentative blueprint for reform - arguably much needed - to Scotland’s qualifications system. Of course, the way forward is up to us.

Mark Priestley is a curriculum expert based at the University of Stirling. This is a version of a piece originally published on his blog