In the wake of the Programme for International Student Assessment (Pisa) results, published in December, a dam burst and the Scottish education system was hit by a veritable tsunami of criticism from politicians, commentators and academics.

The system has long been under pressure but, after the latest Pisa results, new levels of condemnation were reached, with the Scottish Labour leader Anas Sarwar accusing the SNP of “destroying” Scottish education.

Scottish Conservative leader Douglas Ross, meanwhile, accused the government of costing “Scotland its international reputation for excellence in education”. He said Curriculum for Excellence (CfE) - which he branded “the SNP’s curriculum”, even though it has enjoyed cross-party support and was originally formulated under a Labour-Lib Dem administration - should be scrapped.

Those attacks were launched during First Minister’s Questions on Thursday 7 December, two days after the 2022 Pisa results were published. By the Sunday there was an article in the Scottish press by Westminster education secretary Gillian Keegan, describing Scottish education as “on the brink of collapse”. In another newspaper that day, England’s behaviour tsar Tom Bennett called for new guidance on behaviour, and for the CfE to be reviewed and replaced.

Scotland’s Pisa results - which focus on 15-year-olds’ performance in maths, reading and science - showed that Scottish students’ average scores in all three had fallen; Scotland was around the Organisation for Economic Cooperation and Development (OECD) average for science and maths, and above average for reading.

Now that the dust has settled, how much of the outcry over Scotland’s Pisa scores was about political point scoring or education culture wars - and how much was grounded in reality? In other words, is the Scottish education system really in tatters?

‘Clear slide in outcomes’

The consensus among the academics and experts that Tes Scotland spoke to is that, as Pisa chief Andreas Schleicher puts it, there has been “a clear slide in outcomes” in Scotland. However, the experts also broadly agree on the assessment from University of Stirling expert Dr Marina Shapira that “the picture is worrying”, although “at the same time it’s not necessarily the disaster that is being pictured in very sensational terms”.

Something is going wrong but exactly what is harder to pin down, says Christian Bokhove, a University of Southampton professor in mathematics education and specialist in research methodologies.

He is critical of those quick to offer up a diagnosis and questions the credibility of conclusions based on long-held beliefs and hobby horses. For example, he says, for proponents of more prescriptive, knowledge-based curricula, CfE is to blame for Scotland’s poor performance. Meanwhile, among those who dislike the emphasis in Scotland on restorative approaches and reducing exclusions, it is this that is to blame for deteriorating outcomes.

‘Simplistic and naive’ analysis

Bokhove says such analyses are “rather simplistic and naive”, and are often driven by conclusions established years ago, with every subsequent Pisa report becoming “an opportunity for confirmation bias...to confirm what they thought all along”.

“I just don’t like that,” he says. “I don’t think that is very scientific. And I’ve been very surprised how many people who posit themselves as evidence-informed commentators on education do this, because I think it’s the antithesis of being scientific. Basically, the conclusion is ready and you just find the data that fits the conclusion.”

Bokhove also says that England has been “immodest” and “arrogant” in its criticism of Scotland’s Pisa results.

If you are selling cars, to be able to say “the car is worse now - but it’s not as bad as in many other countries” is not a convincing sales pitch. This, he argues, is the position England is in, given that scores there also fell.

Still, according to Pisa, England is doing better in maths, reading and science. So, is Scotland’s more flexible and less prescriptive curriculum the problem?

CfE is “ambitious”, says Schleicher. His question is, do schools and teachers have the capacity to implement it?

There are three curricula in every school system, he says: the intended curriculum, the achieved curriculum and the implemented curriculum.

“We have a good appreciation of the intended curriculum, but where I would put my emphasis is to really study the implemented curriculum,” says Schleicher. “What are instruction practices? What is actually being done in classrooms? How are these intentions being interpreted by teachers in different places, in different classrooms? My suspicion would be that is the broken link.

“When you see that there are capacity constraints then you can look backwards and say ‘maybe we need to have a little bit more prescription in the curriculum to better support struggling teachers and struggling schools’.

“If it was me, that’s where I would be starting.”

The Scottish government’s response to the Pisa results has been to promise a review of the maths curriculum, followed by English and literacy.

But Schleicher reiterates his concern that the implemented curriculum is likely to be the problem, not the intended curriculum set out in policy documents.

“It’s clear that something isn’t working, but the reaction that you just change the curriculum because it’s easiest to do is not necessarily the answer to the question. My hunch would actually be - and it is pure hypothesis - that teacher capacity that exists in the frontline isn’t sufficient to deliver on the ambition of the Curriculum for Excellence.”

Unlike Schleicher, Dr Marina Shapira, an associate professor in sociology, has investigated in great depth how CfE is being implemented.

She was one of a group of University of Stirling researchers who, in February 2023, published research exploring fears that under CfE, the Scottish curriculum was narrowing in S4. The research also looked at factors influencing the subjects and courses offered in Scottish schools.

She suggests it is “a system problem” that is leading to poorer outcomes in Scotland, “not just the curriculum, per se”.

The researchers found that capacity - as suggested by Schleicher - was an issue.

Their report, Choice, Attainment and Positive Destinations: Exploring the impact of curriculum policy change on young people, stated that shortages of teachers, especially in Stem (science, technology, engineering and maths) subjects, and a lack of teacher non-contact time “limit subjects offered and teachers’ capacity for curriculum making”.

It also said that “limited resources” were a problem and that schools were prioritising courses for higher-attaining students at the expense of lower-attaining students.

‘Perverse incentives’ for schools

One serious problem in Scotland, says Shapira, is the focus on “performance indicators” around the proportion of students attaining five National 5s or five Highers. This is creating “perverse incentives” and causing schools to base their curricula not on “what does it mean to be an educated person in the 21st century?” but squarely on improving the attainment data.

Parents told researchers that students were being withdrawn from certain courses “to improve the school’s attainment statistics”. The researchers also found that schools were guilty of “abolishing low-performing subjects” and that curriculum narrowing and reduction of subject choice in S4 was “disproportionally affecting students from disadvantaged socioeconomic backgrounds”.

So, is CfE still the right idea, in Shapira’s view? She finds it hard to disagree with the philosophy that attainment is not the sole purpose of the curriculum and that you are also trying to help young people become “rounded individuals”, but this does not need to come at the expense of “a concrete set of skills which are based on knowledge”.

‘No tolerance of failure’

According to Shapira, the Estonian curriculum is similar - it is also “more skills and competency-based than a knowledge-based curriculum” - but that country, when it comes to Pisa, is doing “really well”. The only countries to outperform Estonia on maths in Pisa 2022 were Singapore, Japan and South Korea.

Schleicher says the aspirations of the Estonian curriculum “are quite similar to Scotland”. Where he sees a big difference is in “academic rigour”: in Estonia, there is “no tolerance of failure”.

“Basically if you go to an Estonian classroom, they won’t make things easier for a student who is struggling in maths or science; actually, they redouble their support. There is a clear expectation that everybody has to deliver really good results.”

But it is not all doom and gloom for Scotland, says Shapira. Scottish teenagers did “very well” on the 2022 Pisa reading test and “extremely well” on the “global competence” test, published in 2020, designed to assess if they could see things from multiple perspectives and identify reliable sources of information.

Scottish students’ mean score on the global-competence cognitive test was 534. Only three OECD members scored higher: Singapore (576), Canada (554) and Hong Kong (542).

Students from 27 countries and economies, including Scotland, took part but, given that around 80 participate in Pisa, many countries that do well in the traditional Pisa assessments of maths, reading and science were not represented.

This could be seen as akin to being top of the class on the day when most of the high-flyers were off ill, but Schleicher doesn’t see it that way: “You can still say that Scotland did better on global competencies than what you would have predicted from its performance in mathematics and science.”

And “whether you can live with people who are different from you, whether you can engage in different ways of thinking, different ways of working” is “very important”, says Schleicher, adding: “This is a very positive outcome for Scotland.”

Ultimately, Shapira would like to see more research into how CfE can be improved as opposed to deciding that it’s time to “forget about it” and “do something different”.

The University of Stirling research contained 23 recommendations, but Shapira’s starting point would be a “move away from this performativity culture”.

“It’s thinking about what we want to measure and how we measure it. And expanding the number of indicators and moving away from just performance on national qualifications...thinking more about what we are trying to achieve.”

The Pisa data itself hints at some other areas that Scotland might want to investigate, she says.

Last week, in a briefing for the Scottish Educational Research Association, a Scottish government statistician said that in many ways, the scores for maths, reading and science are “some of the least interesting” Pisa findings.

He said the study also looked at the factors that support attainment, including students’ backgrounds, their attitudes to learning, their wellbeing and their experience at school and in the classroom.

There were “huge amounts of potential in the data to develop insights to improve education”, he said.

Schleicher says the Pisa data, if analysed properly, could tell Scotland a lot about how the system is working and what needs fixing.

Shapira highlights Pisa data showing that Scottish students were more likely to report skipping school than the OECD average. Life satisfaction and belonging in school, meanwhile, increased on 2018 but were below the OECD average.

“We need to understand what exactly is happening,” she says.

Covid trends

When it came to the pandemic, Scottish students were more likely than the OECD average to agree their teachers were well prepared to provide instruction remotely, but less likely to agree that they were motivated to learn (25.7 per cent compared to an OECD average of 38.5 per cent).

Schleicher also flags this finding as concerning and says that Pisa data suggests that parental engagement is not as strong as previously in Scotland, and that pupils feel less supported by their teachers now.

He highlights that in 2012, 44.2 per cent of headteachers reported at least half of parents initiated contact to discuss their child’s progress with a teacher; by 2022 that had dropped to 25.3 per cent.

“The quality of student-teacher relationships really is a key driver of success in general across countries, particularly for students from more disadvantaged backgrounds, and that has somehow deteriorated,” says Schleicher. “Again, I would read this into the overall decline.”

Smartphones’ impact

How Scottish students use technology is also important, says Schleicher.

“While technology use in classrooms, in a structured way, can be a good thing, you see that students that use their smartphones frequently in lessons do worse,” he says.

Teenagers in Scotland spend a similar amount of leisure time on digital devices to the OECD average. However, a higher-than-average proportion of students in Scotland are distracted by using digital devices in class.

Shapira suggests the figures on motivation to learn during lockdown indicate that Scottish students are “passive learners” who struggled through Covid because, instead of being spoon-fed by teachers, they had to find “huge motivation” and “huge self-discipline”.

“The motivation to learn can’t just be to pass the test,” she says. “You need to want to learn, you need to know why you learn, and just be curious and excited. I think we’ve reduced learning down to something that is maybe not really all that inspiring for young people.”

Others identify this as being a problem with the current system. Douglas Hutchison, Glasgow’s director of education, has an extract from the Christopher Brookmyre novel One Fine Day in the Middle of the Night pinned to the wall of his office. It’s about the character Annette Strachan and her experience of school: “She found the curriculum frustratingly restrictive...with everything so geared towards exam syllabuses and exam technique that learning for its own sake seemed a decadent luxury.”

Hutchison told Tes Scotland in 2022 that Brookmyre had “absolutely nailed the Scottish education system in one”. At the annual conference of the education directors’ body ADES in November, he again used the passage to hammer home the need for “serious change in the education system”.

Bokhove makes the case for “slow reform” so that decisions are not made “on a whim” and it is possible to “analyse the causes” - but he also cautions against “inertia”. In Scotland, though, impatience for change is growing.

Schleicher suspects the gap between the intended and implemented curriculum is to blame for Scotland’s deteriorating Pisa performance. He concedes that he has not got the evidence to support this claim, but it could be argued that Scotland has clearly known this is a problem since an OECD review of CfE was published in 2021.

Too much focus on exams

The review highlighted issues with CfE implementation, particularly in upper secondary school. It largely blamed a qualifications regime that was too focused on exam preparation and rote learning, and not aligned with CfE’s aspirations. The 2021 review also said that teachers’ and schools’ capacity to be “curriculum makers” was hampered by “Scotland’s comparatively high rate of teachers’ class-contact time”.

Since then there has been a slew of further reports making similar points, but little tangible progress.

In a more recent interview last week, on the podcast The Rest is Politics: Leading, Gillian Keegan again took a swipe at Scotland’s Pisa results, saying she would be “horrified” to have its rankings. But in that situation, she added, you just have to get on “and do something”.

And that perhaps is Scotland education’s biggest failing: the problems are well-rehearsed but change is slow to follow.

Emma Seith is senior reporter at Tes Scotland. She tweets @Emma_Seith

For the latest Scottish education news, analysis and features delivered directly to your inbox, sign up to Tes magazine’s The Week in Scotland newsletter