There’s a well-worn script for Pisa in Scotland now, established over many years.
The scores come out. They don’t look great. One set of politicians plays down the significance of those scores. Another says they prove that the once world-bestriding Scottish education system is on its knees.
Rinse and repeat every three years (or four after a global pandemic). It’s like exam results day on a grander scale.
The casual observer would be forgiven for thinking that yesterday’s findings from the Programme for International Student Assessment (to give Pisa its full name) could be easily condensed onto an A4 page, given that almost all the media coverage has focused on headline scores for 15-year-olds’ performance in maths, reading and science.
Yet, Pisa day entails much more than that: several documents published, hundreds of pages to wade through, screeds of complicated data and analysis, myriad issues and trends bubbling under the headline scores published by the Organisation for Economic Cooperation and Development (OECD).
So what else should you know about Pisa 2022?
Well, there was some fascinating insight into that most topical of issues: the impact of mobile devices in schools.
Pisa, in its analysis of data from 81 countries and economies, found that students who spent up to one hour a day using digital devices for learning in school scored 14 points higher in maths than students who spent no time on them, even after taking into account students’ and schools’ socioeconomic profile - a “positive relationship” found in 45 of those countries and economies.
Here, though, comes the big “but”: “Technology used for leisure rather than instruction, such as mobile phones, often seems to be associated with poorer results.”
And, at a time when teachers in Scotland are increasingly concerned about low-level disruption in classrooms, there is the worrying finding that, even if you don’t have a device yourself, it can be a major distraction if a classmate does.
Students who said they had been distracted by other students’ use of digital devices in at least some maths lessons scored 15 points lower than students who reported that this never or almost never happens.
Pisa finds, however, that UK students are significantly less likely to be distracted by digital devices than peers in other OECD countries - and “students are less likely to report getting distracted by digital devices when the use of mobile phones in schools is banned”.
Pupil wellbeing and school staffing issues
Meanwhile, fears about pupils’ mental health and wellbeing will not be assuaged by Pisa. Yesterday’s Pisa analysis found that “the reported life satisfaction scores of UK students had fallen drastically between 2015 and 2022, to the extent that the UK now has the second-lowest average life satisfaction of 15-year-olds across all OECD countries”.
Some commentators suggested that this was the most concerning finding of all to emerge yesterday.
In Scotland specifically, compared to 2018, students reported higher life satisfaction and being more likely to report a sense of belonging in school. However, life satisfaction in Scotland was below the OECD average.
Difficulties in school staffing are also highlighted. In Scotland, some 53.9 per cent of students were in schools where the headteacher reported that a lack of teaching staff had “hindered” the school’s capacity to provide instruction; the OECD average was 46.7 per cent.
Meanwhile, 54.5 per cent of students in Scotland were in schools where the headteacher reported a lack of support staff (“assisting staff” is the term used) was having a detrimental impact on learning and teaching; the OECD average was 37.2 per cent.
To coincide with the various Pisa publications yesterday, Scottish government social researchers published their own report digging into Scotland-specific data in Pisa.
Interestingly, given the issues identified in a landmark report on behaviour in Scottish schools published in November, some behaviour trends - but not all - were going in the right direction. Fewer students, for example, were reporting in 2022 (compared with 2018) that “students do not listen to what the teacher said”, that “there is noise and disorder” or that “the teacher has to wait a long time for students to quiet down”.
Also in Pisa 2022, students in Scotland were more likely than in 2018 to agree that they feel like they belong at school and less likely to report experiencing bullying acts. However, they were more likely in 2022 to report having skipped a day of school recently.
Findings on parental involvement in school life
Tes Scotland recently highlighted concerns that “huge cultural change” is needed if parents are to have more meaningful involvement in school life. However, the social researchers’ Pisa analysis finds that students in Scotland are “more likely to report that their parents or family provide support with their learning than the OECD average”.
Parents in Scotland were significantly more likely than the OECD average to talk to their children about the importance of completing upper secondary education, problems they are having at school, and their future education. They are also more likely to ask how well their children are getting on with other students and to take an interest in what they are learning at school.
The government social researchers also highlight the performance of children of migrant families. For example, the performance of second-generation immigrant students in maths - the main focus of Pisa 2022 - was higher than “non-immigrants” (ie, the student and both parents born in the UK). The performance of first-generation immigrant students, meanwhile, was similar to that of non-immigrant students and second-generation immigrant students.
The thing with Pisa, as one education academic said, is that it’s “like looking at the Mona Lisa through a couple of straws - you can see what you want but you won’t see the full picture”. There is such a raft of data and analysis that all sorts of narratives might emerge, depending on what you choose to focus on.
It is a statistical and unavoidable fact that Scotland’s performance in the Pisa tests of maths, reading and science has, over many years, generally declined.
The reasons for and significance of that trend are pored over at length, by us and many others. But, if you wish to look, there is much else besides that Pisa can tell us.
Henry Hepburn is Scotland editor at Tes. He tweets @Henry_Hepburn
