Policymakers and educators are once again poring over the Programme for International Student Assessment (Pisa) rankings and results.
And, of course, this is an extra-special edition as it comes after the Covid-19 pandemic led to the largest disruption to schooling we have ever experienced. So, what story does Pisa 2022 tell?
The organiser of the Pisa test, the Organisation for Economic Cooperation and Development (OECD), has done its best to report some of the issues surrounding this period.
The results were delayed a year. The difficult circumstances schools faced led to challenges in collecting the data, and the United Kingdom is among the countries whose student response rate and school participation fell below the minimum target set by Pisa technical standards, so we have to be cautious in interpreting the results.
Another issue is that different countries dealt with the pandemic in different ways, both in terms of the duration of school closures and countering its negative effects. For example, perhaps some countries were affected more acutely but were able to address this partially with remediation policies.
More on Pisa 2022:
Pisa 2022: what were the results?
The OECD spends a whole chapter on such aspects. One interesting statistic from this section that leapt out for me was that while 75 per cent of students in the UK felt confident or very confident about using a video communication program, only 47 per cent of students felt confident or very confident about motivating themselves to do school work.
Although overall achievement across the OECD for the three domains - mathematics, reading and science - dropped, there are still quite a few countries that managed to improve. Unfortunately, the UK was not one of them.
Average 2022 results were lower than in 2018 for mathematics and reading, and about the same as in 2018 for science. This brings mathematics and reading back to pre-2018 levels, while for science it now has been a decade-long decline.
The gap between the highest- and lowest-scoring students did not change significantly in mathematics and reading, but widened for science. The gap between the top and the bottom 25 per cent of students in terms of socioeconomic status remained stable in the UK.
What else do the Pisa 2022 results show?
Beyond the headlines, the context questionnaires are always a treasure trove of interesting findings, which could all serve as great starting points for conversations:
- In mathematics - after accounting for socioeconomic status - immigrant students on average do 12 points better than non-immigrant students.
- A relatively high percentage of UK students report not being satisfied with their lives (stable from Pisa 2018).
- UK students report similar disciplinary climate scores to other OECD countries but score much lower on being distracted by digital devices. The OECD also notes, however, that “students were less likely to report getting distracted using digital devices when the use of cell phones on school premises is banned”.
- A relatively high percentage of students in the UK reported having experienced bullying.
- Parental involvement has decreased in the UK; this might be related to the pandemic.
- Finally, principals reported that instruction has been hindered by a shortage of teaching staff for more than half of students, and by inadequate or poorly qualified staff for one in five students.
Correlation, not causation
Of course, we shouldn’t just look at the scores for the whole of the UK. We can readily glean that scores reduced for all countries, but the least for England, most for Wales, and in between for Scotland and Northern Ireland.
No doubt this is going to lead to claims about the relative merits of different curriculum approaches, as was the case with the previous Pisa rankings, but it’s useful to point out that this data can only point out correlations, not causation. Pisa is not the best assessment to check curriculum standards either, as it focuses not on basic skills but literacy.
Another thing to expect in the reaction will be selective comparisons with other countries, with being one point ahead touted as having “passed” other countries. Such differences are likely to be just measurement error.
So, overall, Pisa 2022 again provides an interesting starting point for conversations about education. International comparisons can be interesting. But we have to take into account many cultural and curriculum differences. There also are plenty of changes within countries.
As ever, context is crucial.
Christian Bokhove is professor in mathematics education at the University of Southampton and a specialist in research methodologies
