England had the second-lowest pupil participation rate in the Programme for International Student Assessment (Pisa), according to an analysis by a leading expert.

UCL Institute of Education’s Professor John Jerrim wrote in a blog post: “Even within participating schools, individual pupils can refuse to take part in Pisa - or may be absent on the day of the test.

“In Pisa 2018, this also seems to have been a big problem in England. Around one in six (17 per cent) 15-year-olds across the UK within sampled schools who were meant to take part in Pisa were either absent or refused.

News: England climbs Pisa table as reading and maths improve

Briefing: What is the Pisa test and what does it measure?

Read: Pisa data ‘incredibly underused’, says academic

Analysis: Sampling issues cast doubt on Canada’s Pisa performance

“Indeed, there was only one country with a worse pupil response rate than England (Portugal). This is not good, and again calls into question the quality of the data.”

In England, 83.2 per cent of selected pupils took part in the Pisa test, against a minimum requirement of 80 per cent. But school participation in England was lower than the target set by the Organisation for Economic Cooperation and Development (OECD), which runs the study.

And Professor Jerrim, who wrote England’s official country report for Pisa 2015, is concerned about a lack of transparency surrounding an investigation into potential bias in the sample of England’s schools that eventually participated in Pisa.

“The big problem with this, however, is that no details on the bias analysis have been published by the Department for Education or the OECD,” he writes.

“The national and international Pisa 2018 reports simply say that a bias analysis been done - and that things look OK - but without providing any detail. This is odd, to say the least.”

The school participation rate is calculated as a percentage of the schools initially selected to participate in Pisa that actually take part in the assessment. Other schools can be selected to replace those that are not participating.

In England, the initial response rate was 72 per cent - meaning that more than a quarter of schools which were asked to participate in Pisa refused - and it rose to 86.3 per cent after replacement.

The National Foundation for Educational Research (NFER), which compiled the Pisa England national report, was asked to submit the “non-response bias analysis” - an analysis conducted to ensure that the sample is still representative of the 15-year-old student population in the country.

The bias analysis looked at the differences between responding and non-responding schools, and between schools originally sampled and those selected for replacement.

According to the England national report, the OECD Technical Advisory Group was “satisfied that the analysis demonstrate no notable bias would result from the non-response”, and that the data would, therefore, be fully comparable with other datasets internationally.

Last week, before the publication of the results, Professor Jerrim said that he knew there were difficulties recruiting enough schools to participate in Pisa 2018, and he predicted that the response rate could be less than optimal.

He told Tes: “I know they were having difficulties recruiting sufficient numbers of schools to hit the response rate. [On Tuesday] we’ll find out whether they managed or not.”

He raised doubts as to whether the data would be of good quality and representative enough in the case of a low response rate.

He explained that based on rules from Pisa 2015, countries needed to have 85 per cent of the selected schools to participate in the test. If they didn’t, they could attempt to recruit other schools.

Angela Donkin, chief social scientist at the NFER, said: “The sample NFER recruited was selected following rigorous procedures defined by the OECD, and [it was] content that [the sample] provides a robust, representative picture of schools in the UK which was fully comparable to other countries.

“This was, of course, a challenge. Schools are very busy, and so it is always difficult recruiting them to take part in research studies, even high-profile, important studies such as Pisa. Responses from the schools selected by the OECD were slightly lower than the OECD’s very high target, so we included replacement schools, chosen by the OECD to be as similar as possible to those originally sampled.

“The final school response rate for the UK was 87 per cent (which was slightly below the OECD’s target participation rate) and the pupil response rate was 83 per cent (above the OECD target).

“The UK was asked to submit a non-response bias analysis, analysing differences between responding schools and the population they are intended to represent.

“The OECD’s Technical Advisory Group was satisfied that this analysis demonstrated that no notable bias would result from the non-response, and therefore agreed that the UK data is fully comparable to other countries’ data.”

The DfE and OECD have been contacted for comment.