A low participation rate in the Programme for International Student Assessment (Pisa) in Canada casts doubts on the country’s strong performance, according to a leading academic.

In 2015, the Canadian sample was representative of only about half of the 15-year-old population in the country, compared to about 90 per cent in Japan and South Korea, according to figures cited by UCL Professor John Jerrim.

Analysis: Which pupils are the world’s biggest ‘bullshitters’?

Research: London students do less well in Pisa rankings than GCSE

View: ‘It is ridiculous that Pisa is now making teachers anxious’

The country is the only English-speaking western participant in Pisa able to compete on reading, science and mathematics scores with east Asian nations such as South Korea and Japan.

In 2015, Canada was ranked 10th in maths, seventh in science and third in reading. 

Its performance has been applauded and attributed to various factors, from high student motivation to the strong academic performance of immigrants, Jerrim explains.

But differences in the sample due to higher exclusions or absences can fundamentally affect the pillar of the Pisa exercise, which is to provide a representation of each country’s 15-year-old population, Professor Jerrim said. 

“I believe it could substantially undermine the Canadian results,” Professor Jerrim commented.

Professor Jerrim found three possible reasons why Canada’s participation rate was lower.

First of all, students with special educational needs were more likely to be excluded from taking the Pisa assessment: Professor Jerrim reported a 7.5% exclusion rate in Canada compared to 2.4% in Japan and less than 1% in South Korea.

Absence rates were also much higher than in Japan and South Korea: official figures show that one in five Canadian teenagers were counted as absent on the day of the Pisa test, compared less than 3% in the other two countries.

Canadian schools were also more likely to refuse taking part than schools in other countries. The province of Quebec, in particular, was prominent in the Canadian national report as less than half of its schools approached for Pisa agreed to take part.

“Together, this adds up to a sizeable problem, which I believe significantly undermines our confidence in the PISA 2015 data for Canada,” Professor Jerrim commented.

“I believe that there are particular problems in drawing comparisons to other “high-performing” countries - Japan and South Korea in our example - where a genuinely representative cross-section of children took part.”