The organisation behind Scotland’s national literacy and numeracy tests is warning that there is “clear evidence” that the time of year children sit the assessments can result in a “marked increase” in their performance - and says any comparisons between pupils or groups of pupils must be carefully drawn.

In the first report on the Scottish National Standardised Assessments, published today, the organisation responsible for delivering the tests - ACER International UK Ltd - is warning that “results from all learners should be interpreted with some caution when making any comparative judgements”, given that children who undertake the tests later in the year almost always perform better.

The company says this is reassuring because “it demonstrates children’s and young people’s literacy and numeracy do progress over the course of a school year”.

However, if councils stick to the Scottish government’s policy of allowing teachers to dictate when the tests should be sat, they will be limited in how they can use the data, given that a school or a class that appears to be performing well could simply be putting its pupils through the assessments later.

Fears of school league tables

This barrier to drawing comparisons between schools and councils will be welcomed by those who fear that the test results could be used to create league tables.

University of Strathclyde literacy expert Professor Sue Ellis pointed out recently that teachers dictating when children sit the assessments make it harder “for data to be used to stratify, rather than help, children”.

However, for others the warning will be confirmation that the tests cannot give a valid national picture and are a waste of time.

The report says: “SNSA are designed to allow children and young people to be assessed at any time in the school year that is deemed suitable for the school, class and individual learner. A consequence of the flexible timing is that, when interpreting the results of the assessment at individual, class, school or local authority level, the time in the school year that the assessment was done needs to be taken into account.”

It continues: “Amongst the year groups presenting for SNSA, children in Primary 1 showed a marked increase in capacity in both literacy and numeracy when comparing results from 2017 (August to December) with those from 2018 (January onwards). The same pattern was observed for Primary 4, Primary 7 and Secondary 3, across all subject areas, but with diminishing increases in performance in 2018, for each successive year group. Within each year group, the rate of improvement between the first half and second half of the 2017 to 2018 academic year was similar, regardless of subject area. The only exception to this general pattern of improvement from 2017 to 2018 was for S3 reading, where the overall result was the same.”

The report says, for instance, that if children in P1 sat the numeracy tests in November, the results showed less than half performing at a “high” level. However, if they sat them in March, the majority of pupils attained at this level.

According to the Scottish government’s SNSA website, the purpose of the tests is to give the system information about “which learning and teaching approaches are working best and where changes need to be made”.

The government has been clear that it should be teachers that decide when the tests are sat, with the education secretary, John Swinney, vowing earlier this year to use “every channel available” to ensure that policy is respected, after it emerged that the vast majority of Scottish councils were breaching it.

At the Scottish Learning Festival earlier this year, Allison Skerrett - an associate professor in language and literacy at the University of Texas, Austin, and one of the Scottish government’s international education advisers - said she thought the tests should be rebranded because they were not true standardised tests and were more about “benchmarking”.

She said: “Benchmarking is about seeing where someone is at a particular point in time. It carries the message of trying to understand each child’s pathway or learning, and what is needed in terms of support. ‘Standardised’ gives a different picture of ‘this is what everyone should know and be doing at this point in time’.”