Earlier this year, there was a huge uproar over the key stage 2 Sats reading paper, with concerns levelled around the length of the texts and the difficulty of the questions.

Indeed, a Tes analysis of the paper found that, once pupils had read all the texts and questions, they would be left with just 21 and a half minutes to answer 38 questions - requiring them to answer a question every 34 seconds.

Despite the Department for Education saying the paper was set to “an appropriate level of difficulty”, recently published Sats data revealed a drop in the number of pupils reaching the expected standard for reading.

Specifically, only 73 per cent of pupils met the expected standard, down from 75 per cent last year - while all other papers stayed the same or saw an increase.

What’s more, data on the scores achieved by pupils shows 16,481 more pupils scored between 97 and 99 than the year before and so fell just shy of the 100 scaled score, which means a pupil has reached the “expected standard”.

Sats ‘need reform’

Now we can see the impact this paper had on results. John Jerrim, professor of education and social statistics at UCL Institute of Education - who has published many papers on the impact of the KS2 tests - says it is indicative of a wider issue with the KS2 tests: they need, he says, to “go through some kind of reform”.

“Year after year, there are questions raised about the varying difficulty of the tests or the way the tests are marked,” he notes.

Jerrim feels the tests are too long and asks if “four days of testing” is needed, saying that “it feels like it could and should be cut down”.

And not only that, but Jerrim says their design doesn’t match what they’re intended to measure.

“A key issue with the KS2 tests is that they cover reading and maths only,” he explains.

“They could be designed to cover other areas as well, which would help prevent some of the negative consequences of high-stakes testing - such as narrowing the curriculum.”

Sarah Hannafin, head of policy at the NAHT school leaders’ union, agrees: “The experiences of many schools with KS2 Sats over the past two years has shown clearly that primary statutory assessment must be reviewed and reformed.”

“The current tests and the high-stakes accountability surrounding them do not value the whole child, promote positive mental health or encourage a broad and balanced curriculum.”

‘Wildly inconsistent’

That we’ve reached a tipping point in putting up with the tests is a feeling echoed by school leaders. Tim Roach, vice-principal of Greenacres Primary Academy, says this year’s outcome shows Sats are “wildly inconsistent” and that “greater scrutiny over Sats test papers” is needed.

“Each year, we have texts that drastically vary with their accessibility and diversity,” he says. “The 2023 test felt like a return to the hard papers of the past.”

As well as inconsistency in the texts chosen, Roach says the questions are also too varied as well.

“The questions also are largely inconsistent year to year,” he says. “This makes it harder for teachers to prepare pupils. Questions have to be asked about how this is allowed to happen.”

Suffolk-based headteacher Daniel Woodrow agrees: “When there are significant variations in the amount children are expected to read and the cultural capital knowledge required, it means that some years, the paper is more challenging.”

‘The current tests do not value the whole child or promote positive mental health’

This point was raised by assessment expert Daisy Christodoulou in a blog post in which she said Sats are too open to influence from random knowledge a pupil may or may not have.

“An issue specific to reading is that all reading tests are essentially tests of background knowledge and vocabulary,” Christodoulou wrote.

“If a student takes an unseen reading comprehension test and happens to know a lot about the topic it’s on, they can often end up doing much better than if they sat a test of equivalent difficulty on a topic they knew little about.”

Differences in pupil experience

However, criticism of the KS2 Sats isn’t universal. For example, not everyone agrees that the experience of the 2023 reading paper was too taxing for pupils.

Venessa Willms is director of education at Ark Schools and a former primary school headteacher. Drawing on the experiences of pupils across the 24 primary schools in her trust, Willms says it wasn’t anything out of the ordinary.

“The anecdotal feedback from our schools was that pupils’ experience was similar to previous years and they could certainly access the paper, even though they found it more challenging,” she says.

However, she acknowledges their schools “experienced the national dip in reading score down two percentage points compared to 2022”.

Rigorous process

Given all the concerns raised at the time of the paper, schools minister Nick Gibb promised to look at the situation in May. However, he was soon told by Gillian Hillier, chief executive of the Standards and Testing Agency (STA), which sets the Sats, that the paper was “in line with previous years”.

Hillier did agree, though, that more “transparency” around how tests are put together could help: “We should definitely consider whether there is more that the STA might do to explain to the sector, and the public, the rigorous processes that we go through in developing our tests.”

Information on how Sats are created is actually already available in the KS2 test handbook - although it is somewhat buried away on the STA site and described as “primarily for a technical audience”.

What’s more, for those keen to read through the document, the approach it describes may only serve to convince more people that Sats need revamping.

Specifically, it explains how Sats for each forthcoming year (ie, 2024) are trialled on a random sample of fewer than 2,500 Year 6 pupils the year before (ie, 2023) who, having sat their real KS2 tests one week, then sit sample papers the following week.

The scores these pupils achieve on the sample questions are used in the following year’s paper and are used to calculate the scaled score of 100, which serves as the “working at the expected level” pass point. At no point, though, does any child sit a complete version of the paper before this scale is set.

According to James Pembroke, data analyst and Sats expert, setting the pass mark before seeing how the actual cohort performs means Sats data is unhelpful because it is based on “human decision, fraught with error and bias”.

He elaborates: “The person deciding how many marks on the paper will equate to a scaled score of 100 may over- or underestimate how hard a question is for an 11-year-old pupil, and therefore end up setting the standard too low or too high.”

No ‘worthwhile data’ from Sats

This doesn’t just affect Year 6 pupils and primary school accountability measures. For schools that are meant to use this information to guide pupils’ onward education, it makes life hard, too.

For example, headteacher Vic Goddard has around 25 feeder primary schools for his Year 7 cohort but says “Sats simply don’t give us helpful information”, adding that “there are simply too many variables involved to provide us with worthwhile data”.

Meanwhile, Jonny Uttley, CEO of The Education Alliance - a seven-school trust made up of both primary and secondary settings - says the impact of the inconsistency of Sats is felt everywhere.

“We pay attention to KS2 results, but we use them contextually alongside cognitive ability tests [Cats] testing and our own assessments,” explains Uttley.

“We use the Cats to look for anomalies - where students might have a KS2 result at odds with their Cats result.”

Rigid approach

This inconsistency also seems to be affecting marking, too, with Ofqual announcing this summer that it would conduct a survey asking KS2 Sats markers to what extent tests are marked “accurately and reliably”.

Speaking to Tes anonymously, one Sats marker for the reading paper said she is concerned that the approach is too rigid and not enough is done to reward “unexpected” responses, as is done at GCSE.

“I’ve marked GCSE English and KS2 reading papers for several years, and the key difference is the way the benefit of the doubt is given at GCSE in a way it never is for Sats,” she says.

“At GCSE, if a student has given a correct but unexpected answer, you can reward the mark, and the mark scheme can be updated to include these alternative correct answers, which are very common when assessing inference - whereas Sats it is far more rigid,” she explains.

On the marking side of things, defenders of Sats can point out that guidelines from the STA do allow for movement on the scaled score line: “If the test is more difficult, the raw score required to meet the expected standard will decrease.”

‘Sats simply don’t give us helpful information. There are simply too many variables’

However, for some, this ignores the emotional impact of an overly hard test on pupils, as Cathie Paine, CEO of REAch2, explains.

“This isn’t about compromising on high standards. What it is, instead, is a recognition that further thought could be given to how we set children up for success by considering their experience of that test, rather than just relying on a statistical exercise that happens once the test is over,” she says.

With heads reporting in the summer that children were left “in tears” after the reading paper, this is a point worth considering, too.

All told, it means that 155,232 children started secondary school this September having not met the expected standard in reading.

Pandemic problems

Of course, the pandemic will have played a part in this, too, with Professor Becky Francis, chief executive of the Education Endowment Foundation, noting that “EEF research has shown the pandemic has had a substantial impact on younger pupils’ reading attainment”.

Yet, despite this, secondary schools will receive no extra catch-up funding to help these students reach the same level as their peers, with the Year 7 catch-up premium abandoned in 2020, not set to return.

Hannafin from the NAHT believes this is a prime example of the government failing schools and students: “The removal of the literacy and numeracy catch-up premium to support Year 7 students is yet another way schools are working with one hand tied behind their backs,” she says.

“We now have fewer pupils meeting the expected standard in English and no additional funding to support them as they start secondary school.”

Tes asked the DfE if there were plans for the funding for these students to return in the future, but it declined to comment.

New direction needed

For others, though, talks of funding, scaled scores and marking inconsistency ignore the bigger picture: Sats need to go.

“There is only one solution: to drop Sats for good and introduce assessment that prioritises children’s wellbeing and promotes, rather than destroys, love of learning,” says a spokesperson for the advocacy group More Than a Score.

“Schools should be accountable for all that they do, not just one narrow set of pointless tests taken under GCSE-style exam conditions,” they say.

Jerrim does not agree the tests should go but says they should be evolved, perhaps by moving to adaptative computer-based testing.

“That could - in theory, at least - help to solve this problem [as] it would mean that students could be given questions that are better suited to their ability level, which would help with the issue of questions being too hard,” he says.

This is something others have called for with regard to helping pupils with SEND (special educational needs and disabilities) perform in primary assessments, too, and could well gain traction as more digital assessment trials take place.

Potential changes

Meanwhile, Frank Norris, a former Ofsted inspector and Co-Op Academies Trust CEO who now advises The Northern Powerhouse Partnership, says teacher assessments could be used.

“I would like to see teachers have greater responsibility for assessing their children’s standards,” he says.

“The government could release some exemplar material that could be used for teachers assess against. This would free up the curriculum space to allow for a broader range of subjects to be taught.”

Roach says he would keep the assessments in place, too, but would want to make some key changes.

“I would want to introduce more flexibility with the time given to complete the tests,” says Roach. “Even 15-20 minutes more would make a significant difference.”

Roach makes the point that, as they stand, the tests reward speed “as children are penalised for not completing the test, rather than their being intellectually incapable of reading and comprehending the text”.

As well as changes to the test administration, Roach would also like to see the back of ranking school performance using the KS2 tests.

“League tables should go,” he says. “They are the cause of the gamification and one-upmanship that occurs when schools are pitted against each other. If they went, so would the holiday catch-up clubs, booster groups and narrowing of the curriculum.”

An uncertain future

While not advocating for any specific reforms, Paine at REAch2 says it is right we “constantly question and evaluate whether our national approach to testing is fair and consistent” and work to ensure that any year-to-year variance is being managed as “effectively as possible”.

The DfE also refused to be drawn on whether it would consider reforming Sats in light of the issues the reading paper raised this year. Meanwhile, when the question of reform was raised in Parliament in September this year, Gibb ruled it out.

However, given that, as of this year, KS1 Sats are no longer compulsory and have effectively been replaced by the Reception baseline assessment, the tests’ time in the sun is seemingly limited.

Add to that the negative headlines, multiple calls for reform and a looming election that could usher in numerous policy changes, and the future of KS2 Sats looks to be one punctuated by a rather large question mark.