Why every school should (not) use setting

This article was originally published on 24 April 2024

Setting and streaming is a complex, contested field. But one thing is very clear: English schools do a lot of it.

The latest edition of the Organisation for Economic Cooperation and Development’s Programme for International Student Assessment (Pisa) report found that 97 per cent of its sample of pupils in English secondary schools were in schools that grouped by ability for some or all classes. The figure was 37 per cent across developed nations as a whole.

Looking at the international data from Pisa and the Trends in International Mathematics and Science Study, and how England compares at key stages 2, 3 and 4, John Jerrim, professor of education and social statistics at UCL Institute of Education (IOE), wrote in 2019 that setting or streaming “is now used for maths in almost half of Year 5 classes and in almost every secondary school”, with England having “much higher rates of within-school ability segregation than in other developed countries”.

Yet, in terms of academic research, it’s very hard to find evidence to suggest setting and streaming boost attainment for all pupils. The Education Endowment Foundation’s (EEF) toolkit, which distils educational research for teachers and school leaders, offers this summary on setting and streaming: “no impact” based on “very limited evidence”.

Setting in schools

Based on 58 studies that met its criteria, the EEF gives a score of +1 on the impact of setting and streaming for pupils with high prior attainment - outweighed by the -3 it assigns for impact for pupils with low prior attainment.

Becky Francis, EEF chief executive, says that “when it comes to setting and streaming, the evidence may not be conclusive, but it does show a clear pattern: these practices tend to be good for high-attaining pupils but bad for low-attaining pupils”.

“This disproportionately impacts pupils from certain demographics - for example, more affluent pupils tend to be disproportionately represented in high sets, and socioeconomically disadvantaged pupils in low sets,” she continues.

For the education researchers who spend time studying the impact of setting and streaming, this is a puzzler. Why are so many English schools so attached to something where the evidence (albeit limited) suggests no overall benefit for attainment, and a negative impact for the disadvantaged pupils who ought to be the priority?

For plenty of school leaders and teachers, though, things look very different when you have to do the nitty-gritty of running a school.

Mixed-ability teaching

To start with, we need to be clear on what we’re talking about when we talk about setting and streaming.

IOE and London School of Economics (LSE) researchers who conducted a survey of attainment grouping practices in English schools in 2020 helpfully summed things up.

• Setting means grouping pupils by subject attainment for teaching in that subject.
• Streaming means grouping pupils by attainment across several subjects.
• Mixed-attainment grouping is a different approach - grouping pupils so there is a range of prior attainment in each group (or more simply, not setting or streaming).

In addition, some secondary schools, and many primary schools, use in-class grouping.

The IOE and LSE researchers cautioned against seeing the picture as “a simplistic dichotomy between mixed-attainment and setting”. In practice, they found some schools using mixed-attainment grouping “with a separate bottom or ‘nurture’ group”, or using mixed-attainment grouping “with a separate top group (sometimes known as ‘stretch’ or ‘grammar stream’ groups)”.

Impact on disadvantaged pupils

Becky Taylor, part of that research team and principal research fellow in the IOE’s Centre for Teachers and Teaching Research, echoes Francis - with whom she has been a research collaborator - when it comes to summarising the academic evidence on setting.

It shows there’s “perhaps a slight advantage for pupils in higher-attaining groups, but that’s offset by a bigger disadvantage for those in the lower-attaining groups,” she says.

But Taylor, too, acknowledges the limited nature of the evidence base: it is “not straightforward to draw conclusions about the English system from research that’s largely been done in the US”.

In England, she continues, it has not yet been possible to directly compare outcomes for setting and mixed-attainment grouping - something she and colleagues aim to address with a forthcoming major research project (of which more later).

So the evidence base on outcomes for setting and mixed-attainment grouping is limited. But there is evidence for concern about how schools are making decisions about which pupils are placed in which set.

Maths is a good area to focus on, as the subject area where setting is most dominant.

Setting in maths

David Thomas is a former special adviser to education secretary Gillian Keegan and now chief executive of Axiom Maths, a charity that aims to ensure pupils who were top performers in maths at primary school hit top grades at GCSE - including those from disadvantaged backgrounds.

If setting is used, he says, it should be based on “accurate measurements of whether children are at a similar attainment level”. But, argues Thomas, a former maths teacher and secondary head, that is often not the case.

Giving a “stereotype” that is “not entirely untypical” of how secondary schools set in maths at the start of Year 7, Thomas says a test will come after a summer where most children have not done any maths - apart from those whose parents knew what was coming at secondary and got them to practice in the holidays.

A school’s test could be a 40-mark one pulled together by a teacher “arbitrarily”, with “way too small a number of marks to be able to have narrow confidence intervals around the mark range you put on it”, he adds. A confidence interval is a range around a measurement that conveys how accurate that measurement is.

A school might have, says Thomas, 180 pupils in a year group and for 140 of them, the confidence intervals - “wildly huge” because of the nature of the test used as the basis to set - are overlapping. That would mean it’s not possible to say if the pupil ranked in 40th position is really performing at a higher level than the one ranked 150th.

Nevertheless, Thomas continues: “You’ve said something pretty powerful to that child [ranked 150th] that probably does have a negative consequence. But because your statistics are poor you are not able to have the positive benefit that you would argue made the setting worth it.”

Research into streaming

That concern about students being misallocated to sets is echoed by Taylor, who highlights the socioeconomic dimension and impact of these decisions.

A 2019 paper, on which she and the EEF’s Francis were among the authors, using data gathered from 9,301 Year 7 students, found 31.2 per cent of students had been “misallocated to lower or higher sets than their KS2 results would have warranted”.

The study “didn’t find an effect of disadvantage”, says Taylor. “Young people who received free school meals: they were more likely to be in lower sets, but that was explained by the fact they had lower prior attainment.”

That is “an issue” in setting “that still needs to be addressed”, Taylor continues. But, she adds, what the study did find was the effects of gender and ethnicity.

“The odds of girls being misallocated to lower sets in maths than their prior attainment would warrant was found to be 1.5 times higher than that for boys,” says the paper. And “the odds of Black students being misallocated to lower sets was 2.4 times higher than for White students”.

This suggests, argues Taylor, that schools’ decisions on setting are “not necessarily about ‘ability’ - because the set that you’re in might not be fair”. And evidence that the chances of misallocation are higher for female, Black and Asian students is “a troubling finding”, she adds.

In the face of this level of concern, why are so many schools so attached to setting? Ideas around the practicalities of teaching are a big part of the answer.

Improved teaching

In the national spread of mixed-attainment grouping and setting, the Mossbourne Federation, which includes two secondary schools and a sixth form in the London borough of Hackney, “would be at the extreme end of pure setting - fundamentally, we believe in that, except in art, drama [and] music”, says Peter Hughes, Mossbourne Federation chief executive.

In subjects where there are sets, students in each year are placed in sets numbered one to nine, based on a combination of scores in Sats and cognitive abilities, plus a reading age test taken at secondary.

In his former role as a maths teacher, Hughes taught mixed-attainment groups and sets.

In Year 7 maths, in his experience, a mixed-attainment group could cover “everything from the student who was struggling with two-digit addition” to “students who are comfortably able to expand brackets and manipulate algebra”.

Teacher workload

Teaching a mixed-attainment group means “about double to triple the work, from my experience” and means “essentially teaching multiple classes” within the same classroom, Hughes continues. “For me, that’s what it’s all about: which of these things [mixed ability or setting] will allow my teachers to be brilliant.”

Mossbourne’s pure setting approach means teachers can “pitch precisely to that set and where it is”, he adds. And there’s a focus on “making sure that the standard is high even in our lower sets, where students are still expected to get 4s and 5s [at GCSE], as far down as set 7 and 8 out of 9”.

Those practical concerns about mixed-attainment teaching are widespread. In 2016, a group of researchers led by Taylor and also including Francis published a paper titled Factors deterring schools from mixed attainment teaching practice.

The paper was based on their experience of trying to carry out research comparing results for setting and mixed-attainment in schools, but researchers found it “impossible to recruit the required number of schools to the mixed-attainment trial” despite financial incentives.

Communications with schools reluctant to join “indicate that mixed-attainment is perceived as a risk and evokes a range of fears”, the researchers wrote. Switching timetables was a fundamental obstacle.

But what about the bigger argument set out by people such as Francis of the EEF - that setting sets back lower-attaining pupils?

Progress 8 impact

Mossbourne’s Hughes replies: “I think our outcomes speak for themselves. For us, it’s our ability to allow teachers to be even better is the thing that results in those outcomes.”

Those outcomes include the fact that Mossbourne Victoria Park Academy recorded a Progress 8 (recording progress at secondary school across eight subjects) score of +1 last year for its disadvantaged students - meaning they scored a grade higher in each subject compared with students who started at a similar level.

But “some studies from the broader evidence base…point to longer-term negative effects of setting and streaming on the attitudes and engagement of lower-attaining pupils”, as Francis puts it.

For example, a 2020 paper on which Francis and Taylor were again among co-authors, surveyed around 9,000 students placed in sets on self-confidence at the start of Year 7 and again at the end of Year 8.

“After controlling for prior attainment, the gap in general self-confidence between students in the top and bottom sets for mathematics is shown to widen over time,” said the paper, although “there was not further widening” in specific self-confidence in mathematics.

SEND and mixed-ability teaching

Within this, there’s specific concern about how setting can impact pupils with special educational needs and disabilities (SEND).

Professor Rob Webster is the author of The Inclusion Illusion: How Children with Special Educational Needs Experience Mainstream Schools, based on what it describes as the UK’s “largest longitudinal cohort study of pupils with high-level SEND”.

Webster says that while teaching and expectations are high for pupils in top sets, for those in the lower sets, “you get not-so-good teaching, the curriculum is usually less strong, the expectations are low and the outcomes follow. That’s where you tend to find the kids with SEND”.

The term “bottom set” was widely used by pupils and teachers alike in interviews, Webster found. There was “evidence that some pupils felt there was a stigma in being taught in these classes”, he wrote.

One secondary student told him they didn’t like “being in a lower class. People start being rude, they say rude things, and I want to go to a higher one, so then I can do a proper test”.

Pupils with SEND “are fully aware of the stigma” around lower sets “and how they are perceived by their peers”, says Webster. He adds that the rigidity of sets in many schools means they are “progressing through [their] school career with this label that is hard to shift”.

Allocating teachers to sets

But when it comes to any stigma, Hughes says the key at Mossbourne is that there is no “dumbing down” in lower sets. Does that include putting the best teachers in charge of lower sets?

“We do this even as low as my primary schools,” Hughes replies. “Why would you put your weakest teachers with your children who need them the most?”

He goes on: “You show a child they can succeed, and they are doing well and being challenged…there might be social stigma, but let’s tackle that. Ultimately, when children have some success and they feel like they are learning…then that [stigma] soon dissipates quite quickly.”

Hughes adds that “in schools where I’ve seen that [stigma] it tends to be because it [lower set teaching] has been dumbed down, it’s been turned into something Mickey Mouse - and the kids know it.”

Another source of “fear” about switching to mixed-attainment for schools that Taylor and her colleagues found in looking at deterrent factors was parents. Schools, the researchers said, “fear that parents may respond negatively and results will be endangered”, which are “significant threats within the education market context”.

Good practice role models

For Taylor, another barrier in a shift towards mixed-attainment teaching is a lack of role models for teachers and schools.

Her previous research has found that “a lot of maths teachers in particular have never had experience of teaching mixed-attainment…Because setting is so dominant, there aren’t that many exemplars of good practice out there to show what good mixed-attainment practice could look like”.

What might a better approach look like?

“My challenge to schools is you should be able to back up whatever your system is by showing the statistical validity of the judgements that you make,” says Thomas.

If schools are going to put each year group in sets ordered one to eight, for example, they “need to do quite a lot of assessment to get narrow confidence intervals”, he adds.

But schools might think doing that amount of assessment is not a good use of time, and thus be willing to “live with slightly wider confidence intervals”, he continues. “So maybe rather than having sets one through eight, you have a couple of top sets, a couple of set twos, a couple of set threes, a couple of set fours.”

That should be “driven by the confidence intervals on the pupil data” not just by the desire to arrange pupils in a pre-determined number of sets, he argues.

“I don’t think there is evidence to say, categorically, setting is or isn’t the right approach,” Thomas says.

“Bad mixed-ability [grouping] is bad and bad setting is bad.” But, he adds, “bad setting is far too prevalent, not because of anything educational but because the statistical inferences are poor”.

More and better evidence on the relative merits of setting and mixed-attainment grouping could be another route forward.

Taylor leads the Student Grouping Study, funded by the EEF, alongside Jeremy Hodgen, professor of mathematics education at the IOE.

That study, in the EEF’s description, will measure “attainment outcomes for pupils taught maths in mixed-attainment groups compared to setting”. It has been following a cohort of about 20,000 students through Years 7 and 8.

KS2 outcomes have been used as the starting measure, with pupils taking a short maths test in the summer, alongside self-confidence surveys, to provide final outcomes measures. It will be a year to 18 months until the findings are ready to report, says Taylor.

“We’re not looking for a particular outcome,” she adds. “Our motivation is around improving outcomes for all young people, particularly those who are low-prior-attainers, those from disadvantaged backgrounds, and [to look at] how their experiences can be improved.”

In-class grouping

But Mossbourne’s Hughes has a “challenge” for those who are doing mixed-attainment teaching: “Are you not actually setting yourself?”

“Nearly every time I’ve walked into [a mixed-attainment group], you see the setting happening in the classroom,” he continues.

“The teacher is differentiating and setting the class for themselves” by in-class grouping, Hughes argues. “All I’m doing is doing that on a bigger scale.”

But even if the move was towards in-class grouping, could that bring overall benefit for pupils? While some studies suggest negative impacts from setting for the engagement of pupils with lower attainment, says Francis, “the evidence for ‘within-class grouping’, where pupils of similar attainment levels are flexibly grouped together for specific activities or topics, is more positive for all pupils, on average”.

Though she does acknowledge that the evidence base “is still fairly limited”.

On the question of what all this means for teachers and school leaders, Francis says: “Decisions about how to group pupils should be based on robust evidence and data alongside knowledge of your own school’s context.”

Flexibility is important

If schools do want to group by attainment, says Francis, there are “a few things that are really important to consider to mitigate impacts on lower-attaining pupils”.

“First,” she continues, “you should consider which teachers are allocated to different classes, to ensure that low-attaining pupils receive high-quality teaching. Streaming - where pupils are placed in the same continuous group for all subjects based on an overarching conception of ability - should be avoided”.

“It’s also important to use continuous monitoring to make sure that pupils are in the appropriate sets, with flexible grouping to allow pupils to move easily between classes,” she adds.

There’s clearly a need for more evidence on the relative outcomes for all pupils from setting and mixed-attainment teaching - and the study being led by Taylor aims to address this gap.

However, Webster is sceptical about how much evidence can change things on setting. “A researcher saying the evidence shows this…won’t survive contact with reality because schools are balancing any number of different interests where the evidence might not really matter,” such as parental pressure in favour of sets, he argues.

‘Mixed-ish’ classes

Schools and teachers are under ever greater workload pressure, whether on accountability via Ofsted or the SEND crisis or dealing with the fallout from yawning gaps in social services as local authority funding is slashed.

Particularly in that context, it’s understandable that schools might be sceptical of any shift to mixed-attainment teaching perceived by many as an additional workload burden for teachers.

But, despite a dearth of research evidence as yet on relative outcomes of setting and mixed-attainment teaching, evidence on some of the unfairness in setting decisions and negative impacts of setting might give pause for thought.

As the IOE and LSE researchers argued, it could be a step forward to get a greater sense of the nuances: to have more understanding of the fact that plenty of schools are already finding a middle ground between hard setting and mixed-attainment.

One way forward seen by Thomas, concerned by big setting decisions being made by schools on the basis of poor statistics, is “mixed-ish” - setting to make sure top and bottom outliers on the attainment scale aren’t grouped together, but others are grouped together where a school recognizes it doesn’t have assessment data on which to make reliable judgements about those pupils’ attainment.

Given the potential implications of setting decisions, including for disadvantaged students, perhaps it’s time for schools to reflect more on not just the practicalities of teaching and timetabling, but also on the evidence, the statistical bases on which they set, and the full range of options open to them across pure setting, mixed-attainment, “mixed-ish” or in-class grouping.

John Morgan is a freelance journalist