Ofqual should have given more consideration to colleges with thousands of GCSE resit students when it came to this summer’s assessment model, college leaders have said. Meanwhile, teachers say the process has led to sleepless nights over bias and worries about the impact ranking might have on the student-teacher relationship.
Following the cancellation of exams due to the coronavirus pandemic, Ofqual announced that teachers would be required to calculate grades for each student based on prior attainment and assessment and then rank students according to those grades. Institutions have until 12 June to submit their calculated grades and the rankings.
For some colleges, this has meant ranking thousands of students who were due to resit their GCSE in maths and/or English this summer. Leaders have said that the process has both been challenging and time consuming.
News: Ofsted to review online teaching and learning in FE
Background: Disadvantaged students will lose out over assessment
Student perspective: should we ban all future exams?
Leeds City College has around 6,000 students across GCSE English and maths that required ranking, and principal and chief executive Bill Jones said that Ofqual didn’t consider the challenge for colleges.
He said: “Ofqual have done their best under very difficult circumstances, but the initial guidance did nor properly consider the challenge for large colleges. We have had to find our own solutions.
“All the maths and English staff are working throughout their half term to meet that deadline of 12 June. Devising our own validation process took a lot of staff power and staff hours. We generally approved of the processes that Ofqual suggested but it’s just been a lot of hard work.”
Jonny Kay, head of English and maths at a college in the North East, agreed that more focus should have been given to FE and the needs of colleges in Ofqual’s consultation, specifically those centres with thousands resitting. “It feels that there should have been additional information and guidance,” he said.
Is this system fairer?
In May, further guidance to centres ranking cohorts of 500 students and above was issued. In these centres, the guidance said, students could be given tied rankings when it came to grades 1 to 4.
The guidance sent out by AQA and seen by Tes, said: “It’s just for grades 1-4, as that’s where our analysis shows the bulk of your grades are likely to be. Here’s the process to follow: rank students in groups of 10. The groups of 10 should start at the top of the rank order for each grade (1-10, 11-20 etc). Any remaining students (in a group of less than 10) at the bottom of the rank order will be grouped together automatically. When you come to enter your data on our system, you’ll still need to give a specific ranking (e.g. 22 or 57) to each student within each group of 10, but you can do this randomly as we’ll treat every student within each group equally.”
Eddie Playfair, senior policy manager at the Association of Colleges, said that the granularity of having to assign each individual student a rank - as was suggested in the original guidance - had caused concern.
He said: “It seemed unreasonable to expect hundreds of students in a grade to be individually ranked - that would have provided a level of granularity to marks and would have been excessive.”
But even with that change, the process is still a lengthy one for staff, Mr Jones said.
He said: “When you’ve got 3,000 students, even with all of those sub-grades and subcategories there are still dozens at each level, and then you have to categorise them within that level so that’s been really hard. For colleges, it is just a different order of challenge.”
One GCSE English coordinator and college lecturer, who wished to remain anonymous, said that even with cohorts of under 500, ranking was “difficult, sensitive and painful”.
She said: “You are having to make valued judgements on students not just based on ability, but on how they would have performed if they had sat the exams. For our students how they would have performed in the exams doesn’t necessarily reflect their capability at all.
“I began by assuming it would be possible to have tied rankings, in real life more than one student will gain the same mark. When we realised that that wasn’t the case we did have moments of horror.
“In the end, the whole process is done with professionalism, the best will and the best information to hand. But it cannot substitute really and truly for a student sitting an exam. I do feel that some of our students will have been disadvantaged in this process, and yet some will have had an advantage.”
Mr Kay said that, despite the obvious challenges, the process was fairer. “It is a fairer system, simply because there are students who attend all sessions, really engage and who work through challenging circumstances, but who struggle with exams, and those students will have that performance reflected now. The result is more reflective of the amount of engagement students have shown.”
Unconscious bias
One central criticism of the assessment process has been that of unconscious bias. In April, the University and College Union (UCU) warned that research showed high-achieving, disadvantaged students were more likely to have their grades under-predicted than their wealthier peers.
But Bill Jones said that his staff had gone above and beyond to ensure that they mitigate against unconscious bias. He said: “Our teachers are the most committed in the country to equality, diversity and inclusion, but that’s the point of unconscious bias: you aren’t aware of it. We did a bit of analysis which showed that in line with national findings, we tend to under-predict the performance of our BAME students.
“To mitigate against that, we’ve introduced a range of checks and balances including blind marking and teachers cross-checking results without knowledge of the student’s ethnicity.”
Mr Kay said that he had seen staff across the sector worried about unconscious bias - and that some of his staff had sleepless nights over it.
He said: “I’ve seen staff across the sector worried about unconscious bias and trying to mitigate against it, but it could easily go the other way. For example, in trying to guard against unconscious bias, what if a teacher overcompensates it impacts other students?
“I’ve had staff say they have really struggled with this element of it and have had sleepless nights. They found the process accessible, but how they feel about saying their students are 43rd in grade 3 is something else.”
Student-teacher relationships
Mr Kay said he was also concerned about what the teacher-led rankings would mean for relationships with pupils in September.
He said: “If I’ve ranked Jimmy at grade 3, I’ll likely be teaching him again next year. I can’t share what I predicted him or ranked him at - if he gets that 3 and comes back into my class in September, he’ll feel it wasn’t a more ambiguous, cold, hard exam that he ‘failed at’, he’ll feel I ’failed’ him. He’ll have mixed feelings about being taught by me in future, as he’ll possibly feel that it wasn’t his effort or his achievement that ‘failed’, it was me - his teacher.
“Teaching is subject knowledge, pedagogy etc, but it’s also about relationships. You could be looking at thousands of students who feel an individual teacher failed them, and those relationships are really quite damaged.”
Mr Playfair said that colleges must stress to students that due to national adjustments, the grade they receive in the summer may not be the one calculated for them by their teachers.
He said: “There’s a risk that students may feel the process has some subjectivity in it. Students will need to understand that the grades they get in August may not be the one they were assessed by the centre, it doesn’t necessarily tell them what their teachers predicted.
“In colleges, the process is going through a lot of moderation and challenge within the college so it’s never going to be one teacher on their own predicting a grade, it will be done in context of the whole cohort.
“Students are clearly going to be anxious about the process: it’s not what they expected and it’s not what teachers expected either.”
