When is an exam not an exam?

When it’s a “task” set for this year’s GCSEs and A levels.

Yes, that task may actually be a test paper containing questions set by an exam board that are “similar in style and format to those in normal exam papers”.

And it may be accompanied by an exam board mark scheme, and it may be the key means of ensuring that this year’s grades are set at comparable standards.

But this “task” is not an exam, definitely not an exam. Got that?

GCSEs 2021: Silence on the biggest injustice of all

In full: GCSE and A level 2021 Ofqual and DfE proposals

GCSEs 2021: 4 reasons why more unfairness is inevitable

In full: Williamson’s letter to Ofqual on GCSEs and A levels

Ofqual and the Department for Education aren’t due to report back after going through more than 90,000 consultation responses to their proposals on awarding this summer’s GCSE and A-level grades, until next week. But the question of these non-exams has already emerged as a key point of contention.

It was a big enough concern to prompt Ofqual chief regulator Simon Lebus to publish a new blog tackling the issue “head on” on the last day of the consultation, insisting that the proposed tasks are not “exams by the back door”. 

Heads unconvinced by ‘pseudo exams’

Headteachers however seem unconvinced. Their two unions have referred to the proposed tasks as “mini exams” and a “pseudo exam system” and are calling for teachers to be given more flexibility over their use.

Unfortunately for our exam authorities and government, the linguistic gymnastics they have tried to deploy when describing their proposed replacements for this summer’s cancelled exams are just the start of their problems.

They’ve ended up with a strange contortion of a solution - something there’s no way you would have arrived at if you had started with a clean sheet of paper. It feels like such a forced and elaborate compromise that it could fall apart at any moment (and all the signs are that it already has).

And it feels like you need at least an A-level in subtext, assessment, U-turns and pandemic politics to unpick how we got here.

To understand the current situation let’s start off with the question of why these proposed “tasks” are not being described as what, in so many ways, they actually are - exams.

Cancelling exams was the easy bit

The key reason, of course, is that in January our prime minister decided to cancel all summer exams. It is understood that was not what education secretary Gavin Williamson and his department wanted. But it is what many, though far from all, teachers regarded as inevitable and, in the end, sensible.

How would it be possible or “fair” to run exams as normal when so many of this year’s candidates will have missed out on so much of their exam courses? That was Downing Street’s logic when it decided to cancel all GCSE and A-level exams after closing schools to most pupils, and that case only become more compelling when we learned that schools would not reopen until 8 March at the earliest.

But cancellation was the easy bit. Working out what to do next is when the problems start to close in and you realise that most of the obvious solutions are simply not available.

Algorithms out of the question

Using a statistical algorithm to set grades according to the distributions of results in previous years now has about the same popular and political appeal as a Dominic Cummings DIY eye test.

And simply asking teachers to calculate grades based on what they thought their students would have achieved if the exams had gone ahead is also a non-starter following last year’s U-turn debacle.

Why? Because it is now clear that candidates need some agency in the process if there is to be any hope of avoiding more uproar when results are released. Some students are always going to get grades they were disappointed with, but that is taken to a whole new level if they are failed in exams they were not even allowed to sit.

Using official coursework to decide grades could make sense except that most GCSE and A-level coursework was dispensed with under Michael Gove’s exam reforms.

The advantages of (non) exams

So where does that leave you? The government has said grades must be decided according to teacher assessment. But students need to have some agency in the process. So the grades should be based on something they have done, but coursework is not an option.  So you can see why the DfE and Ofqual, presumably after talking to exam boards, decided that exams - or something very similar to exams - were actually the only real answer.

It may also be because the system is trying to replicate what it is used to. Ofqual chief regulator Simon Lebus - a former exam board chief - has readily admitted that when he agreed to take up what has become a very hot seat, he was expecting to oversee a summer of exams. And when you’re in the kind of crisis precipitated by the sudden cancellation of those exams, the most obvious solution is not to start completely fresh but to modify what you already know.

But it’s not just that - these non-exams ought to be useful in assessment terms. As Mr Lebus has pointed out, they “would help teachers by providing them with an external reference point, giving them greater confidence in the grade they were awarding”.

They also have some inherent advantages for Ofqual and the exam boards who, it is proposed, will be checking the teacher grading process. Externally set “tasks” would allow some level of standardisation and give the boards something they could check and compare against when it came to appeals.

But of course the tasks must not be described as exams because that’s what the government has cancelled. And that’s why we’ve got the proposed GCSE and A level “tasks”. They are essentially exams that have had several changes made, perhaps partly as part of an effort to brand them as something different. But these changes have in themselves the potential to create some big new problems:

Problems

1. Exam boards would not be doing what they are best at

The boards have spent decades developing an efficient system of marking, standardising and grading millions of GCSE and A levels every year. But now the plan is to hand over that Herculean task to teachers, with the exam board experts almost observers in the background, carrying out the odd check here and there where they deem it to be necessary.

Again, you can see how we’ve ended up here. If exam boards did mark and grade their own tasks then how could anyone really argue they weren’t exams - and of course exams have been cancelled. Secondly if you have the “teacher-assessed grades” the government has insisted on then you can’t really risk exam boards undermining or second guessing them with their own grades for “tasks”. And so the whole burden must fall on teachers.

But it does raise the question of what schools will actually be getting for their exam fees, especially as there is now a suggestion that the boards won’t actually be producing any new material.

2. Teacher workload

The profession has probably never been under more pressure than it is today. Many teachers are already effectively having to do two jobs in one - taking lessons in person whilst also offering remote education. On top of that they are also often scared about the prospect of catching Covid as they go into schools while much of the rest of the population works from home.

Now it is being proposed that they take on a huge new area of high stakes responsibility - assessing and grading for this year’s GCSEs and A-levels.

But teachers worried about their workload have described the idea as “utter lunacy” and questioned why they will be expected to carry out a task for free that examiners are usually paid for.

Mr Lebus has sent out mixed messages on this. On the one hand the Ofqual chief has acknowledged that “teacher workload is an issue”.  “Teachers are going to have to take on the heavier responsibility of managing this additional assessment on top of all the extra work they’re having to do to make up for lost learning time,” he has told Tes.

But he has also denied that the “extra marking burden” will be much greater than in normal years.

Some might point out that teachers should be getting used to grading their students for GCSEs and A-levels having gone through the process last year. But as others have already suggested, even without the 2020 “algorithm” and U-turn debacle, this year will be much, much harder.

It’s not that just the 2021 cohort will have been in school less, suffered more disruption and completed less work for teachers to base their grades on. In fact, awarding grades may be only half the battle. It’s students’ reactions after they have received their results that may end up being teachers’ biggest problem.

3. Appeals

In 2020 Ofqual decided that schools, but not students, could appeal against the grades it had moderated. It also took the same position after the infamous U-turn that saw moderation abandoned, sticking to the line that students would not be able to challenge their teachers’ grading decisions.

“Any appeal would have to be undertaken by someone better placed than a student’s teachers to judge their likely grade if exams had taken place and we don’t believe there is any such person,” Ofqual ruled.

But the regulator now apparently believes that there is such a person after all. Under the the Ofqual/DfE plan for this year a student “who believes their teacher has made an error when they assessed their performance in 2021 should be able to appeal to their school or college on that basis”.

It proposes that the appeal should be considered by a teacher not involved in the original assessment, with extra time built into the results timetable to allow such reviews to take place.

They would have to review “the marking of any papers provided by the exam board or the school or college itself, the marking of any non-exam assessment” and “the other evidence used by the teacher to arrive at the overall grade”.

In other words, teachers face a huge triple whammy from these proposals. It’s not just that they will be in the difficult position of having to decide the high stakes grades that will determine their students’ life chances. They will also then face the prospect their decisions being appealed and potentially overturned, with inevitable damaging consequences for their relationships with students.

And if all that wasn’t enough, in addition to the huge extra workload caused by the pandemic, exhausted teachers will also have to take on the burden of deciding on appeals against other teachers’ grading.

4. Huge potential for cheating

It is teachers who are being expected to run this proposed non-exam exams system. And it is teachers who would be expected to ensure that it has some integrity. But under the DfE and Ofqual proposals it is difficult to see how that will be possible.

Their plan acknowledges the risk of cheating but doesn’t seem to do enough to counter it.

It says exam board papers “should be used by teachers within a set period of time”. It cites the obvious reason that: “If students who are completing the papers do so at different times there is a risk that students taking the papers later in the window might be at an advantage, particularly if the content of the papers is leaked.”

But having the papers sat “within a set period of time” is unlikely to cut it. Instant online communication means that unless the papers are sat exactly the same time their contents are surely bound to leak.

The Ofqual/DfE plan to “manage the risk” by having a “menu of papers” for teachers to choose from, “deliberately published shortly before the assessment window opened” feels more like a nod to the problem rather than a concerted attempt to fix it.

And the opportunities for cheating don’t end there. The plan also acknowledges that the pandemic may mean it is not possible for all students to sit these papers and be assessed within their school or college.

So, it says “the papers could be completed at an alternative venue, including a student’s home”. It’s a suggestion that should trigger a huge cheating klaxon. How can you possibly ensure the integrity of a (non) exam if you’re not even in the same building? If the student is in their own home, they could have rigged up any number of aids to give them an advantage over those sitting the test in school.

But fear not, the proposals state that: “If any evidence used to determine a final assessment was not completed under the supervision of a teacher (either directly or remotely), the student (and anyone supervising them) would be required to make an appropriate declaration that they had not received unauthorised assistance.”

That should solve it then. What student would possibly risk cheating in the face of those cast iron safeguards?

And the potential for malpractice only seems to have grown since the plans were first published. This week Tes learned that these non-exams, if they come to fruition, are likely to be based on questions from past papers rather than any new material from exam boards.

But GCSEs have only just been reformed. Depending on the subject, there will be a maximum of three years papers to choose from. That raises the prospect of grades secured by students learning stock answers to past paper questions by rote rather than any actual mastery of their subjects. 

5. Grade inflation

The issue of the level of performance that grades should be pegged to is perhaps the least clear element of the DfE/Ofqual plan, and that’s putting it kindly.

The consultation doesn’t explicitly ask this crucial question at all. And, as I wrote earlier this week, the deeper injustice of differing levels of regional and local Covid disruption has not been addressed.

What has been made very clear is that there will be no “algorithm” used to standardise and moderate teacher grades. “Changes to teachers’ grades should be the exception and will only be if the grade could not legitimately have been given based on the evidence,” the consultation says.

“Grades will only be changed as a result of human judgement, not by an algorithm.”

The political rationale behind this decision is entirely understandable, following the huge controversy last summer. But it does at a stroke remove the comparable outcomes mechanism - the algorithm that has kept GCSE and A-level grading stable for the last decade or so.

Some insiders believe that if teachers are given careful guidance from exam boards and can use exam board test papers then grading will end up some where in line with 2020. But there is no pretence that there will be any way of ensuring this happens. This will be a bottom up process, a question of getting things ready and then seeing what teachers come up with.

And that’s perhaps why one Ofqual adviser has already warned that exam boards will have difficulty restraining “wildly inflated” teacher-assessed GCSE and A-level grades this year, resulting in “Weimar Republic levels” of inflation.

Whatever happens - whether grades end up at pre-pandemic 2019 levels (highly unlikely), in line with the generous record-breaking unmoderated grades of 2020, or even higher - Covid is rapidly ending the idea of our exams system being a common currency, broadly comparable across the years.

No perfect solution to GCSEs and A levels 2021

So, in summary, what’s been proposed is essentially an exams system with added downsides - more workload for teachers with more risk for their relationships with students - and with safeguards against problems like grade inflation and cheating stripped away.

Yes, it might solve last year’s big problem - the uproar created when students got a set of calculated grades they had no agency in. But little has been done on this year’s burning issue, differential Covid learning loss, and it has added big new problems into the mix.

Unsurprisingly there has already been considerable pushback. And insiders are questioning whether the mini exam, non-exam exam, concept will survive at all. But if it doesn’t then what?

You could do what heads are calling for and make the exam board papers optional - the consultation is written as it if that was the plan anyway. But would that leave with teachers any real choice? If they are being expected to do something that must be comparable in demand to exam board papers, with mark schemes, sat at the same time, won’t time pressed teachers just end up using the boards’ papers?

Even the NAHT school leaders’ union, which does not want teachers “constrained by a pseudo exam system” supports “a strong recommendation” for their use of exam board tasks.

But the idea that they could be completely made up of questions from a small set of past papers, brings with it a whole new set of problems as we have already discussed.

So what if you did dispense with them altogether? The first issue is that you would then be back to the issue of students needing to have some agency in their results. If grades are just based on teachers’ opinions - even if they are informed ones - then you risk storing up problems for another summer of discontent.

Secondly exam board papers haven’t been suggested for fun, they have a purpose, setting a standard that teachers can use as guide. Some teachers might welcome the freedom they get without exam board tasks and without an algorithm moderating their grading. But if that results in a free-for-all it will inevitably mean inconsistency and therefore unfairness for some students. It will also further undermine the credibility and worth of the grades they receive.

More to the point, without “mini exams”, hard pressed teachers would not only be doing the assessment, the marking, grading and appeals. They would be setting the tasks as well - effectively running the entire GCSE and A level system

So the Ofqual/DfE proposals may be riddled with problems, but so are the obvious alternatives. It may have become a pandemic cliché, but there really is nothing close to a perfect solution to the 2021 exams conundrum. Whatever form the finalised plan takes when it is released next week, it will inevitably be an unsatisfactory compromise that will need careful explanation and understanding on all sides to make it work.