One of England’s main school exam boards has said that using questions students have answered before to determine GCSE and A level grades this year will not “necessarily” give them an advantage.

Teachers and heads have previously raised concerns over whether students given questions  - set by exam boards to help teachers assign grades this year - that they have already seen will create unfairness through familiarity, or encourage rote learning of stock answers. 

In response to these concerns about this year’s GCSE and A level grading researchers from the OCR board looked into whether predictable questions were advantageous for students, and they found that “it does not seem to radically affect performance”.

Exclusive GCSEs 2021: Grading may rely on past papers

GCSEs 2021: Teachers criticise ‘shocking’ exam material

GCSEs 2021: Teachers mock ‘hilarious’ grade descriptors

An OCR spokesperson told Tes: “There is an interest in this for summer 2021 given that questions from past papers may be available online and some students will have better [or] earlier access to those materials....

‘Not necessarily an unfair advantage’

“Our research around this area seems to show that advanced sight of questions does not necessarily create an unfair advantage.”

OCR revealed that earlier this year its researchers had told a stakeholder forum: “All of the work suggests that people do not necessarily do significantly better on items [questions] which they have seen some time in advance of a full exam or test.

“There are complex mechanisms and effects in play (disappointment at not seeing specific questions appear, wasting time looking for questions which they have done before, trying to retrieve from memory the specific answer (and getting stressed) rather than simply getting down and answering the question from scratch, etc).

“In some settings, the approach to pre-seen questions can be disadvantageous re performance. But the key thing is this - it does not seem to radically affect performance or favour people.”

The researchers acknowledged that candidates could experience relief that particular questions came up in exams.

Less stress ‘a good thing’

But that meant a “reduction in stress” which was “a good thing re accuracy of measurement, since stress is a confounding factor in normal times”.

Research they drew on included a 2015 US National Board of Medical Examiners paper - Repeat Testing Effects on Credentialing Exams: Are Repeaters Misinformed or Uninformed?

Based on 338 medical students retaking a multiple-choice test of 200 questions, it showed that students who sat an identical test in their retake did not receive any statistically significant advantage.

“The lack of advantage for [students who sat the same test a second time] is not surprising given that those retesting have low proficiency, the tests themselves are developed to be long and cover a broad content domain, and only minimal performance feedback is provided,” the US paper says.

“These clearly are not the conditions that foster memorising or learning the content of specific items from an examination,” it adds.

However, the paper is based on students who did not realise they would be sitting the same exam for the second time - which may not be true of GCSE students this year, who may only have three or four past papers in recently-reformed syllabuses they can draw upon.

Candidates ‘might try to memorise questions’ 

“Examinees who knew that the identical form is administered on retesting might alter their test-taking behaviour by trying to memorise questions, which could negate the results from this study,” the 2015 paper acknowledges.

The paper adds that the study is useful in showing that “on a high-stakes credentialing examination same-form examinees do not benefit so long as they don’t know they’ll be encountering the same items again”.

OCR’s research also drew upon a 2008 Ofqual study of question predictability, based on analysis of GCSE and A level exams.

It found that for some subjects, for example A level English literature, “predictability does not necessarily make an examination easier”, because it “produced many answers which were little more than regurgitations of prepared material”.

And for A level psychology, the Ofqual paper found that in some questions “candidates simply repeated the taught material, once again illustrating the point that generally predictable examinations often hinder rather than improve candidates’ responses”.

A further 2020 Ofqual study on question paper predictability found that exam questions should have a “sufficient” amount of predictability and made a “distinction between ‘good’ and ‘bad’ predictability”.

”‘Bad’ predictability is when qualifications are structured in such a way that teachers and students can work out, pretty reliably, what the questions on a future examination paper are going to be about,” Ofqual concluded.

‘Predictability is not always negative’

But they added, “predictability is not always negative and some level of familiarity or predictability is ‘good’”.

The OCR spokesperson added: “It’s worth remembering that student performance on questions from past papers (whether seen or unseen) will be only part of the evidence used this summer to determine teacher assessed grades (some schools may not use the assessment materials at all as use is optional).”

When Tes reported that past papers would be used as part of the grading process for GCSE and A level students this year in February, sources close to the discussions said this raised the prospect of students simply learning stock answers to past paper questions by rote, rather than studying the syllabus.

And last week, when exam boards released assessment materials for GCSE and A level grades, teachers condemned them as “shocking” because of their reliance on past papers, raising concerns that students would be able to learn stock A* answers by rote.

One teacher pointed out that as specifications are “only a couple of years’ old, there’s only about three to four papers in existence for each subject anyway.

‘They will prep the hell out of those few questions’

“It narrows the question range hugely,” they said, adding that in a cohort of able pupils they taught “they will prep the hell out of those few questions and smash them. They will all get A or A*”.

Guidance from the Joint Council for Qualifications on how teachers should approach this year’s grading states that: “It is understood that students may have seen some material previously.

“The purpose of any materials should be considered before they are included in the range of evidence. It would be inappropriate to advise students on the content of any up-coming assessment.

“If a student has recently completed a particular activity there may be little benefit to them completing the same or a very similar activity again.”

An Ofqual spokesperson said: “We expect teachers to use multiple sources of evidence to arrive at a grade, and to use their professional judgement.

“Externally set tasks are optional and just one of the many sources of evidence which will be used by teachers. A wide range of questions will be made available by exam boards so while students will have access to them all in advance, they will not know which ones if any (as the use of exam board materials is optional) their school or college will use.”

“We would expect a teacher confronted with evidence from classwork that gives one view of a student’s performance and evidence from the external tasks that gives a very different view to explore further which is likely to best represent their true level of attainment.”