We are told that the days of pen-and-paper exams are numbered, but that number seems to be a constant: it’ll be 10 years before high-stakes public exams are routinely digital. Next year, no doubt, it’ll still be 10 years. Faith in the necessary transformation amounts to millenarianism. So, to fill time while we await the rapture, let’s consider the implications of ditching treeware for technology.
Merely going paperless is beside the point. Once tests are digitised, interesting things become possible, in principle. Tests no longer need to be fixed in time (cue on-demand testing), nor do they need to be linear, nor one-size-fits-all (enter adaptive and multistage testing).
On-demand testing is a reality in medical and military aptitude assessment and some HE admissions in the US. It seems appropriate for any situation in which baseline mastery or threshold competence is required. Personalised learning would seem to fit well with the concept of entering a candidate when s/he is considered ready, rather than waiting for the official window - which could be months away.
With computerised adaptive testing, a candidate who gets a particular question right is then faced with a more difficult question. If an answer is wrong, the next question would be a less difficult one. This goes on until some “termination criterion” is met and the candidate’s ability is defined. Adaptive tests can be shorter and still maintain precision - research shows that they are more precise at both higher and lower levels of the ability spectrum. And the results are immediate - there’s no need to wait until August…
Of course, there are wrinkles. A huge bank of questions is needed, and they all need to be pretested. Candidates might find it difficult to allocate their time appropriately if they don’t know how difficult subsequent questions are going to be. These sorts of tests are most appropriate for multiple-choice items, or those which, like US SATs, require analogies, fill-in-the-blanks and synonyms.
Multistage testing is another algorithm-based approach in which tests can be built in stages. Groups of items can be presented in a sequence appropriate to the ability of the candidate. This is surely a smarter approach than tiering.
But is any of this remotely relevant to high-stakes public exams such as GCSEs? Arguably, yes, if it leads us to rethink the purpose of GCSE. On-demand, adaptive testing has advantages over fixed-length, linear, pen-and-paper formats when it comes to high-stakes summative exams. What precisely do we want to know from GCSE, given that it is no longer the terminal qualification? It should tell us whether a student has reached a critical threshold of proficiency or mastery. As a summative test, GCSE exists for the purposes of demonstration rather than development.
If we reconceptualise GCSE as a threshold or gateway, where what matters is that a candidate has reached a particular level, then the digital dawn promises a real breakthrough.
We should not look to technology simply as a way of administering our current arrangements more efficiently. We should use the opportunity to transform not just how and when we assess, but more fundamentally, why.
Kevin Stannard is director of innovation and learning at the Girls’ Day School Trust. He tweets @KevinStannard1
