There’s nothing more disappointing in education research than coming across yet another intervention trial that shows null results. 

When this happens, it’s easy to feel disillusioned with the “evidence revolution”. But we shouldn’t be. Each finding, whether positive, negative or neutral, provides insight into what works and what matters, as long as we use those results as a starting point for discussion, not as an endpoint.

I was reminded of this at a recent conference when the results of a reading intervention were presented. Those results were tentative and not yet reviewed by a journal, but it was apparent that the long-term effects were disappointing, even negative, with the benefits not lasting after the intervention had finished. 

This is not uncommon; many interventions suffer from a phenomenon called “fade-out”, where the results don’t persist after the intervention. 

This is illustrated in a 2020 paper, in which Drew H. Bailey and colleagues reviewed the evidence for persistence and fade-out, drawing on evidence from educational interventions. Their study focused on fade-out in the form of decreasing impacts after the end of the intervention. 

The review led to several conclusions. The first was that fade-out is widespread and often coexists with persistence. For example, students learning to solve equations might experience some forgetfulness or setbacks but still be expected to perform better if they receive a particular intervention than if they do not.

In fact, Bailey and colleagues did not find any meta-analyses to support the existence of full persistence; in other words, there was always some degree of fade-out. 

However, they did find that some interventions that were aimed at a broader set of skills or capacities appeared to have generated substantial benefits with some degree of persistence. In these cases, the review showed instances of persistence in educational interventions where fade-out was followed by the emergence of long-term benefits.

A second conclusion was that there are many causes of fade-out. We can’t blame the phenomenon entirely on publication bias, or on differences between what is measured right after the intervention and what is measured later on (for example, measuring concrete skills after intervention and then later looking at exam results). Some explanations of fade-out describe the psychological processes that occur after the interventions, including very normal occurrences of forgetting or failing to transfer learning. 

But while fade-out might be normal, that doesn’t make it desirable. So, what could make interventions more persistent? Bailey and colleagues offer some solutions. 

They recommend that interventions target malleable, fundamental skills that would not have developed in the absence of the intervention. 

They also point out that the environment that is conducive to developing these skills will need to be sustained to maintain those skill advantages. Persistence, therefore, depends on school factors and opportunities within the social context. 

Ideally, there will be an overlap between the intervention and what the school would normally offer. This is different from just continuing an intervention indefinitely - something that is usually not realistic given the costs of maintaining the same support, rigour, mentoring and resourcing available during a particular intervention. 

Instead, we want the effects to sustain without all those scaffolds.

Christian Bokhove is associate professor in mathematics education at the University of Southampton and a specialist in research methodologies