I weigh up my children’s stories from school like a judge sizing up a serial offender. If they tell me that they spent the day being a ninja, perhaps they did. Or perhaps it is a playtime fantasy that has nothing to do with what they were actually learning in class. They don’t have any obvious “tells” and in the world of primary education, anything is possible (and long may that continue!).
So it’s not easy to spot a lie or a part-fabrication. My judgement ends up being a call based on the available evidence, and the lack of that usually makes the final decision only marginally better than a guess: is this more likely to be fact or fiction?
More often than not, in all areas of life, the threshold at which the available evidence compels us to believe is set by context. For ideas that we are opposed to, the threshold is extremely high. For ideas close to our heart, there is barely a threshold at all. I would need a lot of convincing that having one less sugar-rich snack each day would directly improve my quality of life. But studies that show that the food I enjoy leads to improbable health benefits? I’m an instant believer.
This is natural. We shouldn’t beat ourselves up about reflex judgements that we make. But we should always be aware that our view is heavily skewed by bias, and react accordingly.
Education research: What is solid evidence?
What should happen is that we set ourselves an independent threshold for evidence that does not move. If we demand a randomised controlled trial as “proof” in one instance, we should require the same in other circumstances. If we dismiss a sample size as being too small, or too far from the context we wish to apply the findings to, then that should be our requirement at all times.
Which brings us to education. From the education secretary down, this is a sector that now prides itself on being “led by the evidence”. This has, largely, been a welcome change: it seems sensible to reflect on what you do with the help of external stimuli that may challenge your thinking.
But have we ever had a proper conversation about what level of evidence is meaningful in education? Or the level of evidence that should be required before a teacher is asked to change their practice? Or the level of evidence the government must meet in order to enact changes to the system?
It could be argued that such arbitrary thresholds would stifle innovation and, ultimately, be unworkable - after all, it’s not just about evidence of effectiveness but also factors such as cost, ease of adoption, unintended consequences, and so on. In theory, an intervention meeting an arbitrary standard for effectiveness may mean that funding gets stripped from elsewhere and a teacher is left doing double the work.
And if we were to review all practice according to a common threshold, suddenly schools would be banned from doing so much. The research simply isn’t there for so many things that happen in the classroom.
There’s also the risk that minimal-evidence thresholds would devalue an important part of teaching: experience. Teachers run their own studies of thousands of pupils across a career and we should be careful not to dismiss that experience as invalid.
Yet, like it or not, “evidence-led” is now the currency by which a teacher’s role is directed, so we do need a proper discussion about what that actually means. The Education Endowment Foundation has done a tremendous job of attempting a standardisation of evidence by which schools can benchmark their decisions, but rarely do you see it pop up in the justifications for the calls made at any level of the system. Instead, you have each individual working within their own parameters of evidence and, rather than bringing clarity to education, it’s bringing countless arguments.
“Evidence-led’ still too often translates to “ideology-led”, and that is stalling progress.
So, what’s the solution? Schools are difficult places within which to find true answers because of the extraordinary number of variables involved, but if we can control the variable of ideology by recognising our bias and mitigating against it, we would be much more effective than we would be otherwise.
Jon Severs is the editor at Tes. He tweets @jon_severs
