There are a few educational books and articles that have been popular over the past decade among those involved in the evidence-informed “revolution”. In many cases, such books and articles are a great contribution to the translation of research into practice. However, sometimes the situation is less clear cut.
In the past I have written about Rosenshine in this column and how some of his ideas transformed into misconceptions and how some perhaps weren’t as robust as commentators thought. Some people take such comments as a sign that I don’t think his ideas are useful, but that is not the case. I just think that if we want to call ourselves truly evidence-informed, we sometimes need to query our idols.
One popular book is The Hidden Lives of Learners by Graham Nuthall. This book is based on Nuthall’s seminal work and is mostly focused on very detailed exploration of a small number of students’ experiences in the classroom. Firstly, I should first underline that it is an extremely insightful work that will resonate, especially, with every classroom teacher.
Treating education research with caution
Of course, there is nothing inherently wrong with enormously detailed qualitative work with small samples. It does become a problem, however, if claims become stronger than the data analysis actually warrants.
But this only comes to light if you dive deeper into Nuthall’s work, looking beyond just what is reported in this popular book. It was really difficult to follow up some of his claims; referencing could absolutely be better in the book, especially at points when he makes strong claims. Let me lift out two statements.
Read more from Christian Bokhove:
One popular statement in the book concerns the three-times rule: that the equivalent of three complete definitions or descriptions of a concept are needed to construct it in students’ long-term memory. This has been brought up to argue for spaced practice, and that, of course, is reasonable.
It is easy to take findings like these as absolute and generalisable. I have also seen these results reported as replicated, which means that the findings are confirmed in a separate, new study. But, in fact, Nuthall uses the term within his large overarching study in which every study only looked at three or four students. As I said, it’s insightful work, but this is not a firm basis to argue the three-times rule from, in my opinion.
Another widely reported and blogged finding from the book is often used to criticise peer feedback: that 80 per cent of the feedback that students receive daily is from peers and 80 per cent of the time the feedback is inaccurate. I have not been able to confirm this statement.
Now, I’d like to think that I’m pretty experienced in deciphering research, following up references and more, but in this case it was an absolute nightmare. From my current explorations, the percentage seems much lower. But even then, the number of students involved hardly makes it reasonable to make absolute statements on this.
Unfortunately, my observation is that the caveats are seldom added to rather robust statements that peer feedback isn’t a good thing. Furthermore, such statements seem to routinely to ignore other literature on the topic.
Sure, like most pedagogical approaches, peer feedback in the classroom has to be organised well, because if you don’t do that, it might actually backfire and create misconceptions. I even understand that in deciding what approaches to use in the classroom, you might favour other approaches. That’s all perfectly fine.
But if you say you support evidence-informed teaching, and then contort evidence like this or downplay limitations or strengths because this better fits with your diatribe against peer feedback, I would say that’s evidence-uninformed.
Christian Bokhove is professor of mathematics education at the University of Southampton and a specialist in research methodologies
For the latest research, pedagogy and practical classroom advice delivered directly to your inbox every week, sign up to our Teaching Essentials newsletter
