There are so many variables involved in learning that it sometimes feels impossible to take all of them into account.
This can be a point of contention when discussing research findings. For example, one might read a study and think, “Well, they did not take this factor into account.” Because it is unrealistic to include all variables involved in the complexities of learning, commentators probably are right that something is missing.
But does pointing out that something is missing really help to move a discussion forward? Should we stop complaining about studies not taking everything into account?
While no study can include all variables, the limitations of a study might directly influence the strengths of some of its conclusions.
For example, you might find a strong relationship between attainment and motivation, but if this comes from correlational data alone, you can’t suggest this is a one-way relationship. Instead, it’s likely that motivation leads to higher attainment but also that higher attainment makes you more motivated.
Is correlation reliable in research?
Issues around causality have plagued research for centuries. Philosophers David Hume and Immanuel Kant used to debate such topics in the 18th century. The former referred to causality as “a product of our experiences” and a “habit of mind”. In other words, he saw it as a kind of fiction.
Indeed, we only need to look at the website Spurious Correlations to see a raft of examples of data being used to make dubious claims.
Another example is a graph where a country’s ice cream consumption is mapped against its scores in the Programme for International Student Assessment (Pisa), showing a reasonable correlation.
On the one hand, this seems ridiculous. But, on the other hand, it’s not unreasonable to point out that both of these variables are related to a country’s wealth: richer countries do better on Pisa than poorer countries; richer countries also probably consume more ice cream.
Read more:
So where does this leave us when puzzling over the conclusions of education research?
Often we can do little more than try to unpick results as best we can, while remaining open to alternative explanations. This is especially challenging when multiple disciplinary fields come together.
But what is exciting about attempts to do this is that researchers try to expand what we know about education, rather than staying in one particular lane.
I was thinking about all of this recently when I read a study that tried to bring together elements from self-determination theory and cognitive load theory.
Paul Evans and colleagues aimed to advance both fields by “examining the effects of instructional strategies on learners’ experience of cognitive load, motivation, engagement and achievement”. By including multiple variables in their study, which involved more than 1,000 students in four Australian secondary schools, the researchers could look at the relationships between all of these variables.
Their findings suggest that if teachers use strategies to reduce cognitive load, students’ positive autonomous motivation, engagement and attainment will go up. However, the researchers make it clear that this is an “association”, leaving open the possibility that the relationship also works in the other direction. In fact, they note that the way teachers use autonomy, support and structure to motivate students and increase their engagement is also related to reducing their cognitive load.
When it comes to studies that show correlation, then, of course we need to be restrained in pointing to a cause or an effect. But we also shouldn’t dismiss them purely on the basis of not taking everything into account.
It’s not by being fixated on one view but by combining views that we will gain real insights into the complexities of teaching and learning.
Christian Bokhove is professor in mathematics education at the University of Southampton and a specialist in research methodologies
For the latest research, pedagogy and practical classroom advice delivered directly to your inbox every week, sign up to our Teaching Essentials newsletter
