Every now and then, discussions about the value of education research flare up on social media. 

Sometimes, this is because someone has encountered a genuinely “bad” piece of research - although, more often than not, the person also happens to fervently disagree with the findings of that research.

I have no qualms about criticising studies; as a researcher, I do this all the time. But I object to when simply not liking the findings is used as an excuse to write off an entire study. Worse still is when one piece of supposedly “bad” research is used to make generalisations about education research as a whole.

Ironically, one common criticism I see being levelled, especially at qualitative research, is that a limited number of interviews from one school shouldn’t be extrapolated to draw conclusions about education overall. Yet this is exactly what happens when you interpret the quality of one study as being representative of all studies in that field.

We can’t treat education research as one big homogeneous category. Neither can we compare it, as some like to do, with research into cognitive science or the “science of learning” - because this is education research, too.

In fact, any study focused on schools or learning can be considered “education research”. This includes research in fields such as sociology, psychology and economics, as well as classroom studies.

Within each of these fields, there is also a multitude of research methods being used. This means that making an accurate judgement about the quality of a study depends on you knowing enough about that particular study’s design.

For instance, would you really be able to judge a “regression discontinuity design” at face value? Do you know enough about “topic modelling”? And while doing some interviews and a focus group might seem simple, the systematic content analysis that follows certainly isn’t.

Of course, everyone is entitled to like or dislike the findings of the studies they encounter. But criticism about research quality - or indeed about an entire field of research - must be based on more than that. 

Obtaining enough knowledge to make more accurate judgements is, admittedly, not easy, especially given the size of the field. 

One way that I try to expand my own understanding is to immerse myself in broad and diverse education conferences. For example, I recently attended the annual meeting of the American Educational Research Association. Their meetings typically attract over 10,000 participants and show just how diverse education research can be. 

At these conferences, I deliberately seek out sessions covering topics I am less familiar with, or speakers who hold views I don’t necessarily agree with. I think this is important to do because it is all too easy to dismiss ideas without ever engaging with them. 

Sometimes my worries about a piece of research remain, even after fully engaging with a topic, but at least I can now better articulate why I disagree. 

To be more intelligent consumers of research, we all need to step out of our evidence echo chambers. Branching out through conferences is one way to do this. But we can also challenge ourselves to truly engage with those studies that we instinctively want to write off as “bad” - and be honest with ourselves about what lies behind that instinct.

Christian Bokhove is associate professor in mathematics education at the University of Southampton and a specialist in research methodologies