“The research says...”
This is a phrase I read and hear a lot. It is sometimes used as a way of winning an argument: if “research says”, “research” (whatever it may be) must be right. However, whether or not it is right, it should always be questioned.
First ask: who wrote the “research”? Is this person writing about their own research, or are they a “research translator”? If it is the former, I am generally interested; if it is the latter, I want to know a little about the author’s background and engagement with those ideas. Have they engaged with the key texts, and to what extent have they represented the methodology and findings accurately?
If the “research” is a piece of work carried out and written or presented by an individual or group of individuals, my initial focus would be the research questions and the methodology. When I first started reading research, I used to rush to the conclusion, keen to find out the answer.
More: How FE runs edu-research on a shoestring
Background: How should we share quality research in FE?
FE research: It’s time to share pioneering pedagogy
My supervisor told me that as I progressed, I would become more interested in the methodology. At the time, I didn’t believe her (sorry, Gill), but she was so right. Now, before I read anyone’s conclusions, I want to be sure about how, and upon what thinking, they have based their findings. This isn’t always as clear in some work as it should be.
A piece of work by the Education and Training Foundation called Understanding the Role of Advanced Practitioners in English Further Education is a good example. It recommends that “generalist” APs are more effective in developing teaching and learning than subject specialists.
A side note: I have raised my concerns about this piece of work with them so this isn’t just an ambush, and partly as it seems fairer to address a piece of work attributed to an organisation than an individual.
As someone interested in subject specialism and subject-specialist pedagogy, I was fascinated to understand how this judgement was made. Unfortunately, in this case, only the executive summary was published, and this did not have methodological details recorded, so I can’t know.
Was it which AP style had the best feedback from those they supported? Maybe impact on a department? Was it which AP could hop across the room the quickest? It could be any of these, although I very much doubt it’s the third option.
It was important for me to understand this as the research ran contrary to my own thoughts. So much of what I think I know about teaching in the sector connects teaching pedagogy to a subject. Yes, there are “generalist” elements of teaching, but the reading I’ve done - notably the Transforming Learning Cultures project and the Gatsby Foundation’s work on subject-specific pedagogy - suggest to me that advanced practice is tied to subject-specific knowledge, pedagogy and practices.
The importance of a literature review
Which, rather neatly, leads me to what else I look for when I explore what “research says”. Does it look at the ideas and research of others? Good work uses this to demonstrate its understanding of the topic; it ensures it protects the ideas of others and allows their work to build on this. Research without a literature review is like a teacher without a teaching qualification - it demonstrates you don’t perhaps know all you should.
If a piece of research looks at the actions required to develop behaviour management, but doesn’t look at the literature about approaches to management of behaviour, you have to ask how people can make a judgement on the subject without being clear to you as a reader, or themselves as researchers, about what they mean. How do you know a target is hit if you are not explicit about what the target is?
My final question is: where is their data? If you can’t see them, if there are no illustrative quotes, charts or tables, then you can’t make a judgement about the extent to which data or opinion is building the conclusions drawn. If you can’t see the data, you can’t assess how strong the evidence is and you have to wonder what an author may have to hide.
These arguments are perhaps oversimplified, but without consideration of these issues you can never be sure about what “research” really says. We should be more discerning consumers of research and look at who’s written the work, how it has been conducted, how well the concepts used are understood and what assumptions and data the conclusions have been based on.
Sam Jones is a lecturer at Bedford College, founder of FE Research Meet and was FE Teacher of the Year at the Tes FE Awards 2019
