Judging the quality of a study is not an exact science, but there are certain hallmarks that we can look for to figure out how much faith we should place in the findings of a piece of research.
There are two aspects that it’s important to think about here: the design of the study, and whether this makes the research capable of answering the question that you are interested in; and any problems with the study that might represent “threats” to the validity of the findings.
When trying to judge a study, you are ultimately trying to assess whether it has been well designed, well conducted and well reported. It should be based on a clear research question, and it should use a sound research methodology, with limitations fully acknowledged. The data should be collected and analysed carefully, and all the analyses that were conducted should be reported transparently.
How does it work in the classroom?
Speaking to Tes, Professor Becky Francis, chief executive of the Education Endowment Foundation, said that in order for research evidence to have a “positive and empowering effect”, it is vital that all of those involved in education - from teachers and school leaders to policymakers - take the time to really “question, critique and discriminate” between the many claims of evidence-informed approaches that they encounter, rather than simply accepting them at face value.
As responsible professionals, teachers must “exercise professional judgement when assessing the rigour with which research evidence has been produced”, she said, and take time to consider the relevance of a research study to the particular context. Thinking about the feasibility of implementing evidenced approaches in your environment is one way of doing this, she offered, and so it may also be helpful to ask some simple but significant questions:
Will the approach need to be adapted to fit my local context?
How much organisational capacity might it require to embed the practice? Can we afford to make this commitment?
Are teachers and others likely to want to adopt the practice?
The Education Endowment Foundation (EEF) is an independent charity dedicated to breaking the link between family income and educational achievement.
To achieve this, it summarises the best available evidence for teachers; its Teaching and Learning Toolkit, for example, is used by 70 per cent of secondary schools.
The charity also generates new evidence of “what works” to improve teaching and learning, by funding independent evaluations of high-potential projects, and supports teachers and senior leaders to use the evidence to achieve the maximum possible benefit for young people.
