In the US, education researchers are scratching their heads. Three randomised controlled trials of three different professional development programmes (Garet 2008, Garet 2016 and Jacob 2017) have turned up results that nobody can quite explain.

Each of the three professional development programmes was carefully designed around the prevailing research consensus of what constitutes good professional development - the same consensus embodied in the UK’s Standard for Teachers’ Professional Development.

These studies should have confirmed what we already knew: follow the guidelines, and the CPD should be effective. Teachers who participated should gain new skills and their pupils should learn faster as a result.

However, according to the latest research studies, none of these three expensive, carefully-designed CPD interventions increased student learning - in one study, students’ results were actually slightly worse after the CPD. What went wrong?

Some researchers wondered if it had been hard to show an impact because teachers had dropped out of the studies part-way through, unbalancing the experiment. Others have suggested that the interventions were poorly implemented.

But three studies all finding no benefit from apparently well-designed CPD made us stop and think: what if the problem lies in our understanding of what constitutes good CPD in the first place?

We began looking back at the research to try and understand where the consensus had come from. A year of digging has revealed important gaps in our collective knowledge about the characteristics of effective CPD.

Here is what we learned (you can read a full explanation of this in our paper, see box, below, for details):

If you are responsible for CPD at your school, you will no doubt use the government’s Standard for Teachers’ Professional Development for guidance (see bit.ly/StandardTeach).

The standards are clear, evidence-based and reflect something of a consensus within the research community that CPD is more effective if it:

* Is sustained
* Is collaborative 
* Has teacher buy-in
* Is subject specific
* Draws on external expertise
* Is practice-based

But bad CPD is still far too common. Most teachers have listened to “inspiring” speakers with their own agenda, and no idea what’s going on in the school; they have read best-practice case studies a decade out of date; they have attended one-off training unconnected to anything. This CPD is not aligned with the standards: it’s not sustained, subject-focused or practice-based, and teachers are usually not engaged. Little changes as a result.

But how sure are we that CPD that does conform to the guidelines will work? We used to be pretty certain. Then we read about the three experiments summarised above.

All three followed the consensus about what constitutes good CPD: they were sustained, offering at least 40 hours of CPD over a year or more; they were collaborative, involving groups of teachers working together; they encouraged teacher buy-in, often by ensuring participation was voluntary; they drew on the expertise of experienced external trainers. But none of them improved pupil attainment.

So does this mean that the standards are misguided? We looked at the evidence base for the standards. They were developed based on reviews of research on effective CPD programmes, in particular Developing Great Teaching (bit.ly/DevTeach).

This review found 980 relevant studies of CPD, but concluded that the overwhelming majority were insufficiently rigorous. It highlighted one, by Timperley and colleagues, as being the “only fully consistent and rigorous review”, which became “a cornerstone” of the Developing Great Teaching paper.

How do we know if a CPD programme is effective? Take the analogy of testing whether a particular brand of toothpaste works. To discover whether Toothpaste A or Toothpaste B is better at preventing cavities, I might ask some children to use A and some to use B, and then count cavities in their teeth three years later. Crucially, however, this will only be a fair test if the two groups of children are very similar. If one group eats twice as much sugar, the study will be flawed.

The same principle applies to testing CPD programmes: if we are testing Programme A against Programme B, we need to be confident that the teachers in Group A are similar to those in Group B. Otherwise, it’s not a fair test.

Timperley and her colleagues included 11 studies of secondary CPD programmes in their review (from a total of 97). These included:

* A study without a control group (testing Toothpaste A without a comparison group using Toothpaste B)
* Two studies that collected only qualitative data (testing perceptions of the toothpaste, without counting cavities)
* A study that is no longer available on the internet, or from the author (making it unclear which toothpaste was being tested)

The Education Endowment Foundation provides a scale to judge how robust a study is, from zero padlocks to five. Of the studies in Timperley and her colleagues’ paper:

* Three would receive zero padlocks
* Seven would receive one padlock
* One would receive three padlocks

So, the review that forms the “cornerstone” of Developing Great Teaching relies on 11 studies, of which only one would score more than one padlock with the EEF.

As we were reading through the reviews, another concern arose. Developing Great Teaching sought to identify what successful CPD programmes have in common. This sounds reasonable, until we go back to our toothpaste analogy.

Toothpaste has many ingredients. If we look at several kinds of toothpaste that reduce cavities, and identify common ingredients, we will conclude that mint flavouring prevents tooth decay. But, of course, mint flavouring does no such thing: it is not an active ingredient in reducing decay.

Similarly, if we look for common features of apparently effective CPD programmes, we may wrongly conclude that incidental features are key to their success.

Many reviews suggest that “collaboration” is a characteristic of effective CPD, for example. Collaboration could be crucial. But it could also be that many effective programmes bring teachers together because it’s a cheaper way to provide CPD. Concluding that collaboration made such programmes work is like concluding that mint flavouring prevents tooth decay.

Similarly, many reviews advocate “teacher buy-in”, but this is just as likely to be a consequence of experiencing effective CPD as a cause of its effectiveness.

Why are these weaknesses in the consensus view only now emerging? Translating research into practice takes time. It takes years to design, run, analyse and share the findings of a study, for example; and longer still for such studies to be included in reviews and national guidance. It also takes time to sift through and assess the accumulated mountain of evidence.

So if the evidence base for what we are told is effective is shaky, what should we be doing instead? How can we identify characteristics of effective CPD?

Finding programmes that are linked to improved pupil outcomes is a good start, but if we are to identify the active ingredients of these effective programmes, we also need to understand why they work. What mechanisms explain their success?

Take, for example, the way that a link was established between heart disease and smoking. Researchers found evidence of a correlation between smoking and heart disease as early as 1976. But they did not jump to the conclusion that smoking caused heart disease. Perhaps smokers also did less exercise, and therefore had weaker hearts, for example.

Researchers only accepted a causal link when they discovered the mechanism linking the two. Careful research showed that smoking alters cholesterol in the blood, leading it to eventually block blood vessels and cause a heart attack. At this point, there was both evidence of correlation and evidence of a mechanism. It became hard to resist a causal link between smoking and heart disease.

Similarly, identifying the characteristics of effective professional development requires us to both find a correlation (using a strong research design) between a CPD programme and positive results for students and to identify the mechanisms that explain why a programme works.

We believe the most powerful candidates for such mechanisms come from the study of human learning (cognitive science) and human behaviour (behavioural psychology). We give one example here to illustrate our thinking. Interested readers will find more detail in our full paper.

One type of CPD that seems to have a powerful effect on student learning is instructional coaching: frequent, individual, targeted guidance for teachers on small steps to improve.

But what are the active ingredients in coaching? What is essential, like the fluoride in toothpaste, and what is incidental, like the mint flavouring?

Careful studies have shown that CPD often fails to bring about sustained change in teacher practices. Psychologists investigating this have shown that habits - automated behaviours, built up through repetition, and triggered by cues - are the most important reason that people maintain a behaviour. Crucially, we can be confident about this mechanism, because it holds across a very wide range of settings - from car use, to recycling, blood donation and voting.

Coaching incorporates characteristics that are known to promote habit change. Most notably, coaching programmes require teachers to repeatedly practise new skills in their own classrooms. Practising new techniques in the environment in which you aim to reproduce them in future (ie, the classroom) helps to change old habits by overwriting established cue-response relationships. The repeated review and feedback incorporated in coaching models helps to strengthen these new cue-response relationships even further.

This evidence of mechanism (for habit change through repetition in the classroom) combined with evidence of correlation (from trials showing that coaching improves pupil attainment) suggests that this type of embedded practice is an active ingredient of coaching.

This approach provides a more reliable way to identify the characteristics of effective CPD, using a model that is standard practice in medicine.

And it is important that we do find these effective ways of training teachers. It takes years to become an expert teacher. Even the very best initial teacher training cannot hope to give teachers everything they need to master the craft. Good CPD is critical to ensure that teachers can keep getting better. Research suggests that good CPD can also help to improve retention in the profession.

But we cannot provide good CPD unless we know what it looks like. Our year digging through the evidence underpinning the consensus on what constitutes good CPD has revealed some important gaps in our knowledge.

Clear, evidence-informed guidance is useful and the CPD standards are an important innovation that reflected a consensus among researchers. There is also a great deal of useful material in the standards that goes beyond the consensus view we have focused upon here.

But research in CPD has advanced in recent years, and we believe that it’s time to build upon the existing standards to incorporate the latest evidence and thinking.

As we have argued, we believe the best way to make progress on this front is to look for characteristics of professional development that are supported both by evidence of correlation with improved pupil outcomes and evidence of mechanism. We hope policymakers will consider updating the standards based on these principles, to ensure they remain valuable to teachers.

School CPD leaders can make use of this new approach without waiting for the standards to be revised. Teachers’ greatest expertise lies in understanding how people learn and behave. They know that students forget things, form habits and need practice to reach proficiency. CPD leaders should design and commission professional development that reflects these principles.

Good professional development is critical. Let’s build on the foundations that the CPD Standards provide, to ensure teachers get the best on offer.

Harry Fletcher-Wood leads the Fellowship in Teacher Education programme at the Institute for Teaching. He blogs regularly at improvingteaching.co.uk and tweets sporadically at @hfletcherwood; his book, Responsive Teaching: cognitive science and formative assessment in practice, is out now. Sam Sims is a quantitative education researcher, interested in teachers and education policy. He works at the Centre for Education Improvement Science at the UCL Institute of Education. He is also a researcher at Education Datalab

The full story

We hope we have whetted readers’ appetites sufficiently to read our full paper, which can be found here. There are six steps to our argument:

• We show that there is a consensus, by showing how the same key ideas are described as characteristics of effective CPD in a number of reviews
• We discuss the three papers cited above in detail, demonstrating how they followed the consensus carefully, but were ineffective
• We look at how the consensus came about, showing that the reviews included studies which were not sufficiently strong to identify effective interventions 
• Moreover, we show that even if the studies included the reviews had been rigorous, looking for common characteristics of effective interventions is logically flawed
• We propose that characteristics of effective CPD should instead be identified by looking for both 1) effective school CPD programmes that include a specific characteristic 2) evidence of “mechanism” which explains why this characteristic works, across a range of settings.
• We suggest that these mechanisms can be found in psychology, specifically, cognitive psychology and behavioural psychology
• Finally, we explore what this means for the CPD Standards and how they could be revised

The authors of Developing Great Teaching have responded to this article here: https://www.tes.com/news/what-makes-good-cpd