In many ways, teaching is an unusual job. It shares with other professions the requirement that individuals make decisions with imperfect knowledge, but, unlike other professions, there is no shared knowledge base - no set of facts that all involved in doing the job would agree on. The late Professor Ted Wragg, of Exeter University, once suggested that if you got a group of headteachers together, the only thing you could get them to agree on would be the need to pick up litter.

Unlike most other professions, teachers spend much of their working lives without much contact with other professionals, marking or planning work on their own, or with their children in the classroom, and although the widespread use of classroom assistants has changed this a little, in most schools, teamwork is not a major part of many teachers’ typical working day.

Teaching is also different from most other professions in that there is a continuing debate about whether professional qualifications are needed - something that would be unthinkable in medicine or the law. In five years’ time, we will have a shortfall of 7,000 doctors and 66,000 nurses and doctors in the NHS. Solution? Let’s just adapt the “Troops to Teachers” scheme. I don’t think anyone would seriously suggest that.

Tes Podagogy: 10 podcasts to change the way you teach 

Pisa: Comparisons ‘not useful’

Research: ‘Some research is good - but experience is better’

A further difference between teaching and other professions is the role of research, and who creates it. Very few teachers are involved in academic research, and the vast bulk of published research in education is produced by academics in universities who are rarely involved in teaching the students that are the focus of their research. In other professions, much of the published research is done by those who are still practising.

The role of research in education

The traditional explanation of this is that teaching is not really a profession at all. It is, at best a “partial profession”[1] because its practitioners are not as skilled as those in other professions. While such criticisms do have some measure of truth, I think this is at least partly due to the fact that in the professions that appear to be more “professional” this is not because the state of knowledge is any more advanced, but because those professions have focused on the “low-hanging fruit” and choose not to talk very much about the issues where the evidence is less certain.

For example, in medicine, it is widely agreed that while antibiotics can be effective in the treatment of bacterial infections, they are ineffective against viral infections. This is part of the “knowledge base” in medicine. Except we know that medics often prescribe antibiotics for infections that are likely to be viral in origin. This could be for a number of reasons. Doctors may be prescribing antibiotics “just in case” they are wrong about the viral nature of the infection. It may be that busy GPs feel pressured into giving patients treatments that are unlikely to be effective just to be seen to be doing something (which may, after all, be effective because of the placebo effect). Whatever the reason, what actually happens in what Andrew Pickering calls “the mangle of practice”[2] is a lot more complicated that we generally assume.

More importantly, even in areas where there is a reasonably strong research base, like medicine, that evidence does not provide guidance for a lot of what doctors need to do.

For example, there is reasonably strong agreement amongst medical researchers that it is important that patients complete a course of antibiotics rather than stopping as soon as they feel better, because an incomplete course can result in leaving alive the most resistant bacteria, which will then multiply rapidly. Doctors, therefore, need to make sure that patients understand why, even if they feel better, it is important to complete a course of antibiotics. We don’t know much about the best way to do this, because this is an extremely difficult topic to research, requiring an understanding of the quality of the relationship between the patient and the doctor, and a whole host of contextual factors.

In my view, teaching appears to be less “professional” than other professions because the problems that teachers need to solve are just much harder. Physics works because protons and electrons don’t have good days and bad days; they behave consistently, and predictably. As soon as humans are part of the picture, things get a lot more complicated.

And that is why I do not think that teaching will ever be a research-based profession. Classrooms are just too complicated for research ever to tell teachers what to do. Teachers need to know about research, to be sure, so that they can make smarter decisions about where to invest their time, but teachers, and school leaders need to become critical consumers of research - using research evidence where it is available and relevant, but also recognising that there are many things teachers need to make decisions about where there is no research evidence, and also realising that sometimes the research that is available may not be applicable in a particular context.

What this means, I think, that those who call for “evidence-based education” are missing the point. Evidence is important, of course, but what is more important is that we need to build teacher expertise and professionalism so that teachers can make better judgments about when, and how, to use research. Two things seem to me to be particularly important here.

First, I think, is that rather than treating teachers as technicians, where we have researchers figuring out how best to teach, and then telling what teachers to do in Lawrence Stenhouse’s memorable phrase, treating each teacher as a kind of “intellectual navvy”[3] who is told where to dig, but not why - instead we have to recognise that teacher expertise cannot be put into words. The kind of knowledge that expert teachers have is more like the knowledge of how to ride a bicycle than it is the knowledge of how to solve quadratic equations. I can explain to someone how to solve quadratic equations, but I cannot explain to someone how to ride a bicycle. Each person has to figure it out for themselves. There may be guidance I can give, but there is no set of instructions that will be guaranteed to work.

Second, while there are certainly differences in how good teachers are in their first year of teaching - there is such a thing as natural talent for teaching - what matters far more is the improvement that teachers make, and this is why I think we have failed to make the best of the work of Carol Dweck on mindset. She has shown that some students believe that ability at something like mathematics is more or less fixed - you are either good at mathematics or you are not. This “fixed mindset” as she calls it leads students to believe that when you fail, this must mean you have reached your highest level of achievement, and you might as well give up trying to get better. Other students think that ability is more of a malleable quantity - captured nearly by Jeffrey Howard in the phrase, “Smart is not something that you just are, smart is something you can get.”[4] For these students, when they fail, it just means they need to try harder. More importantly, challenging work is not a threat to your self image - a chance to get shown up - but rather a chance to get smarter.

My point is we have been applying Carol Dweck’s work to students, and I think we should be applying it just as much to teachers. We all fail as teachers, because we have such high hopes for our students. But the teacher who does not think she or he can get better blames this failure on the students. “What can you expect from students from that part of town?” “From that estate?” “With those kinds of parents?” They fail because all teachers fail, but teachers with a fixed mindset blame this failure on the students: “I taught it, they just didn’t learn it.”

In contrast, the teacher with a growth mindset says, “What else can I try to help this student learn?” and this has powerful implications for the whole teaching force.

First, a belief that you can get better as a teacher is the key to staying positive about the job. Whatever the management gurus say about the positive possibilities of failure, it is not pleasant. People say “you learn from your mistakes” but that’s not quite right. It is much easier to learn from mistakes than from success, but without a desire to improve, failure is just failure. But with a growth mindset, failure is a chance to learn, to improve. I meet many teachers who have improved their practice over the years, and they tell me that they often feel sorry for the students they taught when they were younger. I feel the same way. But the alternative - that we do not improve as teachers - is much worse. It means that every student we ever teach gets the same deal.

Second, and much more powerful, a focus on improvement can transform the teaching force, because it detoxifies professional development. In many schools, in Scotland and elsewhere in the world, managers often operate a kind of educational triage. Figure out which teachers can’t be helped, and get rid of them. Figure out which teachers are OK, and leave them alone. And for the ones that are left, the ones who are “struggling”, help them get better. With this approach, professional development, or indeed support of any kind, is toxic. Being offered help is a way of saying that you aren’t very good, and the goal is therefore not to be identified as being in need of help, and pretending that everything is OK.

If, on the other hand, we accept that every teacher needs to improve, not because they are not good enough, but because they can be even better, professional development becomes welcome - it is just the way we become better.

And that is why I am so proud to be associated with the work of the Tapestry Partnership, and in particular, to be here tonight, presenting these Professional Recognition Certificates from the General Teaching Council for Scotland (GTCS). The Tapestry Partnership has worked with local education authorities all over Scotland to support teachers in their quest to improve their practice. The teachers here tonight have shown that they are committed to improving their practice. They have undertaken enhanced, significant and sustained enquiry to support the development of their professional learning in a particular area of their practice. They are a model for us all.

After all, if we create a culture where every teacher believes they need to improve, not because they are not good enough, but because they can be even better, there is no limit to what we can achieve.

Dylan Wiliam is emeritus professor of educational assessment at University College London.

This is a transcript of the speech he delivered this afternoon at an awards ceremony for teachers who have taken part in the Glasgow in Partnership with Tapestry’s Supporting Improvement: Pedagogy and Equity Programme. Some 197 Glasgow teachers are receiving GTCS Professional Recognition awards today, while 90 recently received GTCS Professional Recognition for the Tapestry Making Thinking Visible programme.