It all started with a study in the journal Nature Communications. It suggested that teachers were right all along.

Our professional instinct is that pupils succeed best when we teach at a level that is just one step beyond the knowledge with which they are comfortable. Ideally, we want the work we set to be a little difficult, engagingly hard, but not impossibly so. A little failure should be expected.

Scientists at the University of Arizona took that instinct and tested it. By studying machine learning, a type of artificial intelligence that improves through experience - which, for all our insecurities about the superiority of the human intellect, is a plausible metaphor for at least some of it - they found that their computers learned best when, in pattern-matching tests, they obtained the correct answer only 85 per cent of the time. Thus, in machine learning, some failure is clearly both necessary and desirable.

Robert Wilson, one of the scientists involved in the study, told Science Daily that “these were ideas that are out there in the education field - that there is this ‘zone of proximal difficulty’ in which you ought to be maximising your learning; we’ve put that on a mathematical footing”.

Well done, us.

Of course, machine learning is not the same as human learning. And when reading research about education, it’s good to maintain a healthy scepticism. But a success rate of 85 per cent for any given task certainly feels about right. If all the children in a class get everything right, then the level is too easy; if less than half, then the work is too difficult (or there has been a failure in teaching).

And that number strikes a (be it slightly dissonant) chord with research of the 1970s. “Teachers must be effective in diagnosing learning needs and prescribing appropriate activities. Their questions must usually (about 75 per cent of the time) yield correct answers,” stated Brophy and Good in 1984.

There is a rhyme, too, with the goals of so-called mastery learning. A conventional curriculum allocates a fixed amount of time to a predefined unit of learning, and accepts and notes the results of its final test; by contrast, a mastery approach extends the time available to each unit, only moving on to the next stage when all of the pupils have achieved a score in the test of “usually 80 per cent or 90 per cent” (EEF, 2018).

However, looking at that Nature Communications study made me curious: what if we aimed for beyond one step further than a student is capable, and what if we did not hit that 85 per cent figure? What then?

The challenge

A useful challenge presented itself that enabled me to look into these questions. Barnaby Lenon, former headmaster of Harrow and currently dean of education at the University of Buckingham, set the following provocation in his book Much Promise: successful schools in England: “Try teaching your more motivated key stage 3 pupils something at A-level standard and see how far you get” (Lenon, 2017).

Thinking about difficulty as I was, this obviously appealed. Surely, such a task would see the success rate plummet below 85 per cent? Surely, I was about to damage their learning? So I landed on the idea of a rather extreme test: teaching one of my Year 7 classes (Shell C) how to write perfect cadences in four-part harmony. This is a requirement of some but not all A-level syllabuses in music, and in reality is often not sufficiently mastered until the second year of undergraduate studies in music.

Cadences and four-part harmony are tremendously important in the understanding of so-called absolute music (music without the crutch of text or narrative), a fundamental of musical literacy that is either ignored in schools because it is considered to be too difficult, or else presented as an exercise, fenced-off with a sharp tangle of rules and technicalities and prohibitions.

So, could I take such a difficult task and hit 85 per cent success, suggesting our aspirations for our students were often too limited? Or would I fail, and if so, could I find anything useful in the experience of setting the difficulty level of the lessons so high?

The planning stage

I am lucky: I usually teach four-part harmony in Year 10, so I duly dusted off my plans and materials to see what might work with Shell C. Alas, I realised rather quickly that those plans were completely useless for the task ahead.

The first problem with teaching something usually taught in Year 10 to Year 7 is the three years of learning, experience and maturity that are sandwiched in between. We know that new knowledge sticks to old and my Year 10 plans were based on the secure footing I would have provided in the years previous. My Year 7s did not have the benefit of that - and as such, the plans were not usable. So I sat down to create new plans, and a second issue emerged: how much did my students know already, without my teaching?

One of the joys of working at my school is the diversity of the boys - diversity of background, as well as the diversity of their prior musical experience. Some come to us as quite accomplished musicians; others have their first experience of music with me. So I really had no idea about the level on to which I could try to stick this new knowledge.

I thought about how to overcome this. The trick, I decided, would be to allow time and to manage the space in such a way that the more experienced boys’ knowledge and innate generosity could add resonance to, and leverage, my direct instruction.

A carefully considered seating plan, a quick check of the more experienced boys’ work for the most common misconceptions (in order that they did not lead others astray), and a certain tolerance for conversation when boys were clearly teaching each other - all this was to be required here. It also needed me to have the intellectual self-confidence to allow correct and viable explanations to emerge that were not the same as my own.

It appeared that planning was going to be much less important than my questioning in class; my checking for understanding; the moment-by-moment flexibility of my feedback; the subtle responsiveness of my teaching to the boys’ needs; and the understanding that the order in which I presented new material should depend more on the boys’ prior knowledge than on what I had prepared in advance.

The more you think about it, the more true this is of any situation. Here I was, planning for an extreme, but in normal lessons and with more delicately pitched content just outside what we think is the student’s ability, we still don’t know with 100 per cent accuracy the knowledge students bring into the classroom. Thus, our planning for a lesson just outside the reach of students is likely to rarely be that accurate. We need to adapt to this difficulty. And that’s difficult in itself.

The lessons

I began by being honest: the first lesson was used in part to explain what we were going to do, and why; not just explaining why perfect cadences in four-part harmony mattered, but also my intention to write this article, and how we were going to explore learning together. The boys of Shell C were my knowing collaborators in this throughout. I wanted their willing feedback and support to improve the project as it went on.

That done, we were off.

It was a fascinating exercise to telescope at least three years’ teaching and to focus it on four 40-minute lessons. The sequencing of information was a key part of being able to do this.

A student needs to know about scales in order to build chords; but in order to know about scales, he needs to understand simple intervals; and to understand intervals he needs some note-reading skills; and to achieve note-reading skills, a passing familiarity with the piano keyboard is very useful.

This species of ramifications, outwards and backwards, resulted in some of the most useful thinking I’ve ever done about my craft.

So from the sequence to the communication: I knew that time in class to explain would be very limited as there was just so much to get across. I took pains to practise and rehearse the explanations beforehand; practise synchronising my explanations with my board work (dual coding); and make sure each was concise and in accordance with my understanding of the limits of working memory.

Trusting Graham Nuthall’s rule of three - that a student needs, before it is understood, to have encountered a particular piece of information on three separate occasions - I worked out when I would repeat the students’ encounter with each element I considered vital, tried to vary the context in which it was given, and helped the boys to do it for me.

The VLE (virtual learning environment) was a vital ally in all this, a place where I could require the boys to engage in their own time. It was used for testing - both unmarked tests, on which the boys could receive automatic feedback, and which they could repeat for practice; and others on which they received my feedback.

This made it possible to give individual feedback to each of 26 boys, once or twice in each lesson. It made it possible for the most experienced boys to work much faster than those just starting on their musical journey, and for me to spend more time with those who needed the confidence that a teacher’s presence should give.

As you can see, I gave this the very best chance of succeeding: I deployed every tool in the cognitive-science toolbox and optimised every second I could optimise.

Then, suddenly, the four lessons were over and I had to find out whether any of it actually worked.

The results

I wish I could report a neat final test that presented a satisfying narrative in which 85 per cent of the boys had mastered four-part harmony. It would be marvellous to know that a glut of research-informed teaching could lead every student to new heights of attainment; that the 85 per cent rule could still be reached if we extended our aspiration from work being one step beyond capability, to several steps beyond. Reality is messy, though, and more interesting.

A healthy proportion of the boys mastered the task and I’m proud of them for that - unsurprisingly, they were those with the prior knowledge to make the leap that much narrower (though this was certainly still more than one step beyond). For them, it appears true that with a firm grasp of effective teaching tools and flexibility of planning, a teacher can still ensure those with enough prior knowledge can reach an 85 per cent success rate.

But it’s the outcome for the less experienced musicians that is much more important and much more interesting. Although this group cannot yet write four-part harmony, their success in the individual components that the task required was striking, their knowledge of how to do those quite advanced things - which I usually teach as discrete and separate tasks - far better than usual for these groups. Which is to say that they now read the bass clef more fluently, that their understanding has improved of how major scales work, and they name and compose with simple intervals impressively well.

I think this is because those skills were taught in a sequence of lessons as subsidiary parts of a more difficult whole, rather than allotted the same amount of time as standalone lessons. In short, by aiming for a more difficult end goal, and in not hitting the 85 per cent mark, they learned more than in discrete lessons in which I aimed for an 85 per cent success rate across the board.

Of course, this is in no way scientific; there is no meaningful data here, neither are there control groups. I chose Shell C because it was convenient to teach them this on Monday mornings and because it’s particularly fun to work with them; but there are three conclusions I would like tentatively to offer.

First, there is no limit to the potential of children - the boundaries of their knowledge are “mind-forg’d manacles” too often locked by us as teachers - and belief in possibility, confidence in their ability to learn and high expectations can change everything.

Secondly, in my own teaching I am going to talk more often, more openly and rather unflinchingly about big and difficult ideas. I am going to do this with growing confidence, knowing that if I think through carefully enough the preceding stages of knowledge, that if I practise and hone my explanations of the understanding implicit in the big idea, and that if I present the stages in a sequence that leads steadily from one to the next, then the worst that can happen is that I will teach the younger and less experienced musicians some of the foundational stages well. If I can find the level of ideas - through experimentation and practice over time - at which a significant majority of my charges come with me all the way, so much the better.

And my final point is this: engagement with research is a jolly good thing. When we engage with it, we reflect on what we teach and how we teach. Ultimately, working with ideas about teaching, and not just about teaching, will make us better teachers. Even if we engage with that research and understand less than 85 per cent of what is being said.

Martin Leigh is director of music at King Edward’s School, Birmingham

This article originally appeared in the 20 November 2020 issue under the headline “How to fail perfectly”