Despite many years having passed, I still have nightmares about some of my former pupils. The Year 10 boy who made a blow torch under his desk from a deodorant can and a cigarette lighter. The Year 11 boy who threw darts across the classroom. The Year 9 girl who spat chewed up paper at me whenever I turned my back.

The boy I think about most was in Year 8. He sat, facing me, shirt tucked in, tie done neatly, homework completed on time. He watched expectantly, exercise book open and pen poised, as I first tried to get everyone to sit down, then to be quiet, find a pen, sit back down again, be quiet again, not wander off to the toilets, not turn round and chat to their friends, not eat sweets. He’d smile sympathetically.

Tales of fights, drugs and weapons provided fodder for staffroom one-upmanship. But it was the more mundane “low-level” disruption, the continual trickle of pupils arriving late and without equipment, the constant interruptions and infernal chatter, that was demoralising and exhausting.

Now, a report I have written for Policy Exchange sheds light on the extent and impact of disruption in schools. Drawing upon polling data and interviews with pupils, parents and teachers, it comprises one of the most extensive investigations ever conducted into pupil behaviour. The report’s title, ”It just grinds you down”, sums up how one of the teachers we interviewed felt about disruptive behaviour.

The good news is that the most serious bad behaviour - physically attacking or threatening to attack teachers or pupils, taking drugs, smoking or drinking alcohol - seems to be in decline. However, low-level disorder, such as using a mobile phone, listening to music or leaving a lesson without permission, occurs far more regularly. Some 75 per cent of the teachers we polled said low-level disruption and disorder occurred frequently or very frequently in their schools.

‘It’s not fair on the kids who want to learn’

A majority of the teachers we interviewed agreed that the quality of pupils’ education was affected by disrupted lessons. They told us that dealing with disruption takes time away from teaching. As one teacher noted, “You might have 10, 20 disruptions. How many minutes does that take off an hour that you’ve got with that class? How many lessons over the year does that take off? It’s not fair on the kids that are in there to learn.”

Pupils also told us that disruption got in the way of their learning: “Whenever I hear anyone, like just mumbling or giggling at the back, and when it’s constant as well, I feel like that’s all you can focus on when you’re trying to listen to the teacher.” Some pupils were acutely aware of the detrimental impact on their own education: “If you’ve got a question that you want to ask the teacher, and the teacher’s attention is being taken by someone who’s being disruptive, then nobody can really learn from that.”

Although many things have undoubtedly improved since my own days in the classroom, many issues around poor behaviour remain depressingly familiar. Some schools may have lower sets containing a disproportionate number of disruptive pupils. This can mean that a small number of pupils, often those who need the most support with their learning, experience an inordinate amount of disruption. A teacher told us that, “One girl, for example, could be with this group of naughty boys for every lesson of every day, and I think it grinds them down as well.”

The school I taught in had a clear behaviour management policy. Teachers, pupils, parents and the senior leadership team were all well-versed in the theory when it came to good behaviour. Problems arose because the theory was just not put into practice. Some colleagues, perhaps because of workload or exhaustion, opted to befriend rather than discipline badly behaved pupils.

A common complaint among the teachers we interviewed was that behaviour policies were not applied effectively or consistently. Only 23 per cent of teachers thought behaviour policies in their schools were very frequently applied - and then only for the most serious offences. As a result, those teachers who do enforce school rules come to be seen as confrontational by pupils used to an easier life.

Only 27 per cent of teachers told us they were very confident that they would have support from senior members of staff when it came to enforcing discipline. At the school where I taught, the punishment of last resort, doled out to pupils who had committed the most serious offences, was to spend the day sitting outside the staff room. Unsupervised but the centre of attention, most miscreants saw this as a thinly veiled reward. Too often, admitting to struggling with disruptive behaviour was seen as a personal and professional weakness rather than a straightforward call for support.

Our research shows how disruptive behaviour is damaging learning and teaching. We know what needs to be done: as is happening at the best schools already, all teachers need to be fully supported in consistently applying behaviour management policies. Education secretary Damian Hinds is right to say that, “Classrooms should be safe, calm and stimulating places for both our children to learn in and our teachers to teach in.”

Poor behaviour disrupts both learning and teaching, often most keenly affecting disadvantaged young people. The government has correctly pledged £10 million for schools to share best practice on behaviour management so that teachers can focus on the most important task - teaching.

Joanna Williams is head of education at the thinktank Policy Exchange