“Ofsted do not know good teaching when they see it,” a government adviser and expert on teacher quality has claimed. Professor Dylan Wiliam of London University’s Institute of Education is challenging the watchdog to show some “integrity” by subjecting its school inspections to an evaluation of their reliability and publishing the findings.

“This a challenge to Ofsted to say `How much of the variation (in teacher quality) are you capturing? How good is your framework?’,” Professor Wiliam, a member of the national curriculum review expert panel, told TES. “If two inspectors inspect the same school a week apart with no communication between them, would they come to the same ratings? Shouldn’t that matter?”

He argues that the inspectorate should show more “humility” because its short, snapshot observations of lessons cannot possibly be a reliable measure of the impact teachers have on pupil progress. “If I showed you a 30-second clip of a game between Manchester United and Manchester City and I said `Who won the game?’, you would say it’s ridiculous to try to predict the outcome of a 90-minute game on the basis of a 30-second clip,” Professor Wiliam said. “And that is what we are trying to do with Ofsted, isn’t it? Instead of saying a teacher is going to teach a kid for 200 hours, they are looking at 30 minutes.”

The watchdog said it was “very confident” that it accurately captured “the essence of good lessons”. But Mary Bousted, general secretary of teaching union ATL, said Professor Wiliam had Ofsted “bang to rights”.

“He has hit the nail absolutely on the head. It is the thing that Ofsted constantly shies away from: doing a proper evaluation of how good those judgements are, both within inspection teams and between them,” she said. “When you consider that so much rests on judgements and the emphasis moving to classroom performance, Ofsted has to answer these questions about their quality control.”

Professor Wiliam said he knew of schools that were given overall “outstanding” verdicts, with a “good” for teaching, even though their contextual value-added scores showed their pupils were making less than the average national rate of progress.

Ofsted’s new chief inspector Sir Michael Wilshaw and education secretary Michael Gove have both expressed disquiet that many schools have been rated outstanding overall when their teaching did not receive the top rating. Sir Michael signalled that this would end in November, saying: “A `good’ school will have good teaching. An `outstanding’ school will have outstanding teaching.”

Professor Wiliam questioned the inspectorate’s ability to judge teaching at the North of England Education Conference in January. He said that research showed an American framework for judging teaching captured no more than a fifth of the impact that teaching quality could have on pupil progress. The “available evidence” was that Ofsted’s framework was even less accurate.

“I am not saying that Ofsted doesn’t know what it is doing,” Professor Wiliam told TES. “I am saying that teaching and learning is so complex that it’s very hard to predict how much kids are going to learn in a year by looking at a sample of half an hour.”

An Ofsted spokesman said that its criteria for assessing teaching quality were “carefully formulated from international research that identifies those features of teaching that have the greatest impact on pupils’ learning”.

“Inspectors now spend even more time in classrooms,” the spokesman added. “As experienced teachers and headteachers themselves, inspectors know how to recognise lessons that fire the imaginations of pupils and give them the confidence they need to make good progress.”

Photo: Professor Dylan Wiliam by Nick Sinclair

Original headline: Teaching expert challenges Ofsted to prove its `integrity’