I’d like to think the Labour Party’s declaring social mobility a failure combined with their decision to abandon it for social “justice” is a triumph of research over marketing but I’m not that naive. You don’t change the tune by changing one note.

Having waded deeper than most into the social mobility research quagmire, I’ve long understood that schools are not engines of social mobility, that the best they can do is change individual lives. Anyone who seriously expects to see quick intergenerational shifts in family fortune nationally either needs to do something dramatic about the economy or replace compulsory Shakespeare with Jane Austen, and teach children to marry above their station. Nothing else will shift the immensely stubborn forces that longitudinal researchers into social mobility like John H Goldthorpe at Oxford, or Gregory Clark at UC Davis, have shown to be at play.

In his recent book with colleague Erzsébet Bukodi, Social Mobility and Education in Britain, Goldthorpe states unequivocally: “In sum, it is difficult to avoid the conclusion that, rather than the idea of Britain as a low-mobility society being derived from any serious examination of the body of relevant research - whether relating to income or to class mobility or to absolute or relative rates - it is essentially a political construction…an idea that has been found convenient to advance in support of the attempt, made across party political lines, to form a response to increasing inequality of condition in Britain, primarily through a policy emphasis on raising levels of mobility.”

For anyone working in education who has ever felt their heart warmed by the seductive message that they are fuelling social mobility, some of Goldthorpe’s discoveries from his earlier work may come as something of a shock. Using the Office for National Statistics seven NS-SEC categories, which, put simply, are based on employment roles, Goldthorpe discovered that between 1946 and 1980 the proportion of men and women in social classes 1 and 2 tripled and in classes 6 and 7 it halved. In 1972, the so-called golden age of social mobility, of the men in managerial positions in those top two social classes, almost 50 per cent had no formal educational qualifications at all - none, zero. These shifts simply reflect the way the economy changed and employment roles changed within it.

In other words, there are three times as many people today working in the top two employment tiers - employers in large establishments or members of higher professional occupations - than half a century ago, while the number working in routine sales, agricultural or other production occupations, including the unemployed, has halved, because that’s what the economy demands. Increasing numbers of individuals now start out in more advanced social positions and so they are less likely to experience upward mobility and more likely to experience downward mobility. It will be interesting to see what similar longitudinal studies have to say about employment and NS-SEC class changes resulting from the financial crisis of 2008.

Figures like this ought to have been enough to scotch any idea that policies aimed at changing what schools and teachers do would impact on social mobility. But that would ignore the huge industry dedicated to influencing schools and teachers for benefits that have nothing whatsoever to do with children or their education, and everything to do with personal and organisational politics. The forces that combine to impose their will on teachers are potent, well-funded and deeply entrenched in processes completely outside ordinary teachers’ spheres of influence. Against a wind of change on this scale, it’s no surprise that teachers’ professional voices are completely drowned out.

Yet here we are, and one half of a two-party system falling apart at the seams appears to be listening to the evidence.

All of this made me reflect on the widespread international interest that undoubtedly exists to use research evidence to improve education. Even the most enthusiastic voices have recently acknowledged how difficult their optimistic mission is, and how even the most apparently rigorous research projects turn out so often to be both flawed and ineffective.

I’ve long been frustrated with the poor quality of so much educational research design, yet simultaneously, like many others, I’ve a lot of faith in the basic principle that if you try something out and it works, then why shouldn’t other schools and teachers benefit?

I believe it was the Canadian Michael Fullan who some years ago pointed out just how important local conditions were in any school improvement effort and since then the idea that you cannot easily transpose a successful idea from one school to another has become commonplace. Listening this week to some teachers from substantially different schools who had been working collaboratively talk frankly about their successes and failures, it struck me that we have the entire educational research process back to front.

Precisely because the entire influencing industry doesn’t trust teachers as professionals, because it has an agenda it wants to deliver through them and not for them, research is normally externally commissioned, designed and imposed. In stark contrast, through genuine partnership, these teachers had decided on their own goals, designed their own projects and done their best to evaluate them, all tightly within their local context. What they did was designed entirely for the benefit of their schools and their pupils.

What I realised listening to their entirely credible account of the changes they’d achieved was this. That is all we should ever expect in terms of research, from any professional teacher. They shouldn’t have to apologise for not knowing where or how to publish results, or not using statistical methods university-based researchers might use. The evidence they generate merely as professional teachers doing the job should in itself be worthy of consideration by anyone seriously interested in school improvement. The only responsibility they have is to explain what it was they did and what they believed the impact to be, in their schools. The rest is literally none of their business.

So instead of teachers dancing to a tune played by others that leads nowhere, all those eager Pied Pipers should stop playing and start listening. Especially those who think all you need to change is one note.

Joe Nutt is an educational consultant and author. He tweets @joenutt_author