Britain has a teacher recruitment crisis. But it is not truly British. The complaint is much more spectacular in England. In Scotland, teaching is an attractive profession and while recruitment levels are disappointing, the issue is not as profound. The Scottish system is creaking; the English system has fallen over. What explains the difference?

The answer is simple. Scotland values a strong state educational system run by 32 local authorities that is staffed by well-trained and highly valued professionals who stay and grow in a secure and rewarding job. Teachers serve others, for most or all of their working life, in a cooperative profession that supports them to do this to the best of their abilities.

England no longer values these things. About half of its schools are now outside local authority control. England offers a business capital model that invests in education to yield short-term profits and keep down costs through shorter training, weakened security and tenure, and keeping salaries low by letting people go before they cost too much.

By comparison, Scotland models what is called professional capital: bringing in skilled as well as smart people; training them rigorously in university settings connected to practical environments; giving them time and support to collaborate on curriculum and other matters; and paying them to develop their leadership and their careers so that they can make effective decisions together and deliver better outcomes for young people.

In December, the Organisation for Economic Co-operation and Development (OECD) published its review of Scotland’s Curriculum for Excellence. I was one of four people on the review team. Like any system, Scotland’s isn’t perfect. But there is a strong foundation to build on - with a priority placed on valuing and developing teachers’ professional judgment.

When first minister Nicola Sturgeon opened the International Congress for School Effectiveness and Improvement in Glasgow in January, she underlined the importance of the “professional judgment” of teachers. From Glasgow to Stornoway, teachers told our team that teaching today had been like a “breath of fresh air”, which replaced a system of “counting minutes and percentages” that had offered “no room for movement”.

While Scotland is in the vanguard of global educational improvement, England is in the guard’s van, at the back. Worldwide, England’s business capital view is now on the run. The evidence of high-performing nations such as Canada, Singapore and Finland hasn’t been on its side, and countries like Sweden that followed the free-school business model, and saw their results collapse, are reversing course.

In response, there is a market-driven backlash. It takes three lines of attack:

1. Teachers already get too much

If there are different ways to get high quality, we should adopt the cheapest of them. The Department for Education’s mind-numbing “School Efficiency Metric” defines efficiency as “the rate at which organisations turn inputs… into outputs or outcomes. An organisation can become more efficient by…producing the same output with fewer inputs”. In other words, find the lowest-cost example and move everyone down to it. It may not be better, but at least it will be cheap.

This kind of thinking is a global movement, not an English aberration. In 2014, an international report on teacher efficiency was introduced by Lord Adonis but rescinded after former US assistant secretary of education Diane Ravitch blogged that it was the “worst ‘report’ yet”. It compared international performance results in relation to metrics of efficiency such as teachers’ salaries and class sizes. One of its jaw-dropping conclusions was that “if Switzerland, for example, were to decrease its teachers’ salaries by 48.5 per cent, while maintaining the pupil-teacher ratio, it could sit alongside Finland near the top of the Pisa rankings.”

A similar Canadian study concluded that teachers’ salaries should be reduced to the levels of the cheapest of the higher-performing provinces. It’s not hard to imagine how this would affect recruitment and morale.

2. Teachers don’t improve over time…

…and most of the money spent on their professional development is wasted. This argument came to the fore in 2015 in the US, in a report for The New Teacher Project called The Mirage: confronting the hard truth about our quest for teacher development. The report concluded that, despite massive investment in professional development, there was no large-scale systemic evidence that any particular strategies of continuous professional development were effective.

This did not deter the authors from jumping to conclusions that advocated the use of rewards and consequences in development, and that proposed relating compensation and (non-) retention of teachers to clear evaluations of performance. It’s worth noting that The New Teacher Project aggressively promotes non-university-based ways into teaching, and, as we would expect in the business capital model, is bankrolled by major corporate foundations.

3. Professional collaboration is overrated

A global movement has been growing in favour of more time and opportunity for professional collaboration and teacher leadership.

In general, according to the OECD and many research studies, schools and systems do better when teachers have more time out of class to collaborate and when they are involved in school decisions. But a paper by the London-based Centre for the Study of Market Reform of Education appears to conclude otherwise.

Despite its provocative title - Collaborative Overreach: why collaboration probably isn’t key to the next phase of school reform - the report is specifically focused on school-to-school collaboration where it reasonably says that only some of these collaborations are effective. The paper goes on to argue that collaboration is not incompatible with competition; that schools can collaborate and compete at the same time.

My colleagues and I have found the same in our study, Uplifting Leadership. But the sting in the report’s tail is the claim that the future of school collaboration “is both competitive and corporate”. This peculiarly English assertion is, of course, utterly out of line with the evidence of higher-performing countries where the corporate model is weak or non-existent.

So what is it to be for England: the vanguard or the guard’s van of teacher change? With or without free schools, academies and chains, where does England want its teaching profession to go next - to be one that can make high-quality judgments in an increasingly complex environment, or to be a standardised occupation that is flexible and cheap?

Andy Hargreaves is Brennan Chair in Education at Boston College, US. A version of this talk was presented at the Royal Society of Arts on 25 January

This is an article from the 19 February edition of TES. This week’s TES magazine is available in all good newsagents. To download the digital edition, Android users can click here and iOS users can click here

Want to keep up with the latest education news and opinion? Follow TES on Twitter and like TES on Facebook