Our dream - fanciful now as it seems - was to improve the school results of poorer children by empowering teachers through evidence-informed education.

Ten years ago this month, we unveiled the Pupil Premium Toolkit. We couldn’t possibly have imagined how this simple guide, setting out education’s best bets, would revolutionise thinking in classrooms.

And yet a decade on, we’ve become convinced that another step change is needed: we need major reforms to the education system if it is to have any chance of fulfilling the potential of pupils from all backgrounds.

In the winter of 2010, the toolkit was conceived in the depths of the Department for Education’s Sanctuary Buildings in Westminster. The country was still reeling from the great recession of 2008. England’s education secretary, Michael Gove, was busy ushering in reforms that were sweeping away all before them.

It was a DfE event on the technical topic of education effect sizes that brought us together. One of us (Steve) was obsessed with pulling together the findings from summaries of thousands of education studies - so-called metasynthesis. And one of us (Lee) was equally obsessed with translating the complex jargon of academia into accessible language for teachers.

The coalition government had introduced the pupil premium: more than £2 billion allocated to schools every year to help improve the results of pupils on free school meals. As Gove liked to remind everyone at the time, of the 80,000 pupils who were eligible for free school meals, a mere 45 got into Oxford or Cambridge.

In our minds, this flagship policy had a major flaw. It wasn’t what schools spent that mattered, but the way they spent it. This mattered for children whether they were mastering the basics of reading and writing or getting grades to apply to university. We worked together to create the perfect solution.

The toolkit we unveiled in May 2011 was a short 20-page report published by the social mobility charity the Sutton Trust, providing evidence summaries for 20 school approaches. For each approach, we summarised the relevant research studies and presented average impacts on learning in terms of months gained during an academic year.

Following its release, our toolkit talks were attended by thousands of teachers. The findings made for lively debates. Teaching assistants, on average, added little to pupil achievement, based on the evidence at the time; reducing class sizes had surprisingly limited impact on pupil progress; grouping children by “ability” harmed poorer pupils. The standout message from our work resonated with teachers: the greatest learning gains come from improving aspects of the interaction between teachers and their pupils in classrooms, and improving the quality of teaching.

It might have all fizzled out there and then - another interesting report gathering dust on an academic library shelf. But that changed when the newly formed Education Endowment Foundation (EEF) adopted the guide in 2012.

And so the Pupil Premium Toolkit became the Sutton Trust-EEF Teaching and Learning Toolkit. Dedicated to promoting the use of evidence in schools, the EEF has nurtured the toolkit ever since. Much of the basic format and wording from the original report has survived. It is now the 34-strand free interactive website you can view today.

By 2015, two-thirds of school leaders across England reported that they had used the toolkit. It has been replicated across the world. Its findings are now digested by teachers in Santiago, Sydney and Seville - and Scotland as well. As we wrote in our 2019 follow-up book What Works?, teachers now ask questions and write about research in a way they didn’t a decade ago. Hardly a day goes by when one of us doesn’t hear from a teacher who cites the toolkit.

This year, at the end of March, the Department for Education announced that schools will have to reference sources such as the toolkit when demonstrating the evidence used for their pupil premium allocations.

Given this amazing success story (from an initial Sutton Trust grant of just £12,000 for the first toolkit), you’d think it would be time for us to step back and look on admiringly as the education system reaps the benefits of evidence-informed practice. And yet, if we’re honest, the evidence shows otherwise. Our original goals for the toolkit remain a far-off dream.

Alarmingly, even before the pandemic hit last year, attainment gaps between poorer pupils and their more privileged peers had stopped narrowing. The Covid crisis has exposed and exacerbated education inequalities. Attainment gaps may have fallen back to levels observed back in 2010. Social mobility prospects look grim.

As Sir Peter Lampl, chair of the Sutton Trust and EEF, has said, improving outcomes for poorer pupils is akin to pushing water uphill. There is a natural tendency for the divide between the education haves and have-nots to widen in the classroom. This effect is shaped largely by the different conditions experienced by children outside schools. Achievement gaps are wide in countries with high levels of inequality outside the school gates. During the pandemic, children from privileged homes have leaped further ahead.

Meanwhile, surveys of teachers have revealed greater recognition of evidence, but not necessarily more focus on approaches shown by the toolkit to offer the highest chance of improving attainment. A survey published last month by the Sutton Trust found that only one in 50 teachers had prioritised improving feedback between teachers and pupils, an inexpensive measure with the potential to add eight months to pupils’ learning in one year. None at all said they would use peer-to-peer tutoring schemes, another highly promising approach, according to the toolkit. Publishing the evidence doesn’t automatically lead to schools adapting their teaching practices.

Many EEF evaluations of interventions in England’s schools over the past decade have yielded small or insignificant findings. This suggests a number of things. Randomised trials are an important research method, but only one among many others when trying to generate evidence. It is also hard to scale up apparently effective practice. We should expect marginal gains in schools where they are doing lots of good things already.

Meanwhile, the government’s What Works initiative, which used the toolkit as its model, has, at times, overstated claims from evidence. Social science provides powerful insights into what has worked in the classroom. But it can never tell the whole story of the complex, multifaceted interplay between one teacher and their 30 pupils at one particular time and place. We should use the term “evidence-informed” education, not “evidence-based”.

These challenging findings suggest that we need a more fundamental shift in national education policy if we are serious about creating an evidence-informed profession incentivised and supported to help the poorest children in society. It was needed before the Covid crisis, and this need has become even more pressing as the vulnerable have suffered most from the pandemic. In hard times, the disadvantaged take the biggest hit. Let’s hope the crisis ushers in a new dawn of more radical thinking.

First, this means a frank and full discussion about funding. Since 2010, the pupil premium policy has been dogged by accusations that it is not genuine extra money. Schools use it to plug holes elsewhere in their budgets. To make matters worse, the DfE has decided to calculate next year’s pupil premium allocations using a census from last October, not in January 2021, when more pupils would have been eligible.

More, not less, is needed. During the past decade, we’ve been able to generate evidence on the achievement boost that a pupil can enjoy from the current level of £1,000 of pupil premium funds a year - if spent effectively. It suggests that improving progress for poorer pupils would need at least three times this amount: £3,000 annually. We should also have a preschool premium for children during their early years, and a post-school premium for teenagers in further education colleges.

As the DfE prepares for this year’s Whitehall Spending Review, our advice is to think big. If the government is serious about levelling up, it should be investing £10 billion a year into pupil premium funds - quadruple the current amounts.

At the same time, teachers need to look at the costs in terms of time and money of getting particular results. Teachers can’t work any harder. The toolkit should be used to help them work smarter.

The quid pro quo for more money is greater scrutiny over how funds are used and the outcomes for poorer pupils specifically. No school should be judged “outstanding” by Ofsted unless it has demonstrated excellent progress for its most vulnerable pupils - those on free school meals, in care or with special educational needs.

Inspectors should also reward teachers who have used evidence intelligently. Reviews have shown that high-stakes accountability currently leaves little time for research engagement in the classroom. Leaders in schools might believe that research is important, but they are forced to prioritise more immediate concerns and are nervous about adding an extra burden on to busy teachers. The immediate pressures have put paid to risk taking, cooperation with other schools and long-term reflective practice.

Finally, it is time to embed evidence-informed practice in the training of teachers, from the day they start as trainees through their early careers and into senior leadership.

This is an important step in the move to true evidence-informed practice: teachers empowered to use evidence for themselves. That means assessing the general findings from the toolkit and other sources, adapting it, and then scrutinising how approaches are working in their own schools and classrooms.

This was our hope when we first launched the toolkit. Implementation is as important as content. Our Bananarama principle - “it ain’t what you do, it’s the way that you do it” - underscores the power but also the limitations of evidence in helping teachers to decide what to do. Evidence is necessary, but never sufficient.

One thing for sure is that the toolkit is here to stay. Later this year, the EEF will unveil an updated version of the famous guide. After years of painstakingly combing through findings of individual studies, we hope it will be an even more powerful resource to support decision making in schools.

For many teachers who have forged careers over the past decade, the toolkit has been ever present in their working lives. It has revealed an insatiable appetite for evidence. The interactions we enjoy with teachers give us renewed hope for the future. Our pupils - particularly those from the poorest backgrounds - are in good hands.

Lee Elliot Major is professor of social mobility at the University of Exeter. Steve Higgins is professor of education at Durham University. Their book, What Works? Research and Evidence for Successful Teaching, is published by Bloomsbury Education

This article originally appeared in the 21 May 2021 issue under the headline “We’ve got the tools, but we haven’t fixed the problem”