It was 2006, and I was the newly installed schools minister, sat in my office in Sanctuary Buildings, when I recieved a message that Tony Blair wanted to see me in his office at Number 10. This had never happened to me before, as my previous role in the backwaters of the Department for the Environment, Food and Rural Affairs had never captured his attention.

Next minute, I was sat on one of Prime Minster’s famous sofas, alongside the new Secretary of State at Defra, David Miliband, and the public health minister Caroline Flint.

Tony wanted to talk Jamie Oliver.

He told us that we had three months before he was appearing on Jamie’s TV programme about school dinners and, right at that moment, he had nothing worth saying. What could we come up with? Caroline and David were experienced ministers and therefore both turned away from Tony and looked at me.

I walked away from the meeting with a prime-ministerial mandate to work across three government departments to put together a five-point plan. Lists always came in threes but plans, like pledges, always came in fives. As a result, we had new nutritional standards, funding for school kitchens, no fizzy drinks machines, chips only once a week and the important fifth point in the plan.

That was the moment I realised that my new job had real power, but celebrity chefs still might have more…

I only had one year working for Tony as schools minister, before surviving the transition to Gordon, but in that time one of the drivers of attention was publicity. I was responsible for Building Schools for the Future, and remain very proud of the community transformation it delivered, especially in Bristol where it was delivered in full. However the new buildings did also serve well as great backdrops to prime ministers wanting to project future and regeneration.

Publicity was not always positive. My education secretary, Alan Johnson, and I attracted the Prime Minister’s attention over faith schools. David Cameron’s first leader’s speech at party conference was all about repositioning the Tories as being more inclusive. He encouraged church schools to open their admissions to children of other faiths. Alan and I saw a political opportunity to go a bit further than the Opposition and mandate such a move. Alan got clearance from Number Ten to float the idea as long as we had our ducks in a row.

Sadly the ducks turned out to be less orderly than we thought, and we had a bit of an argument. The Monday after what appeared to be a concerted attack on us from the pulpit, we not only had swathes of Labour MPs with large Catholic communities descending on us in the voting lobbies, we had a highly irritated prime minister. We quickly performed a delicate three-point turn. 

But Tony was not all surface - there was a lot going on below the waterline, too. 

I remember holding a constituency surgery in Weymouth Library when he first rang me to offer me the job as schools minister. After I gratefully accepted, his one piece of advice in dealing with this big promotion was to work closely with Andrew (now Lord) Adonis.

Not only was this good advice, as Andrew was an exceptional minister at getting things done, it was also code: make sure the academies programme delivers.

I took over in the middle of a big row about the schools White Paper and trust schools (whatever happened to them?), and I was straight into steering the Education and Inspections Bill onto the statute book. Despite the internal opposition led by John McDonnell (now shadow chancellor), we got there easily enough.

My role soon became one of using my charms to ease through our structural reforms in the Commons, with the unions and in the media, while Andrew did the heavy lifting.

I also drove hard on teacher recruitment and pay, school leadership, BSF, effective use of technology, parental involvement and the ill-fated diploma programme. My day-to-day handling from Tony came in the form of his highly effective adviser, Conor Ryan. Conor would appear and gently advise me of the prime minister’s concerns and I would respond appropriately.

But on academies, we had occasional prime ministerial stocktakes with Tony around the cabinet table. He drilled into the detail, looked you in the eye, and properly held you to account. My recollection of his exasperation in these meetings was that it was mostly pointed towards his neighbour next door in Number 11.

I remain proud of Tony’s record in education. There were mistakes, especially around the Tomlinson Review, but things got better. The literacy and numeracy hours lifted primary results for the first time in a generation, Every Child Matters and Sure Start were brilliant.

The London Challenge and extended schools worked well. And as I visit schools now that is what many teachers tell me we achieved: more staff, better paid, better led and trusted more, with litter bins being used to catch waste paper and not leaks from the ceiling.

Tomorrow’s Tes magazine, available in all good newsagents, has a cover feature analysing Tony Blair’s legacy in schools. To download the digital edition, Android users can click here and iOS users can click here

Jim Knight is chief education adviser to TES Global