The paper last week produced a performance league table based on the percentage of free school meals taken, along with pupils’ raw Higher results in S5, and turned the normal tables upside down.

Westhill Academy in Aberdeenshire is usually in the top 20 Scottish secondaries in terms of exam passes and pushes through 45 per cent of the S5 roll with three or more Highers. Remarkably, it appears in 340th place in the revised table, sixth from bottom.

Derek Thompson, Westhill’s head, ridiculed the use of an approach based on uptake of free meals. “It’s take-up rather than entitlement and the relationship between them is related to the geographical area of the school,” Mr Thompson said.

Many pupils who were entitled to free meals chose to go home or elsewhere for lunch. The focus had to be on each child performing to the best of their potential, Mr Thompson said.

Hamish Vernal, education director in Aberdeenshire, was equally scathing after several top-performing secondaries in terms of Higher results appeared near the bottom of the table.

“I am as impressed with Keith Topping’s calculations as I am with the league tables the other way,” Mr Vernal said.

“We are satisfied that Aberdeenshire schools provide a high quality of education and how do we know? Because we have HMI inspections, local authority quality assurance systems and because pupils, staff and parents tell us so. These kinds of league tables, whether Keith Topping’s or anyone else’s, serve no purpose and in some cases are positively damaging.”

Linda Croxford of Edinburgh University, who has previously attacked the discredited target-setting initiative linked to free school meals, said any approach based on meals was “a lousy measure”.

“There is a tremendous variation across Scotland in free school meals and it has got nothing to do with levels of poverty. What it measures is whether a school has a good kitchen and how easy it is to go somewhere else,” Dr Croxford said.

The Scottish Executive had previously studied how local authorities gathered information on free meals. Entitlement to free meals would be an indicator of low family income but take-up was different. Many people did not apply because of the stigma, because their children preferred to go home or simply because they did not like filling in forms.

Dr Croxford argues that any kind of performance measures have to be associated with the progress individual pupils make or the value added.

That needed data on pupil levels, prior attainment and the results achieved.

Professor Topping produced the formula for The Scotsman which in turn took out its calculator to establish the league table with Hillpark Secondary in Glasgow at the top. Last month, The TES Scotland highlighted the excellent work of the secondary when information on exam results was released.

Four of the five top schools in The ScotsmanTopping table are from Glasgow, which has long maintained its schools do well by pupils, a fact verified by HMI’s inspection of the city’s education department.

Professor Topping admits that the formula he has produced may not be more accurate and that it has its flaws, but counters that it does add balance to tables based on raw results. “If people say they do not believe either of them, fair enough. It has given food for thought. But some schools are doing remarkably well despite economic status and they deserve further study,” he said.

He accepts that the free meal indicator may be unreliable but maintains:

“It’s the best index we have got, although it is still fraught with all sorts of perils.”

Not all schools that do well in the raw results table did badly under his revised method.

Topping and tailing

Keith Topping points out that existing tables give information on the size of roll, percentage of S4 passing five or more Standard grades at level 4 or better, percentage of S5 passing three or more Highers and the percentage of pupils taking free school meals.

He cites a number of problems with such data:

* Percentages (rather than actual numbers) are difficult to compare.

* Free school meals are not a very reliable or valid indicator of socio-economic disadvantage (census details for catchment areas are better, but catchment areas no longer have clear boundaries, especially in urban areas).

* The “scalar properties” of free meals are unknown. Is an uptake of 50 per cent twice as bad as 25 per cent, or 2 per cent twice as bad as 1 per cent?

* Schools may differ in effectiveness internally by department or subject or by teacher. Socio-economic disadvantage is known to be a major variable in pupil achievement, but parent support for education irrespective of poverty is also a significant factor.

“Most analysts would agree that socio-economic disadvantage accounts for at least half the variance in pupil achievement, and is the largest factor,” Professor Topping states.

He also cautions that data on free meals may not be collected consistently and that factors other than socio-economic status and school effectiveness can influence exam results.

In any adjusted tables “the margin of error is likely to be great and any comparisons between schools on the basis of small numerical differences are likely to be of doubtful validity and unlikely to be of statistical significance”.