You will struggle to find a single school in the country that does not collect pupil progress data. While the exact form, quantity and frequency will vary from school to school, I would be amazed if there is an example out there of one that has shunned data altogether.
Having visited a large number of schools over the past few years and heard about their various approaches to collecting and analysing pupil performance data, I have come to the conclusion that there are six golden rules that schools should follow:
1. Focus on assessment first
Before even looking at pupil data, there must be a level of confidence in the assessments that are taking place across the school. If there is not an agreed understanding of what good assessment looks like and an element of moderation taking place, then you run a genuine risk that the data you are analysing is deeply flawed, or even worse - meaningless. It could lead to a lot of time and effort being wasted focusing on the wrong priorities.
2. Minimise the time spent collecting and analysing data
Hours and hours can be spent in the process of gathering, recording, entering and then analysing pupil data. There is a very real “opportunity cost” when it comes to this aspect of a school’s work. We need to remember that every minute spent collecting and analysing data is a minute that could be spent doing something about the very issues it reveals. If you are spending as much time collecting and analysing the data as you are acting upon it, something has gone badly wrong.
3. Don’t lose sight of the wood for the trees
Schools are incredibly data rich, perhaps too much so. At just a click of a button, we can, for example, compare the progress of SEND boys in Year 4 who are also eligible for pupil premium against those who only tick two of those three criteria. It is so easy to end up disappearing down data rabbit holes if one is not very careful. Focus on the big picture, and decide in advance what the key questions are that you need to know the answers to when it comes to the data. Just because your system allows you to analyse the data in hundreds of different ways, with endless permutations, it doesn’t mean you necessarily should.
4. Be very careful of over-analysing small data sets
As discussed above, we are now able to analyse data in countless ways, based on all sorts of pupil characteristics. However, we need to exercise extreme caution when it comes to small groups. It is relatively meaningless to talk about a “33 per cent increase in the number of pupil premium pupils working at age-related expectations” if there are only three pupils eligible for pupil premium in that year group. It is important to take particular care when comparing this sort of data with national figures where the number of pupils in a data set is much larger. Talk of being “50 per cent below the national average” in SEND is not particularly useful if you only have two children with SEND.
5. Lower the stakes
Having recently reviewed Jerry Muller’s book The Tyranny of Metrics for Tes, I have become convinced of the need for schools to lower the stakes when it comes to pupil progress data. Muller shows that in any industry or sector, the higher the stakes that are associated with the data, the more likely it is to become corrupted and, therefore, less useful. With that in mind, if you want the data to provide you with meaningful information, think very carefully before linking it to your performance management and pay system.
6. Use data to raise questions rather than draw conclusions
It is very easy to jump to quick conclusions when looking at school data but it’s advisable to guard against this. The maths data being low in Year 5 is not necessarily proof that there is a problem with maths teaching in that year group. Data that stands out should lead to questions and not definitive answers. Asking questions like: “Why might there be an issue with maths progress in Year 5?” is likely to be a far more helpful approach to take.
It is so important that schools think carefully about both the potential uses and also the limitations of the pupil progress data they collect. Used in the right way, it has the power to be incredibly useful to both teachers and school leaders. Used badly, it could do more harm than good.
James Bowen is director of the NAHT Edge middle leaders’ union, and a former head of an ‘outstanding’ primary. He tweets @JamesJkbowen
