Lies, damned lies, and NUT statistics

I'm a maths teacher and always tell my students that, if nothing else, I want them to learn how to understand data and statistics, so they can hold those in power to account.

It really annoys me when people misuse numbers to try and pull the wool over people's eyes for political or other purposes. It takes the public for mugs, assuming that they will hear the headline and be too busy to drill down into the facts underlying the statement.

Two key checks...

In particular, when teaching students about handling statistics, I try to get them into the habit of carrying out two checks before they do any number crunching:

1) how reliable is the data they are using? And if it's data someone else has collected, what exactly have they measured - is it actually what you think they have measured?

2) what kind of answer might they expect, and what would be unexpected/an anomaly? If you think about this before you have any potential answers, hopefully you'll spot any errors or dodgy underlying data.

A quiet bank holiday weekend...

The other day my ears pricked up then when I heard - amongst the usual sorts of statements made by the NUT during its annual conference - a mention of free schools being set up in areas where there a big surplus of school places is already being forecast.

Suffolk, Central Bedfordshire and Bedford - where I live and where my free school is - were singled out as being particularly bad in this regard.

For Bedford, they said that:

"the local authority is projected to have a 38.2% shortfall in primary places by 2016/17 yet the Bedford Free School – a secondary that opened in September 2012 despite opposition from the local authority – will add to the projected 25.4% surplus in secondary place provision by 2016/17." (1)

My gut instinct was that these figures were totally wrong, for a number of reasons.

First of all, what kind of local authority would allow that kind of shortage to occur in primary without doing something about it? 

Secondly, when we were planning our free school, we used Local Authority and Department for Education data to forecast the number of students of secondary school age out into the future. Our calculations showed a small surplus of places now falling over time as the primary "bulge" worked through the system.

Finally, the three areas that they claimed were most out of kilter are all places that have a mainly 3-tier school system (with lower, middle and upper schools, rather than primary and secondary schools). This made me wonder if they were getting confused by the fact that for DfE categorising purposes, most middle schools (which educate students from years 5-8) are deemed to be secondary schools, even though they cover 2 years of primary education.

Data hunting

It being the Easter holidays, I've had a bit more time on my hands to follow up my instinct that the NUT researcher tasked to produce some "facts" for a press release had been sloppy (or worse).

An hour of hunting down data and playing about with a spreadsheet later, what did I find? (2)

The NUT stats for Bedford were totally wrong.

Why? They had made a simple but fundamental error, and one which they should have spotted if they'd stopped to check their “findings” when their % surpluses and deficits came out so large.

For the demand for school places, they had correctly taken the forecast number of students for the primary & secondary age range (R-6 and 7-13), but for supply they had:

• only taken lower school capacity  to be the supply of primary places (which only covers years R-4, so giving a figure too low), and

• taken middle and upper school capacity to be the supply of secondary school places (which cover years 5 - 13, and so giving a figure too big).

And what do we find when we use the data in the correct way?

A totally different picture in 2016/17:

• over the primary age range, a 2% deficit rather than the 38% the NUT claimed, and

• over the secondary age range, a 2% surplus rather than the 24% they came up with. (3)

Conclusion...

Not quite so shocking, is it?

If anything, the trend would tend to support MORE free schools in the area, to maintain a decent surplus of places and allow some parental choice - not what the NUT were claiming. 

I haven't yet looked at the same data for Central Bedfordshire & Suffolk, the other areas where the NUT reckon there are "massive" surpluses of secondary school places, but I would bet that they've made the same mistake

If this were a project submitted to me by a student of mine what feedback would I give them..? I'll let you decide...

Notes:

(1) The NUT press release can be seen here: http://www.teachers.org.uk/node/17931

(2) All data is from here: http://www.education.gov.uk/researchandstatistics/datasets/a00219162/school-capacity-2011-12  and my spreadsheet with calculations in is here .  

(3) The trends over time show how a rising population will lead to a tightening in the places available:

• in lower schools a current surplus of places falling to zero in 2016/17
• in middle schools a small current surplus turning into a 7% shortage
• in upper schools the current small surplus is virtually unchanged.

If you look at the same data from a "primary/secondary" perspective (sharing out middle school capacity between lower and upper schools) you get the same sort of thing:

• over the primary age range a current 8% surplus falling to a 2% shortage 
• over the secondary age range a current 5% surplus falling to a 2% surplus.

[NB: There are some significant changes planned for Bedford's schools - some lowers becoming primaries, some middles closing, some uppers becoming secondary, some totally new primaries opening - but these weren't taken into consideration in their calculations, and anyway are largely neutral in their effect on places.]