Understanding A-level results

I’ve been struggling to understand the A-level results, a provisional summary of which is published by the Joint Council of Qualifications. Naturally, it’s published in PDF format which is a very useful way of publishing data tables…

That aside, I can’t see how from the data presented one can conclude that the results represent a definitive improvement. I’m not suggesting there hasn’t been an improvement, but I don’t think the data show one.

What the data do show is that for each individual subject, the percentage of entrants achieving higher grades has improved.

A perfectly valid hypothesis would be that candidates were being more selective about the subjects they took and biasing their choices to those subjects in which they would perform better. Or the corollary of that, candidates were dropping subjects in which they couldn’t achieve a high grade.

Let’s take a couple of concrete examples: first, a high flying student is taking four A-levels and expects to achieve AAAB grades. By dropping the fourth subject, these statistics would show an ‘improvement’ though no change in the population’s ability has occurred. Equally, a lower-ranked student may be expecting to pass two A-levels and fail another. Again, by dropping the one she expects to fail the statistics would improve. Again, the population has become neither smarter nor thicker.

I’d suggest that to make sense of these results, we need them to be normalised against the number of A-levels taken, or some other weighting scheme that would remove these biases.

Certainly, the published results don’t justify the director of the JCQ’s conclusion that “the improvement of the results at A-level reflects how well students have done this year.”

Perhaps Dr Sinclair has some other results that do show this which he forgot to publish. Or perhaps he should go back to his school textbooks – I’m sure that they still teach about self-selection biases at A-level.

Wouldn’t it be a service if one of the journalists covering this today asked about this issue? I wouldn’t hold your breath, given that on breakfast time this morning one of them looked flustered on being told that 300 million divided by 300 was one million.