The Grauniad has been running a series of guest posts as a 'Blog Festival (including from SciCurious, usually of this parish). Yesterday's post was called "The simple truth about statistics", which is of my chosen specialist subjects. I'm still trying to work out whether the irony in the errors in the post were deliberate.
The post was written by Matt Parker, who is apparently The Stand-up Mathematician. It might sound odd, but mathematicians can sometimes be confused for statisticians. Whilst statistics has a mathematical core, it is its own discipline with its own dark arts. Some of these have been used (deliberately or not) by Parker in his post.
Parker takes issue with the reporting and interpretation of two reports, one which says "Death rates from breast cancer have fallen more dramatically in the UK than any other European country, cancer researchers have said." (the full paper is available here, BTW).
As he points out, that's true, but the UK started with the highest rates, and we're still the seventh worst country. But it's not clear what his point is - it's about interpretation of statistics, and spin. Not statistics per se. Overall, the pattern is similar in several countries (the UK, Ireland, the Netherlands, Denmark and Belgium) with only small variations between them.
But Parker really shows how not to discuss statistics when he talks about the other story. The headline is "Breast cancer rates in the UK are more than four times higher than those in eastern Africa, the World Cancer Research Fund has revealed.". This is how he describes the problem:
As for the World Cancer Research Fund (WCRF) report that compared the UK with East Africa: it was looking at how many women in 100,000 were diagnosed with breast cancer, not deaths. There are several problems trying to compare UK statistics with East Africa's. The report does mention in passing that the eastern Africa numbers are only reported cases. Much of the population do not have sufficient access to medical support to be diagnosed in the first place.
He's right that the Guardian does concentrate on the lifestyle differences, rather than case recording. But then he starts to go off the rails:
Not only that, but a quick check on the World Health Organisation's website shows that the average life expectancy for women in Zimbabwe is only 42.3 years (compared with the UK's 81.7 years). Most women in East Africa simply do not live long enough to get breast cancer. In the UK, eight out of 10 breast cancers are diagnosed in women aged 50 and over. That women in a different country have half the life expectancy of the UK is the real story, not that our decadent western lifestyle is causing breast cancer.
Zimbabwe? East Africa? Well, that's an understandable confusion: the report talks about Eastern Africa, and includes Zimbabwe in their analysis. But still, why pick out Zimbabwe? Well, if you're familiar with the situation in Africa, you'll know that the country has been falling apart, thanks to some inept government. So the mortality rater might be rather high. Let's look at the figures, from the WHO (pdf), for the Eastern African countries used in the study (these are the expected life expectancies for women born in 2008):
La Réunion (French island); ???
Now, that was naughty. He picked the country with the lowest life expectancy to make the comparison. So he inflates the difference, to make the situation worse than it seems.
Actually, it's even worse than that. A representative from the WCRF (who produced the original report) turns up in the comments to point out that the data aren't the raw reported rates, but are corrected for age. Their report (which Parker actually links to) states:
# The rates per 100,000 people have been age adjusted. This means that for each country they have taken into account the differing proportions of people in different age groups to make sure they are comparing like with like.
# This is important because older people are more likely to develop breast cancer, which would otherwise distort the figures because the UK’s population is older than Eastern Africa’s.
I guess Parker didn't read down to the notes at the bottom, but only saw the headline figures in the second paragraph:
According to the latest cancer statistics, 87.9 women per 100,000 in the UK (adjusted for age) were diagnosed with breast cancer in 2008, compared to just 19.3 women per 100,000 in Eastern Africa, which includes countries such as Kenya and Tanzania.
Adjusted for age? Ooops.
For those of you wondering what these adjustments are, they are a standard technique in statistics. Briefly, the population is split into age groups and the rates (of reporting of breast cancer in this case) are recorded as the number of cases per 1000 people (e.g. 4.5 cases per 1000 women of ages 40-49). These are then multiplied by the number of people in each group taken from a reference population (I think they used a global average) and then summed these numbers. In other words, they ask what would the overall rate be if the same age-specific rates were applied to the global population. If you want a fuller explanation, this is pretty good. The basic point, though, is that it doesn't matter that UK women live longer: this is corrected for in the calculations. And this was clear (and it's a standard thing to do).
Matt Parker ends his piece with this paragraph:
When misused, statistics are less Disraeli's "damned lies" and more another leader's "I did not have sexual relations with that woman". It is by not presenting all of the information and selectively choosing definitions that statistics can appear to lie. But such claims will not stand up under cross-examination.