The headline in the Guardian reads “Hospital death rates in England 45% higher than in US, report finds”, and the story reports on Channel 4 coverage on Wednesday of a new study by Brian Jarman, a professor of health statistics at Imperial College London.

Jarman devised an index called the Hospital Standardised Mortality Ratio (HSMR), which compares a hospital’s mortality rates to expected mortality (given diagnosis). According to a paper by Dr Foster (an indpendent group devoted to providing health care data to the public):

The HSMR is a method of comparing mortality levels in different years,
or for different sub-populations in the same year, while taking
account of differences in casemix. The ratio is of observed to
expected deaths (multiplied conventionally by 100). Thus if mortality
levels are higher in the population being studied than would be
expected, the HSMR will be greater than 100. For all of the 56
diagnosis groups, the observed deaths are the number that have
occurred following admission in each NHS Trust during the specified
time period. The expected number of deaths in each analysis is the sum
of the estimated risks of death for every patient.

The HSMR has become a controversial index. It was credited with bringing to light the Stafford Hospital scandal, which continues to grab the headlines of UK papers with grim stories of how patients were left in their own urine and forced to drink water from flower pots for lack of nursing care. It’s controversial because many people (and not just NHS staff) refuse to believe things can be so bad. Sophisticated apologists for the Trusts poke holes at the methodology of the HSMR index. For example, it’s obviously very sensitive to the way patient’s diagnoses etc are coded, e.g., someone with cancer may be coded as a death from pneumonia.

The latest concerns a cross-sectional HSMR study of the UK and 6 other countries including Canada, Holland, Japan and the US. The UK’s hospital mortality rates are 22% higher than the average of the 7 countries and 45% higher than the US. The comparison with the US is enough for many to dismiss the results right away, as America has a lower life expectancy and its healthcare system is widely distrusted by Brits.

No statistical model is without flaws and data must be interpreted. But what rankles me are those who criticize a quantitative metric that produces uncomfortable results without offering up an alternative. Hospitals must be held accountable to some objective, quantifiable proxy for “quality care” or else the are accountable to nothing at all. The coding-error thesis is particularly pathetic, as that is itself a hospital failure. Imagine a company defending its poor performance by saying that the financial statements are misleading because there were errors in the data provided to the auditors!

And “coding errors” might reveal a different aspect of the problem all together. Maybe it’s not just fat fingers at the keyboard and other flaws in reporting procedures.. maybe patients aren’t being diagnosed properly.