Short CommunicationAre hospital league tables calculated correctly?
Introduction
Standardized mortality ratios (SMRs) are calculated in the ranking of hospitals for the construction of league tables. The SMR, however, is a form of standardization that does not allow a valid comparison of different populations, including those attending hospitals.1 This article will use local authority district mortality data to demonstrate that it is possible for one district to have higher mortality within each age stratum when compared with another district, and yet still enjoy a lower SMR (and consequent ranking).
Section snippets
Data, methods and results
Population and mortality data from 376 local authority districts were obtained for England and Wales from national statistics for 2002. Data were trichotomized into age categories 0–64, 64–74 and 75+ years.
Local authority data were compared with overall data for England and Wales using two methods. First, the SMR was calculated for each authority. This is the ratio of the observed number of deaths in the local population over the expected number of deaths, calculated using the England and Wales
Consequences of the results
The authors have previously recommended using CMFs, and this is an appropriate measure for the data in this paper.3 This paper has shown, with real data, the fallacy inherent in using SMRs, and how false, Simpson-paradox-like events may occur if hospitals are ranked using SMRs for the construction of league tables.
The choice of which statistic to use depends on whether one wishes to have a precise answer to the wrong question or a more approximate answer to the right question. An argument often
Discussion
This paper has highlighted how a local authority district could have higher deaths at each strata level than another district, yet still enjoy a lower overall SMR (and consequent ranking).
The explanation for these anomalies is that subgroup weights used to calculate the denominator of the SMR only depend on the age characteristics of the different districts. If SMRs for several districts are compared, therefore, the relative importance assigned to deaths in different subgroups will differ
References (9)
On some points relating to the vital statistics of occupational mortality
J R Statist Soc
(1934)- et al.
Confounding and Simpson's paradox
BMJ
(1994) - et al.
Why do we continue to use standardised mortality ratios for small area comparisons
J Public Health
(2001) - et al.
League tables and their limitations: statistical issues in comparisons of institutional performance
J R Statist Soc A
(1996)
Cited by (11)
The Use and Misuse of Indirectly Standardized, Risk-Adjusted Outcomes and Star Ratings
2020, Annals of Thoracic SurgeryAre hospital league tables calculated correctly? A commentary
2007, Public HealthSoft play or hard landings in public health?
2007, Public HealthAuthors' response
2007, Public Health