Public HealthFollowing Shipman: a pilot system for monitoring mortality rates in primary care
Section snippets
Statistical Issues
A monitoring system must be designed to quickly detect unusual variations in the underlying mortality rate in any unit. To design such a system requires that expected or acceptable mortality rates be defined, corresponding to what is termed the in-control process in control-chart studies. Criteria must also be decided to define when the observed mortality is sufficiently different from that expected to warrant special attention, corresponding to the mortality rates being out of control. Units
Data linkage
The current death registration process does not record the deceased's family physician. Only the name of the certifying doctor is recorded. To obtain deaths by practice or family physicians, we extracted 7 years of mortality data (1993–99) from the statutory death register held by the Office for National Statistics. We linked the data with general practices' lists of patients held on five English health authority information systems (including that in which Shipman's practice was situated, West
Construction of control charts
A common statistical assumption is that mortality counts follow a Poisson distribution with mean given by the indirectly standardised expected count. However, since we were able to make only limited adjustment of the expected (in-control) mortality counts for case mix, there are probably many unmeasured risk factors leading to variation in the observed data other than that due to random Poisson variation. We obtained estimates of the amount of extra-Poisson variation (overdispersion) in the
Application of charts
The Office for National Statistics provided the NHSIA with 281 777 mortality records from the five pilot areas for the years 1993–99. The success of linking mortality data to registers of lists of patients varied over time and between health authorities. Overall, 92% of mortality records were linked with data on list of patients. The proportion of mortality records successfully linked in all but the West Pennine health authority was similar, with the match rate for the other four health
How charts compare
Various methods for health-care surveillance have been proposed,6, 27, 28 but none seems to have dealt effectively with the monitoring of multiple units over multiple points in time. Nor has the difficulty of how to adjust SPC charts for the inevitable overdispersion in routinely collected health outcome data been previously considered. We have shown the usefulness of log-likelihood ratio cumulative sums for monitoring mortality in primary care with use of real data for a large number of units
Interpretation of charts
Patients' mortality is clearly highly variable at family physician and practice levels. This variability is much greater than would be expected by chance alone. Key explanations for this finding include inadequate adjustment for case-mix and poor data quality yielded from the pilot exercise to link primary care populations and mortality at family physician level. The limited success of the latter was largely due to missing NHS numbers on the Office for National Statistics mortality records for
Conclusions
We envisage cumulative sum charts being used as a governance tool for monitoring performance since they enable a first-pass analysis of the data and can highlight units with unusual outcomes. We caution however, that the charts cannot by themselves shed light on the reasons for apparent poor performance. Methods of operation at a local level will be required that enable the clinical explanations for outliers to be readily identified. Even in this context, use of cumulative sum charts, or any
References (30)
- et al.
Bristol, Shipman and clinical governance: Shewhart's forgotten lessons
Lancet
(2001) - et al.
Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984–86: was Bristol an outlier?
Lancet
(2001) Harold Shipman's clinical practice, 1974–1998
(2001)- et al.
Monitoring mortality rates in general practice after Shipman
BMJ
(2003) The Shipman Inquiry: independent inquiry into the issues arising from the case of Harold Fredrick Shipman
Potential contribution of statistical evidence to the inquiry
(1999)- et al.
Performance league tables: the NHS deserves better
BMJ
(2002) - et al.
Monitoring kidney transplant patients
Statistician
(1983) - et al.
League tables and their limitations: statistical issues in comparisons of institutional performance
J R Stat Soc
(1996) Continuous inspection schemes
Biometrika
(1954)