Public HealthFollowing Shipman: a pilot system for monitoring mortality rates in primary care
Section snippets
Statistical Issues
A monitoring system must be designed to quickly detect unusual variations in the underlying mortality rate in any unit. To design such a system requires that expected or acceptable mortality rates be defined, corresponding to what is termed the in-control process in control-chart studies. Criteria must also be decided to define when the observed mortality is sufficiently different from that expected to warrant special attention, corresponding to the mortality rates being out of control. Units
Data linkage
The current death registration process does not record the deceased's family physician. Only the name of the certifying doctor is recorded. To obtain deaths by practice or family physicians, we extracted 7 years of mortality data (1993–99) from the statutory death register held by the Office for National Statistics. We linked the data with general practices' lists of patients held on five English health authority information systems (including that in which Shipman's practice was situated, West
Construction of control charts
A common statistical assumption is that mortality counts follow a Poisson distribution with mean given by the indirectly standardised expected count. However, since we were able to make only limited adjustment of the expected (in-control) mortality counts for case mix, there are probably many unmeasured risk factors leading to variation in the observed data other than that due to random Poisson variation. We obtained estimates of the amount of extra-Poisson variation (overdispersion) in the
Application of charts
The Office for National Statistics provided the NHSIA with 281 777 mortality records from the five pilot areas for the years 1993–99. The success of linking mortality data to registers of lists of patients varied over time and between health authorities. Overall, 92% of mortality records were linked with data on list of patients. The proportion of mortality records successfully linked in all but the West Pennine health authority was similar, with the match rate for the other four health
How charts compare
Various methods for health-care surveillance have been proposed,6, 27, 28 but none seems to have dealt effectively with the monitoring of multiple units over multiple points in time. Nor has the difficulty of how to adjust SPC charts for the inevitable overdispersion in routinely collected health outcome data been previously considered. We have shown the usefulness of log-likelihood ratio cumulative sums for monitoring mortality in primary care with use of real data for a large number of units
Interpretation of charts
Patients' mortality is clearly highly variable at family physician and practice levels. This variability is much greater than would be expected by chance alone. Key explanations for this finding include inadequate adjustment for case-mix and poor data quality yielded from the pilot exercise to link primary care populations and mortality at family physician level. The limited success of the latter was largely due to missing NHS numbers on the Office for National Statistics mortality records for
Conclusions
We envisage cumulative sum charts being used as a governance tool for monitoring performance since they enable a first-pass analysis of the data and can highlight units with unusual outcomes. We caution however, that the charts cannot by themselves shed light on the reasons for apparent poor performance. Methods of operation at a local level will be required that enable the clinical explanations for outliers to be readily identified. Even in this context, use of cumulative sum charts, or any
References (30)
- et al.
Bristol, Shipman and clinical governance: Shewhart's forgotten lessons
Lancet
(2001) - et al.
Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984–86: was Bristol an outlier?
Lancet
(2001) Harold Shipman's clinical practice, 1974–1998
(2001)- et al.
Monitoring mortality rates in general practice after Shipman
BMJ
(2003) The Shipman Inquiry: independent inquiry into the issues arising from the case of Harold Fredrick Shipman
Potential contribution of statistical evidence to the inquiry
(1999)- et al.
Performance league tables: the NHS deserves better
BMJ
(2002) - et al.
Monitoring kidney transplant patients
Statistician
(1983) - et al.
League tables and their limitations: statistical issues in comparisons of institutional performance
J R Stat Soc
(1996) Continuous inspection schemes
Biometrika
(1954)
The design and analysis of sequential clinical trials
A review and discussion of statistical issues in public health monitoring
J R Stat Soc C
A study of the average run length characteristics of the national notifiable disease surveillance system
Stat Med
A sequential procedure for surveillance of Down's syndrome
Stat Med
Cited by (69)
A new simple primary care morbidity score predicted mortality and better explains between practice variations than the Charlson index
2013, Journal of Clinical EpidemiologyCitation Excerpt :Furthermore, although deprivation is easily accounted for in the analysis of primary care data using area-based measures, our analysis demonstrates that this cannot substitute for adjustment for morbidity, and that taking account of morbidity is more important than deprivation in comparisons between practices. There has been a desire to compare outcomes, including mortality, between individual primary care providers for a number of years, but other authors have identified a number of methodological barriers, including adequate adjustment for case mix and potentially large numbers of false alarms [45,46]. A simple effective morbidity score will help address these concerns by reducing variation owing to case mix.
Early detection of variants of concern via funnel plots of regional reproduction numbers
2023, Scientific ReportsStatistics in times of increasing uncertainty
2022, Journal of the Royal Statistical Society. Series A: Statistics in SocietyManaging Shiga Toxin-Producing E. coli Using Statistical Process Control Charts for Routine Health and Production Monitoring in Pig Farming
2022, Frontiers in Veterinary Science