Mortality variations as a measure of general practitioner performance: implications of the Shipman case

Methods and results

The average list size per general practitioner in England was 1866 in 1998.4 The crude mortality in Britain is currently 1.1% a year. The table shows the number of deaths expected according to practice size, together with the minimum number of deaths for which the lower limit of a 99% confidence interval from the Poisson distribution would exclude the national rate. As there are some 9000 practices in England, the use of 99% confidence intervals would, by definition, necessitate the intense investigation of the 0.5% (approximately 45) practices annually above this limit. In larger practices 30-40 excess deaths a year would not be detected as statistically exceptional. Shipman's practice list of 3600 would allow an excess of 18 deaths a year above the average to pass as unremarkable, which is more than the 15 murders over three years he is currently convicted of (a murder rate of five per year) and also more than the high estimate of 175 murders over more than a decade, although repetitively elevated mortality, even if below the threshold, could, with extra effort, receive additional monitoring. Even if such monitoring were restricted to deaths that occurred outside hospital, random variation would mask considerable illegitimate mortality.

Comment

These findings highlight the difficulty of providing quantitative measures of quality, or of malpractice, within relatively small populations. Our estimates for the expected ranges of deaths within general practices are highly conservative, since different compositions of age, social class, and ethnic group in general practitioners' lists will strongly influence the numbers of deaths: rates could vary several fold between areas according to levels of deprivation,5 and local factors such as communicable disease and the presence of hospices or old peoples' homes will also markedly influence mortality. Therefore, either more than 0.5% of practices would require intense scrutiny each year or the mortality level at which investigation is introduced would need to be raised, allowing the potential concealment of an even greater number of illegitimate deaths. Major changes in mortality in the smallest practices may reach the upper threshold more easily, but the trend towards discouraging singlehanded practice would make this less relevant. In contrast, aberrations in larger practices would be concealed by the wide confidence intervals and by the dilution of individual malpractice within grouped mortality statistics.

Whatever the appeal of seeking hard end points, such as mortality, in judging the quality of care, in the case of the performance of individual general practitioners these may only be a distraction from the more difficult task of measuring the more mundane attributes of care. Routine monitoring of mortality in general practice lists would have had limited benefit in highlighting the Shipman case, and therefore may have little value in preventing future occurrences. Such monitoring would create both false security and a statistical cacophony of false positive suspicion. Strengthened avenues for informal intelligence of aberrant practice from patients, relatives, other doctors, practice staff, pharmacists, coroners, undertakers, and others are the key protection against lethal doctors.