Continuous improvement in health care and elsewhere is not a contentious issue—but the means by which this may be achieved is the subject of much debate. A key aspect of continuous improvement is the measurement, analysis, and interpretation of variation. Consider, for example, the data in table 1 which shows surgeon specific mortality rates after colorectal cancer surgery.¹ Ranking the mortality rates or the adjusted hazard ratios, with or without statistical tests, invites the interpretation that some surgeons are better than others. Furthermore, since a hazard ratio of 1 is defined as neutral, surgeons with a hazard ratio above 1 are considered a hazard to their patients. So, by categorising the hazard ratio as either acceptable or unacceptable, the study concludes that “some surgeons perform less than optimal surgery; some are less competent technically than their colleagues…” To improve outcomes the next logical step is to stop the less competent surgeons from operating and transfer their patients to the more competent surgeons. Surprisingly, perhaps, there is another way of analysing these data using statistical process control (SPC) which leads to very different conclusions.

BACKGROUND TO STATISTICAL PROCESS CONTROL (SPC)

In the 1920s Walter A Shewhart, a physicist, was charged with improving the quality of telephones in Bell Laboratories, USA. His work there won him the accolade of the “father of modern quality control”.² Shewhart developed a theory of variation³ which forms the basis of SPC. His theory is easily illustrated. Consider the first five “QSHC” signatures in fig 1. Two important observations can be made: (1) despite being produced by the same process, they show variation; and (2) the variation is controlled—it lies within certain limits. If nothing is known about the underlying process one would be justified in suggesting that the process appears to be stable. What would traditional approaches to understanding variation tell us about these signatures? The five signatures could be compared to a standard, and some would fall below the standard. A league table could be created, ranking the signatures from best to worst. A statistical test might identify one signature as significantly different from the others. These approaches invite punishment of the writer of the worst signature and reward for the writer of the best signature. Yet, from the point of view of the underlying process, the five signatures are similar. No signature is better or worse than the others. For improvement, the process which produced these signatures must be fundamentally changed.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 Signatures.

Now consider the sixth signature. It is clearly different from the others. A casual look suggests that this variation must have a special cause which needs to be found and eliminated to prevent it from interacting with an otherwise stable process. In this case the sixth signature was produced with the writer using his left hand. By finding this out, a simple lesson is learned—one hand is better than the other at writing signatures.

The critical feature of Shewhart’s theory of variation is that it categorises variation according to the action needed to reduce it. To reduce common cause variation one must act on the process. To reduce special cause variation one must find and act on the special cause(s). To help distinguish between these two kinds of variation Shewhart devised the premier tool of SPC—the control chart (fig 2). Control charts have three important lines. The central line is the mean or median, and the upper and lower lines are termed control limits. Data points outside the control limits (or unusual data patterns) indicate a special cause which should be found and eliminated. However, like any diagnostic tool, the guidance from the control chart is not infallible. Sometimes an outcome resulting from common cause variation is mistakenly taken to indicate a special cause. Sometimes an outcome resulting from a special cause is mistakenly attributed to common cause variation. It is impossible to reduce the frequency of both errors to zero. In the light of this, Shewhart chose to set the control limits at three sigma from the mean (or median), a level that minimised the economic consequences of both kinds of mistakes. This choice was based on a combination of statistical theory, empirical evidence, and pragmatism.³

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

 Control chart of surgeon specific hazard ratios.

Now, if the data in table 1 are reconsidered with the aid of a control chart, profoundly different conclusions are reached from those of the original study. All but one of the surgeons’ results are consistent with common cause variation. The appropriate action to improve their results is to change fundamentally the underlying process of colorectal cancer care—for example, introduce colorectal cancer screening. In contrast, surgeon H’s mortality rate is consistent with special cause variation. This should be investigated, the special causes identified according to a predefined search strategy (such as the pyramid model of investigation^4,5) and, if possible, eliminated.

FEATURES OF A CONTROL CHART

The simplicity of the control chart has inevitably led to its widespread and successful application in manufacturing and service industries.⁶ However, behind this apparent simplicity underlies some important concepts. For instance, the control chart retains the information in the data by plotting (with respect to order, where appropriate) on a graph and so enjoys the ease of communication associated with (good) graphs⁷ while incorporating statistical thinking. The control chart is a guide to continual action—for common and special cause variation. The control limits continually remind us that the major improvement gains lie in reducing common cause variation (fig 2). Furthermore, by allowing for the play of chance and not ranking the data, control charts overcome the fundamental limitations and negative consequences of league tables and comparison with standards. Perhaps its most fundamental advantage is that “… the control chart process takes us through the complete cycle of the scientific method where we develop theories based on data prior to testing them … it is one of very few statistical methods that complete the hypothesis generation–hypothesis testing cycle of the scientific method, which is one reason for its popularity with practitioners. Practitioners have found that they learn new information from the charts, rather than just making a ‘yes/no’ decision.”⁸ It thus generates the knowledge which is the key to improvement. Finally, the usefulness of the control chart is enhanced when it is integrated with other SPC tools/concepts such as cause-effect diagrams, Pareto charts, flow charts, and operational definitions.^6,9

TRANSFER OF SPC TO HEALTH CARE

Why then has SPC not been widely adopted in health care? Firstly, there is evidence that SPC is being increasingly applied in health care—for instance, a keyword literature search (using the term “statistical process control”) of the Medline database found zero hits for 1951–88, two for 1989–91, 26 for 1992–5, and 71 for 1996–2004. In addition, a number of recent publications^10–13 have reported the use of SPC in high profile cases such as the Bristol Inquiry and that of Dr Shipman,^4,14 several health care specific SPC books have been published,^15,16 and organisations such as the Joint Commission on Accreditation of Healthcare Organisations in the USA¹⁵ and the National Health Service Modernisation Agency in the UK have advocated its use.¹⁷ However, these are notable exceptions and not the rule.

In my view the reasons why SPC has been slow to transfer to health care include:

(1) SPC was first used in manufacturing industry so there is a reluctance,¹⁸ despite evidence to the contrary,^4,19 to accept that an approach for improving the quality of “widgets” can be legitimately applied to health care.

(2) Industry has a branch of statistics called industrial statistics (born, incidentally, from Shewhart’s work²⁰); in health care we have medical statistics. Unfortunately, SPC does not feature in the most popular books on medical statistics.

(3) Ultimately, perhaps, the key constraint is that SPC is above all a way of thinking which challenges many of our fundamental assumptions about how to deliver improvement documented by Deming²¹—for example, management by objectives, futility of performance related pay, increased quality means increased costs and less production, local optimisation results in global optimisation. This is where past attempts to introduce SPC into health care have failed—not on the statistical arguments but on our reluctance to face the challenges that SPC makes to our overall management approach. For example: “In Australia in the early 1980s, hospital ‘reform’ was in full swing under the influence of a managerialist management system obsessed with very short term financial objectives and individual performance. The system was one of reducing costs in the short term and judging individuals, not of providing a quality service. SPC was bound to fail.”²²

CONCLUSION

If we are to be more successful at continuous improvement in health care we must recognise that it embodies a science²³ encompassing a range of disciplines from SPC to human psychology. Given that the majority of our healthcare leaders and professionals have not been exposed to this science, we must address this deficiency through widespread education and training. One exemplar model of this is the Veterans Administration (VA) National Quality Scholars Fellowship Program (VAQS) in the USA.²⁴ This is a two year postgraduate fellowship programme (primarily aimed at physicians) which seeks to develop leaders who will apply and develop the science of improvement. Incidentally, the core curriculum draws on the work of Shewhart and Deming. Perhaps the VA has been foremost in realising that, for health care continually to improve: “it is not enough to do your best; you must know what to do and then do your best.”²⁵