Limitations of SPC application in improvement of clinical processes*
| Examples/explanations | ||
|---|---|---|
| *We found statements regarding one or more limitations of SPC application in 22 articles. The limitations formed six categories which are grouped into two broad areas. | ||
| Limitations of the ability to improve clinical processes using SPC | ||
| 1 | Sharing performance data in control chart format does not automatically lead to improvement in healthcare organisations15,67 | Even if control charts can help visualise the performance of a clinical process, and signal a need for improvement, there is no guarantee that such signals prompt improvement. For example, Mertens et al studied the effects of a number of interventions, including control chart format feedback, on oncologist prescribing patterns for the management of delayed chemotherapy-induced nausea and emesis. “In our study, these [control] charts effectively measured the degree of compliance with guideline recommendations, but they were not effective in improving physician compliance .... These findings indicate that statistically valid charting, although useful in measuring compliance, will not achieve improved compliance as a physician feedback tool. However, enhanced compliance may be achieved when adverse patient outcomes are coupled with evidence of poor compliance with evidence-based guidelines” (Mertens et al,67 p 1377) |
| 2 | Statistical control does not necessarily equal clinical control nor desired performance27,35,63,64 | “It is important to note that a chart can be in statistical control, and [the patient can still] be at risk for severe asthma. So statistical control does not equate with clinical control. It simply reflects the absence of special cause variation” (Boggs et al,35 p 555) |
| 3 | Cause and effect relationships are not always obvious, even if a change is identified with statistical confidence45,60,61 | “Although control charts can help detect when the [MRSA] rate has increased or decreased, they typically will not identify the specific cause of the change. Once a change is detected, the infection control team must use its skills to assess the situation, identify possible causes, and promote improvement in practices” (Curran et al,61 p 16) |
| Limitations of the applicability of SPC to clinical processes | ||
| 4 | Differences between patients may limit the appropriateness of combining data about their care onto one control chart25,30,51,52 | “Unlike manufacturing, the health care industry deals with a variable input: Patients differ in their severity of illness on admission. This variability on admission affects care outcomes.[Ref] Therefore, it is imperative to adjust for patients’ risk for adverse outcomes. ... Blindly applying methods of manufacturing to health care may be misleading” (Alemi and Oliver,51 p 2) |
| 5 | The ability of stakeholders to apply SPC correctly may be limited16,35 | Discussing an article which contains a methodological error—irrational subgrouping—Boggs et al emphasise that “the mathematics which defines the upper and lower control limits and the zone lines (from which signals are defined) is derived from the range between the values composing each subgroup. If the daily subgroupings are irrational—i.e., the AM pre- and postbronchodilator and PM pre- and postbronchodilator are included in the same subgrouping—the daily ranges are distorted, the mathematical calculations are erroneous, and the control charts these calculations produce are improperly constructed. An improperly constructed chart is by definition uninterpretable. This error of irrational subgrouping is fatal and invalidates both their analysis and conclusions” (Boggs et al,35 p 560). |
| Another aspect of this limitation is when staff members lack sufficient knowledge of QI in general and SPC in particular16 | ||
| 6 | Limitations regarding data for use in control charts25,26,38,44,45,54,63–65 | Several articles highlight limitations related to the data needed for control charting: |
| • The sample size needs to be “big enough”. A minimum amount of data are needed to produce reliable control charts.26,54 “If a control chart has too few data points, then it may appear that a special cause exists when it really does not” (Caron and Neuhauser,54 p 31) | ||
| • Common types of control charts are not well suited to analysing infrequent events. Such events require special types of control charts64 | ||
| • (Manual) data collection can be prohibitively demanding65 | ||
| • Oversampling of patients whose values are out of control pulls the statistic in the direction of derangements25 | ||
| • Choosing too long a sampling period may delay control chart signalling and delay decision making44 | ||
| • (Retrospective) control chart interpretation can be difficult if charts are not annotated with interventions or other influences on process performance45 | ||
| • Autocorrelation may limit the ability to analyse control chart data. “Autocorrelation is a phenomenon whereby the preceding observation predicts the next observation. ... If yesterday’s blood pressure predicts today’s blood pressure, there is autocorrelation” (Solodky et al,38 p AS14) | ||
| • System-level aggregation of SPC data may prevent local sense making (if the data cannot be disaggregated). For example, performance measures of one primary care medical group practice “are computed on a group practice level. This level of aggregation means that the SPC charts cannot currently be used to evaluate the results of individual and team efforts within individual group practices” (Stewart and Greisler,63 pp 258–9) | ||