Article Text

Download PDFPDF
Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates
  1. Iulian Ilieş1,
  2. Deverick J Anderson2,3,
  3. Joseph Salem1,
  4. Arthur W Baker2,3,
  5. Margo Jacobsen1,
  6. James C Benneyan1,4
  1. 1Healthcare Systems Engineering Institute, Northeastern University, Boston, Massachusetts, USA
  2. 2Duke Center for Antimicrobial Stewardship and Infection Prevention, Durham, NC, USA
  3. 3Division of Infectious Diseases, Duke University School of Medicine, Durham, NC, USA
  4. 4College of Engineering, Northeastern University, Boston, MA, USA
  1. Correspondence to Dr James C Benneyan, Healthcare Systems Engineering Institute, Northeastern University, Boston, Massachusetts, USA; j.benneyan{at}northeastern.edu

Abstract

Objective Surgical site infections (SSIs) are common costly hospital-acquired conditions. While statistical process control (SPC) use in healthcare has increased, limited rigorous empirical research compares and optimises these methods for SSI surveillance. We sought to determine which SPC chart types and design parameters maximise the detection of clinically relevant SSI rate increases while minimising false alarms.

Design Systematic retrospective data analysis and empirical optimisation.

Methods We analysed 12 years of data on 13 surgical procedures from a network of 58 community hospitals. Statistically significant SSI rate increases (signals) at individual hospitals initially were identified using 50 different SPC chart variations (Shewhart or exponentially weighted moving average, 5 baseline periods, 5 baseline types). Blinded epidemiologists evaluated the clinical significance of 2709 representative signals of potential outbreaks (out of 5536 generated), rating them as requiring ‘action’ or ‘no action’. These ratings were used to identify which SPC approaches maximised sensitivity and specificity within a broader set of 3600 individual chart variations (additional baseline variations and chart types, including moving average (MA), and five control limit widths) and over 32 million dual-chart combinations based on different baseline periods, reference data (network-wide vs local hospital SSI rates), control limit widths and other calculation considerations. Results were validated with an additional year of data from the same hospital cohort.

Results The optimal SPC approach to detect clinically important SSI rate increases used two simultaneous MA charts calculated using lagged rolling baseline windows and 1 SD limits. The first chart used 12-month MAs with 18-month baselines and best identified small sustained increases above network-wide SSI rates. The second chart used 6-month MAs with 3-month baselines and best detected large short-term increases above individual hospital SSI rates. This combination outperformed more commonly used charts, with high sensitivity (0.90; positive predictive value=0.56) and practical specificity (0.67; negative predictive value=0.94).

Conclusions An optimised combination of two MA charts had the best performance for identifying clinically relevant small but sustained above-network SSI rates and large short-term individual hospital increases.

  • Infection outbreaks
  • surveillance
  • statistical process control
  • surgical site infections
View Full Text

Statistics from Altmetric.com

Footnotes

  • Funding This study was funded by Agency for Healthcare Research and Quality (R01-HS023821-02).

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.