Reliability of superficial surgical site infections as a hospital quality measure

Background: Although rates of superficial surgical site infection (SSI) are increasingly used as measures of hospital quality, the statistical reliability of using SSI rates in this context is uncertain. We used the American College of Surgeons National Surgical Quality Improvement Program data to determine the reliability of SSI rates as a measure of hospital performance and to evaluate the effect of hospital caseload on reliability.

Study design: We examined all patients who underwent colon resection in hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program in 2007 (n = 18,455 patients, n = 181 hospitals). We first calculated the number of cases and the risk-adjusted rate of SSI at each hospital. We then used hierarchical modeling to estimate the reliability of this quality measure for each hospital. Finally, we quantified the proportion of hospital-level variation in SSI rates due to patient characteristics and measurement noise.

Results: The average number of colon resections per hospital was 102 (SD 65). The risk-adjusted rate of superficial SSI was 10.5%, but varied from 0 to 30% across hospitals. Approximately 35% of the variation in SSI rates was explained by noise, 7% could be attributed to patient characteristics, and the remaining 58% represented true differences in SSI rates. Just more than half of the hospitals (54%) had a reliability >0.70, which is considered a minimum acceptable level. To achieve this level of reliability, 94 cases were required.

Conclusions: SSI rates are a reliable measure of hospital quality when an adequate number of cases have been reported. For hospitals with inadequate caseloads, the National Surgical Quality Improvement Program sampling strategy could be altered to provide enough cases to ensure reliability.