Article Text

Economic evaluation of quality improvement interventions to prevent catheter-associated urinary tract infections in the hospital setting: a systematic review
1. Sara G McCleskey1,2,
2. Lili Shek1,
3. Jonathan Grein1,
4. Hiroshi Gotanda1,
5. Laura Anderson1,3,
6. Paul G Shekelle4,5,
7. Emmett Keeler5,
8. Sally Morton6,
9. Teryl K Nuckols1,5
1. 1 Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California, USA
2. 2 Health Policy & Management, UCLA, Los Angeles, California, USA
3. 3 Center for Observational Research, Amgen, Thousand Oaks, California, USA
4. 4 Department of Medicine, West Los Angeles Vet Administration, Los Angeles, California, USA
5. 5 RAND Corporation, Santa Monica, California, USA
6. 6 Knowledge Enterprise, Arizona State University, Tempe, Arizona, USA
1. Correspondence to Sara G McCleskey, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA; sara.mccleskey{at}cshs.org

Abstract

Background Hospitals have implemented diverse quality improvement (QI) interventions to reduce rates of catheter-associated urinary tract infections (CAUTIs). The economic value of these QI interventions is uncertain.

Objective To systematically review economic evaluations of QI interventions designed to prevent CAUTI in acute care hospitals.

Methods A search of Ovid MEDLINE, Econlit, Centre for Reviews & Dissemination, New York Academy of Medicine’s Grey Literature Report, WorldCat, IDWeek conference abstracts and prior systematic reviews was conducted from January 2000 to October 2020.

We included English-language studies of any design that evaluated organisational or structural changes to prevent CAUTI in acute care hospitals, and reported programme and infection-related costs.

Dual reviewers assessed study design, effectiveness, costs and study quality. For each eligible study, we performed a cost-consequences analysis from the hospital perspective, estimating the incidence rate ratio (IRR) and incremental net cost/savings per hospital over 3 years. Unadjusted weighted regression analyses tested predictors of these measures, weighted by catheter days per study.

Analysis

We conducted separate unadjusted weighted regression analyses to individually identify factors associated with greater effectiveness (lower IRR) and savings (lower incremental net cost). Factors potentially associated with effectiveness were identified a priori, and included intervention type (antimicrobial, policies and practices, or both), study publication year, programme cost per hospital over 3 years and the academic status of the hospitals included in the studies. Factors potentially associated with incremental net costs included the same factors as above, with the addition of quality of economic analysis as measured by mQHES score and intervention effectiveness. Analyses were weighted by the estimated number of catheter days analysed per study. Sensitivity analyses involved jackknife resampling with sequential exclusion of each study to examine if results changed, particularly for larger studies.32

Results

Study selection

We identified 575 publications and selected 69 for full-text review; 14 articles met all eligibility criteria (figure 1).33–44 One article included two separate interventions which we considered as two studies,38 bringing the total number of studies included in our qualitative analysis to 15.

Figure 1

PRISMA flow diagram. Adapted from Page et al.56 CAUTI, catheter-associated urinary tract infection; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; QI, quality improvement.

Of the 55 articles excluded under full-text review, 3 were systematic literature reviews and did not include original data. Nineteen articles did not include an economic evaluation, while 14 articles did not examine a QI intervention designed to prevent CAUTI. Fifteen articles did not measure or model programme costs (ie, implementation costs of the QI intervention). Finally, four articles did not report clinical effectiveness. Queries of grey literature did not identify eligible articles (figure 1).

Study characteristics and quality assessment

Definition of CAUTI

Definitions of CAUTI varied across the 15 studies (table 1). Five studies33 34 38 39 43 used surveillance criteria from the US National Healthcare Safety Network (NHSN) prior to 2015.45 In 2015, the NHSN modified its CAUTI surveillance criteria to improve clinical specificity.45 One study44 used this new surveillance criteria from the NHSN.46 Five studies35–37 42 47 created a custom definition of CAUTI, and three studies40 41 48 did not specify a definition of CAUTI.

Table 1

Definition of catheter-associated urinary tract infection (CAUTI)

QI interventions

The 15 eligible studies each tested one or more strategies recommended by SHEA and IDSA24 49 to prevent CAUTI in acute care hospitals (table 2).33–44 47 48 Strategies involving changes in protocols and practices included written criteria for acceptable catheter indications (two studies),35 40 readily available supplies for aseptic catheter insertion (two studies),41 48 documentation of catheter indications and care (four studies),34 35 40 44 CAUTI surveillance using standardised criteria (three studies),33 35 40 unit-specific feedback (two studies),40 44 meatal cleaning with antiseptic prior to insertion (one study),48 use of bladder scanners (three studies),35 41 44 automated reminders of persistent catheterisation (four studies),34 35 37 40 analysis and reporting of catheter use and adverse events (two studies),40 44 periodical review of catheter necessity (three studies),35 40 44 CAUTI-specific nursing-focused education (seven studies),33–35 40 41 44 47 CAUTI-specific physician-focused education (one study),40 automated catheter stop orders (one study),35 engagement of hospital leadership (four studies),33 37 40 44 a multidisciplinary CAUTI prevention team (four studies),34 35 40 44 use of a ‘bladder bundle’ (two studies),33 44 electronic alerts to confirm catheter necessity (two studies)34 40 and audits on compliance (two studies).34 44 Changes to protocols and practices that are not explicitly included as part of the current recommended prevention strategies by SHEA or IDSA24 49 but were used in the studies included in this review were: use of a CAUTI ‘toolkit’ (one study),33 use of a Bard Tray (one study),47 use of physician champions (three studies),33 40 44 root cause analysis of CAUTI events in real time (two studies),40 47 routine site visits (one study),33 patient education (one study),44 and use of an advanced practice registered nurse and/or specially trained nurses (one study).35

Table 2

Use of strategies designed to prevent catheter-associated urinary tract infections (CAUTIs) in studies with economic evaluations

Strategies involving changes to catheter equipment included silver-impregnated antimicrobial catheters (seven studies),34 36 38–40 42 43 and antibiotic-impregnated antimicrobial catheters (one study).38 It should be noted that antimicrobial impregnated catheters are not recommended by SHEA/IDSA as part of routine CAUTI prevention.24 49

Context

Eight of the 15 unique studies were based in the USA,33 34 36 37 39 40 43 44 3 in the UK,38 41 47 1 in the Netherlands35 and 1 in Australia48 (table 3). One study had no location because it used a hypothetical cohort to create a decision model.42 Most studies were set at a single hospital, although two studies included 24 hospitals,33 38 two included 3 hospitals,47 48 one included 10 hospitals35 and one included 2 hospitals.41 In total, data came from 74 hospitals. Six studies were based at only major academic institutions,36 37 39 41 43 44 four studies were based at only community hospitals,33 34 40 48 one study was based at both academic and community hospitals,35 two studies were based at National Health Service hospitals in the UK38 47 and one study did not state academic status.42 See online supplemental appendix 2 for elements from the Minimum Quality Criteria Set and study funding sources.

Supplemental material

Table 3

Summary of economic evaluations for quality improvement interventions designed to prevent catheter-associated urinary tract infections (CAUTIs) as reported by original study authors

Clinical evaluation

All 15 unique studies compared QI interventions with usual care (table 3). Eight studies used an uncontrolled before–after study design,33 34 36 39–41 44 47 one study used a pretest/post-test design with a non-equivalent control group,37 one study used a time-series analysis,35 one study used a randomised controlled trial design38 and one used a randomised crossover design.43 Finally, two of the studies reported a modelling exercise based on a randomised controlled trial.42 48

Cost evaluation

Most of the 15 unique studies reported cost analyses from the hospital perspective (table 3).33–37 39–41 43 44 47 One study was a cost analysis from the payer perspective,42 one study was a cost-effectiveness analysis from the health system perspective38 and one study was a cost-effectiveness analysis from a societal perspective.48

Supplemental material

Figure 2

Standardised quality improvement (QI) programme, infection-related and incremental net costs of QI interventions. CAUTI, catheter-associated urinary tract infection.

Two studies in particular varied widely in overall net costs when compared with the other studies included in this analysis,39 43 attributed in part to differences in baseline CAUTI rates and overall intervention effectiveness, but importantly one of the studies39 used a high cost estimate per infection (nearly $4000 per CAUTI, compared with$1175 based on a prior meta-analysis).6

Analysis

Using unadjusted regression analysis weighted by the estimated number of catheter days analysed per study, we found that the mean IRR among the 12 studies was 0.57 (95% CI: 0.44 to 0.70), reflecting a 43% decline in infections relative to usual care. Using both types of infection prevention practices at the same time (antimicrobial catheters and CAUTI prevention-related policies and practices) was associated with a statistically significant 89% decline in infections relative to usual care (IRR: 0.11; 95% CI: −0.06 to 0.27; p=0.03), based on two studies. It should be noted that the null value of the CI for a ratio is 1, which means that in this analysis, the null hypothesis can be rejected.50

The mean incremental net cost over 3 years was $52 000 per hospital, but estimates could be as high as saving$288 000 or spending $392 000 (95% CI: −$288 000 to $392 000). Implementing both antimicrobial catheters and CAUTI prevention-related policies and practices was associated with non-significant net costs of −$31 000 over 3 years (95% CI: −$1 708 000 to$1 647 000; p=0.97) when compared with usual care, and this was not significantly different from the net cost observed in studies comparing antimicrobial catheters with usual care ($30 000; 95% CI: −$752 000 to $812 000; p=0.97 for comparison of net costs). We saw no differences in effectiveness or net costs according to QI programme costs, publication year or hospital academic status. We also saw no difference in net costs according to clinical effectiveness or the quality of the economic evaluation (online supplemental appendix 5). Supplemental material These results were robust to sequential exclusion of each study, irrespective of study size, with one exception. This study used both antimicrobial catheters and CAUTI-related policies and practices in its intervention and exhibited an unusually large reduction in infections, with an IRR of 0.08.40 After excluding this study, the use of both intervention types was no longer associated with effectiveness. Discussion On the basis of 15 unique economic evaluations encompassing 74 hospitals, our systematic review and weighted regression analysis showed that QI interventions aimed at reducing rates of CAUTI in acute care hospitals yield highly variable net costs to hospitals with, on average, an insignificant net cost of$52 000 per hospital, even when the QI interventions are clinically effective. This cost assessment does not take into consideration potential penalties or incentives for hospitals with higher or lower CAUTI rates via the Medicare financial incentive programmes.9 10 Among studies including an economic evaluation, we found that overall, QI interventions involving one or more strategies recommended by SHEA and IDSA24 49 were associated with a 43% decline in infections.

To our knowledge, there have been no previous systematic reviews examining the economic implications of CAUTI-related QI interventions, meaning the associated changes in clinical outcomes relative to net cost. A 2013 systematic review and meta-analysis by Zimlichman et al estimated the cost to hospitals of treating a CAUTI event to be $896 (95% CI:$603 to $1189) in 2012 US dollars.6 Our analysis balanced the savings from preventing these events against the investments in QI interventions, which we found had a median programme cost of$131 000 per year.

Over the last decade, hospitals in the USA have come under increasing pressure from policymakers to devote resources to the reduction of hospital-acquired infections, including CAUTI. The Centers for Medicare and Medicaid Services, the federal agency administering Medicare and assisting the states with administering local Medicaid programmes, implemented several policies that created financial incentives for hospitals to reduce rates of CAUTI. These included requiring hospitals to absorb the costs associated with CAUTI and adjusting payments to hospitals based on quality measures for CAUTI and other hospital-acquired infections.8–10 In response to these policies, US hospitals have implemented various CAUTI-prevention practices, and CAUTI rates have subsequently decreased over time.11 12 15 The studies evaluated in this review report similar degrees of effectiveness in reducing rates of CAUTI compared with prior reviews of CAUTI-related QI interventions.14 51–53 Although CAUTI rates have been declining, we found no evidence that net costs or clinical effectiveness differed in recent studies as compared with earlier studies in our study publication year analysis.

Our results suggest that effective interventions are on average a good value for hospitals, despite the initial investment required. With an average 43% reduction in infections and a net cost of \$52 000 over 3 years, some hospitals may find that the programme cost of implementation is fully offset or nearly so by the cost of infections prevented. However, financial outcomes varied greatly in this analysis. Notably, our results also suggest that investing more money in a CAUTI-reduction programme is not necessarily helpful, as higher programme costs did not consistently lead to better outcomes clinically.

A subanalysis based on two of the studies suggested that implementing a multifaceted CAUTI-prevention strategy using both antimicrobial catheters and CAUTI-related policies and practices may be better at reducing infection rates than either strategy alone, and not more costly. Caution should be used in interpreting this result, however, as it is driven largely by one single-centre study where multiple policies and practices were changed in addition to the use of antimicrobial catheters.40 It is not possible to discern from this study which intervention had the largest effect on CAUTI reduction. Furthermore, there is significant debate over the role and value of antimicrobial-impregnated urinary catheters in CAUTI prevention, with multiple conflicting studies. A 2014 Cochrane review did not find clear evidence of the benefit of these types of catheters,54 and the 2014 SHEA/IDSA guidelines for CAUTI prevention explicitly advise against routine use of antimicrobial-impregnated catheters.24

Limitations

Our analysis has several limitations. A limited number of studies have examined the cost of QI interventions related to CAUTI, and most of these used weak uncontrolled before–after designs. Additionally, we used estimates from a systematic review and meta-analysis6 to estimate the infection-related costs in the majority of included studies. There may be methodological limitations to the cost analyses in both the primary studies that we included in our review and in the primary studies included in the meta-analysis of CAUTI-related costs, including inadequate adjustments, residual confounding and time-dependent bias, which can overestimate the costs associated with nosocomial infections.55 This may potentially make these interventions more costly to hospitals than they appear. Additionally, the QI interventions in the included studies had heterogeneous components and were highly complex, limiting our ability to identify the specific elements driving intervention effectiveness and cost-savings. Economic evaluations have not addressed some of the most evidence-based and widely used QI strategies for CAUTI; therefore, further research is needed. Finally, the definitions of CAUTI varied across included studies which may affect CAUTI detection rates but would be less likely to bias estimates of intervention effectiveness since the CAUTI definition is applied equally to the intervention group and the control group. Jackknife resampling with sequential exclusion of each study in our sensitivity analyses, including those studies with unspecified definitions of CAUTI, revealed that results did not change. Additionally, the IRR facilitates standardisation across disparate studies as a ratio of the rate of infection in the intervention group to that in the control group, and it provides a unit-free proportion that ranges from 0 to 1. Using the IRR facilitates standardisation but does not eliminate the problem of different studies using different definitions, since a given intervention might be particularly effective in a subset of CAUTI cases that are not counted using a particular CAUTI definition, for example. Despite these limitations, our findings reflect 74 sites and thousands of catheter days, and the changes in CAUTI rates we observed are consistent with prior reviews.14 51–53

Conclusions

QI interventions that involved antimicrobial catheters and/or changes to CAUTI-related policies and practices were associated with declines in infection rates and net costs to hospitals that varied greatly but that, on average, were not significantly different from zero over 3 years. Future research should seek to tease out specific practices associated with cost-savings and clinical effectiveness, and examine whether or not more comprehensive QI interventions offer hospitals and patients the best value.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplemental information.

Footnotes

• Funding Agency for Healthcare Research and Quality (R01 HS22644-01).

• Competing interests None declared.

• Provenance and peer review Not commissioned; externally peer reviewed.

• Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.