Comparative economic analyses of patient safety improvement strategies in acute care: a systematic review
- Edward Etchells1,3,6,
- Marika Koo2,3,
- Nick Daneman1,3,6,
- Andrew McDonald1,3,6,
- Michael Baker4,6,
- Anne Matlow1,5,6,
- Murray Krahn4,6,
- Nicole Mittmann2,3,6
- 1University of Toronto Centre for Patient Safety Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
- 2Health Outcomes and Pharmacoeconomics (HOPE) Research Centre HOPE Research Centre, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
- 3Sunnybrook Health Sciences Centre, Toronto, Canada
- 4University Health Network Toronto, Ontario, Canada
- 5Hospital for Sick Children, Toronto, Ontario, Canada
- 6University of Toronto, Toronto, Ontario, Canada
- Correspondence to Dr Edward Etchells, 2075 Bayview Avenue H469, Toronto M4N 3M5, Ontario, Canada;
Contributors EE contributed to the conception and design of the study, analysed and interpreted the data, and was one of the principle writers of the research article. MK contributed to the concept and design of the study, collected, analysed and interpreted the data, and contributed to the writing and editing of the research article. ND contributed to the concept and design of the analysis, edited and reviewed each draft of the research article. AMc contributed to the concept and design of the analysis, edited and reviewed each draft of the research article. MB contributed to the concept of the study and edited each draft of the research article. AM contributed to the methodology of the study and participated in editing the research article. MK contributed to the conception and design of the study and reviewed all drafts of the research article. NM contributed to the concept and design of the study and was one of the principle writers of the research article. All authors approved the final version of the research article.
- Accepted 4 January 2012
- Published Online First 22 April 2012
Background The objective was to systematically review comparative economic analyses of patient safety improvements in the acute care setting.
Methods A systematic review of 15 patient safety target conditions and six improvement strategies was conducted. The authors searched the published literature through Medline (2000–November 2011) using the following search terms for costs: ‘costs and cost analysis’, ‘cost-effectiveness’, ‘cost’ and ‘financial management, hospital’. The methodological quality of potentially relevant studies was appraised using Cochrane rules of evidence for clinical effectiveness in quality improvement, and standard economic methods.
Results The authors screened 2151 abstracts, reviewed 212 potentially eligible studies, and identified five comparative economic analyses that reported a total of seven comparisons based on at least one clinical effectiveness study of adequate methodological quality. Pharmacist-led medication reconciliation to prevent potential adverse drug events dominated (lower costs, better safety) a strategy of no reconciliation. Chlorhexidine for vascular catheter site care to prevent catheter-related bloodstream infections dominated a strategy of povidone-iodine for catheter site care. The Keystone ICU initiative to prevent central line-associated bloodstream infections was economically dominant over usual care. Detecting surgical foreign bodies using standard counting compared with a strategy of no counting had an incremental cost of US$1500 (CAN$1676) for each surgical foreign body detected. Several safety improvement strategies were less economically attractive, such as bar-coded sponges for reducing retained surgical sponges compared with standard surgical counting, and giving erythropoietin to reduce transfusion requirements in critically ill patients to avoid one transfusion-related adverse event.
Conclusions Five comparative economic analyses were found that reported a total of seven comparisons based on at least one effectiveness study of adequate methodological quality. On the basis of these limited studies, pharmacist-led medication reconciliation, the Keystone ICU intervention for central line-associated bloodstream infections, chlorhexidine for vascular catheter site care, and standard surgical sponge counts were economically attractive strategies for improving patient safety. More comparative economic analyses of such strategies are needed.
- patient safety
- health quality improvement
- adverse events
- epidemiology and detection
- cognitive biases
- diagnostic errors
- human factors
- medication reconciliation
- patient safety
- infection control
- trigger tools
- adverse events
- epidemiology and detection
- decision making
- clinical guidelines
- evidence-based medicine
- decision analysis
Adverse events have received considerable public, professional, political and scientific attention over the past decade. Although substantial effort has been expended to develop and implement safety improvements, there is uncertainty about which improvement strategies offer the best value.
Comparative economic analyses assess value by comparing the costs and outcomes of safety improvement strategies. An economically dominant improvement strategy is associated with lower costs and greater safety. Other improvement strategies can be associated with greater safety at an increased cost. In these situations, the additional money spent for the gain in patient safety can be calculated, and the cost-effectiveness must be weighed against other interventions. An economically dominated strategy is associated with greater costs and less safety, and should not be adopted.
Comparative economic analyses are conducted using widely accepted frameworks.1–4 There are several key considerations in comparative economic analyses. Evidence of effectiveness is a prerequisite for any comparative economic analysis. The rules of evidence for evaluation of patient safety improvements have been debated,5 but recent guidelines published by the Cochrane collaboration for quality improvement studies balance the need for evaluation with the reality that randomised controlled trials are often impossible or infeasible for safety improvements.5 Second, the choice of economic perspective and time horizon are major determinants of the resources and costs measured. A study with an acute care hospital economic perspective and a short time horizon will focus on the direct costs of providing hospital care for the current visit, but will not consider costs of care after hospital discharge, or societal costs of illness resulting from lost occupational productivity. A significant proportion of the cost of adverse events is accrued after discharge from acute care.6 Third, a comparative cost-effectiveness analysis measures the value of an intervention compared with an alternative strategy. By contrast, budget impact analyses focus narrowly on the costs of developing and implementing an improvement strategy and extrapolating the population of interest, but do not consider the downstream costs, outcomes or effectiveness.7 8
Our goal was to conduct a systematic review of published, comparative economic studies that evaluated the economic burden of a patient safety intervention in the acute care hospital setting using accepted methodological standards for the conduct of comparative economic analyses.
We chose patient safety targets and improvement strategies for the acute care setting based on previous systematic reviews9 and existing national and international safety initiatives.10 11 The patient safety targets were: adverse events; adverse drug events; ventilator-associated pneumonia; nosocomial urinary tract infection; antibiotic-resistant organism colonisation; antibiotic-resistant organism infection; catheter-associated bloodstream infection; nosocomial Clostridium difficile-associated disease; surgical site infection; nosocomial pressure ulcers; wrong site surgery; retained surgical foreign bodies; contrast-induced nephropathy; nosocomial venous thromboembolism; and nosocomial fall-related injuries. We also searched six improvement strategies (hand hygiene, rapid response teams, bundles, checklists, automatic stop orders and bar coding) to ensure that we obtained all relevant economic literature that may not be captured through searches based solely on the patient safety targets.
We sought comparative economic analyses of safety improvement strategies focused on reducing adverse events across broad groups of patients,9 rather than analyses of specific therapeutic approaches for narrow subgroups of patients. For example, we looked for comparative economic analyses of strategies to identify patients at risk of contrast-induced nephropathy and to ensure that such patients received some form of appropriate prophylaxis. We excluded analyses of specific prophylaxis for contrast-induced nephropathy (such as prophylactic haemofiltration) for narrow patient subgroups (such as critically ill patients with serum creatinine concentrations of 265 μmol/l or more undergoing angiography).12
A systematic review of the Medline database for articles between 2000 and November 2011 was conducted. The following search terms for costs were used: ‘costs and cost analysis’, ‘cost-effectiveness’, ‘cost’ and ‘financial management, hospital’. One member of the study team (MK) excluded reviews, editorials and articles with no costing information in the abstract. Two independent members of the study team (MK and EE) reviewed the remaining abstracts and obtained the full publication of any abstract considered potentially relevant by either member. We also searched the AHRQ Patient Safety Network (http://psnet.ahrq.gov) using the term ‘cost’.
Two members of the study team (EE and NM) independently appraised each full publication. We excluded publications that did not analyse an intervention directed at a patient safety target, were not conducted in an acute care setting, or were review papers with no primary data or analysis. We then excluded publications that did not report any costing methodology, or did not provide an incremental economic analysis. We then excluded publications that that did not report, or cite, adequate evidence of effectiveness based on the Cochrane collaboration guidelines for quality improvement effectiveness studies: randomised control studies, controlled clinical trials, controlled before-and-after studies, and interrupted time series.5 Finally we excluded any remaining analyses that focused on specific therapeutic strategies in narrow patient subgroups.
We used the Drummond Checklist3 to evaluate the quality of the comparative economic analyses. The Drummond Checklist rates 35 parameters as present (yes), absent (no), not clear and not applicable. A total score of 35 ‘yes’ ratings indicates that a study contains all the content required for a good economic evaluation. Two independent reviewers (EE and NM) evaluated each manuscript for quality. If the scores were within five points, then the higher of the two scores was taken. Discrepancies between reviews of five or more points were discussed and resolved. We had no difficulty resolving these discrepancies and achieving consensus. We also made some standard assumptions to facilitate our review. For example, most studies took a short-term acute care hospital perspective, so discounting was not relevant.3
We report all cost data using the original currency reported in the manuscript as well as 2010 US dollars for comparative purposes. Each cost was first converted into US dollars of the same year as indicated in the publication using the Bank of Canada currency converter.13 Then, each converted cost was inflated to 2010 US dollars using the USA Department of Labor Bureau of Labor Statistics inflation calculator.14
Our initial search yielded 2151 citations, of which 212 were considered potentially relevant. We reviewed these 212 full manuscripts, and identified five comparative economic analyses that met our inclusion criteria (figure 1). We excluded the remaining 207 articles for the following reasons: they did not study an intervention directed at a patient safety target (n=5), were review papers with no primary data or analysis (n=6), were not conducted in an acute care setting (n=8), did not report any costing methodology (n=101), or did not report a comparative economic analysis (n=66). We then excluded 19 publications that did not report, or cite, adequate evidence of effectiveness based on the Cochrane collaboration guidelines for quality improvement effectiveness studies.5 Of these 13 exclusions, the effectiveness data described or cited were uncontrolled observational cohort studies (n=8),15–22 hypothetical evidence without clinical evidence (n=6),23–28 or uncontrolled before–after studies (n=4).29–33 Finally, we excluded two comparative economic analyses of specific strategies for reducing contrast-induced nephropathy because of narrow patient subgroups.12 34
The methodological characteristics of the five studies included are outlined in table 1. Full details of each study can be found in online appendix 1.
Individual comparative economic analyses involving patient safety targets
The five studies reported a total of seven comparisons that were based on adequate effectiveness data.
Adverse drug events
One comparative analysis studied the impact of various strategies for reducing potential adverse drug events.35 The methodological feature score was 27/35. Pharmacist-led medication reconciliation was the only strategy with adequate effectiveness data, based on one randomised trial and several non-randomised controlled trials.40–44 Pharmacist-led medication reconciliation dominated over a strategy of no reconciliation.35 The main limitation of this analysis was the assumption that reducing potential adverse drug events leads to a reduction in actual adverse drug events.
Transfusion-related adverse events in critically ill patients
One analysis compared the strategy of adding erythropoietin to reduce transfusion-related adverse events to standard care in critically ill patients.36 The methodological feature score was 28/35. Effectiveness data were derived from a randomised clinical trial, where outcomes were measured as units of recombinant human erythropoietin needed to reduce allogeneic blood transfusions.45 The strategy of giving erythropoietin had an incremental cost of US$4 700 000 (CAN$6 816 309) to avoid one transfusion-related adverse event.36
Vascular catheter-associated bloodstream infection
One analysis compared chlorhexidine gluconate and povidone-iodine for catheter site care, with an outcome of catheter-related bloodstream infections in a Thailand hospital.38 The methodological feature score was 25/35. The effectiveness data came from a meta-analysis based on several randomised controlled trials.46 Chlorhexidine gluconate was a dominant strategy over povidone-iodine in both central-line catheter and peripheral-line catheter sites, showing a cost saving of 304.49 baht (CAN$9.98) per central line catheter and 13.56 baht (CAN$0.45) per peripheral catheter, with fewer infections.38 A similar analysis published 3 years earlier yielded a similar result; chlorhexidine was a dominant strategy, showing a cost saving of US$113 (CAN$209) per catheter used, and fewer infections.47
One analysis compared a Keystone ICU patient safety programme in six hospitals. The methodological feature score was 20/35. The effectiveness data came from an interrupted time series study.48 The main finding was that the Keystone ICU patient safety programme had low development and implementation costs. The intervention cost about US$5404 per case of central line-associated bloodstream infection averted, and the cost of such an infection is US$12 208 to $56 167. Therefore the intervention can be considered economically dominant.39
Retained surgical foreign body
One comparative economic analysis was related to retained surgical foreign bodies.37 This analysis compared seven strategies: no sponge tracking; standard counting; universal radiography without counting; universal radiography with standard counting; selective mandatory radiography for high-risk operations; bar-coded sponges; and radiofrequency-tagged sponges. The methodological feature score was 24/35. The effectiveness data came from a randomised control study and diagnostic test studies.49–51 Detection of surgical foreign bodies can be considered a diagnostic test; some of the evidence for effectiveness came from studies that evaluated the sensitivity and specificity of standard surgical counting to other detection methods, such as routine postoperative radiography. Standard counting was predicted to prevent 82% of retained surgical sponges with an incremental cost of US$1500 (CAN$1676) for each surgical foreign body detected, compared with a strategy of no counting. Bar-coded sponges would prevent 95% of retained surgical sponges, with an incremental cost of US$95 000 (CAN$106 132) for each surgical foreign body detected, compared with a strategy of standard counting. Selective mandatory radiography for high-risk operations, universal radiography without counting, and universal radiography with standard counting were less effective and more expensive than bar-coded sponges. The downstream costs of retained surgical foreign bodies were not included in this study, as these costs have not been described. If these downstream costs were included, then standard counting would probably be the dominant strategy compared with no counting, and bar-coded sponges would be more economically attractive.37
We did not identify any eligible comparative economic analyses for the remaining patient safety targets: ventilator-associated pneumonia; nosocomial urinary tract infection; antibiotic-resistant organism colonisation; antibiotic-resistant organism infection; nosocomial C difficile-associated disease; surgical site infection; nosocomial pressure ulcers; wrong site surgery; nosocomial venous thromboembolism; and nosocomial fall-related injuries. In addition, there were no comparative analyses for six improvement strategies (hand hygiene, rapid response teams, bundles, checklists, automatic stop orders, and bar coding).
We found few good-quality comparative economic analyses of safety improvement strategies in the acute care setting. We found five analyses that reported a total of seven comparisons based on adequate effectiveness data. Among the seven comparisons, pharmacist-led medication reconciliation to prevent potential adverse drug events dominated (lower costs, better safety) a strategy of no reconciliation. Chlorhexidine for vascular catheter site care to prevent catheter-related bloodstream infections dominated (lower costs, better safety) a strategy of povidone-iodine for catheter site care. The Keystone ICU initiative to prevent central line-associated bloodstream infections was economically dominant over usual care. Detecting surgical foreign bodies using standard counting compared with a strategy of no counting had an incremental cost of US$1500 (CAN$1676) for each surgical foreign body detected.37 Several safety improvement strategies were less economically attractive. A strategy of bar-coded sponges cost US$95 000 (CAN$106 132) for each retained surgical sponge prevented compared with standard surgical counting.37 The strategy of giving erythropoietin to reduce transfusion requirements in critically ill patients had an incremental cost of US$4 700 000 (CAN$6 816 309) to avoid one transfusion-related adverse event.36
Our results are consistent with previous reviews of the economics of patient safety in the acute care setting. A 2005 review identified 165 patient safety articles that included an economic analysis as an objective, but 35% of these articles provided no economic analysis, and 25% provided no primary economic data. The remaining studies had significant gaps in their costing methodology, and only 16% conducted sensitivity analyses that could address these limitations.52 Another review of economic evaluations of patient safety programmes identified 40 studies published between 2001 and 2004, none of which provided sufficient information about both the cost of the prevention programme and the cost of the adverse event being targeted.53 A 2005 review of comparative economic analyses related to bloodstream infections found that the existing analyses were characterised by low data quality, lack of transparency, short time horizons, and narrow economic perspectives.54
We did not identify comparative economic analyses for several high-profile patient safety improvement strategies. Some strategies, such as rapid response teams or smart infusion pumps, have not been consistently effective in evaluative studies.55 56 Other improvement strategies have adequate evidence of effectiveness, but have not been subjected to a full comparative economic analysis. We did not find any recent comparative economic analyses on improvement programmes in venous thromboembolism, although there are several older analyses showing that some form of prophylaxis is economically dominant compared with no prophylaxis in many patient subgroups. We found one recent comparative economic analysis of implementation of clinical guidelines for venous thromboembolism prophylaxis, but this analysis used effectiveness estimates from a single-site uncontrolled before-and-after study. The analysis assumed that guideline implementation would be 100% effective, with no incremental costs beyond the cost of administering prophylaxis.30 Implementing venous thromboembolism risk assessment and ensuring adequate prophylaxis for medical and surgical patients would probably reduce total costs by US$1.9 million (CAN$2.0 million) from the perspective of a national healthcare system, according to a large budget impact analysis published by the National Institute for Health and Clinical Effectiveness in the UK in 2010. This was not a comparative economic analysis, because it did not model the effectiveness of guideline implementation, and did not consider any incremental costs of developing, organising, implementing and maintaining national and local venous thromboembolism prevention improvement programmes.57
We did not identify an eligible comparative economic analysis on hand hygiene, although we identified several analyses with interesting findings. One comparison found that an alcohol-based hand hygiene product was cheaper and faster, and yielded better hand hygiene compliance, than a detergent-based antiseptic.58 Rapid methicillin-resistant Staphylococcus aureus (MRSA) detection by PCR was more costly, but not more effective, than standard culture methods.59 Failure to perform hand hygiene by a healthcare worker moving between two patients of unknown MRSA status incurred a mean cost per non-compliant event of US$1.98 (CAN$2.16) (if leaving a room with unknown MRSA status) to US$52.53 (if leaving a room of a patient known to be MRSA-positive).60
Our review found that medication reconciliation by a clinical pharmacist was a dominant strategy, based on reductions in potential adverse drug events. We excluded one comparative analysis61 because it did not cite effectiveness data of sufficient quality on bar coding or rounding clinical ward pharmacists. However, such data exist and could be incorporated into future analyses. One comparative economic analysis of a computerised order entry system was excluded because of lack of effectiveness data for the hospital's computerised physician order entry system.62 When effectiveness data from other systems were incorporated, the incremental cost was US$12 700 (CAN$18 704) per adverse drug event prevented, a moderately attractive healthcare intervention.62 As expected, this result was sensitive to the effectiveness and cost of the computerised physician order entry system, as well as the baseline rate of adverse drug events at the hospital.
Our findings indicate that greater attention needs to be paid to the methodological standards for comparative economic analyses in patient safety, and the safety community needs to conduct more comparative economic analyses of safety improvement strategies. Better knowledge of the economics of patient safety improvement strategies will inform decisions about health policy and patient safety research programmes. Most of the studies we identified considered only the acute care hospital economic perspective, but the economic perspective should extend beyond the acute care hospital, as only 22–66% of the economic burden of adverse events in acute care are borne by the acute care hospital.63 64 Economic evaluations should explicitly consider all costs of improvement strategies such as development, implementation and maintenance, as well as economic consequences from multiple perspectives and time frames. There are additional nuances to the economics of patient safety that warrant further attention, such as the impact of patient safety on staff safety, staff retention, staff absenteeism and patient (market) retention.
Our review has several important limitations. First, we focused on studies published between 2000 and November 2011 and indexed in Medline. We focused only on the acute care setting, so similar reviews of other healthcare settings would be valuable. We excluded studies that lacked adequate effectiveness data, but we acknowledge that these standards are not necessarily accepted by all. We did not evaluate the inter-rater reliability of our methodological reviews. Our review method was designed to yield higher methodological ratings, as we always took the higher rating of the two reviewers, yet we still identified a significant lack of methodological features.
We found only five comparative economic analyses that reported seven comparisons based on at least one effectiveness study of adequate methodological quality. On the basis of these limited studies, pharmacist-led medication reconciliation, chlorhexidine for vascular catheter site care, and standard surgical sponge counts were economically attractive patient safety improvement strategies. In contrast, other interventions—namely bar-coded sponges, radiography and erythropoietin in transfusion—were not considered cost-effective strategies. More comparative economic analyses of patient safety improvement programmes are needed.
We acknowledge Kaveh Shojania, Peggy Kee, Evelyn Worthington, William Geerts, Damon Scales and Andrew Simor for their assistance with this project.
Funding Unrestricted grant from the Canadian Patient Safety Institute.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement This research article is a literature review. All data presented in this article have been previously published.