Article Text
Abstract
Background Adverse drug events (ADEs) are a major cause of morbidity in hospitalised and ambulatory patients. Computerised provider order entry (CPOE) combined with clinical decision support systems (CDSS) are being widely implemented with the goal of preventing ADEs, but the effectiveness of these systems remains unclear.
Methods We searched the specialised database Agency for Healthcare Research and Quality (AHRQ) Patient Safety Net to identify reviews of the effect of CPOE combined with CDSS on ADE rates in inpatient and outpatient settings. We included systematic and narrative reviews published since 2008 and controlled clinical trials published since 2012.
Results We included five systematic reviews, one narrative review and two controlled trials. The existing literature consists mostly of studies of homegrown systems conducted in the inpatient setting. CPOE+CDSS was consistently reported to reduce prescribing errors, but does not appear to prevent clinical ADEs in either the inpatient or outpatient setting. Implementation of CPOE+CDSS profoundly changes staff workflow, and often leads to unintended consequences and new safety issues (such as alert fatigue) which limit the system's safety effects.
Conclusions CPOE+CDSS does not appear to reliably prevent clinical ADEs. Despite more widespread implementation over the past decade, it remains a work in progress.
- computerized provider order entry
- clinical decision support systems
- medication errors
- adverse drug events
- patient safety
- health information technology
Statistics from Altmetric.com
- computerized provider order entry
- clinical decision support systems
- medication errors
- adverse drug events
- patient safety
- health information technology
Introduction
Adverse drug events (ADEs) are a major source of preventable harm in both hospitalised and ambulatory patients. Preventable ADEs—which are primarily a result of prescribing errors1 ,2—occur in 7–10 of every 100 hospital admissions,3–5 and may be even more common in the ambulatory setting.6
Computerised provider order entry (CPOE) systems require clinicians to directly place orders for medications, tests or studies into an electronic system, which then transmits the order directly to the recipient responsible for carrying out the order (the pharmacy, laboratory or radiology department). These systems have the potential to greatly reduce prescribing errors, since at a minimum they ensure standardised, legible and complete orders. Initially implemented in the inpatient setting with the goal of preventing prescribing errors and ADEs, the use of CPOE systems has increasingly been broadened to include entry of all types of orders in both the inpatient and outpatient settings.
Clinical decision support systems (CDSS) are often integrated with CPOE, and aim to optimise the safety and quality of clinical decisions by providing clinicians with reminders or recommendations at the point of care. For example, a medication CDSS may offer default values for doses, routes of administration and frequency for commonly used drugs. In more advanced forms, CDSS can also check for drug allergies or drug–drug interactions, provide reminders for appropriate laboratory monitoring (eg, checking renal function if a patient is prescribed a potentially nephrotoxic medication) or even suggest appropriate orders based on patient factors (eg, ordering broad-spectrum antibiotics in a patient admitted with sepsis). At the highest level of sophistication, the combination of CPOE and CDSS—hereafter referred to as CPOE+CDSS—has the potential to prevent ADEs by preventing errors of commission and errors of omission.
The pace of uptake of CPOE and CDSS has been relatively slow in both the inpatient and outpatient environments,7–9 but is increasing in the USA due to the federal Health Information Technology for Economic and Clinical Health (HITECH) Act. HITECH stipulates that healthcare providers must demonstrate the ‘meaningful use’ of electronic health records (EHRs) by 2015, and has provided incentive payments for implementing such systems, switching to penalties for failing to achieve the meaningful use standard by 2016. The meaningful use criteria require in part that EHRs must include a CDSS system. In the UK, universal implementation of CPOE in both inpatient and ambulatory settings is an area of priority for the National Health Service. Despite this increased usage, questions about the effectiveness of CPOE+CDSS at preventing clinical ADEs remain. A recent systematic review10 estimated that CPOE can reduce medication prescribing errors in hospitalised patients by 48% on average, but acknowledged that ‘it is unclear whether this translates into reduced harm for patients’.
The 2001 Making Health Care Safer report, published by the US Agency for Healthcare Research and Quality, reviewed evidence on the effectiveness of CPOE+CDSS, as well as isolated CDSS, at improving medication safety.11 The review defined level 1 outcomes as ADEs, and level 2 and 3 outcomes as medication errors and change in prescribing practices, respectively. These definitions were used in order to distinguish the effects of CPOE and CDSS on clinical outcomes (eg, preventable ADEs) and surrogate outcomes that may not have caused patient harm (eg, medication errors). The review identified four studies, all of which found improvement in level 2 and 3 outcomes, but did not document a reduction in preventable ADEs. The review also identified significant problems with generalisability of these results, as all of the studies evaluated ‘homegrown’, institution-specific systems (as opposed to commercial systems purchased from vendors) and often focused on safety of a specific medication or medication class. The review assigned CPOE+CDSS a ‘medium strength of evidence’ recommendation, noting the limited effect of CPOE+CDSS on preventing clinical patient harm. This conclusion proved to be somewhat controversial, with follow-up commentaries arguing that as clinical ADEs are rare, conducting randomised trials of CPOE+CDSS would be costly and impractical, and that proof of clinical benefit should not be required before wider adoption given the face validity of such systems.12 The evidence report's authors responded that rigorous research to evaluate the effectiveness and generalisability of CPOE+CDSS—and all patient safety practices—was necessary in order to appropriately assess and prioritise methods of improving safety.13
Since this debate, CPOE+CDSS usage has indeed been more widely adopted, and more high-quality research has been undertaken and disseminated. In view of the increasing use of these systems, we performed a narrative review to assess the state of the evidence regarding the effectiveness of CPOE+CDSS at preventing clinically significant ADEs, and evaluate factors such as implementation issues and unintended consequences that may play a role in the success or failure of CPOE+CDSS systems.
Methods
This review was performed as part of Making Healthcare Safer II, a series of comparative effectiveness reviews of patient safety strategies commissioned by the US Agency for Healthcare Research and Quality (AHRQ).14 We sought to narratively synthesise the results of recent systematic reviews of the effect of CPOE+CDSS on medication safety. We identified eligible articles by searching the specialised database AHRQ Patient Safety Net (http://psnet.ahrq.gov), which indexes published literature and grey literature on patient safety, using the keywords ‘CPOE’, ‘clinician decision support system’, ‘health information technology (HIT)’, ‘electronic health record’ and ‘electronic medical record’. For the AHRQ technical report on which this article is based, we included systematic and narrative reviews specifically assessing the effect of CPOE+CDSS on clinical ADEs, as well as articles that measured costs, unintended consequences or specific implementation issues relating to CPOE+CDSS implementation. For this article, we updated the systematic review search, and also included controlled studies (randomised or non-randomised) published after the most recent inclusion date of the systematic reviews.
Results
Effect of CPOE+CDSS on ADEs
We identified five systematic reviews and one narrative review published since 2008 that met our inclusion criteria (table 1). Wolfstadt et al15 identified 10 trials of CPOE+CDSS (none of which were randomised controlled trials (RCTs)), and concluded that CPOE+CDSS may be effective at reducing ADEs, as five of the 10 studies found a statistically significant reduction in ADEs and four others reporting a non-significant improvement. However, most of these studies utilised homegrown systems, and nine of the 10 included studies were conducted in the inpatient setting.
Schedlbauer et al16 identified 20 studies (including four RCTs) that evaluated a total of 27 forms of CDSS. The authors classified the CDSS alerts as ‘basic’ (including only information about allergies, drug–drug interactions and default dosing), ‘advanced’ (including alerts targeting errors of omission and patient-specific dosing and safety guidelines) and ‘complex’ (including features of both basic and advanced systems). Only four of these studies—only one of which used a ‘complex’ alert—evaluated the effect of CPOE+CDSS on clinical ADEs; three of them did find statistically significant reductions in preventable ADEs. Ammenwerth et al’s review17 also identified seven observational studies of the effect of CPOE+CDSS on ADE rates; the four studies reporting a significant reduction in ADEs all utilised a CDSS with patient-specific alerts.
Van Rosse et al’s review18 specifically focused on the effectiveness of CPOE+CDSS in adult and paediatric intensive care units. The 12 observational studies they identified collectively demonstrated reductions in medication prescribing errors; however, no overall effect was found on ADEs or mortality rates.
As part of a larger systematic review of the effectiveness of HIT on medication management, McKibbon et al19 identified 78 RCTs of various HIT-based interventions on medication safety, including 10 trials specifically evaluating the effect of CPOE+CDSS on ADE rates. Although the studies yielded no conclusive evidence that any form of HIT can improve clinical outcomes, they did find that CDSS that targeted specific clinical problems (eg, ordering blood tests to monitor the safety of anticoagulants) appeared to be more effective than those offering general decision support (eg, providing non-specific recommendations for cardiovascular risk reduction).
Paediatric inpatients are particularly vulnerable to prescribing errors. A narrative review by Stultz and Nahata20 identified 11 studies evaluating the effectiveness of CPOE+CDSS in this population. They concluded that CPOE+CDSS may reduce ADEs, but were unable to identify specific CDSS features that were associated with ADE prevention.
These reviews identified several common weaknesses in the CPOE+CDSS literature base. The vast majority of studies have been conducted in the inpatient setting, often within a single hospital, and consisted of relatively small patient populations evaluated for short intervention periods. Many studies also evaluated homegrown systems, with comparatively less evaluation of commercial CPOE+CDSS systems. We did identify two recent controlled trials that studied the effectiveness of systems developed by commercial vendors. Westbrook et al21 conducted a controlled study of the implementation of two different commercial CPOE+CDSS systems in two teaching hospitals, and found that both systems reduced the incidence of serious medication prescribing errors (those considered to have a high likelihood of causing patient harm). The study did not formally assess clinical outcomes, however. Leung et al22 evaluated the effect of a commercial CPOE+CDSS system in five community hospitals over a 5-year period. The overall rate of ADEs actually increased during the study period, although this was primarily due to non-preventable events; the rate of preventable ADEs fell (from 10.7 to 7.0 ADEs per 100 admissions, p<0.001).
The use of CPOE+CDSS in the ambulatory care setting is less extensively studied. Two recent studies23 ,24 conducted in large, community-based practice settings found that mandatory use of CPOE+CDSS achieved reductions in prescribing errors, but not clinical ADEs—mirroring the evidence from the inpatient setting.
Taken together, these data indicate that while CPOE+CDSS clearly reduce medication prescribing errors, their effect on prevention of clinical ADEs is inconsistent. Two recent systematic reviews that evaluated the effect of CDSS on a broad range of outcomes provide insight into why CDSS, with or without CPOE, may be less effective than originally thought. A meta-analysis and meta-regression by Roshanov et al25 reviewed 162 RCTs of various CDSS with the goal of identifying the features of effective CDSS. Somewhat surprisingly, systems that integrated CDSS within CPOE were actually less effective, compared with those employing other ways of delivering reminders. Bright et al26 reviewed 148 RCTs of CDSS that provided decision support for a variety of clinical processes. Although CDSS improved the ordering of preventive services, clinical studies and appropriateness of prescribing, the magnitude of these effects was small to moderate, and did not appear to improve clinical outcomes. These disappointing findings are likely explained by the complexity of integrating CPOE+CDSS into the busy clinical environment, and the unintended consequences that can result from flawed implementation of these systems.
Unintended consequences and adverse effects associated with CPOE+CDSS
The growth in use of CPOE+CDSS has yielded a more nuanced appreciation of the unintended consequences of the technology. These unintended consequences, which include unfavourable workflow changes, generation of new types of errors and overdependence on the technology, were classified in a seminal 2006 article27 and are summarised in table 2. Clinicians perceive these unintended consequences to be common, and to adversely affect care.28
A detailed discussion of the various potential adverse effects of CPOE is beyond the scope of this review, but one particular problem deserves comment, as it likely explains in part why CPOE+CDSS has not yet delivered its promised safety benefits. CPOE+CDSS systems generate alerts for clinicians during the ordering process in order to warn them of the possibility of harmful prescribing, and approximately 3–6%29 ,30 of orders generate alerts in a typical system. Since a busy clinician could enter hundreds of orders in a day, he or she will likely receive multiple, perhaps dozens, of alerts every day. ‘Alert fatigue’ refers to the phenomenon whereby users of a CDSS that generates an excessive number of warning messages tend to ignore many of these alerts—even the ones that warn of potentially serious errors. Alert fatigue is well documented in both the inpatient and ambulatory settings,29 as the CPOE+CDSS literature consistently shows that clinicians override the majority of alerts, even those ‘critical’ alerts warning of potentially severe drug–drug interactions. In one study of an outpatient CPOE+CDSS system, clinicians overrode nearly 90% of ‘high-severity’ alerts,30 and another hospital-based study found that clinicians ignored 75% of even ‘critical’ drug–drug interaction alerts.31 This problem arises in part because most existing CPOE+CDSS systems favour providing comprehensive alerts for all potential drug safety problems rather than focusing alerts on the most clinically significant problems. Studies have shown that as many as 40% of drug interaction alerts may represent false positives,32 and when surveyed, clinicians often cite the questionable clinical significance of alerts as a major reason for overriding them.33
There is consensus that alert fatigue diminishes the potential safety effects of CPOE+CDSS, but no standardised approach exists to avert this problem. Some studies have successfully ‘tailored’ alerts by incorporating patient-specific characteristics into algorithms for displaying drug warnings. For example, Seidling et al34 achieved a reduction in prescribing errors by tailoring drug alerts at a German hospital. In this study, providers accepted nearly 25% of warnings, much higher than rates generally reported in the literature. However, efforts to tailor drug warnings are currently limited by the lack of standardised consensus definitions for drug–drug interactions that are likely to lead to ADEs, and unclear malpractice implications for users and manufacturers of CDSS systems35 should patients be harmed if an alert is not provided. Recent commentaries35 ,36 have called for better guidance and legal protection to allow greater tailoring of alerts, and a recent consensus conference37 identified the key issues in developing more effective alert mechanisms. Initial recommendations have also been published that list specific drug–drug interactions that pose minimal clinical risk and therefore should not require interruptive alerts.38
Implementation and costs
Healthcare organisations must pay very close attention to how CPOE+CDSS is configured and implemented, as failure to effectively implement CPOE+CDSS can lead to substantial frustration on the part of clinicians,39 decreased efficiency,40 and even clinical harm. Information technology implementation is much more than a technical intervention, as it profoundly affects workflow for both clinical and non-clinical staff.41 The workflow changes that result—which include the need to continuously interact with the system, shifting of roles among providers, and creation of workarounds to avert problems with the system—have been implicated in the high-profile abandonment of some systems after implementation,42 and have been associated with patient harm in others.43 Even when systems are implemented successfully, specific aspects of system configuration can lead to unintended consequences. For example, one study44 evaluated the effect of a ‘hard-stop’ warning that essentially prevented co-prescribing of the anticoagulant warfarin and the antibiotic trimethoprim–sulfamethoxazole—a combination associated with serious bleeding risks. The warning was effective at its intended aim, but was abandoned after 6 months because four patients experienced delays in needed treatment with one of the drugs. The technical aspect of how alerts are configured is thus critical to ensuring their effectiveness, as overly rigid rules could lead to delays in therapy, whereas more frequent but less consequential warnings run the risk of inducing alert fatigue.
Unfortunately, no clear consensus exists on the optimal implementation methods in either the hospital or ambulatory setting. It has become clear that EHR implementation in general must take into account the principles of human factors engineering, tailoring the introduction of the systems so as to minimise disruptions to existing clinician workflow and to avoid problems such as alert fatigue.45 Ongoing support and follow-up with frontline users is also required in order to respond to clinicians’ concerns and make system improvements, since user perceptions of information technology invariably change over time.46 The AHRQ has published the online ‘Guide to reducing unintended consequences of electronic health records’ (http://www.ucguide.org), and several case studies of implementation of commercial CPOE+CDSS systems have also been published.47–51
The cost–effectiveness of CPOE+CDSS is also unclear; we did not identify any formal cost–effectiveness analyses of CPOE+CDSS published in the past 5 years. Individual institutions with homegrown CPOE+CDSS systems have estimated considerable cost savings52 due to ADE prevention and optimising medication use, but these data may not be generalisable to other settings and systems. In the USA, the HITECH Act's considerable financial incentives have shifted the cost–benefit equation markedly despite the absence of formal cost–effectiveness data.
Conclusions
CPOE+CDSS are effective at reducing medication prescribing errors, but there is no clear evidence that these systems reduce clinical ADEs in either the inpatient or outpatient setting. Implementation issues, including failure to adequately tailor warnings and the resultant alert fatigue, may explain this lack of success. Unfortunately, a decade of wider CPOE+CDSS implementation and intensive research does not appear to change the conclusion reached more than a decade ago when the effectiveness of these systems was first systematically reviewed11: despite considerable promise and face validity, these systems do not clearly prevent patient harm.
The use of CPOE+CDSS will certainly continue to increase worldwide as part of the move towards EHR. Despite their mixed record thus far, it is conceivable that overall safety performance may improve with more widespread use. Multiple studies have demonstrated that increased user experience with EHR is required before safety benefits are achieved—for example, one recent inpatient study50 found that prescribing errors consistently decreased as users became more familiar with the system, and another primary care-based study53 found that clinicians’ perceptions of the system's patient safety effects markedly improved after 1 year of experience. The adverse effects currently associated with CPOE+CDSS implementation may therefore decrease as organisations gain more experience with tailoring these systems to better integrate with clinician workflow and minimise problems such as alert fatigue. However, it is clear that further research and dissemination of best practices in EHR implementation is sorely needed, especially for commercial applications. Until such approaches have been defined, the actual process of implementing CPOE+CDSS will remain exercises in trial and error.
Although CPOE has specifically been recommended as a patient safety strategy by a wide range of influential organisations, it may not be capable of preventing ADEs on its own. CPOE+CDSS systems are designed to primarily target prescribing errors, and in specific clinical circumstances may also target appropriate monitoring of therapies. However, they have no effect on medication administration errors, which account for a large proportion of medication errors in inpatients. ‘Closed loop’ medication management systems, which combine CPOE+CDSS with other proven technologies to prevent errors at each step of the medication pathway, show great promise for preventing clinical ADEs.54 Fundamentally, though, improving medication safety cannot be viewed as an isolated safety issue. Analyses of serious medication errors55 invariably reveal other underlying system flaws, such as human factors engineering issues and impaired safety culture, which allow individual prescribing or administration errors to reach the patient and cause serious harm. Despite the promise of technological approaches to medication safety, the potential for error will remain unless these systems are carefully implemented and greater attention is paid to developing safer systems of care.
Acknowledgments
The authors gratefully acknowledge Yimdriuska Magan, BS for assistance with article preparation, and Lisha Lo, BA for assistance with article abstraction.
References
Footnotes
-
Contributors SRR was responsible for acquisition of data, synthesis of data, drafting and revision of the manuscript. SR was responsible for acquisition and synthesis of data. RMW was responsible for drafting and revision of the manuscript.
-
Funding This study was funded by the Agency for Healthcare Research and Quality, US Department of Health and Human Services (contract no. HHSA-290-2007-100621). The Agency for Healthcare Research and Quality reviewed contract deliverables to ensure adherence to contract requirements and quality, and a copyright release was obtained from the Agency for Healthcare Research and Quality before submission of this manuscript. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.
-
Competing interests None.
-
Provenance and peer review Not commissioned; internally peer reviewed.