The unintended consequences of computerized provider order entry: Findings from a mixed methods exploration

https://doi.org/10.1016/j.ijmedinf.2008.07.015Get rights and content

Abstract

Objective

To describe the foci, activities, methods, and results of a 4-year research project identifying the unintended consequences of computerized provider order entry (CPOE).

Methods

Using a mixed methods approach, we identified and categorized into nine types 380 examples of the unintended consequences of CPOE gleaned from fieldwork data and a conference of experts. We then conducted a national survey in the U.S.A. to discover how hospitals with varying levels of infusion, a measure of CPOE sophistication, recognize and deal with unintended consequences. The research team, with assistance from experts, identified strategies for managing the nine types of unintended adverse consequences and developed and disseminated tools for CPOE implementers to help in addressing these consequences.

Results

Hospitals reported that levels of infusion are quite high and that these types of unintended consequences are common. Strategies for avoiding or managing the unintended consequences are similar to best practices for CPOE success published in the literature.

Conclusion

Development of a taxonomy of types of unintended adverse consequences of CPOE using qualitative methods allowed us to craft a national survey and discover how widespread these consequences are. Using mixed methods, we were able to structure an approach for addressing the skillful management of unintended consequences as well.

Introduction

When our study began in October of 2003, the unintended consequences of computerized provider order entry (CPOE) were a little-discussed area. Patterson et al. had identified and described what they called “side effects” of a different, but similar, kind of system, bar code medication administration (BCMA) [1]. They used a relatively structured ethnographic approach to studying BCMA in Veterans Administration hospitals and identified side effects which they believed could lead to adverse drug events (ADEs). BCMA is supposed to prevent ADEs, but there are numerous unintended consequences they documented that could lead to mistakes. They offered suggestions about how to “eliminate these side effects before they contribute to adverse outcomes” ([1], p. 540).

Problems related to clinical decision support had likewise been described in the literature. Although decision support is often cited as a reason for implementing CPOE, there has been controversy about the appropriate number of alerts and reminders, since too many tend to overwhelm and annoy users [2]. One 1989 report described an experiment where, to reduce the time between alert posting and review by the clinician, a flashing light mechanism was placed on top of the computer and designed to flash to let a user know when an alert was present. The system was extremely effective in encouraging a rapid response to the alert, reducing the average acknowledgment time from 28 to 1 h, but users insisted the experiment be halted because the lights were too annoying [3]. This is a dramatic example of a negative unintended consequence of an otherwise effective system.

Medical error reduction is a prime reason for implementing CPOE, but users are also concerned that new kinds of errors are being made because of clinical systems. Many papers written about CPOE gave brief mention to this concern or cited anecdotes, but there were no published studies about mistakes that could be caused by CPOE. The Physician Order Entry Team (POET), a group of researchers based at Oregon Health & Science University in Portland, Oregon, U.S.A., was conducting a study of success factors for implementing computerized physician order entry (CPOE), defined as direct entry of orders into the computer by physicians or others with the same ordering privileges, when we began noticing unintended consequences (UCs) that might lead to errors. The clearest example is entry of an order for the wrong patient because of what we call a “juxtaposition error” when an item near the one actually desired is clicked by mistake.

Colleagues doing similar qualitative studies in Australia and The Netherlands were discovering these UCs as well, and a collaborative effort in 2002 produced a general description of kinds of adverse consequences caused by clinical information systems (CIS) [4]. This was a rather startling revelation at a time when CPOE was being touted as the “leap” that hospitals should take in the interest of patient safety [5] and little attention was being paid to problems caused by CPOE. In three separate observational studies, these research teams had continually witnessed “wrong patient” juxtaposition errors. Similar errors occur when one clicks on a test or medication listed on the screen next to the one needed. The summary paper by Ash et al. highlighted the phrase “unintended consequences of CPOE,” which has become widely accepted and used [4]. This paper was influenced by several monographs that dramatically describe the unintended consequences of technology in general [6], [7], [8], [9]. Since publication of the Ash et al. paper [4], numerous papers in both medical and the medical informatics journals have further described the unintended consequences of health information technology [10], [11], [12], [13], [14], [15], [16], [17], [18].

With funding from the U.S. National Library of Medicine, POET has been able to conduct an in-depth study over the past 4 years utilizing both qualitative and quantitative methods to discover more about these UCs of CPOE. Data were gathered via two expert panel conferences, fieldwork at a total of six sites (one outpatient and five primarily inpatient), and a national telephone survey of all CPOE sites in the U.S.A. The aims were to identify types of UCs and strategies for preventing, managing or overcoming them, and to provide tools to help implementers address them. The following presents a summary of the research foci, methods, and results, along with general conclusions about the overall project.

Section snippets

Sample selection

The main criterion for site selection for fieldwork was that organizations have a reputation for excellence in using clinical information systems. Excellent organizations learn from their mistakes [19], [20] and therefore staff members in those organizations have analyzed the issues and are knowledgeable about strategies for overcoming obstacles. We were seeking sites with personnel who would be willing to (1) describe surprises they have experienced and managed, and (2) be observed during the

The initial schema

The initial broad schema of consequences related to CPOE included categories of intended and unintended consequences, desirable and undesirable, direct and indirect, and “two sided” consequences that could be either desirable or undesirable depending on one's point of view [22]. This schema provided a valuable framework for fieldwork because it assured that we would not limit our foci to adverse consequences. While most informaticians are interested in undesirable consequences because they need

Discussion

In this team's earlier research on success factors for implementing CPOE, we cast a wide net because little was known at that time about factors leading to success or lack thereof. A rigorous yet open-ended grounded theory [24] approach was deemed most appropriate. As research questions became more focused, however, our qualitative methods became more structured. To investigate UCs of CPOE, we started with a broad schema of types and iteratively refined the schema by consulting experts,

Conclusion

While it is hard summarizing results of an intense 4-year study of UCs, we can draw some general conclusions about both methods and UCs. First, the selected methods served us well for this study. The more structured and rapid techniques such as the anticipation survey efficiently augmented other kinds of fieldwork. Second, development of a taxonomy of types and subtypes not only allowed us to craft survey questions, but was also useful in structuring an approach for addressing management of

Acknowledgments

This paper is based on a presentation given at MedInfo 2007. This work was supported by grant LM06942 and training grant ASMM10031 from the U.S. National Library of Medicine, National Institutes of Health. The funding agency had no role in development of this paper and none of the authors has conflicts to declare. All authors contributed to data gathering, analysis, and writing.

Summary points

What was known before this study

  • CPOE with CDS can reap benefits for hospitals, but use is not widespread

References (43)

  • D.F. Sittig et al.

    Assessing the anticipated consequences of computer-based provider order entry at three community hospitals using an open-ended, semi-structured survey instrument

    Int. J. Med. Inform.

    (2008)
  • E.S. Patterson et al.

    Improving patient safety by identifying side effects from introducing bar coding in medication administration

    J. Am. Med. Inform. Assoc.

    (2002)
  • The Leapfrog Group, Factsheet: computer physician order entry....
  • E. Tenner

    Why Things Bite Back: Technology and the Revenge of Unintended Consequences

    (1996)
  • D. Doerner

    The Logic of Failure: Recognizing and Avoiding Error in Complex Situations

    (1996)
  • C. Perrow

    Normal Accidents: Living with High-Risk Technologies

    (1999)
  • J.R. Chiles

    Inviting Disaster: Lessons from the Edge of Technology

    (2002)
  • R. Koppel et al.

    Role of computerized physician order entry systems in facilitating medication errors

    JAMA

    (2005)
  • R.L. Wears et al.

    Computer technology and clinical work: still waiting for Godot

    JAMA

    (2005)
  • J.R. Nebeker et al.

    High rates of adverse drug events in a highly computerized hospital

    Arch. Int. Med.

    (2005)
  • Y.Y. Han et al.

    Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system

    Pediatrics

    (2005)
  • Cited by (160)

    • Unintended consequences of pharmacy information systems: A case study

      2023, International Journal of Medical Informatics
    View all citing articles on Scopus
    View full text