You are here

  • Home
  • Archive
  • Volume 27, Issue 4
  • Reasons for computerised provider order entry (CPOE)-based inpatient medication ordering errors: an observational study of voided orders

Article Text

Article menu

Download PDFPDF

Original research
Reasons for computerised provider order entry (CPOE)-based inpatient medication ordering errors: an observational study of voided orders
  1. Joanna Abraham1,
  2. Thomas G Kannampallil2,
  3. Alan Jarman1,
  4. Shivy Sharma1,
  5. Christine Rash3,
  6. Gordon Schiff4,
  7. William Galanter3,5,6
  1. 1 Department Biomedical and Health Information Sciences, University of Illinois, Chicago, Illinois, USA
  2. 2 Department of Family Medicine, University of Illinois at Chicago, Chicago, Illinois, USA
  3. 3 Department of Pharmacy Practice, University of Illinois at Chicago, Chicago, USA
  4. 4 Department of General Medicine, Brigham and Women’s Hospital, Boston, Massachusetts, USA
  5. 5 Department of Medicine, University of Illinois, Chicago, Illinois, USA
  6. 6 Department of Pharmacy Systems, Outcomes and Policy, University of Illinois at Chicago, Chicago, IL, USA
  1. Correspondence to Dr Joanna Abraham, Department of Biomedical and Health Information Sciences, University of Illinois, 1919 W Taylor Street, Chicago, Illinois 60612,USA; abrahamj{at}uic.edu

Abstract

Objective Medication voiding is a computerised provider order entry (CPOE)-based discontinuation mechanism that allows clinicians to identify erroneous medication orders. We investigated the accuracy of voiding as an indicator of clinician identification and interception of a medication ordering error, and investigated reasons and root contributors for medication ordering errors.

Method Using voided orders identified with a void alert, we conducted interviews with ordering and voiding clinicians, followed by patient chart reviews. A structured coding framework was used to qualitatively analyse the reasons for medication ordering errors. We also compared clinician-CPOE-selected (at time of voiding), clinician-reported (interview) and chart review-based reasons for voiding.

Results We conducted follow-up interviews on 101 voided orders. The positive predictive value (PPV) of voided orders that were medication ordering errors was 93.1% (95% CI 88.1% to 98.1%, n=94). Using chart review-based reasons as the gold standard, we found that clinician-CPOE-selected reasons were less reflective (PPV=70.2%, 95% CI 61.0% to 79.4%) than clinician-reported (interview) (PPV=86.1%, 95%CI 78.2% to 94.1%) reasons for medication ordering errors. Duplicate (n=44) and improperly composed (n=41) ordering errors were common, often caused by predefined order sets and data entry issues. A striking finding was the use of intentional violations as a mechanism to notify and seek ordering assistance from pharmacy service. Nearly half of the medication ordering errors were voided by pharmacists.

Discussion We demonstrated that voided orders effectively captured medication ordering errors. The mismatch between clinician-CPOE-selected and the chart review-based reasons for error emphasises the need for developing standardised operational descriptions for medication ordering errors. Such standardisation can help in accurately identifying, tracking, managing and sharing erroneous orders and their root contributors between healthcare institutions, and with patient safety organisations.

  • Medication Safety
  • Patient Safety
  • Qualitative Research

http://dx.doi.org/10.1136/bmjqs-2017-006606

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background and significance

    Medication errors are reported to cause approximately 1.5 million preventable adverse drug events (pADEs) in the USA every year.1 Electronic prescribing using computerised provider order entry (CPOE) systems has been highlighted by clinicians, researchers and policy makers as a mechanism to prevent and mitigate potential medication errors.2 3 Improvements in medication safety are achieved through CPOE infrastructures that support coordination of clinical tasks among patient care teams,4 reduction of misinterpretation of orders,5 minimisation of illegible orders,6 assistance with medication dosage calculations7 and decision support using alerts (eg, drug–drug interaction alerts, drug allergy alerts, patient problem-based indication alerts and dosage suggestions).7–9

    Notwithstanding these advantages, several concerns associated with CPOE use continue to surface, including unintended consequences of CPOE use,10 11 adverse drug events12 and other medication safety challenges.8 13 With over 26.1% (range: 16.0%–53.6%) of medication orders  initiated using a CPOE system in acute care hospitals in the USA ,14 the importance of devising effective mechanisms to identify, characterise and track medication ordering errors is of significant concern. The value of recording and tracking medication errors has been highlighted in an Institute of Medicine (IOM; now, the National Academy of Medicine) committee report, Health IT and Patient Safety: Building Safer Systems for Better Care,15 and in the 2014 US Food and Drug Administration Safety and Innovation Act.16 Furthermore, medication safety experts have emphasised the ‘worrisome lack of effort to learn from medication ordering errors’ and characterised the current state of unsafe medication ordering practices to be ‘still a work in progress’.17 This is especially important given the high rates of pADEs that occur during medication ordering.18 19 For example, Nebeker et al 20 reported that 61% of pADEs occurred during the medication ordering stage.

    To address this, recent initiatives have focused on ways to identify and classify medication ordering errors.21 However, there are limited, if any, approaches for automatically flagging, tracking and aggregating medication ordering error data in real time.

    Informed by a recent retrospective analysis of CPOE-based medication orders, we identified a potentially viable approach for identifying intercepted erroneous medication orders. This approach relies on tracking medication orders that were discontinued using the ‘void’ function, indicating that an order was in error (detailed description is provided in the section on ‘Medication order voiding’).22 Based on a retrospective study of nearly six million medication orders over a 6-year period, Kannampallil and colleagues23 found that an estimated 70% of voided orders were medication ordering errors. However, clinician-selected reasons documented at the time of voiding were reasonably predictive of the actual cause of medication ordering errors only for duplicate orders (72%), but not for any of the other menu of choices for voided order error reasons.

    Leveraging these results, we sought to better characterise the underlying causes and reasons for medication ordering errors identified through the medication voiding process. For voided orders, using an observational study, we asked the following research questions: (1) What are the reasons for CPOE-based medication ordering errors based on clinician perceptions and patient chart reviews? (2) Are there mismatches between clinician-perceived reasons (based on interviews) and actual reasons (based on chart reviews) for these errors? (3) What factors contribute to the generation of, and identification of, medication ordering errors?

    Medication order voiding

    Medication order voiding is a CPOE-integrated discontinuation function in the Cerner electronic health record (EHR) that allows clinicians to identify and remove erroneous orders from a patient’s active medication list.22 23 By choosing to void a medication, a clinician indicates that an order was erroneously placed. Within the CPOE, medication order voiding involves the selection of a medication order to be voided and choosing the ‘void’ option. After this, the clinician is prompted to select from one of eight choices as their reason for voiding: blank (no reason is provided), order on wrong encounter, wrong patient, incorrect ordering physician, duplicate order, system date error, voiding student order and improperly composed order (see online supplementary appendix figure 1 for the voiding workflow). This set of eight reasons was the default, vendor-provided options at the time of CPOE implementation. We refer to these as the ‘clinician-CPOE-selected reasons for voiding’. Although clinicians have multiple options to discontinue medication orders, including ‘cancel’, ‘modify’ or ‘complete’, the ‘void’ function is intended to be used for medication orders that a clinician identifies and flags as being placed in error (eg, order on the wrong patient or wrong drug). Evaluation of such voided orders thus can provide insights into the nature, reasons and causes of medication ordering errors.

    Supplementary appendix

    Method

    Setting

    This study was conducted at a Midwestern academic medical centre in the USA that houses a 495-bed hospital with approximately 22 000 hospitalisations per year. Medication orders were placed with a CPOE using Cerner PowerChart. The configuration of Cerner at this site included the use of Multum drug–drug interaction and duplicate order alerts, along with clinical decision support for drug–laboratory (eg, when ordering potassium or an ACE inhibitor, serum potassium test results are evaluated), drug–diet and drug–disease alerts.

    Physicians (attending, fellows and residents), pharmacists, nurses and medical students can place medication orders. As per state laws, pharmacists and nurses can place medication orders based on verbal, written or protocol-based physician orders. Although a physician cosignature is required, these orders are immediately actionable. Medication orders placed by medical students are not actionable without a physician cosignature.

    Medication voiding data were collected from inpatient units, the setting where most of the preventable prescribing errors have been reported.24 Recent IOM reports have suggested that a hospitalised patient experiences an average of one medication error per day during their stay.25 26 Additionally, due to the longitudinal nature of care provided in inpatient units, it was relatively easier to contact clinicians who were voiding medication orders on hospitalised patients.

    This study was approved by the institutional review board of the university, and verbal consent was obtained from all participants.

    Void alert tool

    For identifying and tracking inpatient voided orders, we developed a Void Alert Tool (VAT). VAT generated an alert based on the following conditions: (1) voiding action performed by a clinician for a patient in an inpatient unit and (2) for a patient whose age >18 years.

    Each VAT trigger initiated a secure email transferring the details of the voided order — medical record number, patient name, medication name, ordering clinician and voiding clinician — to the clinical inbox of the first author (JA). A void alert secure message for a test patient is shown in online supplementary appendix figure 2.

    Data collection

    We followed up on a convenience sample of 289 voided orders (37%), mostly between the hours of 07:00 and 18:00 during week days (Monday through Friday). Follow-up interviews were conducted on 101 voided orders.

    Clinician interviews

    Within 24 hours of a VAT trigger, a researcher attempted to contact the clinicians — both the original ordering clinician (ie, clinician who placed the order that was later voided) and the voiding clinician (ie, clinician who voided the original order) — through the hospital paging system and telephone calls.

    Clinicians who responded to the page with telephone calls were recruited for participation in this study. Recruited clinicians were interviewed using a semistructured interview template. The primary purpose of the follow-up telephone interview was to develop a contextual perspective on the medication order and its subsequent voiding by obtaining first-hand reports from the involved clinicians. Similar approaches have been successfully used for prospectively studying medication ordering errors.13

    For the ordering clinician, interview questions focused on the characteristics of the medication order, reasons for placing the original order, possible reasons for its eventual voiding and factors that could have led to the order being voided. For the voiding clinician, interview questions were related to the reasons for their voiding, how they reached their voiding decision and potential factors that led to the voiding of the order. In some cases, where the ordering and voiding clinicians were the same individual, questions were modified to gather insights on reasons for both ordering and voiding. Interviews lasted approximately 5–7 min and were audio-recorded. All interviews were conducted either by a medical student or a physician-researcher. The interview guide is provided in section 2 of the online supplementary appendix.

    Patient chart reviews of voided orders

    Patient charts of all voided orders that had an interview with at least one clinician (n=101 voided orders) were reviewed. The purpose of the chart review was to (1) identify whether a voided medication order was an error, and (2) if it was an error, the reason for the error (we refer to this as ‘medication ordering error reason (chart)’).

    For characterising a medication ordering error, we used the following generic definition: ‘any preventable event that may cause or lead to inappropriate medication use or patient harm’.27 Specifically, we investigated whether any signed voided order may have resulted in (or led to) inappropriate medication use or harm. The reviewers relied mainly on progress notes, other medication orders, problem lists and laboratory tests to make this determination. We used the chart review findings as the gold standard basis for all comparisons. Prior research, based on a systematic review, has shown that chart review-based evaluation is used extensively to identify medication-related problems.28

    Chart reviews were conducted in two phases by trained chart abstractors, using an abstraction protocol for determining (1) and (2) described above; the difference between the phases was that additional demographic data were captured in the first phase. In the first phase, a physician (SS) reviewed patient charts associated with each voided order to determine (1) and (2) for each voided order. This reviewer also gathered the following data elements from the patient chart: clinician role for both ordering and voiding (physician, nurse, pharmacist, medical student), time of ordering and voiding, patient demographics (age, gender and race) and medication details (name, route, dosage and frequency of use).

    In the second phase, a clinical pharmacist and a physician (CR and WG, reviewing 50 and 51 charts, respectively) independently reviewed each voided order to determine (1) and (2) for each order. In case of disagreements between the first and second phases on either (1) or (2), a third adjudicator independently reviewed the chart, and a final disposition was based on a majority decision among the three reviewers. The reviewers had access to the entire patient chart during the reviews. The data collection process is illustrated in figure 1.

    Figure 1
    Figure 1

    Data collection and analysis process. On the left (A), the generation of void alerts, follow-up interviews with clinicians and the outcomes are shown (ie, clinician-reported reasons for voiding (interview) and factors contributing to medication ordering errors). On the right (B), the chart review and the outcomes from the chart review process are shown. The study was conducted between 27 January 2016 and 29 July 2016. The frequencies of total inpatient medication orders (age >18), voided medication orders, interviews conducted and medication ordering errors identified are also provided.

    Data analysis

    Analysis of interviews

    Interviews were transcribed verbatim and deductively analysed using the structured framework developed by Schiff et al.21 The framework included three themes: what happened, why it happened and possible prevention strategies. The ‘what happened’ theme was used to categorise the clinician-reported reasons for voiding during the interview. We refer to these as ‘clinician-reported reasons for voiding (interview)’; ‘why it happened’ theme was used for coding the contributing factors for the order and its eventual voiding; and finally, ‘prevention strategies’ were the suggestions provided by clinicians for preventing similar medication ordering errors.

    We further classified clinician-reported reasons (interview) along two dimensions: first, along four core categories representing the primary factors contributing to the medication ordering errors: CPOE order entry issues, user-related issues, system limitations and transition issues. Next, for each core category, we identified specific subcategories. For example, user-related issues were further coded into subcategories such as data entry errors, lack of user clinical knowledge and communication issues. The categories and subcategories were adapted from published frameworks on reasons for medication errors.21 29 Descriptions for the core and subcategories are provided in table 1.

    View this table:
    Table 1

    Descriptions of core categories and subcategories related to the ‘why it happened’ theme (core categories were adapted from refs 21 29)

    Initial coding of the interview data was performed by the third and fourth authors (AJ and SS), and all coding was independently (ie, blinded from initial coding) performed by the first author (JA). Any discrepancies between the coders were discussed (less than 5%) and 100% consensus was reached.

    Statistical analysis

    We first determined the positive predictive value (PPV) of voided orders that were actual medication ordering errors (from chart review) and its 95% CI. A Standard Error of Proportions (SEP) was also determined.

    We also investigated mismatches between the clinician-CPOE-selected, clinician-reported (interview) and medication ordering error reasons (chart) (see detailed descriptions of each in table 2). Using the chart review-based error reasons as the gold standard, we determined the PPV of similarity in the reasons between clinician-CPOE-selected reasons and medication ordering error reasons (chart), and clinician-reported reasons (interview) and medication ordering error reasons (chart). PPVs were computed based on the number of medication ordering errors identified from chart review.

    View this table:
    Table 2

    Description of the data sources used for analysis

    Results

    During the 6-month study period (27 January 2016 to 29 July 2016), a total of 787 void alerts were generated (4.3 alerts per day). The overall rate of medication order voiding was 0.21% during this period (787 voided orders among 377 480 inpatient medication orders for patients >18 years of age).

    Interviews were conducted on 101 voided orders: for 95 voided orders, interviews were conducted with at least the voiding clinician; six orders had interviews with just the ordering clinician. Eighty unique clinicians were interviewed: 30 physicians, 28 pharmacists and 22 nurses. A detailed summary of the sample characteristics is provided in section 3 of the online supplementary appendix.

    Reasons for medication ordering errors

    Based on chart reviews of 101 voided orders, the PPV of voided orders that were medication ordering errors was 93.1% (95% CI 88.1% to 98.1%, n=94 erroneous orders out of 101 voided orders; ie, 7 of the voided orders were not medication ordering errors). The predominant reasons for medication ordering errors for 94 orders identified as medication ordering errors were duplicate orders (n=44), followed by improperly composed orders (n=41), wrong drug (n=3), wrong encounter (n=2), voiding student order (n=2), wrong patient (n=1) and wrong time selected (n=1). Definitions for these error reasons are provided in online supplementary appendix table 6.

    For the 94 medication ordering errors, we also describe the clinician-CPOE-selected reasons and clinician-reported reasons for voiding (interview). For the clinician-CPOE-selected reasons, the most prevalent reason was duplicate order (n=58), followed by improperly composed order (n=25), voiding student order (n=6), wrong patient (n=4) and incorrect ordering physician (n=1).

    For clinician-reported reasons for voiding (interview) contributing to medication ordering errors, clinicians attributed the reason for voiding as duplicate order (n=43), improperly composed order (n=38), wrong drug (n=5), wrong encounter (n=2), discontinuation issues (n=1), delays in order processing (n=1), voiding student order (n=1), wrong patient (n=1), incorrect ordering physician (n=1) and wrong time selected for order (n=1). A summary of the error reasons using the three data sources is provided in figure 2 (additional data are also provided in online supplementary appendix table 5).

    Figure 2
    Figure 2

    Proportion of reasons for medication ordering errors based on clinician-CPOE-selected reasons, clinician-reported reasons (interview) and based on chart review. Note that the proportions for reasons are based only on those voided orders identified as medication ordering errors after chart review (n=94). The set of reasons (x-axis) includes all unique reasons that were identified across the three methods. However, some reasons appeared only in one method. For example, ‘discontinuation issues’ was identified only in the clinician-reported reasons (interview) method and not the others. Refer to online supplementary appendix table 5 for the proportions and frequency for each reason. CPOE, computerised provider order entry.

    Comparing the reasons for medication ordering errors

    Using the chart review-based reasons as the gold standard for comparison, we found that the PPV of similarity between clinician-CPOE-selected reasons and chart review was 70.2% (95% CI 61.0% to 79.4%); PPV between clinician-reported reasons (interview) and chart review was 86.1% (95% CI 78.2% to 94.1%). We found no significant association between clinician-CPOE-selected and clinician-reported (interview) reasons (χ2(1)=2.95, p=0.086).

    Factors leading to medication ordering errors

    Based on interviews, we identified several contributing factors for each of the reasons for medication ordering errors (see table 1 for the list of core and subcategories).

    The primary factors that contributed to the medication ordering errors were user-related issues (57%, n=54) and order entry process-related issues (31%, n=29), followed by system limitations (10%, n=9) and transition issues (2%, n=2). Among these, user-related issues were predominantly caused by data entry errors (37%, n=20 of 54), creation of multiple orders by multiple clinicians (26%, n=14 of 54) and lack of clinical knowledge (13%, n=7 of 54). Meanwhile, order entry issues were triggered by visual interface issues (48%, n=14 of 29) and predefined order sets or medication protocol issues (38%, n=11 of 29).

    We organised these factors under the various chart-based medication ordering errors (see online supplementary appendix table 6). Duplicate ordering errors were caused primarily by order entry (50%, n=22 of 44) and user-related issues (45%, n=20 of 44). Order entry issues arose as a result of CPOE user interface or order set template challenges. During interviews, clinicians stated that at the time of entry of medication orders, they were unable to view the complete list of active medication orders to ascertain duplication. One clinician remarked that “because originally I didn’t see that the order was placed. So, I ordered it before I tried to send him [patient] upstairs, and as I was charting it I saw it was already ordered this morning, so I voided it. I ordered it and it was a duplicate because I didn’t realize it was already ordered.” With regard to the order set/template issues, duplicate orders were placed because clinicians failed to remove selections of medications (ie, by unchecking boxes) that were included in an order set profile. As one participant stated “…a lot of it has to do with duplicate orders, a lot of it has to do with just not checking a box [i.e., by selecting a checkbox] in the whole order set. Because when they go in, they will check what they want but they won’t uncheck [i.e., by removing unnecessary checkbox selections] what’s already on the profile.”

    User-related issues contributing to duplicate orders were related to multiple clinicians placing the same medication order. This was often the case when clinicians working in a team were unaware of the other team members’ orders on their patients. This was exemplified by a voiding clinician who stated that “It [medication order] was the same thing, duplicate orders, two physicians ordered the same product. They obviously are not paying attention on what the other one is ordering.”

    Similar to duplicate ordering, medication ordering errors related to improperly composed orders were caused predominantly by user-related workflow and process issues (70%, n=29 of 41 improperly composed ordering errors), with approximately half of these being data entry errors. A voiding clinician stated: “I voided that order because I accidentally wrote it as not a PRN [pro re nata] as to administer or as scheduled, and the second I hit send I realized that I did that, so I immediately copied the order, voided the old one, and changed it to a PRN.”

    As previously described, we had a limited number of wrong drug, wrong encounter, wrong patient, voiding student order and wrong time ordering errors within our sample. A brief description of the errors, their frequencies and examples is provided in online supplementary appendix table 6.

    Factors leading to medication ordering error detection and interception

    Besides the factors contributing to the generation of medication ordering errors, we also gathered insights from voiding clinicians on factors and strategies leading to their timely identification and interception. One of the interesting findings from our interviews was the surveillance role of inpatient pharmacists in the interception of ordering errors (n=48 by pharmacists). For instance, a pharmacist stated that: “I voided it [medication order] after I spoke to the APN [Advance Practice Nurse] because we have the credit [payment], they just entered it as a non-formulary medication when they should have ordered it through the respiratory care pathways, it was not properly ordered. I look at orders all day…” In another instance, a pharmacist voided a medication order for heparin because “there were two separate orders for heparin around the same timeframe so I just voided one of the duplicate orders, because it’s part of a protocol that they do and part of order sets so that one tends to be duplicate ordered on a pretty regular basis.”

    Closely related to the pharmacists’ surveillance function was their beneficial role to clinicians as a workaround mechanism for medication ordering. In a number of cases, physicians placed improperly composed orders (eg, an extremely large dose), with the expectation that pharmacists would intercept these ‘intentional violations’30 and enter the appropriate medication order. One of the pharmacists described an instance of intentional violation as follows: “physician entered an order that did not make sense. It was 1% dextrose, it’s not something we made in the pharmacy, it was never dispensed to the patient. What the physician was trying to do was let us know that she wanted us to write the TPN [Total Parenteral Nutrition] order and that was her way to alert us. It was an impossible order; we would never have made that solution… I voided it so that the patient would not get charged for it. She wanted us, the pharmacist, to place the order, so I put in the appropriate order.”

    Discussion

    Our review of voided orders was facilitated by an automated VAT, which helped in identifying and investigating potential medication ordering errors. The PPV of medication ordering errors among void alerts was high (93%), translating to approximately four intercepted medication ordering errors per day. There was nearly 86% concordance between clinician interviews and chart review regarding the reasons for these errors. However, as described in prior research, clinician-CPOE-selected voiding reasons for medication ordering errors did not correlate with their verbally reported reasons or those found in charts.23 This worrisome mismatch could be due to limited choices available on the CPOE drop-down list of reasons. Alternatively, it also suggests that voiding clinicians either lacked a good understanding or were not careful in their drop-down selections. This discrepancy points to the need for improved taxonomy/choices for characterising medication ordering errors within the voiding process. Developing standardised operational descriptions of the reasons for voiding can help in better understanding, ascertaining and sharing of medication ordering errors and their root contributors among local and national patient safety organisations.23 31

    Consistent with prior research, the leading underlying causes for errors related to CPOE ordering were order entry process errors and user issues, resulting from a complex set of human factors, usability and system-related challenges.7 17 21 29 32–34 Similar to Wetterneck et al,32 we also found that duplicate orders were common, primarily caused by order entry process-related issues from predefined order sets and partial visual displays of active medication lists. Efforts to improve such usability issues with order sets and interface screen layouts are ongoing.13 29 35–39

    A novel insight from this study was the uncovering of intentional violations. Several clinicians described deliberately composing incorrect or improper orders for the purpose of obtaining assistance from pharmacists for identifying the right dosage, type or form for a drug, or solution such as hyperalimentation. Such ordering workarounds represent deviations from expected or standard operating procedures.30 40 The Quality in Australian Health Care Study found that deliberate violation workarounds caused up to 4.8% of adverse events; however, it is unknown how many of these events were related to medications.41 Although most of these violation workarounds did not cause patient harm, they remain a source of concern as local attitudes, motivations, procedures and technological systems allow (and, perhaps drive) them to occur.30 42 These intentional violations also underscore the critical surveillance role played by clinical pharmacists in inpatient settings that has been highlighted in prior research.19 43 44

    Using void or equivalent functions, clinicians can easily document intercepted errors within their clinical workflow and correspondingly add metadata regarding the identified error. Based on informal discussions with users and administrators of prominent EHR systems, the voiding function or a closely related functionality is available in both Cerner and Epic EHRs. The outpatient Longitudinal Medical Record system at Partners Healthcare has a similar functionality labelled as ‘Error (erroneous entry)’. It must, however, be noted that such functions are optional and are not enabled at all institutions or installations.23

    The void function can easily be turned on across institutions to capture and record medication safety data, without disrupting the clinical workflows. The voided order data set can be used to complement other error reports for understanding the rates and reasons for medication errors and for tailoring surveillance mechanisms. In addition to formalising the reasons for medication ordering errors, it is also likely that clinician education may help with streamlining the process of medication voiding and the accuracy of the selected reasons for voiding. With a growing emphasis on the need for a culture of safety to create learning health systems, clinicians may be engaged to provide more accurate reasons for voiding that can benefit their own work as well as others in the institution.

    The limitations of our study highlight the challenges of capturing highly contextual, real-time data on CPOE use and medication errors. As this study was conducted at a single academic medical centre using a single EHR, the findings may not translate to other settings or EHR systems. Nevertheless, we believe that insights on the reasons for medication ordering errors identified through voiding is generalisable to other systems. The response rate to our efforts to contact clinicians was only 37%, which is substantial, but clearly imperfect. This was likely due to several factors. First, given the real-time nature of this study, we had considerable difficulty in reaching certain clinicians (especially pharmacists and nurses) who did not carry pagers or did not respond. In other cases, nurses or pharmacists had left the unit by the time we attempted to contact them. Additionally, we did not follow up on voided orders that were placed >24 hours earlier, causing more sampling of voided orders from Sunday through Thursday. Due to the retrospective nature of interviews, there is a possibility that some of the clinicians may have misquoted or failed to recall their reasons for voiding. However, all follow-up calls included in our data set were made within a 24-hour period, potentially minimising this limitation.

    There were several instances of the same clinician performing the multiple voiding tasks for the same patient. In these cases, interview follow-up was based on the first voided order. This resulted in a loss of approximately 10% of the voided orders. In 36 orders, the ordering and voiding clinicians were the same. In these cases, the participant was asked the set of questions provided in online supplementary appendix section 2 (see questions under ‘interview guide for same voiding and ordering clinician’). Although we did not perform any analysis on this subgroup of interviews, there were no observable differences between these and other interviews.

    Chart reviewers were not blinded to the clinician-CPOE-selected reasons for voiding, which may have caused a bias. However, based on our prior research, we found that the clinician-CPOE-selected reasons were not indicative of the actual reasons for medication ordering errors.23 Hence, during the chart review training, we had instructed chart reviewers not to rely on the clinician-CPOE-selected reasons (present in the order details of the patient’s chart) during their review.

    Finally, as previously mentioned, ‘void’ is one of multiple ways clinicians can discontinue erroneous medication orders (cancel, modify or complete being other options). Hence, use of the ‘void’ function is dependent on the clinician awareness and willingness to use it. Although our previous study23 showed widespread awareness and use among clinicians, it is likely that clinicians also used other mechanisms to discontinue erroneous medication orders.

    In spite of these limitations, our study demonstrates the value of near-real-time identification of medication ordering errors and interviewing the clinicians involved to gather perspectives on the process of medication ordering error creation to interception.

    Conclusion

    Tracking medication order voiding using alerts can provide a potential mechanism for identifying, investigating and tracking medication ordering errors. We found that timely follow-up with clinicians on the contributing causes for ordering errors can help in developing a more robust taxonomy of its voiding reasons and in developing interventions that can mitigate CPOE-related challenges. Medication voiding function provides a viable reporting mechanism to develop a generalisable and standardised set of CPOE-based medication ordering error reasons that can be adopted and shared across healthcare institutions and patient safety organisations. Additionally, the surveillance role of pharmacists for monitoring inpatient orders and the mechanisms to prevent intentional violations are interesting topics that require further investigation.

    Acknowledgments

    We would like to thank Dr. Imade Ihianle for herhelp with the analysis, and Michael Jhattu for his assistance with the VAT.

    References

    1. ↵
      2. Bates DW
      . Preventing medication errors: a summary. Am J Health Syst Pharm 2007;64:S3–S9.doi:10.2146/ajhp070190
      OpenUrlAbstract/FREE Full Text
    2. ↵
      2. Gandhi TK ,
      3. Weingart SN ,
      4. Borus J , et al
      . Adverse drug events in ambulatory care. N Engl J Med 2003;348:1556–64.doi:10.1056/NEJMsa020703
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. Kaushal R ,
      3. Kern LM ,
      4. Barrón Y , et al
      . Electronic prescribing improves medication safety in community-based office practices. J Gen Intern Med 2010;25:530–6.doi:10.1007/s11606-009-1238-8
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      2. Potts AL ,
      3. Barr FE ,
      4. Gregory DF , et al
      . Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics 2004;113:59–63.doi:10.1542/peds.113.1.59
      OpenUrlAbstract/FREE Full Text
    5. ↵
      2. Cordero L ,
      3. Kuehn L ,
      4. Kumar RR , et al
      . Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol 2004;24:88–93.doi:10.1038/sj.jp.7211000
      OpenUrlCrossRefPubMed
    6. ↵
      2. Ash JS ,
      3. Sittig DF ,
      4. Poon EG , et al
      . The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007;14:415–23.doi:10.1197/jamia.M2373
      OpenUrlCrossRefPubMed
    7. ↵
      2. Slight SP ,
      3. Eguale T ,
      4. Amato MG , et al
      . The vulnerabilities of computerized physician order entry systems: a qualitative study. J Am Med Inform Assoc 2016;23:311–6.doi:10.1093/jamia/ocv135
      OpenUrlCrossRefPubMed
    8. ↵
      2. Bates DW ,
      3. Leape LL ,
      4. Cullen DJ , et al
      . Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280:1311–6.doi:10.1001/jama.280.15.1311
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      2. Galanter WL ,
      3. Bryson ML ,
      4. Falck S , et al
      . Indication alerts intercept drug name confusion errors during computerized entry of medication orders. PLoS One 2014;9:e101977.doi:10.1371/journal.pone.0101977
    10. ↵
      2. Campbell EM ,
      3. Sittig DF ,
      4. Ash JS , et al
      . Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006;13:547–56.doi:10.1197/jamia.M2042
      OpenUrlCrossRefPubMed
    11. ↵
      2. Ash JS ,
      3. Sittig DF ,
      4. Dykstra R , et al
      . The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform 2009;78(suppl 1):S69–S76.doi:10.1016/j.ijmedinf.2008.07.015
      OpenUrlCrossRefPubMed
    12. ↵
      AHRQ Patient Safety Network. Computerized Provider Order Entry. Secondary Computerized Provider Order Entry. 2015. http://psnet.ahrq.gov/printviewPrimer.aspx?primerID=6
    13. ↵
      2. Koppel R ,
      3. Metlay JP ,
      4. Cohen A , et al
      . Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293:1197–203.doi:10.1001/jama.293.10.1197
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      2. Radley DC ,
      3. Wasserman MR ,
      4. Olsho LE , et al
      . Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc 2013;20:470–6.doi:10.1136/amiajnl-2012-001241
      OpenUrlCrossRefPubMed
    15. ↵
      2. Chou D
      . Health IT and patient safety: building safer systems for better care. JAMA 2012;308:2282–82.doi:10.1001/jama.308.21.2282-a
      OpenUrl
    16. ↵
      US Food and Drug Administratio. 2014. FDASIA Health IT Report: Proposed Strategy and Recommendations for a Risk-Based Framework.
    17. ↵
      2. Schiff GD ,
      3. Hickman TT ,
      4. Volk LA , et al
      . Computerised prescribing for safer medication ordering: still a work in progress. BMJ Qual Saf 2016;25:315–9.doi:10.1136/bmjqs-2015-004677
      OpenUrlFREE Full Text
    18. ↵
      2. Bates DW ,
      3. Boyle DL ,
      4. Vander Vliet MB , et al
      . Relationship between medication errors and adverse drug events. J Gen Intern Med 1995;10:199–205.doi:10.1007/BF02600255
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      2. Kaushal R ,
      3. Shojania KG ,
      4. Bates DW
      . Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163:1409–16.doi:10.1001/archinte.163.12.1409
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      2. Nebeker JR ,
      3. Hoffman JM ,
      4. Weir CR , et al
      . High rates of adverse drug events in a highly computerized hospital. Arch Intern Med 2005;165:1111–6.doi:10.1001/archinte.165.10.1111
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      2. Schiff GD ,
      3. Amato MG ,
      4. Eguale T , et al
      . Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015;24:264–71.doi:10.1136/bmjqs-2014-003555
      OpenUrlAbstract/FREE Full Text
    22. ↵
    23. ↵
      2. Kannampallil TG ,
      3. Abraham J ,
      4. Solotskaya A , et al
      . Learning from errors: analysis of medication order voiding in CPOE systems. J Am Med Inform Assoc 2017.doi:10.1093/jamia/ocw187
    24. ↵
      2. Dean B ,
      3. Barber N ,
      4. Schachter M
      . What is a prescribing error? Qual Health Care 2000;9:232–7.doi:10.1136/qhc.9.4.232
      OpenUrlAbstract/FREE Full Text
    25. ↵
      2. Aspden P ,
      3. Wolcott J ,
      4. Bootman JL , et al
      . Preventing medication errors: quality chasm series. National Academies Press, 2006.
    26. ↵
      Institute of Medicine. Preventing Medication Errors. Committee on Identifying and preventing medication Errors Board on Health Care Services. Institute of Medicine, 2006.
    27. ↵
      Taxonomy of Medication Errors. National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP), 2001. http://www.nccmerp.org/sites/default/files/taxonomy2001-07-31.pdf
    28. ↵
      2. Manias E
      . Detection of medication-related problems in hospital practice: a review. Br J Clin Pharmacol 2013;76:7–20.doi:10.1111/bcp.12049
      OpenUrlCrossRefPubMed
    29. ↵
      2. Brown CL ,
      3. Mulcaster HL ,
      4. Triffitt KL , et al
      . A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc 2017;24:432–40.doi:10.1093/jamia/ocw119
      OpenUrl
    30. ↵
      2. Amalberti R ,
      3. Vincent C ,
      4. Auroy Y , et al
      . Violations and migrations in health care: a framework for understanding and management. Qual Saf Health Care 2006;15(suppl 1):i66–i71.doi:10.1136/qshc.2005.015982
      OpenUrlAbstract/FREE Full Text
    31. ↵
      2. Amato MG ,
      3. Salazar A ,
      4. Hickman TT , et al
      . Computerized prescriber order entry-related patient safety reports: analysis of 2522 medication errors. J Am Med Inform Assoc 2017;24:316–22.doi:10.1093/jamia/ocw125
      OpenUrl
    32. ↵
      2. Wetterneck TB ,
      3. Walker JM ,
      4. Blosky MA , et al
      . Factors contributing to an increase in duplicate medication order errors after CPOE implementation. J Am Med Inform Assoc 2011;18:774–82.doi:10.1136/amiajnl-2011-000255
      OpenUrlCrossRefPubMed
    33. ↵
      2. Patel VL ,
      3. Kannampallil TG
      . Cognitive informatics in biomedicine and healthcare. J Biomed Inform 2015;53:3–14.doi:10.1016/j.jbi.2014.12.007
      OpenUrl
    34. ↵
      2. Patel VL ,
      3. Kannampallil TG ,
      4. Shortliffe EH
      . Role of cognition in generating and mitigating clinical errors. BMJ Qual Saf 2015;24:468–74.doi:10.1136/bmjqs-2014-003482
      OpenUrlAbstract/FREE Full Text
    35. ↵
      2. Khajouei R ,
      3. Jaspers MW
      . The impact of CPOE medication systems' design aspects on usability, workflow and medication orders: a systematic review. Methods Inf Med 2010;49:3–19.doi:10.3414/ME0630
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      2. Thiel SW ,
      3. Asghar MF ,
      4. Micek ST , et al
      . Hospital-wide impact of a standardized order set for the management of bacteremic severe sepsis. Crit Care Med 2009;37:819–24.doi:10.1097/CCM.0b013e318196206b
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      2. Horsky J ,
      3. Kuperman GJ ,
      4. Patel VL
      . Comprehensive analysis of a medication dosing error related to CPOE. J Am Med Inform Assoc 2005;12:377–82.doi:10.1197/jamia.M1740
      OpenUrlCrossRefPubMed
    38. ↵
      2. Khajouei R ,
      3. Peek N ,
      4. Wierenga PC , et al
      . Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform 2010;79:690–8.doi:10.1016/j.ijmedinf.2010.08.001
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      2. Reckmann MH ,
      3. Westbrook JI ,
      4. Koh Y , et al
      . Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009;16:613–23.doi:10.1197/jamia.M3050
      OpenUrlCrossRefPubMed
    40. ↵
      2. Runciman W ,
      3. Hibbert P ,
      4. Thomson R , et al
      . Towards an International classification for Patient Safety: key concepts and terms. Int J Qual Health Care 2009;21:18–26.doi:10.1093/intqhc/mzn057
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      2. Wilson RM ,
      3. Runciman WB ,
      4. Gibberd RW , et al
      . The Quality in Australian Health Care Study. Med J Aust 1995;163:458–71.
      OpenUrlPubMedWeb of Science
    42. ↵
      2. Taylor JA ,
      3. Loan LA ,
      4. Kamara J , et al
      . Medication administration variances before and after implementation of computerized physician order entry in a neonatal intensive care unit. Pediatrics 2008;121:123–8.doi:10.1542/peds.2007-0919
      OpenUrlAbstract/FREE Full Text
    43. ↵
      2. Blum KV ,
      3. Abel SR ,
      4. Urbanski CJ , et al
      . Medication error prevention by pharmacists. Am J Hosp Pharm 1988;45:1902–3.
      OpenUrlPubMedWeb of Science
    44. ↵
      2. Leape LL ,
      3. Cullen DJ ,
      4. Clapp MD , et al
      . Pharmacist participation on physician rounds and adverse drug events in the intensive care unit. JAMA 1999;282:267–70.doi:10.1001/jama.282.3.267
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors JA, TGK and WG conceptualised and designed the study. JA, TGK, WG, SS and AJ collected the data. JA, TGK, WG, SS, CR, and AJ were involved in the organisation and analysis of data. JA, TGK, WG and GS were involved in interpreting the results. All authors were involved critically reviewing, revising and finalising the manuscript.

    • Funding This study was supported by a pilot grant fundedby the College of Applied Health Sciences at the University of Illinois atChicago

    • Competing interests None declared.

    • Patient consent No patients were recruited for this study. Interviews were conducted with consented clinicians only.

    • Ethics approval University of Illinois at Chicago Office for the Protection of Research Subjects.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Linked Articles