Understanding handling of drug safety alerts: a simulation study

https://doi.org/10.1016/j.ijmedinf.2010.01.008Get rights and content

Abstract

Purpose

To study correctness of drug safety alert handling and error type in a computerized physician order entry (CPOE) system in a simulated work environment.

Methods

Disguised observation study of 18 physicians (12 from internal medicine and 6 from surgery) entering 35 orders of predefined patient cases with 13 different drug safety alerts in a CPOE. Structured interviews about how the generated drug safety alerts were handled in the simulation test and resemblance of the test to the normal work environment. Handling and reasons for this were scored for correctness and error type.

Results

Thirty percent of alerts were handled incorrectly, because the action itself and/or the reason for the handling were incorrect. Sixty-three percent of the errors were categorized as rule based and residents in surgery used incorrect justifications twice as often as residents in internal medicine. They often referred to monitoring of incorrect substances or parameters. One alert presented as a second alert in one screen was unconsciously overridden several times. One quarter of residents showed signs of alert fatigue.

Conclusion

Although alerts were mainly handled correctly, underlying rules and reasoning were often incorrect, thereby threatening patient safety. This study gave an insight into the factors playing a role in incorrect drug safety alert handling that should be studied in more detail. The results suggest that better training, improved concise alert texts, and increased specificity might help. Furthermore, the safety of the predefined override reason ‘will monitor’ and double alert presentation in one screen is questioned.

Introduction

Overriding of drug safety alerts in computerized physician order entry systems (CPOEs) is very common and occurs in 49–96% of cases [1]. However, frequent overriding may cause alert fatigue, important alerts being overridden along with unimportant ones, thus impairing patient safety. Research into overriding has been focused on the extent of overriding, reasons and causes for overriding in general, effects of overriding and suggestions for useful alerts [1]. Studies on the role played by cognitive processes in overriding drug safety alerts are lacking [1]. It is not clear which cognitive level is used in interpretation and handling of drug safety alerts, which kind of errors are made and which factors determine these processes. Understanding the reasons for, and causes of, overriding in particular cases is necessary for the development of effective alerting systems that are safe and acceptable to users [1].

The aim of this study is to gain an insight into errors occurring in drug safety alert handling. The research questions are:

  • 1.

    How many, and which type of, errors are made in handling drug safety alerts?

  • 2.

    Are there any signs of alert fatigue or dependency on the alerting system?

Section snippets

Background

The handling of drug safety alerts can be divided into several steps: the alert has to be read and understood, the consequences of the intended plan have to be weighed, and the intended action has to be performed [1]. In each step different types of errors can be made.

Reason divides human errors into slips, lapses and mistakes [2], [3]. Slips are acts not intended, nor attended; errors in which the right intention is incorrectly carried out, for example clicking the override button instead of

Setting

The Erasmus University Medical Center in Rotterdam, the Netherlands is a 1237-bed academic medical center that started introducing CPOE in December 2001. Since March 2005 all inpatient wards, intensive care units excluded (1107 beds), have been using the Windows-based CPOE system Medicatie/EVS® (iSOFT, Leiden, the Netherlands) [8].

Test development

A simulation test was developed with 6 patient cases, 35 orders and 13 drug safety alerts of different types (2 duplicate orders, 8 drug–drug interactions (DDIs), and

Results

Ninety residents (33 from internal medicine, 57 from surgery) were invited for the study. Nineteen residents (21%) completed the test, 13 (39%) from internal medicine and 6 (11%) from surgery. One internal medicine resident had to be excluded because he misinterpreted the task to be performed in the study. He thought the order entry had to do with continuation of drugs the patient was already taking and he admitted to have overridden alerts for that reason. The 18 residents included were all in

Errors in alert handling

This study in a simulated work environment showed that physicians handled many drug safety alerts incorrectly at the RB level, using many different rules. Respondents often justified their overrides referring to monitoring serum levels or patient conditions, but the substances mentioned or patient parameters were often incorrect. Four respondents said they would monitor renal function, diuresis or electrolytes when blood pressure monitoring was appropriate, others referred to drug serum level

Conclusions

Drug safety alerts were mainly handled rule based, but incorrect rules or reasoning were often used to justify actions. Residents in surgery justified their actions incorrectly twice as often as residents in internal medicine. Main causes of errors were rules that were incorrect or not applicable, such as monitoring of incorrect serum levels or patient parameters, among others. Insufficient training and low specificity played a role in erroneous alert handling. Furthermore, a second alert in

Authors’ contributions

Heleen van der Sijs was involved in conception and design of the study, analysis and interpretation of data and writing the manuscript. Teun van Gelder and Jos Aarts were involved in conception and design of the study, analysis and interpretation of data, revising the manuscript critically, and for final approval. Marc Berg and Arnold Vulto were involved in conception and design of the study, revising the manuscript critically and for final approval.

Acknowledgements

The authors thank all respondents for their participation. The authors thank Shantie Anant for performing the tests.

Summary points

What was already known on the topic?

  • Drug safety alerts in CPOE are frequently overridden.

  • Alert fatigue may compromise patient safety.

What this study added to our knowledge?

  • Physicians use many different reasons justifying alert overriding.

  • Rules and reasoning justifying alert overriding are often incorrect, such as monitoring of incorrect substances or patient

References (23)

  • A. Hobbs et al.

    Skills, rules and knowledge in aircraft maintenance: errors in context

    Ergonomics

    (2002)
  • Cited by (62)

    • Influence of smart real-time electronic alerting on glucose control in critically ill patients

      2015, Journal of Critical Care
      Citation Excerpt :

      These computerized insulin algorithms showed improvement in time to target glycemia and numbers of aberrant glycemia events, while reducing workload [25]. On the other hand, too many, irrelevant or false-positive alerts will lead to alert fatigue and, consequently, nonacceptance [27]. Hence, it is of utmost importance to find a perfect balance between sensitive and relevant alerting.

    • Evaluation of medication dose alerts in pediatric inpatients

      2013, International Journal of Medical Informatics
      Citation Excerpt :

      However, these systems are associated with new challenges, and may introduce new types of errors in the prescribing and administration process [24,25]. A significant amount of information is available on adult dose checking systems [26,27]. While large scale pediatric dose range checking systems have been described [28], to our knowledge there has been only one report on pediatric provider compliance to automated warnings, which did not include medication dose range alerts [29].

    View all citing articles on Scopus
    View full text