The practice of emergency medicine/original research
Usability Study of Two Common Defibrillators Reveals Hazards

Presented in part at the Human Factors and Ergonomics Society 48th annual meeting, September 2004, New Orleans, LA; and the National Association of EMS Physicians meeting, January 2006, Tucson, AZ.
https://doi.org/10.1016/j.annemergmed.2007.03.029Get rights and content

Study objective

This usability study evaluates the user interface of 2 common monitor-defibrillators, the Lifepak10 and Lifepak12, to identify use-related hazards.

Methods

Fourteen paramedics familiar with both devices completed 4 EMS simulator scenarios using each device. The scenarios involved “quick look” and monitoring, defibrillation, synchronized cardioversion, and replacing paper. Qualitative and quantitative data were collected, including both participant self-evaluation (scored 1 to 9) and expert observer evaluation (scored 0 to 4).

Results

Participant ratings demonstrated that for performing a quick look, the Lifepak10 was easier to use (mean 8.0 versus 7.1), and for synchronized cardioversion the Lifepak12 was easier (mean 6.7 versus 5.3). Participants performed better on the Lifepak12 than the Lifepak10 for synchronized cardioversion (mean 3.1 versus 1.6) and replacing paper (mean 3.0 versus 2.1). One participant did not complete the final questionnaire. Of the remaining 13, 11 (85%) participants preferred the Lifepak12 for use on a regular basis. Eight (62%) paramedics thought that the Lifepak12 would be more effective in an emergency; 9 (69%) believed that the Lifepak10 is quicker to learn. Paramedics reported difficulty using the devices with gloves and confusion in “sync” mode. Of note, 50% of participants inadvertently delivered an unsynchronized countershock for supraventricular tachycardia.

Conclusion

Although the Lifepak10 is easier to learn, the Lifepak12 is perceived as easier to use and more effective in emergencies. The high failure rate in synchronized cardioversion indicates a need to reevaluate the user interface design for this function. Limitations of this study include the use of simulation.

Introduction

Medical providers often depend on medical devices such as monitor-defibrillators in critical and time-dependent situations. It is important that these devices be designed with an emphasis on reducing the potential for adverse events.1 Although the traditional response to adverse events and near misses in medicine has been to blame the provider, experts in patient safety have demonstrated that there is often a deficiency in a system component, such as the user-interface design of a medical device, that is the actual root cause.2, 3, 4, 5, 6, 7, 8, 9, 10

Medical providers interact with medical devices through the user interface, which typically consists of visual and auditory displays (to communicate information to the user) and controls (to communicate instructions to the device). A good user-interface design follows human-factors engineering design standards and takes into consideration the capabilities and limitations of the user, as well as any limitations imposed by the environment(s) in which the device is intended to be used.

The user interface has a surprisingly powerful ability to facilitate and avert hazards.6, 11, 12 The evaluation of user-interface design is a well-established component of safety engineering in other complex industries, but its role in the medical industry is underrecognized.1, 13, 14

Usability testing is a method used by human factors engineers to evaluate a device’s user interface and its effect on user performance and safety.14, 15, 16, 17, 18, 19 Few usability studies are found in the medical literature, and we are aware of none that examine manual monitor-defibrillator devices.20, 21, 22

Emergency medical services (EMS) providers are concerned about adverse events.23 Monitor-defibrillators are complex medical devices, and their use has been shown to save lives in the out-of-hospital environment, but they also have potential for harm.9, 24 Identification of use-related hazards and subsequent optimization of user-interface design are essential for safe operation of these devices.

The objectives of this study were to evaluate and compare the usability of 2 commonly used manual monitor-defibrillators and to identify user-interface-related hazards that may lead to adverse events.

Section snippets

Study Design

This is a prospective crossover study of paramedic use of manual monitor-defibrillator devices in a simulated EMS environment. In contrast to traditional research methods, in usability testing the object of the research is the device; the goal is not to assess users’ performance but rather to identify design characteristics that could lead to hazards in use. The study was approved by the University of Rochester Research Subjects Review Board.

Setting

Usability testing was conducted at the Monroe County

Results

The 14 participants were 21% women and had an average of 9 years’ experience as advanced life support EMS providers (range 1 to 24).

Participants’ ease of use ratings are summarized in the Table. There were similar ratings between devices for tasks 2 and 4. The newer Lifepak12 was deemed less easy to use for the routine monitoring task yet easier to use for the more complex synchronized cardioversion task. Observer ratings, also shown in the Table, revealed similar success between devices for

Limitations

Several limitations to our study must be recognized. First, the devices are capable of performing more functions than were tested in our scenarios. However, we chose routine and emergency tasks that are typical of those performed by paramedics and some that incorporate multiple functions. We did not control for previous experience with the individual devices in our analyses. But all participants reported familiarity (and field experience) with both models, and the need to use both models is a

Discussion

Traditionally in medicine when an adverse event occurs the natural reaction is to assign fault to a person. The systems approach to reducing adverse events emphasizes that latent factors exist that either facilitate hazards or fail to protect patients from the effects of hazards. Failures in medical device user-interface design serve as an example of latent errors that can lead to adverse events.

Although usability testing has been recognized as an important component of medical device

References (34)

  • M.S. Bogner

    There is more to error in healthcare than the care provider

    Proceedings of the Human Factors and Ergonomics Society 49th annual meeting, Orlando, FL

    (2005)
  • T.E. Lum et al.

    Misplaced femoral line guidewire and multiple failures to detect foreign body on chest x-ray

    Acad Emerg Med

    (2005)
  • M.E. Wiklund

    Eleven keys to designing error-resistant medical devices

    MDDI

    (2002)
  • A.W. Kushniruk et al.

    The relationship of usability to medical error: an evaluation of errors associated with usability problems in the use of a handheld application for prescribing medications

    Medinfo

    (2004)
  • American National Standards Institute, Association for the Advancement of Medical Instrumentation. HE 74: 2001—Human...
  • CDRH. Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management. Guidance for Industry...
  • A.M. Wichansky

    Usability testing in 2000 and beyond

    Ergonomics

    (2000)
  • Cited by (38)

    • The role of human factors in neonatal patient safety

      2019, Seminars in Perinatology
      Citation Excerpt :

      Human factors techniques can be used to develop processes that will ensure the most reliable integration and operation of a device or task. Finally, human factors principles can be used to understand how designs can predispose to errors and accidents.8 Some interactions are linear and deterministic with predictable outcomes; others are complex and adaptive with unpredictable outcomes.

    • Defibrillator Design and Usability May Be Impeding Timely Defibrillation

      2018, Joint Commission Journal on Quality and Patient Safety
      Citation Excerpt :

      Fairbanks et al. evaluated the use of two manual-mode defibrillators (the Lifepak 10 and Lifepak 12) by paramedics in a simulated environment.21 For both devices, the difficulty of performing a defibrillation was rated 6.9 on a scale of 1–9 (1 = “very difficult”; 9 = “very easy”).21 This contrasts with our results in which difficulty of performing a defibrillation with the Philips HeartStart XL was rated 3.9.

    • Design: A Neglected Modality for Improvement

      2017, Annals of Emergency Medicine
    View all citing articles on Scopus

    Supervising editor: Robert L. Wears, MD, MS

    Author contributions: RJF and SHC conceived the study, designed the study, and obtained research funding. RJF, SHC, and MNS supervised the conduct of the trial and data collection. RJF, PAB, AMM, and MNS undertook recruitment of participating paramedics. RJF, SHC, and MNS managed the data. All investigators participated in the qualitative analysis. MNS provided statistical advice and RJF, SHC, and MNS analyzed the data. RJF and SHC drafted the article, and all authors contributed substantially to its revision. RJF takes responsibility for the paper as a whole.

    Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article, that may create any potential conflict of interest. See the Manuscript Submission Agreement in this issue for examples of specific conflicts covered by this statement. This research was supported by the University of Rochester Department of Emergency Medicine research seed funding. At the time of the study, Drs. Fairbanks and Shah were supported by funding from the National Institutes of Health (1R41NR009592) and Agency for Healthcare Research and Quality (1U18HS015818), and Dr. Shah, by funding from Health Resources and Services Administration.

    Available online May 11, 2007.

    Reprints not available from the authors.

    1

    Mr. Marks is now with Bridgeborn Government Solutions, Arlington, VA.

    View full text